Home

Awesome

Mapillary Street-level Sequences

:newspaper: News

Description

Mapillary Street-level Sequences (MSLS) is a large-scale long-term place recognition dataset that contains 1.6M street-level images.

🔥 Using MSLS

We've included an implementation of a PyTorch Dataset in datasets/msls.py. It can be used for evaluation (returning database and query images) or for training (returning triplets). Check out the demo to understand its usage.

📊 Standalone evaluation script

A standalone evaluation script is available for all tasks. It reads the predictions from a text file (example) and prints the metrics.

Here we show results of models consisting of a Resnet50 backbone followed by Generalized Mean Layer. The models are trained with either the standard triplet loss or the uncertainty-aware Bayesian triplet loss. All models are trained with standard hard negative mining on image resolution 224x224.

Results on test set (Miami, Athens, Buenos Aires, Stockholm, Bengaluru, Kampala):

LossR@1R@5R@10R@20M@1M@5M@10M@20
Triplet Loss0.3720.5220.5820.6360.3720.2610.2340.228
Bayesian Triplet Loss0.3660.5130.5740.6290.3660.2530.2290.222

Results on validation set (San Francisco, Copenhagen)

LossR@1R@5R@10R@20M@1M@5M@10M@20
Triplet Loss0.6230.7800.8300.8590.6230.4320.3800.372
Bayesian Triplet Loss0.6180.7460.8050.8390.6180.4190.3690.360

📊 Evaluate on the test set

For evaluating on the test set, we've hosted a Codalab competition here

Make sure to submit a single .zip file containing a single .csv file following the format described above for the standalone evaluation script. The (example) in this repository is for the validation set, you must make sure to submit predictions for the test set or your submission will fail.

If your submission fails, read the log to find out why. Known reasons:

If you get a different error, please file a Github Issue

📦 Package structure

All the archives can be extracted in the same directory resulting in the following tree:

The meta files include the following information:

⚖ Legal

Copyright © Meta Platforms, Inc

This repository is MIT licensed.

Terms of Use

Privacy Policy