Home

Awesome

<img src="/imgs/grid_img.png" align="top" width="1028" height="234"/>

Foodi-ML dataset

<img src="/imgs/Glovo_logo.png" align="right" width="320" height="180"/> This is the GitHub repository for the Food Drinks and groceries Images Multi Lingual (FooDI-ML) dataset. This dataset contains over 1.5M unique images and over 9.5M store names, product names, descriptions and collection sections gathered from the Glovo application. The data made available corresponds to food, drinks and groceries products from over 37 countries in Europe, the Middle East, Africa and Latin America. The dataset comprehends 33 languages, including 870k samples of languages of countries from Eastern Europe and West Asia such as Ukrainian and Kazakh, which have been so far underrepresented in publicly available visio-linguistic datasets. The dataset also includes widely spoken languages such as Spanish and English.

License

The FooDI-ML dataset is offered under the BY-NC-SA license.

1. Download the dataset

The FooDI-ML dataset is hosted in a S3 bucket in AWS. Therefore AWS CLI is needed to download it. Our dataset is composed of:

1.1. Download AWS CLI

If you do not have AWS CLI already installed, please download the latest version of AWS CLI for your operating system.

1.2. Download FooDI-ML

  1. Run the following command to download the DataFrame in ENTER_DESTINATION_PATH directory. We provide an example as if we were going to download the dataset in the directory /mnt/data/foodi-ml/.

    aws s3 cp s3://glovo-products-dataset-d1c9720d/glovo-foodi-ml-dataset.csv ENTER_DESTINATION_PATH --no-sign-request

    Example: aws s3 cp s3://glovo-products-dataset-d1c9720d/glovo-foodi-ml-dataset.csv /mnt/data/foodi-ml/ --no-sign-request

  2. Run the following command to download the images in ENTER_DESTINATION_PATH/dataset directory (please note the appending of /dataset). This command will download the images in ENTER_DESTINATION_PATHdirectory.

    aws s3 cp --recursive s3://glovo-products-dataset-d1c9720d/dataset ENTER_DESTINATION_PATH/dataset --no-sign-request --quiet

    Example: aws s3 cp --recursive s3://glovo-products-dataset-d1c9720d/dataset /mnt/data/foodi-ml/dataset --no-sign-request --quiet

  3. Run the script rename_images.py. This script modifies the DataFrame column to include the paths of the images in the location you specified with ENTER_DESTINATION_PATH/dataset.

    pip install pandas
    python scripts/rename_images.py --output-dir ENTER_DESTINATION_PATH
    
  4. Run the script scripts/dataset_preprocess.py in order to filter the dataset:

python scripts/dataset_preprocess.py --dataset-path <ENTER_PATH_TO_DATSET_FOLDER>

Getting started

Our dataset is managed by the DataFrame glovo-foodi-ml-dataset.csv. This dataset contains the following columns:

Dataset Statistics

A notebook analyzing several dataset statistics is provided in notebooks/FooDI-ML Dataset Stats Analytics.ipynb.

Benchmarks

Our paper includes 3 benchmarks: Text to Image/Image to Text Retrieval

Conditional Image Generation

Citation

You can cite our paper in arxiv: https://arxiv.org/abs/2110.02035