Awesome

Python API and Evaluation Code for v1.0 release of the VQA dataset.

This release of the dataset consists of

• Real
  • 82,783 MS COCO training images, 40,504 MS COCO validation images and 81,434 MS COCO testing images (images are obtained from [MS COCO website] (http://mscoco.org/dataset/#download))
  • 248,349 questions for training, 121,512 questions for validation and 244,302 questions for testing (3 per image)
  • 2,483,490 answers for training and 1,215,120 answers for validation (10 per question)
• Abstract
  • 20,000 training images, 10,000 validation images and 20,000 MS COCO testing images
  • 60,000 questions for training, 30,000 questions for validation and 60,000 questions for testing (3 per image)
  • 600,000 answers for training and 300,000 answers for validation (10 per question)

There are two types of tasks

• Open-ended task
• Multiple-choice task (18 choices per question)

Requirements

• python 2.7
• scikit-image (visit this page for installation)
• matplotlib (visit this page for installation)

Files

./Questions

• For both real and abstract, download the question files from the VQA download page, extract them and place in this folder.
• Question files from Beta v0.9 release (123,287 MSCOCO train and val images, 369,861 questions, 3,698,610 answers) can be found below
  • training question files
  • validation question files
• Question files from Beta v0.1 release (10k MSCOCO images, 30k questions, 300k answers) can be found here.

./Annotations

• For both real and abstract, download the annotation files from the VQA download page, extract them and place in this folder.
• Annotation files from Beta v0.9 release (123,287 MSCOCO train and val images, 369,861 questions, 3,698,610 answers) can be found below
  • training annotation files
  • validation annotation files
• Annotation files from Beta v0.1 release (10k MSCOCO images, 30k questions, 300k answers) can be found here.

./Images

• For real, create a directory with name mscoco inside this directory. For each of train, val and test, create directories with names train2014, val2014 and test2015 respectively inside mscoco directory, download respective images from MS COCO website and place them in respective folders.
• For abstract, create a directory with name abstract_v002 inside this directory. For each of train, val and test, create directories with names train2015, val2015 and test2015 respectively inside abstract_v002 directory, download respective images from VQA download page and place them in respective folders.

./PythonHelperTools

• This directory contains the Python API to read and visualize the VQA dataset
• vqaDemo.py (demo script)
• vqaTools (API to read and visualize data)

./PythonEvaluationTools

• This directory contains the Python evaluation code
• vqaEvalDemo.py (evaluation demo script)
• vqaEvaluation (evaluation code)

./Results

• OpenEnded_mscoco_train2014_fake_results.json (an example of a fake results file to run the demo)
• Visit [VQA evaluation page] (http://visualqa.org/evaluation) for more details.

./QuestionTypes

• This directory contains the following lists of question types for both real and abstract questions. In a list, if there are question types of length n+k and length n with the same first n words, then the question type of length n does not include questions that belong to the question type of length n+k.
• mscoco_question_types.txt
• abstract_v002_question_types.txt

References

• VQA: Visual Question Answering
• Microsoft COCO

Developers

• Aishwarya Agrawal (Virginia Tech)
• Code for API is based on MSCOCO API code.
• The format of the code for evaluation is based on MSCOCO evaluation code.