Awesome

EPIC-KITCHENS Multi-Instance Retrieval (MIR) baselines

Train/Test splits for the Multi-Instance Retrieval Challenge are available here

To participate and submit results to the challenge, refer to the Multi-Instance Retrieval Codalab Challenge

Full information about how to use the jPoSE code can be found here.

Full information about how to use the MI-MM code can be found here.

Trained Models

Trained models can be found within data/models for both MLP and JPoSE by downloading the data folder here. These were trained using PyTorch with the arguments within the args.txt files.

Trained models can be found within data/models for MI-MM by downloading the data folder here.

Video Features

Features extracted using TBN[1] can be found at on dropbox separated for train and test (~10GB). Both are pickle files containing the features for the Multi-Instance Retrieval train and test splits respectively.

Each represents a python dictionary containing the 'RGB', 'Flow' and 'Audio' features as a matrix of size nx25x1024 where n is the number of videos (67,217/9,668). The ordering of the videos is the same as in EPIC_100_retrieval_train.pkl and EPIC_100_retrieval_test.pkl found in the EPIC-KITCHENS-100 repo.

Temporally grouped features (mean/max) can be found in the data folder here.

Features extracted using S3D trained on HowTo100M can be found within data/features by downloading the data folder here.

Training and Evaluation

Information for training and evaluating models can be found in the JPoSE repo and MI-MM repo, including how to generate a submission file for the codalab challenge.