Awesome

MM-SADA_Domain_Adaptation_Splits

This repository contains the annotations for the domain adaptation dataset used in the paper Multi-Modal Domain Adaptation for Fine-Grained Action Recognition.

BibTeX

If this repository was utilised, please cite:

@InProceedings{munro20multi,
author = "Munro, Jonathan and Damen, Dima",
title = "{M}ulti-modal {D}omain {A}daptation for {F}ine-grained {A}ction {R}ecognition",
booktitle = "Computer Vision and Pattern Recognition (CVPR)",
year = "2020"
}

Annotations

Three domains are defined as D1, D2 and D3 from individual kitchens in the EPIC Kitchens dataset (P08, P01 and P22 respectively).

D*_train.pkl - Contains action segments for either a labelled source or unlabelled target domain. For an unlabelled target domain only video id's and timestamps should be used.

D*_test.pkl - Contains action segments for evaluation only.

verb_class is a numeric id used as the ground truth action prediction in this work.

Each pickle file contains a pandas.DataFrame with 10 columns:

Column Name	Type	Example	Description
`uid`	int	`12917`	Unique ID of the segment.
`video_id`	string	`P08_01`	Video the segment is in.
`narration`	string	`close fridge`	English description of the action provided by the participant.
`start_timestamp`	string	`00:00:07.29`	Start time in `HH:mm:ss.SSS` of the action.
`stop_timestamp`	string	`00:00:08.95`	End time in `HH:mm:ss.SSS` of the action.
`start_frame`	int	`437`	Start frame of the action (WARNING only for RGB frames extracted as detailed in Video Information).
`stop_frame`	int	`537`	End frame of the action (WARNING only for RGB frames extracted as detailed in Video Information).
`participant_id`	string	`P08`	ID of the participant.
`verb`	string	`close`	Parsed verb from the narration.
`verb_class`	int	`3`	Numeric ID of the parsed verb's class.

Flow modality start and stop times

Optical Flow was calcuated with a stride=2 in EPIC Kitchens, therefore the start and stop frames for the Flow modality are (start_frame/2, stop_frame/2).

Downloading Frames

download_script.sh will download the frames from the relevent participants P08, P02 and P22 into the below directory structure. Unless an argument is specfied, the directory structure will be created in "$HOME/Downloads/EPIC_KITCHENS_UDA".

~/Downloads/EPIC_KITCHENS_UDA/
└── frames_rgb_flow
    ├── rgb
    │   ├── test
    │   │   ├── D1
    │   │   │   ├── P08_10.tar
    │   │   │   ├── ...
    │   │   ├── D2
    │   │   │   ├── P01_11.tar
    │   │   │   └── ...
    │   │   └── D3
    │   │       ├── P22_01.tar
    │   │       └── ...
    │   └── train
    │       ├── D1
    │       │   └── ...
    │       ├── D2
    │       │   └── ...
    │       └── D3
    │           └── ...
    └── flow
        ├── ... same file structure as rgb