Home

Awesome

Attention in Attention: Modeling Context Correlation for Efficient Video Classification (IEEE TCVST 2022)

This is an official implementaion of paper "Attention in Attention: Modeling Context Correlation for Efficient Video Classification", which has been accepted by IEEE TCVST 2022. Paper link

<div align="center"> <img src="demo/AIA.jpg" width="1100px"/> </div>

Updates

Apr 20, 2022

Content

Prerequisites

The code is built with following libraries:

For video data pre-processing, you may need ffmpeg.

Data Preparation

Code

Pretrained Models

Here we provide some of the pretrained models.

Something-Something

Something-Something-V1

ModelFrame * view * clipTop-1 Acc.Top-5 Acc.Checkpoint
AIA(TSN) ResNet508 * 1 * 148.5%77.2%link

Something-Something-V2

ModelFrame * view * clipTop-1 Acc.Top-5 Acc.Checkpoint
AIA(TSN) ResNet508 * 1 * 160.3%86.4%link

Diving48

ModelFrame * view * clipTop-1 Acc.Checkpoint
AIA(TSN) ResNet508 * 1 * 179.3%link
AIA(TSM) ResNet508 * 1 * 179.4%link

EGTEA Gaze

ModelFrame * view * clipSplit1Split2Split3
AIA(TSN) ResNet508 * 1 * 163.7%62.1%61.5%
AIA(TSN) ResNet508 * 1 * 164.7%63.3%62.2%

Train

Test

Contributors

GC codes are jointly written and owned by Dr. Yanbin Hao and [Dr. Shuo Wang].

Citing

Acknowledgement

Thanks for the following Github projects: