Awesome

FocusMAE

This is the official implementation for the CVPR 2024 paper FocusMAE: Gallbladder Cancer Detection from Ultrasound Videos with Focused Masked Autoencoders.

DATA PREPARATION

Refer to the instructions in VideoMAE v2 Repository for this step.

Additionally, we suggest using the --test_randomization argument while testing for best results.

For the region priors using FasterRCNN model, obtain the region proposals in a JSON file for each video using this code.

Specify the path for folder containing the json files in the dataloader.

DATASET

We contribute additional videos to our Ultrasound video dataset (GBUSV). The complete dataset comprises of 59 videos with malignancy and 32 videos which are benign. The dataset donload instructions are available in this link.

The COVID-19 CT Dataset can be obtained here

INSTALLATION

The required packages are in the file requirements.txt, and you can run the following command to install the environment

conda create --name videomae python=3.8 -y
conda activate videomae

conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 -c pytorch

pip install -r requirements.txt

Note:

• The above commands are for reference only, please configure your own environment according to your needs.
• We recommend installing PyTorch >= 1.12.0, which may greatly reduce the GPU memory usage.
• It is recommended to install timm == 0.4.12, because some of the APIs we use are deprecated in the latest version of timm.

USAGE INSTRUCTIONS

The folder scripts contains files for Finetuning and Pre-training.

In each script specify the following:

OUTPUT_DIR :

• Working directory name which saves all the checkpoints
• Each working directory folder structure looks like this:- Dataset_folder/work_dir/output_dir_name/checkpoint_folder
• You can either download the model checkpoints and pretrained folders in the same format, or download individual checkpoint from the links in the table and place them in the folder structure desscribed above.

MODEL_PATH :

• Specify the path of the pretrained model to finetune from
• You can download the pretrained models and arrange then in the folder structure shown above.

Our pretrained models and checkpoints can be downloaded from this link : CVPR Weigths

Additionally, we provide our training and testing scripts as examples which can be used as follows bash scripts/finetune_train.sh

Acknowledgements

We thank VideoMAE, VideoMAEv2, and AdaMAE authors for publicly releasing their code. We have built our code-base on top of these fabulous repositories.