Home

Awesome

BAMM: Bidirectional Autoregressive Motion Model (ECCV 2024)

arXiv

The official PyTorch implementation of the paper "BAMM: Bidirectional Autoregressive Motion Model".

Please visit our webpage for more details.

If our project is helpful for your research, please consider citing :

@inproceedings{pinyoanuntapong2024bamm,
  title={BAMM: Bidirectional Autoregressive Motion Model}, 
  author={Ekkasit Pinyoanuntapong and Muhammad Usama Saleem and Pu Wang and Minwoo Lee and Srijan Das and Chen Chen}, 
  booktitle="Computer Vision -- ECCV 2024",
  year={2024},
}

1. Setup Env & Download Pre-train MoMask

Our model is devloped based on MoMask. You can follow the setup instructions provided there, or refer to the steps below. (The environment setup is the same but we change conda env name to "BAMM")

<details>

1. Conda Environment

conda env create -f environment.yml
conda activate BAMM
pip install git+https://github.com/openai/CLIP.git

We test our code on Python 3.7.13 and PyTorch 1.7.1

Alternative: Pip Installation

<details> We provide an alternative pip installation in case you encounter difficulties setting up the conda environment.
pip install -r requirements.txt

We test this installation on Python 3.10

</details>

2. Models and Dependencies

Download Pre-trained Models

bash prepare/download_models.sh

Download Evaluation Models and Gloves

For evaluation only.

bash prepare/download_evaluator.sh
bash prepare/download_glove.sh

Troubleshooting

To address the download error related to gdown: "Cannot retrieve the public link of the file. You may need to change the permission to 'Anyone with the link', or have had many accesses". A potential solution is to run pip install --upgrade --no-cache-dir gdown, as suggested on https://github.com/wkentaro/gdown/issues/43. This should help resolve the issue.

(Optional) Download Manually

Visit [Google Drive] to download the models and evaluators mannually.

3. Get Data

You have two options here:

(a). Full data (text + motion)

HumanML3D - Follow the instruction in HumanML3D, then copy the result dataset to our repository:

cp -r ../HumanML3D/HumanML3D ./dataset/HumanML3D

KIT-Download from HumanML3D, then place result in ./dataset/KIT-ML

</details>

2. Download Pre-train BAMM and Move MoMask to "log" folder

bash prepare/download_models_BAMM.sh

3. Generate

python gen_t2m.py \
    --res_name tres_nlayer8_ld384_ff1024_rvq6ns_cdp0.2_sw \
    --name 2024-02-14-14-27-29_8_GPT_officialTrans_2iterPrdictEnd \
    --text_prompt "the person crouches and walks forward." \
    --motion_length -1 \
    --repeat_times 3 \
    --gpu_id 0 \
    --seed 1 \
    --ext generation_name_nopredlen

4. Evaluation

<details>
python eval_t2m_trans_res.py \
    --res_name tres_nlayer8_ld384_ff1024_rvq6ns_cdp0.2_sw \
    --name 2024-02-14-14-27-29_8_GPT_officialTrans_2iterPrdictEnd \
    --gpu_id 1 \
    --ext LOG_NAME
</details>