Home

Awesome

Can We Use Diffusion Probabilistic Models for 3D Motion Prediction?

A repository of a paper named "Can We Use Diffusion Probabilistic Models for 3D Motion Prediction?", accepted to ICRA 2023.

Installation

  1. Clone this repository, and move to root of this repo.
  2. Prepare human3.6m dataset for euler angle based experiments. Save Folder h3.6m/ into ./data.
    • From issues, we just found out that the previous link for h3.6m has been shut down. One would be required to download the original dataset from human3.6m.
    • Inside the h3.6m folder, there should be sub-folders for each human subject, and that folder should contain .txt files (i.e., h3.6m/S1/directions_1.txt.)
  3. Activate your conda environment, train pytorch based on your system, run pip install -r requirements.txt.
    • In my case, since I use cuda=11.6, I used command conda install pytorch torchvision torchaudio pytorch-cuda=11.6 -c pytorch -c nvidia to install pytorch.
  4. Clone DLow repository.
  5. Prepare dataset (human3.6m and humaneva) as DLow suggests, and put *.npz files to ./data and ./DLow/data.
  6. Copy ./src/dlow/eval.py of our repository into DLow/motion_pred/eval.py.

Preprocess

Training

python train.py --cfg h36m_euler_series_20step
python train.py --cfg h36m_euler_parallel_20step
python train.py --cfg h36m_xyz_series_20step
python train.py --cfg h36m_xyz_parallel_20step
python train.py --cfg humaneva_xyz_series_20step
python train.py --cfg humaneva_xyz_parallel_20step

Quantitative Evaluation

Qualitative Evaluation

Pretrained model

Make comparison with DLow

  1. Copy ./src/dlow/eval.py of our repository into DLow/motion_pred/eval.py.

  2. Copy your preprocessed data (i.e., data_3d_humaneva15.npz) into DLow/data.

  3. Run DLow's training code as their README file instructs.

    python motion_pred/exp_vae.py --cfg h36m_nsamp50
    python motion_pred/exp_vae.py --cfg humaneva_nsamp50
    python motion_pred/exp_dlow.py --cfg h36m_nsamp50
    python motion_pred/exp_dlow.py --cfg humaneva_nsamp50
    
  4. Run DLow's testing code as their README file instructs.

    python motion_pred/eval.py --cfg h36m_nsamp50 model stats
    python motion_pred/eval.py --cfg humaneva_nsamp50 model stats
    
  5. To compare visualization between DLow and Diffusion model, run below code at the root of our repo.

    python viz_comp_dlow.py