Home

Awesome

This repository has become incompatible with the latest and recommended version of Tensorflow 2.0 Instead of refactoring this code painfully, I created a new fresh repository with some additional features like:

Here is the link to the new repo: tensorflow_Realtime_Multi-Person_Pose_Estimation


Realtime Multi-Person Pose Estimation (DEPRECATED)

This is a keras version of Realtime Multi-Person Pose Estimation project

Introduction

Code repo for reproducing 2017 CVPR paper using keras.

This is a new improved version. The main objective was to remove dependency on separate c++ server which besides the complexity of compiling also contained some bugs... and was very slow. The old version utilizing rmpe_dataset_server is still available under the tag v0.1 if you really would like to take a look.

Results

<p align="center"> <img src="https://github.com/michalfaber/keras_Realtime_Multi-Person_Pose_Estimation/blob/master/readme/dance.gif", width="720"> </p> <div align="center"> <img src="https://github.com/michalfaber/keras_Realtime_Multi-Person_Pose_Estimation/blob/master/sample_images/ski.jpg", width="300", height="300"> &nbsp; <img src="https://github.com/michalfaber/keras_Realtime_Multi-Person_Pose_Estimation/blob/master/readme/result.png", width="300", height="300"> </div>

Contents

  1. Converting caffe model
  2. Testing
  3. Training
  4. Changes

Require

  1. Keras
  2. Caffe - docker required if you would like to convert caffe model to keras model. You don't have to compile/install caffe on your local machine.

Converting Caffe model to Keras model

Authors of original implementation released already trained caffe model which you can use to extract weights data.

Testing steps

Training steps

Changes

25/06/2018

26/10/2017

Fixed problem with the training procedure. Here are my results after training for 5 epochs = 25000 iterations (1 epoch is ~5000 batches) The loss values are quite similar as in the original training - output.txt

<p align="center"> <img src="https://github.com/michalfaber/keras_Realtime_Multi-Person_Pose_Estimation/blob/master/readme/losses.png", width="700"> </p>

Results of running demo_image --image sample_images/ski.jpg --model training/weights.best.h5 with model trained only 25000 iterations. Not too bad !!! Training on my single 1070 GPU took around 10 hours.

<p align="center"> <img src="https://github.com/michalfaber/keras_Realtime_Multi-Person_Pose_Estimation/blob/master/readme/5ep_result.png", width="300"> </p>

22/10/2017

Augmented samples are fetched from the server. The network never sees the same image twice which was a problem in previous approach (tool rmpe_dataset_transformer) This allows you to run augmentation locally or on separate node. You can start 2 instances, one serving training set and a second one serving validation set (on different port if locally)

Related repository

Citation

Please cite the paper in your publications if it helps your research:

@InProceedings{cao2017realtime,
  title = {Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields},
  author = {Zhe Cao and Tomas Simon and Shih-En Wei and Yaser Sheikh},
  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2017}
  }