Home

Awesome

Viewpoints And Keypoints

Shubham Tulsiani and Jitendra Malik. Viewpoints and Keypoints. In CVPR, 2015.

0) Setup

bash initSetup.sh

(if PASCAL VOC server is not working, uncomment the corresponding lines from the script and move a local copy to the desired location)

cd external/caffe
cp Makefile.config.example Makefile.config
make -j 8
#edit MATLAB_DIR in Makefile.config
make matcaffe
cd ../..

1) Viewpoint Prediction

Preprocessing :

We first need to create some data-structures which store the annotations for each object category. To do this run in matlab -

mainVpsPreprocess 

Network Training :

./build/tools/caffe.bin train -solver ../../prototxts/vggJointVps/solver.prototxt -weights PATH_TO_PRETRAINED_VGG_CAFFEMODEL
./build/tools/caffe.bin train -solver ../../prototxts/vggAzimuthVps/solver.prototxt -weights PATH_TO_PRETRAINED_VGG_CAFFEMODEL

Predciting Pose for PASCAL VOC

mainVpsPredict

(computing pose for all R-CNN detections might take a while, you can comment the corresponding lines if you just want to reproduce the evaluation given ground-truth boxes)

Evaluation and Analysis

mainRigidViewpoint
runAvpExperiments
perfCharachteristics = smallVsLarge() ;
perfModes = errorModes();

2) Keypoint Prediction :

Preprocessing :

We first need to create some data-structures which store the annotations for each object category. To do this run in matlab -

mainKpsPreprocess

Network Training :

./build/tools/caffe.bin train -solver ../../prototxts/vggConv6Kps/solver.prototxt -weights PATH_TO_PRETRAINED_VGG_CAFFEMODEL
./build/tools/caffe.bin train -solver ../../prototxts/vggConv12Kps/solver.prototxt -weights PATH_TO_TRAINED_VGG_6_X_6_KPS_CAFFEMODEL

Note that for training the finer scale model, we initialize from a coarse scale model. An alternate is to finetune from a classification VGG model but this requires the use of cumulative gradients and a much longer training time.

Predciting Pose for PASCAL VOC

mainKpsPredict

(computing pose for all R-CNN detections might take a while, you can comment the corresponding lines if you just want to reproduce the evaluation given ground-truth boxes)

Evaluation and Analysis

mainRigidPck
mainRigidApk
objectCharacteristics