Home

Awesome

semantic_affordance

Code for reproducing the results described in the paper:

Mining Semantic Affordances of Visual Object Categories
Yu-Wei Chao, Zhan Wang, Rada Mihalcea, and Jia Deng
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015

If you use this code, please cite our work:

@INPROCEEDINGS{chao:cvpr2015,
  author = {Yu-Wei Chao and Zhan Wang and Rada Mihalcea and Jia Deng},
  title = {Mining Semantic Affordances of Visual Object Categories},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year = {2015},
}

Check out the project site for more details.

Quick start

  1. Download a copy of our affordance dataset and unzip the file.

  2. Get the source code by cloning the repository: git clone https://github.com/ywchao/semantic_affordance.git

  3. Change into the source code directory cd semantic_affordance and start MATLAB matlab. You should see the message added paths for the experiment! followed by the MATLAB prompt >>.

  4. Change the path data_dir in config.m to the downloaded folder affordance_data/.

    data_dir = '/z/ywchao/datasets/affordance_data/';

  5. Run setup to prepare the required files.

    • Generate ground-truth binary labels from afforadance data
    • Generate WordNet similarity measures for 91 MS-COCO object categories
    • Download KPMF code

  6. Run pca_2d_run to visualize the object categories in the 2D affordance space.

  7. Run demo_cf_nn and demo_cf_kpmf to reproduce the NN and KPMF results.

    • In the default setting, the code will reproduce the paper's results on 20 PASCAL object categories. To run on 91 MS-COCO object categories, change the variable param.n_set from 'pascal' to 'mscoco'
    • If you download the affordance dataset before 2015-08-07, please re-download it as the previous one does not support the MS-COCO experiment.