Home

Awesome

Probabilistic Neural-symbolic Models

Code for our ICML 2019 paper:

Probabilistic Neural-Symbolic Models for Interpretable Visual Question Answering
Ramakrishna Vedantam, Karan Desai, Stefan Lee, Marcus Rohrbach, Dhruv Batra, Devi Parikh

Checkout our package documentation at kdexd.github.io/probnmn-clevr!

probnmn-model

If you find this code useful, please consider citing:

@inproceedings{vedantam2019probabilistic,
  title={Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering},
  author={Ramakrishna Vedantam and Karan Desai and Stefan Lee and Marcus Rohrbach and Dhruv Batra and Devi Parikh},
  booktitle={ICML},
  year={2019}
}

Usage Instructions

  1. How to setup this codebase?
  2. How to train your ProbNMN?
  3. How to evaluate or run inference?

Pre-trained Checkpoint

Pre-trained checkpoints and corresponding config files (with all the hyper-parameters) for all training phases is available with v1.0 release of this repository. Check out the Releases!

Acknowledgments

We thank the developers of:

  1. @davidmascharka/tbd-nets for providing a very clean implementation of our core Neural Module Network.

  2. @allenai/allennlp for providing an awesome framework which indeed takes masking and padding seriously.

  3. @rbgirshick/yacs for providing an efficient package-wide configuration management.

  4. @pytorch/pytorch, this needs no reason.