Home

Awesome

Comparative Study of Inference Methods for Bayesian Nonnegative Matrix Factorisation

This project contains an implementation of the Bayesian non-negative matrix factorisation and tri-factorisation models presented in the paper Comparative Study of Inference Methods for Bayesian Nonnegative Matrix Factorisation, published at the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2017). For both models we implement four different inference methods (Gibbs sampling, variational Bayesian inference, iterated conditional modes, and non-probabilistic inference), and for the Bayesian models we also provide automatic model selection (using the automatic relevance determination prior). We furthermore provide all datasets used (including the preprocessing scripts), and Python scripts for experiments.

<img src="./images/gm_bnmf.png" width="43%"/> <img src="./images/gm_bnmtf.png" width="55%"/>

Paper abstract

In this paper, we study the trade-offs of different inference approaches for Bayesian matrix factorisation methods, which are com- monly used for predicting missing values, and for finding patterns in the data. In particular, we consider Bayesian nonnegative variants of matrix factorisation and tri-factorisation, and compare non-probabilistic infer- ence, Gibbs sampling, variational Bayesian inference, and a maximum- a-posteriori approach. The variational approach is new for the Bayesian nonnegative models. We compare their convergence, and robustness to noise and sparsity of the data, on both synthetic and real-world datasets. Furthermore, we extend the models with the Bayesian automatic rele- vance determination prior, allowing the models to perform automatic model selection, and demonstrate its efficiency.

Authors

Thomas Brouwer, Jes Frellsen, Pietro Lio'. Contact: thomas.a.brouwer@gmail.com.

Installation

If you wish to use the matrix factorisation models, or replicate the experiments, follow these steps. Please ensure you have Python 2.7 (3 is currently not supported).

  1. Clone the project to your computer, by running git clone https://github.com/ThomasBrouwer/BNMTF_ARD.git in your command line.

  2. In your Python script, add the project to your system path using the following lines.

    project_location = "/path/to/folder/containing/project/"
import sys
sys.path.append(project_location)

    For example, if the path to the project is /johndoe/projects/BNMTF_ARD/, use project_location = /johndoe/projects/. If you intend to rerun some of the paper's experiments, those scripts automatically add the correct path.

  3. You may also need to add an empty file in /johndoe/projects/ called __init__.py.

  4. You can now import the models in your code, e.g.

from BNMTF_ARD.code.models.nmf_np import nmf_np
model = nmf_np(R=numpy.ones((4,3)), M=numpy.ones((4,3)), K=2)
model.initialise()
model.run(iterations=10)

Examples

You can find good examples of the models running on data in the convergence experiment on the toy data, e.g. nonnegative matrix factorisation with Gibbs sampling.

Citation

If this project was useful for your research, please consider citing our paper.

Thomas Brouwer, Jes Frellsen, and Pietro Lió (2017). Comparative Study of Inference Methods for Bayesian Nonnegative Matrix Factorisation. Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2017).

@inproceedings{Brouwer2017b,
	author = {Brouwer, Thomas and Frellsen, Jes and Li\'{o}, Pietro},
	booktitle = {Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD)},
	title = {{Comparative Study of Inference Methods for Bayesian Nonnegative Matrix Factorisation}},
	year = {2017}
}

Project structure

<details> <summary>Click here to find a description of the different folders and files available in this repository.</summary> <br>

/code/

Python code, for the models, cross-validation methods, and model selection.

/models/: Python classes for the BNMF and BNMTF models: Gibbs sampling, Variational Bayes, Iterated Conditional Modes, and non-probabilistic versions. Each class contains both the version with ARD, and without.

/grid_search/: Classes for doing cross-validation, and nested cross-validation, on the Bayesian NMF and NMTF models

/data/

Contains the datasets, as well as methods for loading them in.

/toy/: Contains the toy data, and methods for generating toy data.

/drug_sensitivity/: Contains the drug sensitivity datasets (GDSC IC50, CCLE IC50, CCLE EC50, CTRP EC50). For more details see description.md.

/experiments/

/plots/

The results and plots for the experiments are stored in this folder, along with scripts for making the plots.

/tests/

py.test unit tests for the code and classes in /code/. To run the tests, simply cd into the /tests/ folder, and run pytest in the command line.

/images/

The images at the top of this README.

</br> </details>