Awesome
NPEET_LNC
Mutual information Estimation with Local Non-uniformity Correction (a branch of <a href="https://github.com/gregversteeg/NPEET/">NPEET</a> Non-parametric Entropy Estimation Toolbox)
This package contains Python code implementing mutual information estimation functions for continuous variables. This estimator gives a correction term to the traditional kNN estimator and can estimate mutual information more accurately than Kraskov estimator for strongly dependent variables with limited samples.
To use this package, it requires <a href="http://www.scipy.org">scipy</a> 0.12 or greater.
Example installation and usage:
git clone https://github.com/BiuBiuBiLL/NPEET_LNC.git
>>> from lnc import MI
>>> import numpy as np
>>> x = [1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0]
>>> y = [1.0,2.0,3.0,4.0,5.0,6.0,7.0,8.0,9.0,10.0,11.0]
>>> MI.mi_LNC([x,y],k=5,base=np.exp(1),alpha=0.25) #Apply LNC estimator
Output: 25.29758574548632
>>> MI.mi_Kraskov([x,y],k=5,base=np.exp(1)) #Same data applied to Kraskov estimator
Output: 0.62745310245310382
To run the test example in test.py
, simply type the following in terminal
$python test.py
Output:
Testing 2D linear relationship Y=X+Uniform_Noise
noise level=1e-07, Nsamples = 500
True MI(x:y) 16.0472260191
Kraskov MI(x:y) 5.79082342999
LNC MI(x:y) 15.9355050641
Testing 2D quadratic relationship Y=X^2+Uniform_Noise
noise level=1e-07, Nsamples = 1000
True MI(x:y) 15.8206637545
Kraskov MI(x:y) 6.48347086055
LNC MI(x:y) 11.4586276609
Testing 3D linear relationship Y=X+Uniform_Noise, Z=X+Uniform_Noise
noise level=1e-07, Nsamples = 500
True MI(x:y:z) 32.2836569188
Kraskov MI(x:y:z) 11.58164686
LNC MI(x:y:z) 32.1846129957
Testing 3D quadratic relationship Y=X^2+Uniform_Noise, Z=X^2+Uniform_Noise
noise level=1e-07, Nsamples = 500
True MI(x:y:z) 31.5020968975
Kraskov MI(x:y:z) 11.57764686
LNC MI(x:y:z) 25.6686276941
One need to specify the thresholding parameter alpha when using LNC estimator. This parameter is related to the nearest-neighbor parameter k
and dimensionality d
, see alpha.xlsx
for the detailed alpha value to use.
Also see the references on the implemented estimators.
A Kraskov, H Stögbauer, P Grassberger.
http://pre.aps.org/abstract/PRE/v69/i6/e066138
Estimating Mutual Information
PRE, 2004.
Shuyang Gao, Greg Ver Steeg and Aram Galstyan
http://arxiv.org/abs/1411.2003
Efficient Estimation of Mutual Information for Strongly Dependent Variables
AISTATS, 2015.