Awesome

ARC (Adaptive and Reliable Classification)

This package provides some statistical wrappers for machine learning classification tools in order to construct prediction sets for the label of a new test point with provably valid marginal coverage and approximate conditional coverage.

Accompanying paper (https://papers.nips.cc/paper/2020/hash/244edd7e85dc81602b7615cd705545f5-Abstract.html):

"Classification with Valid and Adaptive Coverage"
Y. Romano, M. Sesia, E. Candès
NeurIPS 2020 (spotlight).

arc/ Python package implementing our methods and some alternative benchmarks.
third_party/ Third-party Python packages imported by our package.
examples/ Jupyter notebooks with introductory usage examples.
experiments_sim_data Code for the experiments with simulated data discussed in the accompanying paper.
experiments_real_data Code for the experiments with real data discussed in the accompanying paper.

Third-party packages

This package builds upon the following non-standard Python packages provided in the "third-party" directory:

nonconformist https://github.com/donlnz/nonconformist
cqr https://github.com/yromano/cqr
cqr-comparison https://github.com/msesia/cqr-comparison

Prerequisites

Prerequisites for the arc package:

numpy
scipy
sklearn
skgarden
torch
tqdm

Additional prerequisites for example notebooks:

pandas
matplotlib
seaborn

Installation

The development version is available from GitHub:

git clone https://github.com/msesia/arc.git

License