Awesome

Benchmarking atlas-level data integration in single-cell genomics

This repository contains the code for the scib package used in our benchmarking study for data integration tools. In our study, we benchmark 16 methods (see Tools) with 4 combinations of preprocessing steps leading to 68 methods combinations on 85 batches of gene expression and chromatin accessibility data.

Workflow

Resources

The git repository of the scib package and its documentation.
The reusable pipeline we used in the study can be found in the separate scib pipeline repository. It is reproducible and automates the computation of preprocesssing combinations, integration methods and benchmarking metrics.
On our website we visualise the results of the study.
For reproducibility and visualisation we have a dedicated repository: scib-reproducibility.

Please cite:

Luecken, M.D., Büttner, M., Chaichoompu, K. et al. Benchmarking atlas-level data integration in single-cell genomics. Nat Methods 19, 41–50 (2022). https://doi.org/10.1038/s41592-021-01336-8

Package: scib

We created the python package called scib that uses scanpy to streamline the integration of single-cell datasets and evaluate the results. The package contains several modules for preprocessing an anndata object, running integration methods and evaluating the resulting using a number of metrics. For preprocessing, scib.preprocessing (or scib.pp) contains functions for normalising, scaling or batch-aware selection of highly variable genes. Functions for the integration methods are in scib.integration or for short scib.ig and metrics are under scib.metrics (or scib.me).

The scib python package is available on PyPI and can be installed through

pip install scib

Import scib in python:

import scib

Optional Dependencies

The package contains optional dependencies that need to be installed manually if needed. These include R dependencies (rpy2, anndata2ri) which require an installation of R integration method packages. All optional dependencies are listed under setup.cfg under [options.extras_require] and can be installed through pip.

e.g. for installing rpy2 and bbknn dependencies:

pip install 'scib[rpy2,bbknn]'

Optional dependencies outside of python need to be installed separately. For instance, in order to run kBET, install it via the following command in R:

install.packages('remotes')
remotes::install_github('theislab/kBET')

Metrics

We implemented different metrics for evaluating batch correction and biological conservation in the scib.metrics module.

<table class="docutils align-default"> <colgroup> <col style="width: 50%" /> <col style="width: 50%" /> </colgroup> <thead> <tr class="row-odd"><th class="head">Biological Conservation</th> <th class="head">Batch Correction</th> </tr> </thead> <tbody> <tr class="row-even" > <td><ul class="simple"> <li>Cell type ASW</li> <li>Cell cycle conservation</li> <li>Graph cLISI</li> <li>Adjusted rand index (ARI) for cell label</li> <li>Normalised mutual information (NMI) for cell label</li> <li>Highly variable gene conservation</li> <li>Isolated label ASW</li> <li>Isolated label F1</li> <li>Trajectory conservation</li> </ul></td> <td><ul class="simple"> <li>Batch ASW</li> <li>Principal component regression</li> <li>Graph iLISI</li> <li>Graph connectivity</li> <li>kBET (K-nearest neighbour batch effect)</li> </ul></td> </tr> </tbody> </table>

For a detailed description of the metrics implemented in this package, please see our publication and the package documentation.

Integration Tools

Tools that are compared include:

BBKNN 1.3.9
Combat paper
Conos 1.3.0
DESC 2.0.3
FastMNN (batchelor 1.4.0)
Harmony 1.0
LIGER 0.5.0
MNN 0.1.9.5
SAUCIE
Scanorama 1.7.0
scANVI (scVI 0.6.7)
scGen 1.1.5
scVI 0.6.7
Seurat v3 3.2.0 CCA (default) and RPCA
TrVae 0.0.1
TrVaep 0.1.0

Development

For developing this package, please make sure to install additional dependencies so that you can use pytest and pre-commit.

pip install -e '.[test,dev]'

Please refer to the setup.cfg for more optional dependencies.

Install pre-commit to the repository for running it automatically every time you commit in git.

pre-commit install