Home

Awesome

<!-- README.md is generated from README.Rmd. Please edit that file -->

Build
Status Lifecycle doi ℹ️ Tutorials Β  <br><img src="package/man/figures/logo.png" align="right" width="125" height="144" />

Benchmarking trajectory inference methods

This repo contains the scripts to reproduce the manuscript

A comparison of single-cell trajectory inference methods <strong> Wouter Saelens* </strong> <a href='https://orcid.org/0000-0002-7114-6248'><img src='https://github.com/dynverse/dynmethods/raw/master/man/figures/orcid_logo.svg?sanitize=true' height='16'></a> <a href='https://github.com/zouter'><img src='https://github.com/dynverse/dynmethods/raw/master/man/figures/github_logo.png' height='16'></a>, <strong> Robrecht Cannoodt* </strong> <a href='https://orcid.org/0000-0003-3641-729X'><img src='https://github.com/dynverse/dynmethods/raw/master/man/figures/orcid_logo.svg?sanitize=true' height='16'></a> <a href='https://github.com/rcannood'><img src='https://github.com/dynverse/dynmethods/raw/master/man/figures/github_logo.png' height='16'></a>, Helena Todorov <a href='https://github.com/Helena-todd'><img src='https://github.com/dynverse/dynmethods/raw/master/man/figures/github_logo.png' height='16'></a>, <em> Yvan Saeys </em> <a href='https://github.com/saeyslab'><img src='https://github.com/dynverse/dynmethods/raw/master/man/figures/github_logo.png' height='16'></a>
doi:10.1038/s41587-019-0071-9 altmetric

Dynverse

Under the hood, dynbenchmark makes use of most dynverse package for running the methods, comparing them to a gold standard, and plotting the output. Check out dynverse.org for an overview!

Experiments

From start to finish, the repository is divided into several experiments, each with their own scripts and results. These are accompanied by documentation using github readmes and can thus be easily explored by going to the appropriate folders:

#idscriptsresults
1DatasetsπŸ“„βž‘πŸ“Šβž‘
2MetricsπŸ“„βž‘πŸ“Šβž‘
3MethodsπŸ“„βž‘πŸ“Šβž‘
4Method testingπŸ“„βž‘πŸ“Šβž‘
5ScalingπŸ“„βž‘πŸ“Šβž‘
6BenchmarkπŸ“„βž‘πŸ“Šβž‘
7StabilityπŸ“„βž‘πŸ“Šβž‘
8SummaryπŸ“„βž‘πŸ“Šβž‘
9GuidelinesπŸ“„βž‘πŸ“Šβž‘
10Benchmark interpretationπŸ“„βž‘πŸ“Šβž‘
11Example predictionsπŸ“„βž‘πŸ“Šβž‘
12ManuscriptπŸ“„βž‘πŸ“Šβž‘
VariaπŸ“„βž‘

We also have several additional subfolders:

Guidelines

Based on the results of the benchmark, we provide context-dependent user guidelines, available as a shiny app. This app is integrated within the dyno pipeline, which also includes the wrappers used in the benchmarking and other packages for visualising and interpreting the results.

dynguidelines

Datasets

The benchmarking pipeline generates (and uses) the following datasets:

datasets

Methods

All wrapped methods are wrapped as both docker and singularity containers. These can be easily run using dynmethods.

Installation

dynbenchmark has been tested using R version 3.5.1 on Linux. While running the methods also works on on Windows and Mac (see dyno), running the benchmark is currently not supported on these operating system, given that a lot of commands are linux specific.

In R, you can install the dependencies of dynbenchmark from github using:

# install.packages("devtools")
devtools::install_github("dynverse/dynbenchmark/package")

This will install several other β€œdynverse” packages. Depending on the number of R packages already installed, this installation should take approximately 5 to 30 minutes.

On Linux, you will need to install udunits and ImageMagick:

Docker or Singularity (version β‰₯ 3.0) has to be installed to run TI methods. We suggest docker on Windows and MacOS, while both docker and singularity are fine when running on linux. Singularity is strongly recommended when running the method on shared computing clusters.

For windows 10 you can install Docker CE, older Windows installations require the Docker toolbox.

You can test whether docker is correctly installed by running:

dynwrap::test_docker_installation(detailed = TRUE)
## βœ” Docker is installed

## βœ” Docker daemon is running

## βœ” Docker is at correct version (>1.0): 1.39

## βœ” Docker is in linux mode

## βœ” Docker can pull images

## βœ” Docker can run image

## βœ” Docker can mount temporary volumes

## βœ” Docker test successful -----------------------------------------------------------------

## [1] TRUE

Same for singularity:

dynwrap::test_singularity_installation(detailed = TRUE)
## βœ” Singularity is installed

## βœ” Singularity is at correct version (>=3.0): v3.0.0-13-g0273e90f is installed

## βœ” Singularity can pull and run a container from Dockerhub

## βœ” Singularity can mount temporary volumes

## βœ” Singularity test successful ------------------------------------------------------------

## [1] TRUE

These commands will give helpful tips if some parts of the installation are missing.