Home

Awesome

Exact

PyPI version fury.io MIT license

This is a collaborative online tool for labeling image data.

ScreenShot

Reference

This paper describes the EXACT-Server in depth. Please cite if you use this tool in your research:

Marzahl et al. EXACT: A collaboration toolset for algorithm-aided annotation of almost everything

@Article{marzahl2021exact,
title={EXACT: a collaboration toolset for algorithm-aided annotation of images with annotation version control},
author={Marzahl, Christian and Aubreville, Marc and Bertram, Christof A. and Maier, Jennifer and Bergler, Christian and Kr{\"o}ger, Christine and Voigt, J{\"o}rn and Breininger, Katharina and Klopfleisch, Robert and Maier, Andreas},
journal={Scientific Reports},
year={2021},
month={Feb},
day={23},
volume={11},
number={1},
pages={4343},
abstract={In many research areas, scientific progress is accelerated by multidisciplinary access to image data and their interdisciplinary annotation. However, keeping track of these annotations to ensure a high-quality multi-purpose data set is a challenging and labour intensive task. We developed the open-source online platform EXACT (EXpert Algorithm Collaboration Tool) that enables the collaborative interdisciplinary analysis of images from different domains online and offline. EXACT supports multi-gigapixel medical whole slide images as well as image series with thousands of images. The software utilises a flexible plugin system that can be adapted to diverse applications such as counting mitotic figures with a screening mode, finding false annotations on a novel validation view, or using the latest deep learning image analysis technologies. This is combined with a version control system which makes it possible to keep track of changes in the data sets and, for example, to link the results of deep learning experiments to specific data set versions. EXACT is freely available and has already been successfully applied to a broad range of annotation tasks, including highly diverse applications like deep learning supported cytology scoring, interdisciplinary multi-centre whole slide image tumour annotation, and highly specialised whale sound spectroscopy clustering.},
issn={2045-2322},
doi={10.1038/s41598-021-83827-4},
url={https://doi.org/10.1038/s41598-021-83827-4}
}


Features

Documentation

Issues with the notebooks on GitHub? Please use NBViewer

DescribtionVideo
EXACT installation with Docker (en)Datasets
EXACT First steps (en)Datasets
EXACT Product and Annotationtype defintion (en)Datasets
Study and annotation modes (en)AnnotationModes <br> Code
AnnotationMaps (en)Inference <br> Code
DensityMaps (en)Inference <br> Code
Cluster annotations (en)Inference <br> Code
Cluster sounds (en)Inference
AnnotationVersioning (en)AnnotationVersioning
Inference (en)Inference <br> [Code](doc/Inference Asthma.ipynb)
Segmentation (en)Inference <br> Code
EXACT Media Files (en)Datasets
Datasets (en)Datasets
Explains how teams, products and annotation types are created (de)Datasets
Describes how to create, view and edit image sets and upload images. (en)Datasets
Describes basic viewer and plugin functions. (en)Datasets
Describes the image registration functionalityDatasets
Explains collaboratory annotation features (de)Datasets
Shows multiple types of datasets (de)Datasets
Explains the screening plugin for WSI (de)Datasets
Syncronisation with the offline tool SlideRunner (en)Datasets
Advanced polygon annotation operations (de)Datasets
REST-API Examplepip install EXCAT-Sync <br> Code <br> Notebooks <br> Browsable-API

Install

Docker

Install Docker

Checkout the latest release:

git clone https://github.com/DeepMicroscopy/Exact.git

Production

Additional features:

Copy the files env.dev and env.dev.db, rename to env.prod and env.prod.db and change settings according to your preferences.

Copy and rename settings.py.example to settings.py in the exact folder.

Enable caching in the settings.py according to the documentation.

Build and run the container:


docker-compose -f docker-compose.prod.yml up -d --build
docker-compose -f docker-compose.prod.yml exec web python3 manage.py migrate --noinput 
docker-compose -f docker-compose.prod.yml exec web python3 manage.py createsuperuser
docker-compose -f docker-compose.prod.yml exec web python3 manage.py collectstatic --no-input --clear
docker-compose -f docker-compose.prod.yml logs -f

Navigate to http://localhost:1337/

Development

Copy and rename settings.py.example to settings.py in the exact folder.

Modify the configuration files: env.dev and env.dev.db or use the default configuration.

Build and run the container:

docker-compose -f docker-compose.yml up -d --build
docker-compose logs -f 

Navigate to http://localhost:8000/ For default the super user login is:

User: exact
Pw: exact

Cloud

To use cloud services like amazon fargate. The exact server has to connect to a cloud database like amazon rds.

Copy the files env.dev and env.dev.db, rename to env.prod and env.prod.aws-db and change settings according to your preferences.

Copy and rename settings.py.example to settings.py in the exact folder.

Build and run the container:

docker-compose -f docker-compose.prod.aws-db.yml up -d --build
docker-compose -f docker-compose.prod.aws-db.yml exec web python3 manage.py migrate --noinput
docker-compose -f docker-compose.prod.aws-db.yml exec web python3 manage.py createsuperuser
docker-compose -f docker-compose.prod.aws-db.yml exec web python3 manage.py collectstatic --no-input --clear
docker-compose -f docker-compose.prod.aws-db.yml logs -f

Send container to AWS ECR

Invoke-Expression -Command (aws ecr get-login --no-include-email)

docker tag exact_nginx:latest **************.dkr.ecr.eu-central-1.amazonaws.com/exact_nginx:latest
docker tag exact_web:latest **************.dkr.ecr.eu-central-1.amazonaws.com/exact:latest

docker push **************.dkr.ecr.eu-central-1.amazonaws.com/exact_nginx:latest
docker push **************.dkr.ecr.eu-central-1.amazonaws.com/exact:latest

Windows

The Server is also runnig on Windows but I would recommend to use docker in that case.

MacOS or Linux

Ubuntu20.04 has a known issue with openslide. Fix: Build pixman

Install libvips and verify by executing the comand vips:

https://libvips.github.io/libvips/
[Ubuntu](https://github.com/libvips/libvips/wiki/Build-for-Ubuntu)

Checkout the latest release:

git clone https://github.com/DeepMicroscopy/Exact.git

In our production Senty is used for error reporting (pip install raven). django-auth-ldap is used for login via ldap uwsgi is used to serve the app to nginx

Install Python Dependencies:

pip3 install -r requirements.txt

Copy settings.py.example to settings.py in the exact folder:

cp exact/exact/settings.py.example exact/exact/settings.py

and customize the settings.py.

The following settings should probably be changed:

For the database, postgresql is used. Install it by running sudo apt install postgresql

You will also need opencv3 and other packages. You can install that by running apt-get update && apt-get install python3-pip dos2unix python3-openslide python3-opencv libvips libvips-dev

Initialize the database cluster with sudo -iu postgres initdb --locale en_US.UTF-8 -D '/var/lib/postgres/data'.

Note: It may be that initdb is not in your current PATH (seems to be default for postgresql >= 10), in this case, you have to specify the proper path to initdb, e.g: sudo -iu postgres /usr/lib/postgresql/*/bin/initdb --locale en_US.UTF-8 -D '/var/lib/postgresql/data' (for Ubuntu systems)

To start the postgresql server, run sudo systemctl start postgresql.service. If the server should always be started on boot, run sudo systemctl enable postgresql.service.

Then, create the user and the database by running

sudo -iu postgres psql

and then, in the postgres environment

CREATE USER exact PASSWORD 'exact';
CREATE DATABASE exact WITH OWNER exact ENCODING UTF8;

where of course the password and the user should be adapted to the ones specified in the database settings in the settings.py.

To initialize the database, run python3 manage.py migrate

To create an administrator user, run python3 manage.py createsuperuser.

python3 manage.py runserver starts the server with the configuration given in the settings.py file.

To create annotation types, log into the application and click on Administration at the very bottom of the home page.

For production systems it is necessary to run the following commands after each upgrade

python3 manage.py migrate
python3 manage.py compilemessages
python3 manage.py collectstatic

Our production uwisgi config is

[uwsgi]
socket = /tmp/exact.socket
chmod-socket = 666
chdir = /srv/exact/Exact/exact/
master = true
binary-path = /usr/bin/uwsgi
virtualenv = /srv/exact/virtualenv/exact
module = exact.wsgi
uid = exact
gid = exact
processes = 6
#async = 10
threads = 1
#logto = /var/log/exact.log
plugins = python3,logfile
logger = file:/var/log/exact.log


Example Nginx Config:

upstream exact {
    server web:8000;
}

server {

    listen 80;

    client_max_body_size 10000M;
    keepalive_timeout 65;

    proxy_connect_timeout       6000s;
    proxy_send_timeout          6000s;
    proxy_read_timeout          6000s;
    send_timeout                6000s;
    client_body_timeout     6000s;

    location / {
        proxy_pass http://exact;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $host;
        proxy_redirect off;
    }
	
	location /static/ {
        expires 1h;
        alias /home/app/web/static/;
    }
	
	
    location /media/ {
        expires 1h;
        alias /home/app/web/media/;
    }

}

Verification E-Mails

Please add credentials to the settings.py

EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_HOST = 'smtp.googlemail.com' # for example gmail
EMAIL_PORT = '587'
EMAIL_HOST_USER = 'example@gmail.com'
EMAIL_HOST_PASSWORD = 'example'
EMAIL_USE_TLS = True
EMAIL_USE_SSL = False

Upgrade

pip install -U -r requirements.txt
python3 manage.py migrate

for additional steps on some releases see instructions in UPGRADE.md

Key shortcuts

Viewer

KeyModifierFunction
Del,xDelete annotation
EscapeCancel editing
EnterConfirm / Save
Ctrl + zUndo
cToggle annotation mode
yToggle annotation visibility
bPush currently selected annotation type into the background
Ctrl,aDraw annotation on top of existing one
0,1,2,3,4Change label of local annotations
0,1,2,3,4ShiftChange label of global annotations
qPrevious image
qShiftPrevious frame
eNext image
eShiftNext frame
rRotate image (Warning: annotations are not affected)
fFlip image (Warning: annotations are not affected)
sScissor (Delete from selected object)
gGlue (Add to selected object)
dKnife (Draw a line to split objects)
Mouse wheelShiftPaint brush
Arrow keysMove viewing window
Screening Viewer
KeyModifierFunction
ascreen left tile
wscreen up tile
sscreen down tile
dscreen right tile
jnavigate to left tile
inavigate to up tile
knavigate to down tile
lnavigate to right tile
Sync Plugin
KeyModifierFunction
otoggle visibility

REST-API

OpenAPI-Schema

GET /api/v1/openapi

Examples

https://github.com/rsinger86/drf-flex-fields
https://django-filter.readthedocs.io/en/master/

Token

$ curl -X POST -d '{"username": "exact","password": "top_secret"}' -H
'Content-Type: application/json'  http://127.0.0.1:8000/api/auth/token/login/

All suported classes

GET /api/v1/

{
    "users/users": "http://127.0.0.1:8000/api/v1/users/users/",
    "users/teams": "http://127.0.0.1:8000/api/v1/users/teams/",
    "users/team_membership": "http://127.0.0.1:8000/api/v1/users/team_membership/",
    "images/images": "http://127.0.0.1:8000/api/v1/images/images/",
    "images/image_sets": "http://127.0.0.1:8000/api/v1/images/image_sets/",
    "images/set_tags": "http://127.0.0.1:8000/api/v1/images/set_tags/",
    "images/screening_modes": "http://127.0.0.1:8000/api/v1/images/screening_modes/",
    "annotations/annotations": "http://127.0.0.1:8000/api/v1/annotations/annotations/",
    "annotations/annotation_types": "http://127.0.0.1:8000/api/v1/annotations/annotation_types/",
    "annotations/annotation_media_files": "http://127.0.0.1:8000/api/v1/annotations/annotation_media_files/",
    "annotations/verifications": "http://127.0.0.1:8000/api/v1/annotations/verifications/",
    "annotations/log_image_actions": "http://127.0.0.1:8000/api/v1/annotations/log_image_actions/",
    "administration/products": "http://127.0.0.1:8000/api/v1/administration/products/"
}

Filter all image_sets with "Katze" in name and expand products and main_annotation_type

GET /api/v1/images/image_sets/?name__contains=Katze&expand=product_set,main_annotation_type

{
    "count": 1,
    "next": null,
    "previous": null,
    "results": [
        {
            "id": 3,
            "name": "EIPH-Katze",
            "path": "exact_1_3",
            "location": null,
            "description": "",
            "images": [
                17,
                20,
                22,
                23,
                26,
                27,
                30
            ],
            "product_set": [
                {
                    "id": 2,
                    "name": "EIPH",
                    "description": "",
                    "team": 1,
                    "creator": 1,
                    "imagesets": [
                        2,
                        3,
                        16
                    ],
                    "annotationtype_set": [
                        10,
                        11,
                        12,
                        13,
                        14
                    ]
                }
            ],
            "main_annotation_type": {
                "id": 10,
                "name": "0",
                "vector_type": 1,
                "node_count": 0,
                "enable_concealed": false,
                "enable_blurred": false,
                "color_code": "#0000FF",
                "default_width": 120,
                "default_height": 120,
                "sort_order": 0,
                "closed": true,
                "area_hit_test": true,
                "product": 2
            },
            "set_tags": [],
            "team": 1,
            "creator": 1
        }
    ]
}

Include(fields) or Exclude(omit) fields

GET /api/v1/images/image_sets/?name__contains=Katze&fields=name

{
    "count": 1,
    "next": null,
    "previous": null,
    "results": [
        {
            "name": "EIPH-Katze"
        }
    ]
}

GET /api/v1/images/image_sets/?name__contains=Katze&omit=images,product_set

{
    "count": 1,
    "next": null,
    "previous": null,
    "results": [
        {
            "id": 3,
            "name": "EIPH-Katze",
            "path": "exact_1_3",
            "location": null,
            "description": "",
            "main_annotation_type": 10,
            "set_tags": [],
            "team": 1,
            "creator": 1
        }
    ]
}

Used dependencies

The exact relies on the following plugins, libraries and frameworks:

NameVersionLicense
Django3.0BSD
Pillow5.4.1Standard PIL License
asgiref3.2.3BSD
confusable-homoglyphs3.2.0MIT
django-friendly-tag-loader1.3.1MIT
django-registration3.0.1MIT
django-widget-tweaks1.4.3MIT license
djangorestframework3.11.0BSD
fasteners0.14.1ASL 2.0
gunicorn19.9.0MIT
imagecodecs-lite2019.12.3BSD
monotonic1.5Apache
numpy1.17.4BSD
opencv-python4.1.2.30MIT
openslide-python1.1.1GNU Lesser General Public License, version 2.1
psycopg2-binary2.7.7LGPL with exceptions or ZPL
pytz2018.9MIT
six1.12.0MIT
sqlparse0.3.0BSD
tifffile2019.7.26.2BSD
Bootstrap4.4BSD
jQuery3.4.1MIT
jQuery-Autocomplete1.4.1
jQuery-File-Upload10.7.0MIT
OpenSeadragon2.4.1BSD-3

We are grateful to the maintainers and contributors of the respective projects.

ImageTagger Reference

This paper describes the Bit-Bots imagetagger we build on in depth. Please cite if you use this tool in your research:

FIEDLER, Niklas, et al. imagetagger: An Open Source Online Platform for Collaborative Image Labeling. In: RoboCup 2018: Robot World Cup XXII. Springer, 2018.

@inproceedings{imagetagger2018,
   author={Fiedler, Niklas and Bestmann, Marc and Hendrich, Norman},
   year={2018},
   title={Imagetagger: An Open Source Online Platform for Collaborative Image Labeling},
   booktitle={RoboCup 2018: Robot World Cup XXII},
   organization={Springer}
}