Awesome
py-clesperanto
py-clesperanto is a prototype for clesperanto - a multi-platform multi-language framework for GPU-accelerated image processing. We mostly use it in the life sciences for analysing 3- and 4-dimensional microsopy data, e.g. as we face it developmental biology when segmenting cells and studying their individual properties as well as properties of compounds of cells forming tissues.
Image data source: Daniela Vorkel, Myers lab, MPI-CBG, rendered using napari
clesperanto uses OpenCL kernels from CLIJ.
For users convenience, there are code generators available for napari and Fiji. Also check out the napari workflow optimizer for semi-automatic parameter tuning of clesperanto-functions.
Reference
The preliminary API reference is available here. Furthermore, parts of the reference are also available within the CLIJ2 documentation.
Installation
- Get a conda/python environment, e.g. via mamba-forge.
- If you never used python/conda environments before, please follow these instructions first.
conda create --name cle_39 python=3.9
conda activate cle_39
- Install pyclesperanto-prototype using mamba / conda:
mamba install -c conda-forge pyclesperanto-prototype
OR using pip:
pip install pyclesperanto-prototype
Troubleshooting: Graphics cards drivers
In case error messages contain "ImportError: DLL load failed while importing cl: The specified procedure could not be found" see also or "clGetPlatformIDs failed: PLATFORM_NOT_FOUND_KHR", please install recent drivers for your graphics card and/or OpenCL device. Select the right driver source depending on your hardware from this list:
Sometimes, mac-users need to install this:
mamba install -c conda-forge ocl_icd_wrapper_apple
Sometimes, linux users need to install this:
mamba install -c conda-forge ocl-icd-system
Linux user may have to install packages such as intel-opencl-icd or rocm-opencl-runtime depending on their GPU.
Computing on Central Processing units (CPUs)
If no OpenCL-compatible GPU is available, pyclesperanto-prototype can make use of CPUs instead. Just install oclgrind or pocl, e.g. using mamba / conda. Oclgrind is recommended for Windows systems, PoCL for Linux. MacOS typically comes with OpenCL support for CPUs.
mamba install oclgrind -c conda-forge
OR
mamba install pocl -c conda-forge
Owners of compatible Intel Xeon CPUs can also install a driver to use them for computing:
Example code
A basic image processing workflow loads blobs.gif and counts the number of objects:
import pyclesperanto_prototype as cle
from skimage.io import imread, imsave
# initialize / select GPU with "TX" in their name
device = cle.select_device("TX")
print("Used GPU: ", device)
# load data
image = imread('https://imagej.nih.gov/ij/images/blobs.gif')
# process the image
inverted = cle.subtract_image_from_scalar(image, scalar=255)
blurred = cle.gaussian_blur(inverted, sigma_x=1, sigma_y=1)
binary = cle.threshold_otsu(blurred)
labeled = cle.connected_components_labeling_box(binary)
# The maximium intensity in a label image corresponds to the number of objects
num_labels = labeled.max()
print(f"Number of objects in the image: {num_labels}")
# save image to disc
imsave("result.tif", labeled)
Example gallery
<table border="0"> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/screenshot_select_GPU.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/jupyter.png" width="300"/> </td><td>Image processing in Jupyter Notebooks
</td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/screenshot_count_blobs.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/voronoi_otsu_labeling.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/segmentation_3d.png" width="300"/> </td><td> </td></tr> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/segmentation_2d_membranes.png" width="300"/> </td><td>Cell segmentation based on membranes
</td></tr> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/counting_nuclei_multichannel.png" width="300"/> </td><td>Counting nuclei according to expression in multiple channels
</td></tr> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/differentiate_nuclei_intensity.png" width="300"/> </td><td>Differentiating nuclei according to signal intensity
</td></tr> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/bead_segmentation.png" width="300"/> </td><td>Detecting beads and measuring their size
</td></tr> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/label_statistics.png" width="300"/> </td><td> </td></tr> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/parametric_maps.png" width="300"/> </td><td> </td></tr> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/intensities_along_lines.png" width="300"/> </td><td> </td></tr> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/screenshot_crop_and_paste_images.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/screenshot_inspecting_3d_images.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/screenshot_affine_transforms.png" width="300"/> </td><td>Rotation, scaling, translation, affine transforms
</td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/screenshot_deskew.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/screenshot_multiply_vectors_and_matrices.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/screenshot_multiply_matrices.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/screenshot_spots_pointlists_matrices_tables.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/mesh_between_centroids.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/mesh_between_touching_neighbors.png" width="300"/> </td><td>Mesh between touching neighbors
</td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/mesh_with_distances.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/mesh_nearest_neighbors.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/ipgraph_networkx.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/neighborhood_definitions.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/tissue_neighborhood_quantification.png" width="300"/> </td><td>Tissue neighborhood quantification
</td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/neighbors_of_neighbors.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/screenshot_voronoi_diagrams.png" width="300"/> </td><td> </td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/shape_descriptors_based_on_neighborhood_graphs.png" width="300"/> </td><td>Shape descriptors based on neighborhood graphs
</td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/distance_to_other_labels.png" width="300"/> </td><td>Measuring distances between labels in two label images
</td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/screenshot_tribolium_napari.png" width="300"/> </td><td>Tribolium morphometry + Napari
</td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/screenshot_tribolium_morphometry.png" width="300"/> </td><td>Tribolium morphometry (archived version)
</td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/screenshot_napari_dask.png" width="300"/> </td><td>napari+dask timelapse processing
</td></tr> </table>Technical insights
<table border="0"><tr><td> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/browse_operations.png" width="300"/> </td><td> </td></tr> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/interactive_widgets.gif" width="300"/> </td><td> </td></tr> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/optimize_blobs_segmentation.png" width="300"/> </td><td>Automatic workflow optimization
</td></tr> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/memory_management.png" width="300"/> </td><td>Tracing memory consumtion on NVidia GPUs
</td></tr> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/switching_gpus.png" width="300"/> </td><td>Exploring and switching between GPUs
</td></tr> <tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/cupy_compatibility.png" width="300"/> </td><td> </td></tr> <tr><td> <img src="./docs/images/dask.jpg" width="300"/> </td><td>Big data handling with Dask GPU clusters
</td></tr> </table>Related projects
<table border="0"><tr><td> <img src="https://github.com/clesperanto/napari_pyclesperanto_assistant/raw/master/docs/images/screenshot.png" width="300"/> </td><td>napari-pyclesperanto-assistant: A graphical user interface for general purpose GPU-accelerated image processing and analysis in napari.
</td></tr><tr><td> <img src="https://github.com/haesleinhuepf/napari-accelerated-pixel-and-object-classification/raw/main/images/screenshot.png" width="300"/> </td><td>napari-accelerated-pixel-and-object-classification: GPU-accelerated Random Forest Classifiers for pixel and labeled object classification
</td></tr><tr><td> <img src="https://github.com/clEsperanto/pyclesperanto_prototype/raw/master/docs/images/napari-clusters-plotter.png" width="300"/> </td><td>napari-clusters-plotter: Clustering of objects according to their quantitative properties
</td></tr></table>Benchmarking
We implemented some basic benchmarking notebooks allowing to see performance differences between pyclesperanto and some other image processing libraries, typically using the CPU. Such benchmarking results vary heavily depending on image size, kernel size, used operations, parameters and used hardware. Feel free to use those notebooks, adapt them to your use-case scenario and benchmark on your target hardware. If you have different scenarios or use-cases, you are very welcome to submit your notebook as pull-request!
- Affine transforms
- Background subtraction
- Gaussian blur
- Convolution
- Otsu's thresholding
- Connected component labeling
- Extend labels
- Statistics of labeled pixels / regionprops
- Histograms
- Matrix multiplication
- Pixel-wise comparison
- Intensity projections
- Axis transposition
- Nonzero
See also
There are other libraries for code acceleration and GPU-acceleration for image processing.
Feedback welcome!
clesperanto is developed in the open because we believe in the open source community. See our community guidelines. Feel free to drop feedback as github issue or via image.sc