Awesome
A Soma Segmentation Benchmark in Full Adult Fly Brain
Accepted by CVPR-2023
Xiaoyu Liu, Bo Hu, Mingxing Li, Wei Huang, Yueyi Zhang, and Zhiwei Xiong*
University of Science and Technology of China (USTC), Hefei, China
Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China
*Corresponding Author
Abstract
Neuron reconstruction in a full adult fly brain from high-resolution electron microscopy (EM) data is regarded as a cornerstone for neuroscientists to explore how neurons inspire intelligence. As the central part of neurons, somas in the full brain indicate the origin of neurogenesis and neural functions. However, due to the absence of EM datasets specifically annotated for somas, existing deep learning-based neuron reconstruction methods cannot directly provide accurate soma distribution and morphology. Moreover, full brain neuron reconstruction remains extremely time-consuming due to the unprecedentedly large size of EM data. In this paper, we develop an efficient soma reconstruction method for obtaining accurate soma distribution and morphology information in a full adult fly brain. To this end, we first make a high-resolution EM dataset with fine-grained 3D manual annotations on somas. Relying on this dataset, we propose an efficient, two-stage deep learning algorithm for predicting accurate locations and boundaries of 3D soma instances. Further, we deploy a parallelized, high-throughput data processing pipeline for executing the above algorithm on the full brain. Finally, we provide quantitative and qualitative benchmark comparisons on the testset to validate the superiority of the proposed method, as well as preliminary statistics of the reconstructed somas in the full adult fly brain from the biological perspective.
Enviromenent
This code was tested with Pytorch 1.0.1 (later versions may work), CUDA 9.0, Python 3.7.4 and Ubuntu 16.04.
If you have a Docker environment, we strongly recommend you to pull our image as follows:
docker pull registry.cn-hangzhou.aliyuncs.com/renwu527/auto-emseg:v3.1
Implement the full brain data processing pipeline
1. Full Brain Data Division
cd ./Full_Brain_Soma_Segmentation_Pipeline/full_data_process2.0_seg
python script/submit_task.py -tn=divide_block
2. Intra-block Segmentation
python script/submit_task.py -tn=segmentation
3. Inter-block Stitching
cd ./Full_Brain_Soma_Segmentation_Pipeline/full_data_process2.0_stitch
python script/submit_task.py -tn=sort_ids
python scripts/submit_task.py -tn=stitching -sd=x0
python scripts/submit_task.py -tn=stitching -sd=x1
python scripts/submit_task.py -tn=stitching -sd=y0
python scripts/submit_task.py -tn=stitching -sd=y1
python scripts/submit_task.py -tn=stitching -sd=z0
python scripts/submit_task.py -tn=stitching -sd=z1
python scripts/submit_task.py -tn=concat
python scripts/submit_task.py -tn=global
python scripts/submit_task.py -tn=remap
<!-- ## Train Localization Network
```shell
cd Localization
python train.py
```
## Train Segmentation Network
```shell
cd Segmentation
python train.py
``` -->
Model Zoo
We provide the trained models at GoogleDrive, including the pre-trained localization model and two segmentation models with different backnones (3D UNet and Swin UNETR).
Models | Download |
---|---|
localization_model.pth | GoogleDrive |
segmentation_model_swin.pth | GoogleDrive |
segmentation_model.pth | GoogleDrive |
Inference on a 3D block
<!-- We have released our trained models, including the localization network and the segmentation network. You can download these model at [[Models](https://drive.google.com/drive/folders/1KpbS3s4-B45O27FI96Um3nW9LlAusj_k)] -->If you want to test the model on a block, you can implement the following two commands for 3D UNet and Swin UNETR model, respectively :
python inference_unet.py
python inference_swin.py
Examples
We provide an example of their corresponding segmentation result predicted by our trained model in [Examples]
patch-level example
block-level example
Dataset
The annotated dataset EMADS is released and can be downloaded at [Dataset]
Overview of data annotation
Visualization of our soma reconstruction
We also make a 4K video based on the image sequence of the full brain. We set the time duration of each image as 0.01 second, and stack the image sequence into a video chronological. The 4K video is released and can be downloaded at [Video]
Soma statistics of the full brain
Contact
If you have any problem with the released code and dataset, please contact me by email (liuxyu@mail.ustc.edu.cn).
Citation
@inproceedings{liu2023soma,
title={A Soma Segmentation Benchmark in Full Adult Fly Brain},
author={Liu, Xiaoyu and Hu, Bo and Li, Mingxing and Huang, Wei and Zhang, Yueyi and Xiong, Zhiwei},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={7402--7411},
year={2023}
}