Awesome
SA<u>LE</u>M<sup>2</sup>: Segment Anything in <u>L</u>ight and <u>E</u>lectron <b>M</b>icroscopy via <b>M</b>embrane Guidance
This repository has been archived. Please instead refer SAEM2 in SAvEM3.
Comparison with General-purpose Methods
Quantitative comparison: SAM, HQ-SAM, Mobile-SAM (from HQ-SAM) and Micro-SAM, as well as trained SAM and HQ-SAM in both LM and EM datasets.
Comparison with Specialized Supervised Methods
Qualitative comparison: CellPose in LM, ilastik (pretrained models from https://bioimage.io/#/?partner=ilastik) and Superhuman (onnx models from https://github.com/seung-lab/DeepEM/releases) in EM.
Elongated cells in LM:
<table> <tr> <td><img src="assets/Elong-528-raw.jpg" width="200"></td> <td><img src="assets/Elong-528-label.jpg" width="200"></td> <td><img src="assets/Elong-528-CellPose.png" width="200"></td> <td><img src="assets/Elong-528-OmniPose.png" width="200"></td> <td><img src="assets/Elong-528.png" width="200"></td> </tr> <tr> <td><p align="center">Raw</p></td> <td><p align="center">GT</p></td> <td><p align="center">CellPose<br>(<a href="http://dx.doi.org/10.1038/s41592-020-01018-x">Nat. Methods 2021</a>)</p></td> <td><p align="center">OmniPose<br>(<a href="http://dx.doi.org/10.1038/s41592-022-01639-4">Nat. Methods 2022</a>)</p></td> <td><p align="center">SALM<sup>2</sup><br>(Ours)</p></td> </tr> </table>Weak boundaries in LM:
<table> <tr> <td><img src="assets/Weak-738-raw.jpg" width="200"></td> <td><img src="assets/Weak-738-label.jpg" width="200"></td> <td><img src="assets/Weak-738-CellPose.jpg" width="200"></td> <td><img src="assets/Weak-738.jpg" width="200"></td> </tr> <tr> <td><p align="center">Raw</p></td> <td><p align="center">GT</p></td> <td><p align="center">CellPose<br>(<a href="http://dx.doi.org/10.1038/s41592-020-01018-x">Nat. Methods 2021</a>)</p></td> <td><p align="center">SALM<sup>2</sup><br>(Ours)</p></td> </tr> </table>CREMI-B (blur and misalignment) in EM:
<table> <tr> <td><img src="assets/NcremiB_origin.gif" width="200"></td> <td><img src="assets/NcremiB_labels.gif" width="200"></td> <td><img src="assets/CREMI-B15-Superhuman.jpg" width="200"></td> <td><img src="assets/CREMI-B15.jpg" width="200"></td> </tr> <tr> <td><p align="center">Raw<br>(sections 12-18)</p></td> <td><p align="center">GT<br>(sections 12-18)</p></td> <td><p align="center">Superhuman (section 15)<br>(<a href="http://dx.doi.org/10.1109/TMI.2021.3097826">IEEE Trans. Med. Imaging 2021</a>)</p></td> <td><p align="center">SAEM<sup>2</sup> (section 15)<br>(Ours)</p></td> </tr> </table>CREMI-C (blur and missing sections) in EM:
<table> <tr> <td><img src="assets/NcremiC_origin.gif" width="200"></td> <td><img src="assets/NcremiC_labels.gif" width="200"></td> <td><img src="assets/CREMI-C103-ilastik.jpg" width="200"></td> <td><img src="assets/CREMI-C103.jpg" width="200"></td> </tr> <tr> <td><p align="center">Raw<br>(sections 100-106)</p></td> <td><p align="center">GT<br>(sections 100-106)</p></td> <td><p align="center">ilastik (section 103)<br>(<a href="http://dx.doi.org/10.1038/s41592-019-0582-9">Nat. Methods 2019</a>)</p></td> <td><p align="center">SAEM<sup>2</sup> (section 103)<br>(Ours)</p></td> </tr> </table>More results can be found in Google Drive: BBC039, NeurIPS22-CellSeg and CREMI. We used CellPose and OmniPose with the configurations of nuclei (flow_threshold=0.3) for Bare Nuclei, tissuenet (flow_threshold=0.6) for Weak Boundary, nuclei (flow_threshold=0.6) and bact_phase_omni for Elongated. We used the open-sourced trained models of ilastik and Superhuman with alternative preprocessing steps.
Contributors
Thanks to the other authors and MiRA Team for their support and resources.
Acknowledgments
Thanks for their public code and released models.