Awesome

MSEG-VCUQ: Multimodal SEGmentation with Enhanced Vision Foundation Models, Convolutional Neural Networks, and Uncertainty Quantification for High-Speed Video Phase Detection Data

This repository contains the codebase and resources for the MSEG-VCUQ framework, as detailed in the paper, "MSEG-VCUQ: Multimodal SEGmentation with Enhanced Vision Foundation Models, Convolutional Neural Networks, and Uncertainty Quantification for High-Speed Video Phase Detection Data." The framework integrates vision foundation models and convolutional neural networks (CNNs) with uncertainty quantification (UQ) to advance segmentation accuracy and reliability across diverse HSV modalities.

📁 Repository Structure

This folder focuses on U-Net Convolutional Neural Networks (CNNs) and Uncertainty Quantification (UQ) for HSV segmentation. Highlights include:

• Automated segmentation pipelines using U-Net for high-speed video data.
• Quantification of pixel-level discretization errors to evaluate boiling metrics such as contact line density and dry area fraction.
• Comprehensive UQ analyses to enhance experimental reproducibility.
• Refer to the folder-specific README.md for detailed instructions.

This folder contains the VideoSAM framework, which integrates convolutional neural networks (CNNs) with the transformer-based Segment Anything Model (SAM). It represents the vision foundation model component of the MSEG-VCUQ framework, specifically tailored for multimodal segmentation tasks. Key features include:

• Advanced segmentation capabilities across complex HSV modalities, including Argon, Nitrogen, and FC-72.
• Zero-shot generalization on unseen datasets, demonstrating robust adaptability to varying fluid dynamics.
• Comprehensive evaluation of segmentation performance using metrics like IoU and F1 Score.
• Detailed implementation and usage instructions are available in the folder-specific README.md.

This folder contains:

• The research paper outlining the MSEG-VCUQ framework, experimental results, and key findings.
• Supporting figures, tables, and datasets used in the study.

This file provides a comprehensive overview of the repository and guides users in navigating its structure.

🚀 Getting Started

To begin using MSEG-VCUQ, follow these steps:

1. Clone the repository:

   git clone https://github.com/chikap421/mseg_vcuq.git
   cd mseg_vcuq
   

2. Explore the subdirectories (cnn_uq, videosam, paper) for specific tools, datasets, and models.

3. Install required dependencies:

   pip install -r requirements.txt
   

4. Follow folder-specific README.md files for instructions on replicating results or running experiments.

🔗 Links and Resources

• MIT Red Lab: Experimental data and resources for this project.
• VideoSAM Repository: Standalone implementation of the VideoSAM framework.
• CNN-UQ Repository: Implementation of U-Net CNNs and Uncertainty Quantification for HSV segmentation.

📜 License

This repository is licensed under the MIT License. See the LICENSE file for details.

🖋️ Citations

If you use this repository in your research, please cite:

@misc{maduabuchi2024msegvcuqmultimodalsegmentationenhanced,
      title={MSEG-VCUQ: Multimodal SEGmentation with Enhanced Vision Foundation Models, Convolutional Neural Networks, and Uncertainty Quantification for High-Speed Video Phase Detection Data}, 
      author={Chika Maduabuchi and Ericmoore Jossou and Matteo Bucci},
      year={2024},
      eprint={2411.07463},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2411.07463}, 
}

🌟 Acknowledgments

We acknowledge the contributions of the MIT Red Lab, collaborators, and funding agencies that supported this research.