Awesome

Masking HSI With Pretrained Models

HZDR - Hif_Exploration

Overview

The presence of undesired background areas associated with potential noise and unknown spectral characteristics degrades the performance of hyperspectral data processing. Masking out unwanted regions is key to addressing this issue. Processing only regions of interest yields notable improvements in terms of computational costs, required memory, and overall performance.

The proposed method offers a solution for masking out unwanted regions and objects in the images. The method is built upon SAM and Grounding Dino in addition to further computer vision techniques.

To illustrate the efficacy of the masking procedure, the proposed method is deployed on three challenging applications scenarios that demand accurate masking i.e. shredded plastics characterization, drill core scanning, and litter monitoring

This work builds upon Segment Anything and Grounding Dino

Requirements

Clone and set up the Segment-Anything and Grounding Dino repositories.

Please check the dependency requirements in the two repositories.

The implementation is tested under python 3.8, as well as pytorch 1.12 and torchvision 0.13. We recommend an equivalent or higher pytorch version.

Input Data

For data privacy, the hyperspectral images used in the paper are not fully provided. Only 3 channel representations and their ground truths can be found in the images folder.

For reference, please cite: E. Arbash et al., "Masking Hyperspectral Imaging Data with Pretrained Models," 2023 13th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS), Athens, Greece, 2023, pp. 1-5, doi: 10.1109/WHISPERS61460.2023.10431074. keywords: {Image segmentation;Filtering;Grounding;Pipelines;Data processing;Plastics;Hyperspectral imaging;Hyperspectral Imaging;Classification;Masking;SAM;Grounding Dino},