Awesome
DF4LCZ
This repository hosts the code for the DF4LCZ model which is a sam-empowered data fusion framework for scene-level local climate zone classification. The relevant paper is DF4LCZ: A SAM-Empowered Data Fusion Framework for Scene-Level Local Climate Zone Classification
Usage
Install
- Python 3.8
- Tensorflow 2.11.0
- Keras 2.11.0
- Spektral 1.3.0
- albumentations 1.3.1
Train and test
- Change the current directory to
DF4LCZ
. - For the Sentinel-2 stream, run
single_fix_augment.py
using
python ./single_fix_augment.py --model 'resnet11_3D' --batch_size 64 --initial_lr 0.002 --decay_factor 0.4 --patience 40 --epoch 100
- For the Google Earth stream, run
GNN_train.py
using
python ./GNN_train.py
- For the DF4LCZ classification,
- Add the weights file paths derived from the two streams into the
fusion.py
, - and run
fusion.py
- Add the weights file paths derived from the two streams into the
SAM and Graph construction
- Download the SAM weights file
sam_vit_h_4b8939.pth
from segment anything and place it in theweights
folder ofDF4LCZ
. - Obtain the original Google patches from the link in the
LCZC-GES2 DATASET
section of this README.md and store them in the folderDF4LCZ
. - Change the current directory to
DF4LCZ
. - Run
ins_gen.py
for SAM-based instance extraction (make sure to create an output folder beforehand). - Execute
testmasks.py
for graph construction; the generated graphs are provided in the foldergg_nodes_refine
.
LCZC-GES2 DATASET
-
The LCZC-GES2 dataset comprises 19,088 pairs of image patches. Each pair includes a Google Earth RGB image and a Sentinel-2 multispectral image patch. The data can be found in the folders
sen2_img_patches
andgg_nodes_refine
.(gg_nodes_refine
contains all graphes after the graph construction procedure, the original google patches please refer to Google drive files) -
The folder
patches_split
contains the spliting results from two sampling strategies. The filepartition_polygons_1125.npz
represents one strategy, named “splitting the polygon pool,” whilepartition_random.npz
represents the other strategy, named “splitting the sample pool.”
Latest Updates
- [2024/05/07]: Uploaded a batch of code, including the implementation of DF4LCZ and several comparative models.
- [2024/10/07]: Uploaded a batch of code, including the implementation of SAM-based instance extraction and graph construction.
Citation
If you use this code in your research, please consider citing the following paper:
@ARTICLE{10556641,
author={Wu, Qianqian and Ma, Xianping and Sui, Jialu and Pun, Man-On},
journal={IEEE Transactions on Geoscience and Remote Sensing},
title={DF4LCZ: A SAM-Empowered Data Fusion Framework for Scene-Level Local Climate Zone Classification},
year={2024},
volume={62},
number={},
pages={1-16},
keywords={Feature extraction;Internet;Earth;Task analysis;Image segmentation;Spatial resolution;Meteorology;Data fusion;local climate zone (LCZ) classification;segment anything model (SAM)},
doi={10.1109/TGRS.2024.3414143}}
Contact
For any inquiries or further information, please contact me.
Other Works
There are some other works in our group:
SAM-Assisted Remote Sensing Imagery Semantic Segmentation with Object and Boundary Constraints(TGRS under review)Code