Awesome
Paper - SS-SFDA: Semi Supervised Source Free Domain Adaptation for Road Segmentation for Road Segmentation in Hazardous Environments
Project Page - https://gamma.umd.edu/researchdirections/autonomousdriving/weathersafe/
Watch the entire video here
Please cite our paper if you find it useful.
@article{kothandaraman2020ss,
title={SS-SFDA: Self-Supervised Source-Free Domain Adaptation for Road Segmentation in Hazardous Environments},
author={Kothandaraman, Divya and Chandra, Rohan and Manocha, Dinesh},
journal={arXiv preprint arXiv:2012.08939},
year={2020}
}
Table of Contents
- Paper - <a href="link to paper" rel="nofollow"><strong>SS-SFDA: Semi Supervised Source Free Domain Adaptation for Road Segmentation for Road Segmentation in Hazardous Environments</strong></a>
- Repo Details and Contents
- Our network
- Acknowledgements
Repo Details and Contents
Python version: 3.7
Code structure
Dataloaders <br>
| Dataset | Dataloader | List of images |
|---|---|---|
| Clear weather CityScapes | dataset/cityscapes.py | dataset/cityscapes_list (train_images, val_images, train_labels, val_images) |
| Synthetic Fog | dataset/cityscapes_fog.py | dataset/cityscapes_list (train_rain_fog, val_rain_fog) |
| Synthetic Rain | dataset/cityscapes_rain.py | dataset/cityscapes_list (train_rain_fog, val_rain_fog) |
| Synthetic Rain | dataset/cityscapes_rain.py | dataset/cityscapes_list (train_rain_fog, val_rain_fog) |
| Real Fog - Foggy Zurich | dataset/foggy_zurich/train(test).py | dataset/foggy_zurich/lists_file_names |
| Real, Night Driving - Dark Zurich | dataset/dark_zurich/train(test).py | dataset/dark_zurich/lists_file_names |
| Heterogeneous Real, Rain + Night - Raincouver | dataset/raincouver/raincouver.py | dataset/raincouver (train_rain_fog, val_rain_fog) |
| Heterogeneous Real, Berkeley Deep Drive | dataset/bdd/bdd_{train,val}.py | dataset/bdd/bdd_list |
Models
model/drnd38.py - DRN-D-38 model <br> model/drnd38_attention.py - DRN-D-38 model with self-attention
Pretrained models
The pretrained models can be found here. Use eval.py to test them.
Our network
<p align="center"> <img src="main_architecture.png"> </p>Training your own model
Stage 1: The network is pre-trained on a clear weather dataset such as CityScapes. <br>
python train_stage1.py
Stage 2: SS-SFDA
I. Arrange the images in the increasing order of intensity (of rain/fog/snow/light), and divide into minibatches - provided by the dataset in most cases.
II. For each minibatch:
a. Initialize the network with weights from the minibatch (For the first minibatch, initialize from Stage 1)
b. python train_stage2step1.py
c. python train_stage2step3.py.
Stage 3: Further finetuning for heterogeneous real datasets.
I. Pick a small subset (random) of images (of the order of 5-10).
II. Initialize the network with Stage 2 weights.
III. Finetune the network with knowledge distillation
python train_stage3.py.
Evaluation:
python eval.py
Make sure to set appropriate paths to the folders containing the datasets, and the models in the training and evaluation files.
Datasets
- Clear weather: CityScapes
- Synthetic: Rain and Fog
- Real dataset (night): Dark Zurich
- Real dataset (fog): Foggy Zurich
- Heterogeneous Real dataset (rain+night): Raincouver
- Heterogeneous Real dataset (multiple weather, lighting conditions): Berkeley Deep Drive
Dependencies
PyTorch <br> NumPy <br> SciPy <br> Matplotlib <br>
Acknowledgements
This code is heavily borrowed from AdaptSegNet, and SAGAN