Awesome
Learning Across Domains and Devices: Style-Driven Source-Free Domain Adaptation in Clustered Federated Learning
Official implementation of Learning Across Domains and Devices: Style-Driven Source-Free Domain Adaptation in Clustered Federated Learning
by Donald Shenaj<sup>*,1</sup>, Eros Fanì<sup>*,2</sup>, Marco Toldo<sup>1</sup>, Debora Caldarola<sup>2</sup>, Antonio Tavera<sup>2</sup>, Umberto Micheli<sup>†,1</sup>, Marco Ciccone<sup>†,2</sup>, Pietro Zanuttigh<sup>†,1</sup> and Barbara Caputo<sup>†,2</sup>
Corresponding authors: donald.shenaj@dei.unipd.it, eros.fani@polito.it
<sup>*</sup> Equal contribution. <sup>†</sup> Equal supervision. <sup>1</sup> Authors supported by University of Padova, Padua, Italy. <sup>2</sup> Authors supported by Politecnico di Torino, Turin, Italy.
<img src="teaser.png" alt="drawing" width="800"/>Citation
If you find our work relevant to your research, or use this code, please cite our WACV 2023 paper:
@inproceedings{shenaj2023ladd,
title={"Learning Across Domains and Devices: Style-Driven Source-Free Domain Adaptation in Clustered Federated Learning},
author={Shenaj, Donald and Fan\`i, Eros and Toldo, Marco and Caldarola, Debora and Tavera, Antonio and Michieli, Umberto and Ciccone, Marco and Zanuttigh, Pietro and Caputo, Barbara},
booktitle={Winter Conference on Applications of Computer Vision (WACV)},
year={2023},
organization={IEEE}
}
Summary
In this work we propose a novel realistic scenario for Semantic Segmentation in Federated Learning: Federated source-Free Domain Adaptation (FFreeDA). In FFreeDA, the server can pre-train the model on labeled source data. However, as in the Source-Free Domain Adaptation (SFDA) setting, further accessing the source data is forbidden. Clients can access only their unlabeled target dataset, but cannot share it with other clients nor with the server. Moreover, there are many clients in the system and each of them has only a limited amount of images, to emulate real-world scenarios. Therefore, after the pre-training phase, the training is fully unsupervised. To address the FFreeDA problem, we propose LADD, a novel federated algorithm that assumes the presence of multiple distributions hidden among the clients. LADD partitions the clients into clusters based on the styles of the images belonging to each client, trying to match them with their actual latent distribution. LADD shows excellent performance on all benchmarks with a source dataset (GTA5) and three different targets (Cityscapes, CrossCity, Mapillary), with diversified splits of the data across the clients.
Setup
Preliminary operations
- Clone this repository.
- Move to the root path of your local copy of the repository.
- Create the
LADD
new conda virtual environment and activate it:
conda env create -f environment.yml
conda activate LADD
Datasets
- Download the the Cityscapes dataset from here (
gtFine_trainvaltest.zip
andleftImg8bit_trainvaltest.zip
files). - Ask for the Crosscity dataset here.
- Download the Mapillary Vistas dataset from here.
- Download the GTA5 dataset from here.
- Extract all the datasets' archives. Then, move the datasets' folders in
data/[dataset_name]/data
, where[dataset_name]
is one of{cityscapes, crosscity, mapillary, gta5}
.
data
├── cityscapes
│ ├── data
│ │ ├── leftImg8bit
│ │ │ ├── train
│ │ │ ├── val
│ │ ├── gtFine
│ │ │ ├── train
│ │ │ ├── val
│ ├── splits
├── gta5
│ ├── data
│ │ ├── images
│ │ ├── labels
│ ├── splits
├── crosscity
│ ├── data
│ │ ├── cities
│ ├── splits
├── mapillary
│ ├── data
│ │ ├── training
│ │ ├── testing
│ │ ├── validation
│ ├── splits
Experiments' logging
Make a new wandb account if you do not have one yet, and create a new wandb project.
How to run
In the configs folder, it is possible to find examples of config files for some of the experiments to replicate the results of the paper. Run one of the exemplar configs or a custom one from the root path of your local copy of the repository:
./run.sh [path/to/config]
We provide config files to replicate the experiments in the paper:
- run a pre-training script — e.g.
./run.sh configs/pretrain_crosscity.txt
and take the corresponding exp_id from wandb - run our method — e.g.
./run.sh configs/crosscity_LADD_classifier.txt
. Make sure to setload_FDA_id=[exp_id_pretrain]
N.B. change the wandb_entity
argument with the entity name of your wandb project.
N.B. always leave a blank new line at the end of the config. Otherwise, your last argument will be ignored.
Results
GTA5 → Cityscapes
Setting | Method | mIoU (%) |
---|---|---|
centralized | Oracle | 66.64 ± 0.33 |
centralized | Source Only | 24.05 ± 1.14 |
centralized | FTDA | 65.74 ± 0.48 |
centralized | MCD | 20.55 ± 2.66 |
centralized | DAFormer | 42.31 ± 0.20 |
FL-UDA | MCD | 10.86 ± 0.67 |
FFreeDA | FedAvg<sup>†</sup> + Self-Tr. | 35.10 ± 0.73 |
FFreeDA | LADD (cls) | 36.49 ± 0.13 |
FFreeDA | LADD (all) | 36.49 ± 0.14 |
<sup>†</sup> Same pretrain as LADD.
GTA5 → Crosscity
Setting | Method | mIoU (%) |
---|---|---|
centralized | Source Only | 26.49 ± 1.46 |
centralized | MCD | 27.15 ± 0.87 |
FL-UDA | MCD | 24.80 ± 1.56 |
FFreeDA | FedAvg<sup>†</sup> + Self-Tr. | 33.59 ± 1.25 |
FFreeDA | LADD (cls) | 39.87 ± 0.14 |
FFreeDA | LADD (all) | 40.09 ± 0.19 |
<sup>†</sup> Same pretrain as LADD.
GTA5 → Mapillary
Setting | Method | mIoU (%) |
---|---|---|
centralized | Oracle | 61.46 ± 0.21 |
centralized | Source Only | 32.40 ± 0.71 |
centralized | MCD | 31.93 ± 1.89 |
federated | Oracle | 49.91 ± 0.49 |
FL-UDA | MCD | 19.15 ± 0.75 |
FFreeDA | FedAvg<sup>†</sup> + Self-Tr. | 38.97 ± 0.21 |
FFreeDA | LADD (cls) | 40.16 ± 1.02 |
FFreeDA | LADD (all) | 38.78 ± 1.82 |
<sup>†</sup> Same pretrain as LADD.