Home

Awesome

Bias-to-Text (B2T): Visual Biases as Keywords

Implementation of the bias-to-text (B2T) algorithm described in "Bias-to-Text: Debiasing Unknown Visual Biases through Language Interpretation." B2T identifies and mitigates visual biases in image classifiers and text-to-image generative models using language descriptions.

Example Results

plot See the link for more detailed results in bias discovery and debiasing.

Method Overview

plot

Installation

Download datasets.

Clone our repository.

$ git clone https://github.com/Erena-Kim/b2t.git

Run below to create virtual environment for b2t and install all prerequisites.

$ pip install pipenv
$ pipenv --python 3.8
$ pipenv install

To run our code, you need to place datasets and model checkpoints to right directory.
You can download the ClipCap pretrained model here and place the model to [root_dir]/function. (Note that our paper uses the model that trained on Conceptual Captions)
The main point of entry to the code is b2t.py

Arguments

CelebA

Our code expects the following files/folders in the [root_dir]/data/celebA directory:

A sample command to run b2t on CelebA with pretrained erm model is:

$ python b2t.py --dataset celeba --model best_model_CelebA_erm.pth

Waterbirds

Our code expects the following files/folders in the [root_dir]/data/cub directory:

A sample command to run b2t on Waterbirds with pretrained erm model is:

$ python b2t.py --dataset waterbird --model best_model_Waterbirds_erm.pth

Debiasing classifiers with B2T

To reproduce the debiasing classifier experiments, see b2t_debias.

Diffusion models with B2T

To reproduce the diffusion model experiments, see b2t_diffusion.