Awesome

We plan to create a very interesting demo by combining Segment Anything and a series of style transfer models! We will continue to improve it and create more interesting demos. Interesting ideas, results, and contributions are warmly welcome!

Demo

Different contents can specify different styles in a style image. Left click to select an area and right click to exclude an area.
Middle click and drag your mouse to specify a bounding box.
You can also draw a contour with mouse and specify a style region for it.

Installation

python -m pip install torch
python -m pip install -e segment_anything
python -m pip install opencv-python

The code is tested on the environment with Ubuntu 18.04, python 3.6.9, torch 1.8.1+cuda10.2, opencv-python 4.5.5, and a 2080Ti GPU.

Get Started

Clone this repo:

git clone https://github.com/Huage001/Transfer-Any-Style.git
cd Transfer-Any-Style

Download the model checkpoint of Segment Anything:

wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
mv sam_vit_h_4b8939.pth segment-anything/

Download the model checkpoint of AdaAttN from here and unzip it to directory of this repo:
```
mv [DOWNLOAD_PATH]/ckpt.zip .
unzip ckpt.zip
rm ckpt.zip
```
Run the following command:
```
transfer_any_style.py \
--content_path CONTENT_PATH \
--style_path STYLE_PATH \
--resize
```
Follow the instruction printed on the console to run the interactive demo.
Full usage:
- Command: python transfer_any_style.py [-h] --content_path CONTENT_PATH --style_path STYLE_PATH [--output_dir OUTPUT_DIR] [--resize] [--keep_ratio]
- Optional Arguments:
  - --content_path: Path to a single content img
  - --style_path: Path to a single style img
  - --output_dir: Output path
  - --resize: Whether resize images to the 512 scale, which is the training resolution of the model and may yield better performance
  - --keep_ratio: Whether keep the aspect ratio of original images while resizing

Future Work

Support more style images at a time.
Integrate with more state-of-the-art style transfer methods.
More user-friendly and stable user interface.
...

:cupid: Acknowledgement

Citation

If you find this project helpful for your research, please consider citing the following BibTeX entry.

@article{kirillov2023segany,
    title={Segment Anything}, 
    author={Kirillov, Alexander and Mintun, Eric and Ravi, Nikhila and Mao, Hanzi and Rolland, Chloe and Gustafson, Laura and Xiao, Tete and Whitehead, Spencer and Berg, Alexander C. and Lo, Wan-Yen and Doll{\'a}r, Piotr and Girshick, Ross},
    journal={arXiv:2304.02643},
    year={2023}
}

@inproceedings{liu2021adaattn,
  title={AdaAttN: Revisit Attention Mechanism in Arbitrary Neural Style Transfer},
  author={Liu, Songhua and Lin, Tianwei and He, Dongliang and Li, Fu and Wang, Meiling and Li, Xin and Sun, Zhengxing and Li, Qian and Ding, Errui},
  booktitle={Proceedings of the IEEE International Conference on Computer Vision},
  year={2021}
}