Awesome

Awesome-Image-Colorization

A collection of Deep Learning based Image Colorization papers and corresponding source code/demo program, including Automatic and User Guided (i.e. with User Interaction) colorization, as well as video colorization.

Feel free to create a PR or an issue. (Pull Request is preferred)

examples

Outline

Automatic Image Colorization
- Software / Demo
- Papers
User Guided Image Colorization
Techniques of Improving Image Colorization
Video Colorization

1. Automatic Image Colorization

1.1 Software / Demo

Name	Author/Owner	Code/Project Link
DeOldify	Jason Antic	[link]
Palette.fm	Emil Wallner	[link]

1.2 Papers

Natural images

Paper	Source	Code/Project Link
Learning Large-Scale Automatic Image Colorization	ICCV 2015	[project] [code]
Deep Colorization	ICCV 2015
Learning Representations for Automatic Colorization	ECCV 2016	[project] [code]
Colorful Image Colorization	ECCV 2016	[project] [code]
Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification	SIGGRAPH 2016	[project] [code]
Learning Diverse Image Colorization	CVPR 2017	[code]
Structural Consistency and Controllability for Diverse Colorization	ECCV 2018
Coloring With Limited Data: Few-Shot Colorization via Memory Augmented Networks	CVPR 2019	[project]<br/><br/> Unofficial:<br/> [code (PyTorch)]
ChromaGAN: Adversarial Picture Colorization with Semantic Class Distribution	WACV 2020	[code]
Instance-aware Image Colorization	CVPR 2020	[code] [project]
Pixelated Semantic Colorization	IJCV 2020
Colorization Transformer	ICLR 2021	[code]
Focusing on Persons: Colorizing Old Images Learning from Modern Historical Movies	ACM MM 2021	[code]
Towards Vivid and Diverse Image Colorization with Generative Color Prior	ICCV 2021	[code]
SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution (also reference based)	AAAI 2022
Bridging the Domain Gap towards Generalization in Automatic Colorization	ECCV 2022	[code]
ColorFormer: Image Colorization via Color Memory assisted Hybrid-attention Transformer	ECCV 2022	[code]
BigColor: Colorization using a Generative Color Prior for Natural Images	ECCV 2022	[project] [code]
CT2: Colorization Transformer via Color Tokens	ECCV 2022	[code]
Disentangled Image Colorization via Global Anchors	SIGGRAPH Asia 2022	[code] [project]
UniColor: A Unified Framework for Multi-Modal Colorization with Transformer (also multi-modal)	SIGGRAPH Asia 2022	[project]
Improved Diffusion-based Image Colorization via Piggybacked Models	arxiv 23.04	[project]
DDColor: Towards Photo-Realistic and Semantic-Aware Image Colorization via Dual Decoders	ICCV 2023	[code] [project]
Diffusing Colors: Image Colorization with Text Guided Diffusion (also text-based)	SIGGRAPH Asia 2023	[project]
Automatic Controllable Colorization via Imagination	CVPR 2024	[code] [project]
Control Color: Multimodal Diffusion-based Interactive Image Colorization	arxiv 24.02	[code] [project]
MultiColor: Image Colorization by Learning from Multiple Color Spaces	ACM MM 2024

Line arts

Paper	Source	Code/Project Link
Region Assisted Sketch Colorization	TIP 2023

2. User Guided Image Colorization

2.1 Based on scribble

Software / Demo

Image Type	Name	Author/Owner	Code/Project Link
Line art	Petalica Paint (Old version: PaintsChainer)	Preferred Networks, Inc.	[Petalica Paint (Online service)] [PaintsChainer V1 code]
Line art	Style2Paints (SEPA)	Style2Paints Research	[link]

Papers

Image Type	Paper	Source	Code/Project Link
Manga	Manga colorization	SIGGRAPH 2006	[Project]
Line art / Sketch / Manga	LazyBrush: Flexible Painting Tool for Hand-drawn Cartoons	Eurographics 2009	[Project Home & Software] <br/><br/>Unofficial:<br/> [code(Matlab)]
Line art / Sketch	Outline Colorization through Tandem Adversarial Networks	1704.08834	[code]
Natural Gray-Scale	Real-Time User-Guided Image Colorization with Learned Deep Priors	SIGGRAPH 2017	[project] [code1] [code2]
Sketch	Scribbler: Controlling Deep Image Synthesis with Sketch and Color	CVPR 2017
Line art / Sketch	Auto-painter: Cartoon Image Generation from Sketch by Using Conditional Generative Adversarial Networks	Neurocomputing 2018	[code]
Line art	User-Guided Deep Anime Line Art Colorization with Conditional Adversarial Networks	ACM MM 2018	[code]
Line art	Two-stage Sketch Colorization	SIGGRAPH Asia 2018	[Project]
Line art	User-Guided Line Art Flat Filling with Split Filling Mechanism	CVPR 2021	[Project] [code]
Line art	Dual Color Space Guided Sketch Colorization	TIP 2021
Natural Gray-Scale	iColoriT: Towards Propagating Local Hint to the Right Region in Interactive Colorization by Leveraging Vision Transformer	WACV 2023	[project] [code]

2.2 Based on reference image

Image Type	Paper	Source	Code/Project Link
Manga	Comicolorization: Semi-Automatic Manga Colorization	SIGGRAPH Asia 2017 Technical Briefs	[code]
Sketch	TextureGAN: Controlling Deep Image Synthesis with Texture Patches	CVPR 2018	[code]
Natural Gray-Scale	Deep Exemplar-based Colorization	SIGGRAPH 2018	[code]
Natural Gray-Scale	Example-Based Colourization Via Dense Encoding Pyramids (also palette based)	Pacific Graphics 2018	[code]
Natural Gray-Scale	A Superpixel-based Variational Model for Image Colorization	TVCG 2019
Natural Gray-Scale	Automatic Example-based Image Colourisation using Location-Aware Cross-Scale Matching	TIP 2019
Line art / Sketch	Adversarial Colorization Of Icons Based On Structure And Color Conditions	ACM MM 2019	[Code]
Line art / Sketch	Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence	CVPR 2020	[project]
Natural Gray-Scale	Stylization-Based Architecture for Fast Deep Exemplar Colorization	CVPR 2020	[code]
Manga	Manga Filling Style Conversion with Screentone Variational Autoencoder	SIGGRAPH Asia 2020	[project]
Line art / Sketch	Colorization of Line Drawings with Empty Pupils	PG 2020
Line art / Sketch	Active Colorization for Cartoon Line Drawings	TVCG 2020
Natural Gray-Scale	Gray2ColorNet: Transfer More Colors from Reference Image	ACM MM 2020
Line art / Sketch	Line Art Correlation Matching Feature Transfer Network for Automatic Animation Colorization	WACV 2021
Natural Gray-Scale	Globally and Locally Semantic Colorization via Exemplar-Based Broad-GAN	TIP 2021
Natural Gray-Scale	Yes, "Attention Is All You Need", for Exemplar based Colorization	ACM MM 2021
Natural Gray-Scale	SCSNet: An Efficient Paradigm for Learning Simultaneously Image Colorization and Super-Resolution (also automatic)	AAAI 2022
Line art / Sketch	Style-Structure Disentangled Features and Normalizing Flows for Diverse Icon Colorization	CVPR 2022
Line art / Sketch	Eliminating Gradient Conflict in Reference-based Line-Art Colorization	ECCV 2022	[code]
Natural Gray-Scale	Semantic-Sparse Colorization Network for Deep Exemplar-based Colorization	ECCV 2022
Line art / Sketch	Self-driven Dual-path Learning for Reference-based Line Art Colorization under Limited Data	TCSVT 2023
Natural Gray-Scale	Unsupervised Deep Exemplar Colorization via Pyramid Dual Non-local Attention	TIP 2023	[code]
Line art / Sketch	FlexIcon: Flexible Icon Colorization via Guided Images and Palettes (also palette based)	ACM MM 2023
Line art / Sketch	AnimeDiffusion: Anime Face Line Drawing Colorization via Diffusion Models	TVCG 2024	[code]
Natural Gray-Scale	Lightweight Deep Exemplar Colorization via Semantic Attention-Guided Laplacian Pyramid	TVCG 2024

2.3 Based on palette

Image Type	Paper	Source	Code/Project Link
Natural Image	Palette-based Photo Recoloring	SIGGRAPH 2015	[project]
Natural Gray-Scale	Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation (also text based)	ECCV 2018	[code]
Natural Gray-Scale	Example-Based Colourization Via Dense Encoding Pyramids (also reference based)	Pacific Graphics 2018	[code]
Natural Gray-Scale	PalGAN: Image Colorization with Palette Generative Adversarial Networks	ECCV 2022	[code]
Line art / Sketch	FlexIcon: Flexible Icon Colorization via Guided Images and Palettes (also reference based)	ACM MM 2023
Line art / Sketch	SketchDeco: Decorating B&W Sketches with Colour	arxiv 24.05	[code] [webpage]

2.4 Based on language or text

Image Type	Paper	Source	Code/Project Link
Natural Gray-Scale / Sketch	Language-Based Image Editing with Recurrent Attentive Models	CVPR 2018	[code]
Natural Gray-Scale	Coloring with Words: Guiding Image Colorization Through Text-based Palette Generation (also palette based)	ECCV 2018	[code]
Natural Gray-Scale	Learning to Color from Language	NAACL 2018	[code]
Line art	Tag2Pix: Line Art Colorization Using Text Tag With SECat and Changing Loss	ICCV 2019	[code] [code2 (GUI)]
Scene Sketch	Language-based Colorization of Scene Sketches	SIGGRAPH Asia 2019	[code] [project]
Line art	Line Art Colorization Based on Explicit Region Segmentation	Pacific Graphics 2021	[code]
Natural Gray-Scale	L-CoDe: Language-based Colorization using Color-object Decoupled Conditions	AAAI 2022	[code]
Natural Gray-Scale	L-CoDer: Language-based Colorization with Color-object Decoupling Transformer	ECCV 2022	[code]
Natural Gray-Scale	L-CoIns: Language-based Colorization with Instance Awareness	CVPR 2023
Natural Gray-Scale	L-CAD: Language-based Colorization with Any-level Descriptions	NeurIPS 2023	[code]
Sketch	Adding Conditional Control to Text-to-Image Diffusion Models	ICCV 2023	[code]
Natural Gray-Scale	Diffusing Colors: Image Colorization with Text Guided Diffusion	SIGGRAPH Asia 2023	[project]

2.5 Multi-modal

Image Type	Paper	Input	Source	Code/Project Link
Natural Gray-Scale	Interactive Deep Colorization Using Simultaneous Global and Local Inputs	stroke + palette	ICASSP 2019
Natural Gray-Scale	UniColor: A Unified Framework for Multi-Modal Colorization with Transformer	stroke + exemplar + text	SIGGRAPH Asia 2022	[project]
Line art	Two-Step Training: Adjustable Sketch Colourization via Reference Image and Text Tag	exemplar + text	CGF 2023	[code]
Natural Gray-Scale	Control Color: Multimodal Diffusion-based Interactive Image Colorization	text, stroke, exemplar	arxiv 24.02	[code] [project]
Natural Gray-Scale	Versatile Vision Foundation Model for Image and Video Colorization	text, scribble, exemplar	SIGGRAPH 2024

2.6 Interactive Colorization

Image Type	Paper	Source	Code/Project Link
Line art	Guiding Users to Where to Give Color Hints for Efficient Interactive Sketch Colorization via Unsupervised Region Prioritization	WACV 2023

3. Techniques of Improving Image Colorization

Paper	Source	Code/Project Link
Deep Edge-Aware Interactive Colorization against Color-Bleeding Effects	ICCV 2021	[project] [code(metric)]
Line Art Colorization Based on Explicit Region Segmentation	Pacific Graphics 2021	[code]
FlatGAN: A Holistic Approach for Robust Flat-Coloring in High-Definition with Understanding Line Discontinuity	ACM MM 2023	[code]

4. Video Colorization

4.0 Survey

Image Type	Paper	Source	Code/Project Link
	Video Colorization: A Survey	JCST 2024

4.1 Automatically

Image Type	Paper	Source	Code/Project Link
Natural Gray-Scale	Fully Automatic Video Colorization with Self-Regularization and Diversity	CVPR 2019	[code]

4.2 Based on reference

Image Type	Paper	Source	Code/Project Link
Natural Gray-Scale	Switchable Temporal Propagation Network	ECCV 2018
Natural Gray-Scale	Tracking Emerges by Colorizing Videos	ECCV 2018	[code]
Natural Gray-Scale	Deep Exemplar-based Video Colorization	CVPR 2019	[code]
Natural Gray-Scale	DeepRemaster: Temporal Source-Reference Attention Networks for Comprehensive Video Enhancement	SIGGRAPH Asia 2019	[code] [project]
Natural Gray-Scale	Reference-Based Video Colorization with Spatiotemporal Correspondence	2011.12528
Line art	The Animation Transformer: Visual Correspondence via Segment Matching	ICCV 2021	[App]
Line art	Line Art Correlation Matching Feature Transfer Network for Automatic Animation Colorization	WACV 2021
Line art	Reference-Based Deep Line Art Video Colorization	TVCG 2022
Line art	Learning Inclusion Matching for Animation Paint Bucket Colorization	CVPR 2024	[webpage] [code]
Natural Gray-Scale	BiSTNet: Semantic Image Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-based Video Colorization	TPAMI 2024	[code] [project]
Natural Gray-Scale	Exemplar-based Video Colorization with Long-term Spatiotemporal Dependency	KBS 2024
Natural Gray-Scale	ColorMNet: A Memory-based Deep Spatial-Temporal Feature Propagation Network for Video Colorization	arxiv 24.04	[code] [project]
Line art	ToonCrafter: Generative Cartoon Interpolation	arxiv 24.05	[code] [webpage]

4.3 Based on scribble

Image Type	Paper	Source	Code/Project Link
Natural Gray-Scale	SVCNet: Scribble-based Video Colorization Network with Temporal Aggregation	TIP 2023	[code]

4.4 Based on text

Image Type	Paper	Source	Code/Project Link
Natural Gray-Scale	Video Colorization with Pre-trained Text-to-Image Diffusion Models	arxiv 23.06	[code] [project]
Natural Gray-Scale	Towards Photorealistic Video Colorization via Gated Color-Guided Image Diffusion Models	ACM MM 2024