Home

Awesome

Awesome-Sketch-Based-Applications

A collection of sketch based applications.

Feel free to create a PR or an issue.

Outline


0. Survey

PaperSourceCode/Project Link
Deep Learning for Free-Hand Sketch: A SurveyTPAMI 2022[code]

1. Sketch Based Image Synthesis

1.1 Automatic Synthesis

PaperSourceCode/Project Link
SketchyGAN: Towards Diverse and Realistic Sketch to Image SynthesisCVPR 2018[code]
Image Generation from Sketch Constraint Using Contextual GANECCV 2018[code] [project]
Multi-Instance Sketch to Image Synthesis With Progressive Generative Adversarial NetworksIEEE Access 2019
Interactive Sketch & Fill: Multiclass Sketch-to-Image TranslationICCV 2019[code] [project]
SketchyCOCO: Image Generation from Freehand Scene SketchesCVPR 2020[code]
Unsupervised Sketch-to-Photo SynthesisECCV 2020[code] [project]
Sketch Your Own GANICCV 2021[code] [webpage]
Adversarial Open Domain Adaption for Sketch-to-Photo SynthesisWACV 2022[code]
Customizing GAN Using Few-Shot SketchesACM MM 2022
DiffSketching: Sketch Control Image Synthesis with Diffusion ModelsBMVC 2022[code]
MaskSketch: Unpaired Structure-guided Masked Image GenerationCVPR 2023[project] [code]
Picture that Sketch: Photorealistic Image Generation from Abstract SketchesCVPR 2023[project] [code]
PaperSourceCode/Project Link
LinesToFacePhoto: Face Photo Generation from Lines with Conditional Self-Attention Generative Adversarial NetworkACM MM 2019
DeepFaceDrawing: Deep Generation of Face Images from SketchesSIGGRAPH 2020[project] [code]
DeepFacePencil: Creating Face Images from Freehand SketchesACM MM 2020[project] [code]
Controllable Sketch-to-Image Translation for Robust Face SynthesisTIP 2021
DrawingInStyles: Portrait Image Generation and Editing with Spatially Conditioned StyleGANTVCG 2022
DeepPortraitDrawing: Generating Human Body Images from Freehand SketchesC&G 2023
Semantics-Preserving Sketch Embedding for Face GenerationTMM 2023[project] [code]
Parsing-Conditioned Anime Translation: A New Dataset and MethodTOG 2023[code]
Sketch2Human: Deep Human Generation with Disentangled Geometry and Appearance ConstraintsTVCG 2024
PaperSourceCode/Project Link
3D-aware Conditional Image SynthesisCVPR 2023[project] [code]

1.2 Style-conditioned

PaperSourceCode/Project Link
TextureGAN: Controlling Deep Image Synthesis with Texture PatchesCVPR 2018[code]
Multimodal Unsupervised Image-to-Image TranslationECCV 2018[code]
SketchPatch: Sketch Stylization via Seamless Patch-level SynthesisSIGGRAPH Asia 2020
Self-Supervised Sketch-to-Image SynthesisAAAI 2021[code]
CoGS: Controllable Generation and Search from Sketch and StyleECCV 2022
Adaptively-Realistic Image Generation from Stroke and Sketch with Diffusion ModelWACV 2023[project] [code]
DemoCaricature: Democratising Caricature Generation with a Rough SketchCVPR 2024[project] [code]
SMFS-GAN: Style-Guided Multi-class Freehand Sketch-to-Image SynthesisCGF 2024

1.3 Text-conditioned

PaperSourceCode/Project Link
Sketch-Guided Text-to-Image Diffusion ModelsSIGGRAPH 2023[project]
Adding Conditional Control to Text-to-Image Diffusion ModelsICCV 2023[code]
UniControl: A Unified Diffusion Model for Controllable Visual Generation In the WildNeurIPS 2023[code] [project]
Stable Doodle (ClipDrop)[demo]
CustomSketching: Sketch Concept Extraction for Sketch-based Image Synthesis and EditingPG 2024
Block and Detail: Scaffolding Sketch-to-Image GenerationUIST 2024
One-Step Image Translation with Text-to-Image Modelsarxiv 24.03[code]
It's All About Your Sketch: Democratising Sketch Control in Diffusion ModelsCVPR 2024[code]
T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion ModelsAAAI 2024[code]
Training-Free Sketch-Guided Diffusion with Latent Optimizationarxiv 24.09
Scribble-Guided Diffusion for Training-free Text-to-Image Generationarxiv 24.09[code]

2. Sketch Based Image Editing

PaperSourceCode/Project Link
Free-Form Image Inpainting with Gated Convolution (DeepFill v2)ICCV 2019[project] [code]
Fashion Editing with Multi-scale Attention Normalization1906.00884
DeFLOCNet: Deep Image Editing via Flexible Low-level ControlsCVPR 2021[code]
DeepSIM: Image Shape Manipulation from a Single Augmented Training SampleICCV 2021[code] [project]
SketchEdit: Mask-Free Local Image Manipulation with Partial SketchesCVPR 2022[code] [project]
Draw2Edit: Mask-Free Sketch-Guided Image ManipulationACM MM 2023[code]
Towards Interactive Image Inpainting via Robust Sketch RefinementTMM 2024[code] [project]
PaperSourceCode/Project Link
FaceShop: Deep Sketch-based Face Image EditingSIGGRAPH 2018[project]
CaricatureShop: Personalized and Photorealistic Caricature SketchingTVCG 2018
Sparse, Smart Contours to Represent and Edit ImagesCVPR 2018[project]
Example-Guided Style-Consistent Image Synthesis from Semantic LabelingCVPR 2019[code]
SC-FEGAN: Face Editing Generative Adversarial Network With User's Sketch and ColorICCV 2019[code]
Intuitive, Interactive Beard and Hair Synthesis with Generative ModelsCVPR 2020
Deep Plastic Surgery: Robust and Controllable Image Editing with Human-Drawn SketchesECCV 2020[code] [project]
DeepFaceEditing: Deep Face Generation and Editing with Disentangled Geometry and Appearance ControlSIGGRAPH 2021[code] [project]
SketchHairSalon: Deep Sketch-based Hair Image SynthesisSIGGRAPH Asia 2021[project]
Paint2Pix: Interactive Painting based Progressive Image Synthesis and EditingECCV 2022[code] [project]
NeRFFaceEditing: Disentangled Face Editing in Neural Radiance FieldsSIGGRAPH Asia 2022[project]
SketchFaceNeRF: Sketch-based Facial Generation and Editing in Neural Radiance FieldsSIGGRAPH 2023
PaperSourceCode/Project Link
AniFaceDrawing: Anime Portrait Exploration during Your SketchingSIGGRAPH 2023[project]
PaperSourceCode/Project Link
DeepFaceVideoEditing: Sketch-based Deep Editing of Face VideosSIGGRAPH 2022[project]

3. Sketch Based Image Retrieval (SBIR)

PaperSourceCode/Project Link
Sketch Me That ShoeCVPR 2016[code-caffe] [code-tf] [project]
Deep Multi-task Attribute-driven Ranking for Fine-grained Sketch-based Image RetrievalBMVC 2016
Deep Sketch Hashing: Fast Free-hand Sketch-Based Image RetrievalCVPR 2017[code]
Deep Spatial-Semantic Attention for Fine-Grained Sketch-Based Image RetrievalICCV 2017[project]
Zero-Shot Sketch-Image HashingCVPR 2018
SketchMate: Deep Hashing for Million-Scale Human Sketch RetrievalCVPR 2018
Generative Domain-Migration Hashing for Sketch-to-Image RetrievalECCV 2018[code]
A Zero-Shot Framework for Sketch Based Image RetrievalECCV 2018[code]
Generalising Fine-Grained Sketch-Based Image RetrievalCVPR 2019
Doodle to Search: Practical Zero-Shot Sketch-Based Image RetrievalCVPR 2019[project] [code]
LiveSketch: Query Perturbations for Guided Sketch-Based Visual SearchCVPR 2019
Semantically Tied Paired Cycle Consistency for Zero-Shot Sketch-Based Image RetrievalCVPR 2019[code]
Learning Structural Representations via Dynamic Object Landmarks Discovery for Sketch Recognition and RetrievalTIP 2019
Semantic-Aware Knowledge Preservation for Zero-Shot Sketch-Based Image RetrievalICCV 2019
TC-Net for iSBIR: Triplet Classification Network for Instance-level Sketch Based Image RetrievalACM MM 2019
Sketch-Based Image Retrieval With Multi-Clustering Re-RankingTCSVT 2019
Semi-Heterogeneous Three-Way Joint Embedding Network for Sketch-Based Image RetrievalTCSVT 2019
Zero-Shot Sketch-Based Image Retrieval via Graph Convolution NetworkAAAI 2020
Sketch Less for More: On-the-Fly Fine-Grained Sketch Based Image RetrievalCVPR 2020[code]
Solving Mixed-modal Jigsaw Puzzle for Fine-Grained Sketch-Based Image RetrievalCVPR 2020
Fine-Grained Instance-Level Sketch-Based Image RetrievalIJCV 2020
StyleMeUp: Towards Style-Agnostic Sketch-Based Image RetrievalCVPR 2021
More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch-Based Image RetrievalCVPR 2021[code]
DLA-Net for FG-SBIR: Dynamic Local Aligned Network for Fine-Grained Sketch-Based Image RetrievalACM MM 2021
Domain-Aware SE Network for Sketch-based Image Retrieval with Multiplicative Euclidean Margin SoftmaxACM MM 2021[code]
Relationship-Preserving Knowledge Distillation for Zero-Shot Sketch Based Image RetrievalACM MM 2021
Transferable Coupled Network for Zero-Shot Sketch-Based Image RetrievalTPAMI 2021[project]
TVT: Three-Way Vision Transformer through Multi-Modal Hypersphere Learning for Zero-Shot Sketch-Based Image RetrievalAAAI 2022
Sketching without Worrying: Noise-Tolerant Sketch-Based Image RetrievalCVPR 2022[code]
Sketch3T: Test-time Training for Zero-Shot SBIRCVPR 2022
Augmented Multi-Modality Fusion for Generalized Zero-Shot Sketch-based Visual RetrievalTIP 2022[code]
Adaptive Fine-Grained Sketch-Based Image RetrievalECCV 2022[code]
Conditional Stroke Recovery for Fine-Grained Sketch-Based Image RetrievalECCV 2022[code]
A Sketch Is Worth a Thousand Words: Image Retrieval with Text and SketchECCV 2022[code] [project]
Multi-Level Region Matching for Fine-Grained Sketch-Based Image RetrievalACM MM 2022[code]
Prototype-based Selective Knowledge Distillation for Zero-Shot Sketch Based Image RetrievalACM MM 2022
DLI-Net: Dual Local Interaction Network for Fine-Grained Sketch-Based Image RetrievalTCSVT 2022[code]
Data-Free Sketch-Based Image RetrievalCVPR 2023[code]
Zero-Shot Everything Sketch-Based Image Retrieval, and in Explainable StyleCVPR 2023[code]
Exploiting Unlabelled Photos for Stronger Fine-Grained SBIRCVPR 2023[project]
CLIP for All Things Zero-Shot Sketch-Based Image Retrieval, Fine-Grained or NotCVPR 2023[project]
Cross-Domain Alignment for Zero-Shot Sketch-Based Image RetrievalTCSVT 2023
Semi-transductive Learning for Generalized Zero-Shot Sketch-Based Image RetrievalAAAI 2023
Text-to-Image Diffusion Models are Great Sketch-Photo MatchmakersCVPR 2024[project]
You'll Never Walk Alone: A Sketch and Text Duet for Fine-Grained Image RetrievalCVPR 2024[project]
How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval?CVPR 2024[project]
Asymmetric Mutual Alignment for Unsupervised Zero-Shot Sketch-Based Image RetrievalAAAI 2024
Composite Sketch+Text Queries for Retrieving Objects with Elusive Names and Complex InteractionsAAAI 2024[project]
Freeview Sketching: View-Aware Fine-Grained Sketch-Based Image RetrievalECCV 2024
Elevating All Zero-Shot Sketch-Based Image Retrieval Through Multimodal Prompt LearningECCV 2024[code]
Zero-Shot Sketch Based Image Retrieval via Modality Capacity GuidanceIJCAI-24[code]
PaperSourceCode/Project Link
SceneSketcher: Fine-Grained Image Retrieval with Scene SketchesECCV 2020
Partially Does It: Towards Scene-Level FG-SBIR with Partial InputCVPR 2022
SceneSketcher-v2: Fine-Grained Scene-Level Sketch-Based Image Retrieval using Adaptive GCNsTIP 2022
FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in ContextECCV 2022[code] [Dataset]
Scene-Level Sketch-Based Image Retrieval with Minimal Pairwise SupervisionAAAI 2023
FreestyleRet: Retrieving Images from Style-Diversified QueriesECCV 2024[code]
PaperSourceCode/Project Link
Fine-Grained Instance-Level Sketch-Based Video RetrievalTCSVT 2020
Fine-Grained Video Retrieval with Scene SketchesTIP 2023[project]

4. Sketch Based 3D Shape Retrieval

PaperSourceCode/Project Link
Sketch-based 3D Shape Retrieval using Convolutional Neural NetworksCVPR 2015
Learning Cross-Domain Neural Networks for Sketch-Based 3D Shape RetrievalAAAI 2016
Deep Correlated Metric Learning for Sketch-based 3D Shape RetrievalAAAI 2017
Learning Barycentric Representations of 3D Shapes for Sketch-based 3D Shape RetrievalCVPR 2017
Deep Cross-modality Adaptation via Semantics Preserving Adversarial Learning for Sketch-based 3D Shape RetrievalECCV 2018
Unsupervised Learning of 3D Model Reconstruction from Hand-Drawn SketchesACMMM 2018
Towards 3D VR-Sketch to 3D Shape Retrieval3DV 2020[code] [project]
Uncertainty Learning for Noise Resistant Sketch-Based 3D Shape RetrievalTIP 2021
Towards Fine-Grained Sketch-Based 3D Shape RetrievalTIP 2021
Domain Disentangled Generative Adversarial Network for Zero-Shot Sketch-Based 3D Shape RetrievalAAAI 2022
Retrieval-Specific View Learning for Sketch-to-Shape RetrievalTMM 2023
Doodle to Object: Practical Zero-Shot Sketch-Based 3D Shape RetrievalAAAI 2023[code]
Democratising 2D Sketch to 3D Shape Retrieval Through PivotingICCV 2023

5. Sketch Based 3D Shape Modeling

PaperSourceCode/Project Link
3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks3DV 2017[code] [project]
Shape Synthesis from Sketches via Procedural Models and Convolutional NetworksTVCG 2017
DeepSketch2Face: A Deep Learning Based Sketching System for 3D Face and Caricature ModelingSIGGRAPH 2017[project] [code]
BendSketch: Modeling Freeform Surfaces Through 2D SketchingSIGGRAPH 2017[project]
Robust Flow-Guided Neural Prediction for Sketch-Based Freeform Surface ModelingSIGGRAPH Asia 2018[code] [project]
DeepSketchHair: Deep Sketch-based 3D Hair ModelingTVCG 2019
Interactive Liquid Splash Modeling by User SketchesSIGGRAPH Asia 2020
Monster Mash: A Single-View Approach to Casual 3D Modeling and AnimationSIGGRAPH Asia 2020[project] [code] [demo]
SAniHead: Sketching Animal-like 3D Character Heads Using a View-surface Collaborative Mesh Generative NetworkTVCG 2020
Towards Practical Sketch-Based 3D Shape Generation: The Role of Professional SketchesTCSVT 2020
Sketch2Model: View-Aware 3D Modeling from Single Free-Hand SketchesCVPR 2021
Sketch2Mesh: Reconstructing and Editing 3D Shapes from SketchesICCV 2021
Real-time Skeletonization for Sketch-based ModelingSMI 2021[code]
Sketch2Pose: Estimating a 3D Character Pose from a Bitmap SketchSIGGRAPH 2022[project] [code]
Sketch2PQ: Freeform Planar Quadrilateral Mesh Design via a Single SketchTVCG 2022
Structure-aware Editable Morphable Model for 3D Facial Detail Animation and ManipulationECCV 2022[code]
SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth SamplingECCV 2022[code]
Deep Reconstruction of 3D Smoke Densities from Artist SketchesEG 2022[project] [code]
A Diffusion-ReFinement Model for Sketch-to-Point ModelingACCV 2022[code]
RaBit: Parametric Modeling of 3D Biped Cartoon Characters with a Topological-consistent DatasetCVPR 2023[project]
SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face ModelingTVCG 2023[project] [code]
Reality3DSketch: Rapid 3D Modeling of Objects from Single Freehand SketchesTMM 2023
GA-Sketching: Shape Modeling from Multi-View Sketching with Geometry-Aligned Deep Implicit FunctionsPG 2023[code]
Doodle Your 3D: From Abstract Freehand Sketches to Precise 3D ShapesCVPR 2024[project] [code]
SENS: Part-Aware Sketch-based Implicit Neural Shape ModelingEG 2024[code] [project]
3D Reconstruction from a Single Sketch via View-dependent Depth SamplingTPAMI 2024[code]
S2TD-Face: Reconstruct a Detailed 3D Face with Controllable Texture from a Single SketchACM MM 2024[code]
Sketch2Vox: Learning 3D Reconstruction from a Single Monocular SketchECCV 2024
PaperSourceCode/Project Link
Sketch2Scene: Automatic Generation of Interactive 3D Game Scenes from User's Casual Sketchesarxiv 24.08[webpage]
GroundUp: Rapid Sketch-Based 3D City MassingECCV 2024[code]
External Knowledge Enhanced 3D Scene Generation from SketchECCV 2024
PaperSourceCode/Project Link
Sketch2NeRF: Multi-view Sketch-guided Text-to-3D Generationarxiv 24.01
SKED: Sketch-guided Text-based 3D EditingICCV 2023[project] [code]
SketchDream: Sketch-based Text-to-3D Generation and EditingSIGGRAPH 2024[project]
PaperSourceCode/Project Link
Lifting Freehand Concept Sketches into 3DSIGGRAPH Asia 2020[project] [code]
Sketch2CAD: Sequential CAD Modeling by Sketching in ContextSIGGRAPH Asia 2020[project] [code]
Free2CAD: Parsing Freehand Drawings into CAD CommandsSIGGRAPH 2022[project] [code]
Symmetry-driven 3D Reconstruction From Concept SketchesSIGGRAPH 2022[project]
Piecewise-smooth Surface Fitting Onto Unstructured 3D SketchesSIGGRAPH 2022[project]
Reconstruction of Machine-Made Shapes from Bitmap SketchesSIGGRAPH Asia 2023[project]
CAD-SIGNet: CAD Language Inference from Point Clouds using Layer-wise Sketch Instance Guided AttentionCVPR 2024
SfmCAD: Unsupervised CAD Reconstruction by Learning Sketch-based Feature Modeling OperationsCVPR 2024[code]

6. Sketch Based Garment Design

PaperSourceCode/Project Link
Sketching Folds: Developable Surfaces from Non-Planar SilhouettesTOG 2015[project]
FoldSketch: Enriching Garments with Physically Reproducible FoldsSIGGRAPH 2018[project]
Learning a Shared Shape Space for Multimodal Garment DesignSIGGRAPH Asia 2018[project]
Garment Ideation: Iterative view-aware sketch-based garment modeling3DV 2022[code]
Controllable Visual-Tactile SynthesisICCV 2023[project] [code]
Controllable Garment Image Synthesis Integrated with Frequency Domain FeaturesPG 2023
Toward Intelligent Interactive Design: A Generation Framework Based on Cross-domain Fashion ElementsACM MM 2023
FashionDiff: A Controllable Diffusion Model Using Pairwise Fashion Elements for Intelligent DesignACM MM 2023
TexControl: Sketch-Based Two-Stage Fashion Image Generation Using Diffusion ModelNICOGRAPH 2024

7. Sketch Completion

PaperSourceCode/Project Link
SketchGAN: Joint Sketch Completion and Recognition with Generative Adversarial NetworkCVPR 2019
SketchHealer: A Graph-to-Sequence Network for Recreating Partial Human SketchesBMVC 2020[code]
Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch GestaltCVPR 2020[code]
Generative Sketch HealingIJCV 2022
PaperSourceCode/Project Link
Joint Gap Detection and Inpainting of Line DrawingsCVPR 2017[project] [code]
Detecting Viewer-Perceived Intended Vector Sketch ConnectivitySIGGRAPH 2022[project]

8. Sketch Restoration, Retargeting and Inpainting

PaperSourceCode/Project Link
Exploiting Aliasing for Manga RestorationCVPR 2021[webpage] [code]
Seamless Manga Inpainting with Semantics AwarenessSIGGRAPH 2021[webpage] [code]
Screentone-Preserved Manga Retargetingarxiv 22.03
Manga Rescreening with Interpretable Screentone Representationarxiv 23.06
PaperSourceCode/Project Link
Restoration of Hand-Drawn Architectural Drawings using Latent Space Mapping with Degradation GeneratorCVPR 2023

9. Sketch and Shading

PaperSourceCode/Project Link
Learning to Shadow Hand-drawn SketchesCVPR 2020[project] [code]
SmartShadow: Artistic Shadow Drawing Tool for Line DrawingsICCV 2021[project]

10. Sketch Enhancement / Beautification

PaperSourceCode/Project Link
SketchMan: Learning to Create Professional SketchesACM MM 2020[code]
PaperSourceCode/Project Link
Sketch Beautification: Learning Part Beautification and Structure Refinement for Sketches of Man-made ObjectsTVCG 2024
PICASSO: A Feed-Forward Framework for Parametric Inference of CAD Sketches via Rendering Self-Supervisionarxiv 24.07
PaperSourceCode/Project Link
APISR: Anime Production Inspired Real-World Anime Super-ResolutionCVPR 2024[code]

11. Sketch-Guided Object Localization

PaperSourceCode/Project Link
Sketch-Guided Object Localization in Natural ImagesECCV 2020[project] [code]
Localizing Infinity-shaped fishes: Sketch-guided object localization in the wildarxiv 21.09[code]
What Can Human Sketches Do for Object Detection?CVPR 2023[project]
Sketch-based Video Object Segmentation: Benchmark and AnalysisBMVC 2023[code]
Query-guided Attention in Vision Transformers for Localizing Objects Using a Single SketchWACV 2024[project] [code]

12. Sketch-Guided Video Synthesis

PaperSourceCode/Project Link
Deep Sketch-guided Cartoon Video InbetweeningTVCG 2021[code]
ToonCrafter: Generative Cartoon InterpolationSIGGRAPH Asia 2024[code] [webpage]
LVCD: Reference-based Lineart Video Colorization with Diffusion ModelsSIGGRAPH Asia 2024[webpage]

13. Sketch Recognition

PaperSourceCode/Project Link
Sketch-a-Net that Beats HumansBMVC 2015[code]
Sketch-a-Net: A Deep Neural Network that Beats HumansIJCV 2017[code]
Deep Self-Supervised Representation Learning for Free-Hand SketchTCSVT 2020[code]
S3Net:Graph Representational Network For Sketch RecognitionICME 2020[code]
Sketchformer: Transformer-based Representation for Sketched StructureCVPR 2020[code]
Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch GestaltCVPR 2020[code]
Multi-Graph Transformer for Free-Hand Sketch RecognitionTNNLS 2021[code]
Sketch-R2CNN: An RNN-Rasterization-CNN Architecture for Vector Sketch RecognitionTVCG 2021[code]
Vectorization and Rasterization: Self-Supervised Learning for Sketch and HandwritingCVPR 2021[code]
SketchAA: Abstract Representation for Abstract SketchesICCV 2021
SketchLattice: Latticed Representation for Sketch ManipulationICCV 2021[code]
Multi-Stage Spatio-Temporal Networks for Robust Sketch RecognitionTIP 2022

14. Sketch Segmentation and Perceptual Grouping

<table> <tr> <td><strong>Type</strong></td> <td><strong>Paper</strong></td> <td><strong>Source</strong></td> <td><strong>Code/Project Link</strong></td> </tr> <tr> <td rowspan=4"><strong>Pixelwise</strong></td> <td> <a href="https://ieeexplore.ieee.org/abstract/document/8784880">SPFusionNet: Sketch Segmentation Using Multi-modal Data Fusion</a> </td> <td> ICME 2019 </td> <td> </td> </tr> <tr> <td> <a href="http://openaccess.thecvf.com/content_ECCV_2018/papers/Changqing_Zou_SketchyScene_Richly-Annotated_Scene_ECCV_2018_paper.pdf">SketchyScene: Richly-Annotated Scene Sketches</a> (scene-level) </td> <td> ECCV 2018 </td> <td> <a href="https://github.com/SketchyScene/SketchyScene">[code]</a> </td> </tr> <tr> <td> <a href="https://ieeexplore.ieee.org/abstract/document/9686584">Exploring Local Detail Perception for Scene Sketch Semantic Segmentation</a> (scene-level) </td> <td> TIP 2022 </td> <td> </td> </tr> <tr> <td> <a href="https://arxiv.org/abs/2312.12463">Open Vocabulary Semantic Scene Sketch Understanding</a> (scene-level) </td> <td> CVPR 2024 </td> <td> <a href="https://github.com/AhmedBourouis/Scene-Sketch-Segmentation">[code]</a> <a href="https://ahmedbourouis.github.io/Scene_Sketch_Segmentation/">[project]</a> </td> </tr> <tr> <td rowspan="11"><strong>Stroke-level</strong></td> <td> <a href="https://www.microsoft.com/en-us/research/wp-content/uploads/2012/01/16-2012-eccv-sketch-segmenation.pdf">Free Hand-Drawn Sketch Segmentation</a> </td> <td> ECCV 2012 </td> <td> </td> </tr> <tr> <td> <a href="https://scholars.cityu.edu.hk/files/15839162/9_5915645_TOG_Data_driven_postprint.pdf">Data-driven Segmentation and Labeling of Freehand Sketches</a> </td> <td> SIGGRAPH Asia 2014 </td> <td> <a href="http://sweb.cityu.edu.hk/hongbofu/projects/SketchSegmentationLabeling_SA14/src_global_interpretation.zip">[code]</a> <a href="http://sweb.cityu.edu.hk/hongbofu/projects/SketchSegmentationLabeling_SA14">[project]</a> <a href="http://sweb.cityu.edu.hk/hongbofu/projects/SketchSegmentationLabeling_SA14/Sketch_dataset.zip">[dataset]</a> </td> </tr> <tr> <td> <a href="http://homes.esat.kuleuven.be/~konijn/publications/2016/a151-schneider.pdf">Example-Based Sketch Segmentation and Labeling Using CRFs</a> </td> <td> TOG 2016 </td> <td> </td> </tr> <tr> <td> <a href="https://ieeexplore.ieee.org/abstract/document/8766108">SketchSegNet+: An End-to-End Learning of RNN for Multi-Class Sketch Semantic Segmentation</a> </td> <td> IEEE Access 2019 </td> <td> </td> </tr> <tr> <td> <a href="http://sweb.cityu.edu.hk/hongbofu/doc/fast_sketch_segmentation_CGA2019.pdf">Fast Sketch Segmentation and Labeling With Deep Learning</a> </td> <td> CGA 2019 </td> <td> </td> </tr> <tr> <td> <a href="https://arxiv.org/pdf/1901.03427.pdf">Stroke-based sketched symbol reconstruction and segmentation</a> </td> <td> CGA 2020 </td> <td> </td> </tr> <tr> <td> <a href="http://sweb.cityu.edu.hk/hongbofu/doc/SketchGNN_TOG21.pdf">SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks</a> </td> <td> TOG 2021 </td> <td> <a href="https://github.com/sYeaLumin/SketchGNN">[code]</a> </td> </tr> <tr> <td> <a href="https://arxiv.org/abs/2112.10838">One Sketch for All: One-Shot Personalized Sketch Segmentation</a> </td> <td> TIP 2022 </td> <td> </td> </tr> <tr> <td> <a href="https://ieeexplore.ieee.org/abstract/document/10214525">Sketch-Segformer: Transformer-Based Segmentation for Figurative and Creative Sketches</a> </td> <td> TIP 2023 </td> <td> <a href="https://github.com/PRIS-CV/Sketch-SF">[code]</a> </td> </tr> <tr> <td> <a href="https://ieeexplore.ieee.org/abstract/document/10471272">CreativeSeg: Semantic Segmentation of Creative Sketches</a> </td> <td> TIP 2024 </td> <td> <a href="https://github.com/PRIS-CV/Sketch-CS">[code]</a> </td> </tr> <tr> <td> <a href="https://openaccess.thecvf.com/content/CVPR2024/papers/Wang_ContextSeg_Sketch_Semantic_Segmentation_by_Querying_the_Context_with_Attention_CVPR_2024_paper.pdf">ContextSeg: Sketch Semantic Segmentation by Querying the Context with Attention </a> </td> <td> CVPR 2024 </td> <td> <a href="https://enigma-li.github.io/projects/contextSeg/contextSeg.html">[webpage]</a> </td> </tr> </table> <table> <tr> <td><strong>Type</strong></td> <td><strong>Paper</strong></td> <td><strong>Source</strong></td> <td><strong>Code/Project Link</strong></td> </tr> <tr> <td rowspan="3"><strong>Stroke-level</strong></td> <td> <a href="https://ieeexplore.ieee.org/abstract/document/6738056">Sketching by perceptual grouping</a> </td> <td> ICIP 2013 </td> <td> </td> </tr> <tr> <td> <a href="https://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Qi_Making_Better_Use_2015_CVPR_paper.pdf">Making Better Use of Edges via Perceptual Grouping</a> </td> <td> CVPR 2015 </td> <td> </td> </tr> <tr> <td> <a href="http://openaccess.thecvf.com/content_ECCV_2018/papers/Ke_LI_Universal_Sketch_Perceptual_ECCV_2018_paper.pdf">Universal Sketch Perceptual Grouping</a> / <a href="https://ieeexplore.ieee.org/abstract/document/8626530">Toward Deep Universal Sketch Perceptual Grouper</a> </td> <td> ECCV 2018 / TIP 2019 </td> <td> <a href="https://github.com/KeLi-SketchX/Universal-sketch-perceptual-grouping">[code]</a> </td> </tr> </table> <table> <tr> <td><strong>Type</strong></td> <td><strong>Paper</strong></td> <td><strong>Source</strong></td> <td><strong>Code/Project Link</strong></td> </tr> <tr> <td rowspan="6"><strong>Vector</strong></td> <td> <a href="https://proceedings.neurips.cc/paper/2021/hash/cdf1035c34ec380218a8cc9a43d438f9-Abstract.html">Recognizing Vector Graphics without Rasterization</a> </td> <td> NeurIPS 2021 </td> <td> <a href="https://github.com/microsoft/YOLaT-VectorGraphicsRecognition">[code]</a> </td> </tr> <tr> <td> <a href="https://openaccess.thecvf.com/content/ICCV2021/papers/Fan_FloorPlanCAD_A_Large-Scale_CAD_Drawing_Dataset_for_Panoptic_Symbol_Spotting_ICCV_2021_paper.pdf">FloorPlanCAD: A Large-Scale CAD Drawing Dataset for Panoptic Symbol Spotting</a> </td> <td> ICCV 2021 </td> <td> <a href="https://floorplancad.github.io/">[project]</a> </td> </tr> <tr> <td> <a href="https://arxiv.org/abs/2201.00625">GAT-CADNet: Graph Attention Network for Panoptic Symbol Spotting in CAD Drawings</a> </td> <td> CVPR 2022 </td> <td> </td> </tr> <tr> <td> <a href="https://openaccess.thecvf.com/content/CVPR2022/papers/Fan_CADTransformer_Panoptic_Symbol_Spotting_Transformer_for_CAD_Drawings_CVPR_2022_paper.pdf">CADTransformer: Panoptic Symbol Spotting Transformer for CAD Drawings</a> </td> <td> CVPR 2022 </td> <td> <a href="https://github.com/VITA-Group/CADTransformer">[code]</a> </td> </tr> <tr> <td> <a href="https://openaccess.thecvf.com/content/CVPR2023/papers/Yang_VectorFloorSeg_Two-Stream_Graph_Attention_Network_for_Vectorized_Roughcast_Floorplan_Segmentation_CVPR_2023_paper.pdf">VectorFloorSeg: Two-Stream Graph Attention Network for Vectorized Roughcast Floorplan Segmentation</a> </td> <td> CVPR 2023 </td> <td> <a href="https://github.com/DrZiji/VecFloorSeg">[code]</a> </td> </tr> <tr> <td> <a href="https://ieeexplore.ieee.org/abstract/document/10508965">Hierarchical Recognizing Vector Graphics and A New Chart-based Vector Graphics Dataset</a> </td> <td> TPAMI 2024 </td> <td> <a href="https://github.com/microsoft/YOLaT-VectorGraphicsRecognition">[code]</a> </td> </tr> </table>

15. Sketch Representation Learning

PaperSourceCode/Project LinkData MannerRelated Applications
Learning Deep Sketch AbstractionCVPR 2018Vector/stroke-levelFG-SBIR
Goal-Driven Sequential Data AbstractionICCV 2019Vector/stroke-levelSketch recognition
Pixelor: a competitive sketching AI agent. So you think you can sketch?SIGGRAPH Asia 2020[Project] [Code]Vector/stroke-levelSketch synthesis and recognition
SketchAA: Abstract Representation for Abstract SketchesICCV 2021Vector/stroke-levelSketch recognition, FB-SBIR, sketch healing
CLIPasso: Semantically-Aware Object SketchingSIGGRAPH 2022[project] [code]Vector/stroke-level
Abstracting Sketches through Simple PrimitivesECCV 2022[code]Vector/stroke-levelSketch recognition, FG-SBIR
Learning Geometry-aware Representations by SketchingCVPR 2023[code]Vector/stroke-levelobject attribute classification, domain transfer, stroke-based generation, FG-SBIR
SketchXAI: A First Look at Explainability for Human SketchesCVPR 2023[project]Vector/stroke-levelsketch recognition
Prediction with Visual Evidence: Sketch Classification Explanation via Stroke-Level AttributionsTIP 2023Vector/stroke-level
What Sketch Explainability Really Means for Downstream TasksCVPR 2024Vector/stroke-level
Do Generalised Classifiers really work on Human Drawn Sketches?ECCV 2024Pixel-level
PaperSourceCode/Project LinkData MannerRelated Applications
A Neural Representation of Sketch Drawings (Sketch-RNN)ICLR 2018[code]Vector/stroke-levelReconstruction and interpolation
SketchHealer: A Graph-to-Sequence Network for Recreating Partial Human SketchesBMVC 2020[code]Vector/stroke-levelSketch recognition, retrieval, completion and analogy
Sketchformer: Transformer-based Representation for Sketched StructureCVPR 2020[code]Vector/stroke-levelSketch classification, SBIR, reconstruction and interpolation
CoSE: Compositional Stroke EmbeddingsNeurIPS 2020[Code]Vector/stroke-levelAuto-completing diagrams
SketchLattice: Latticed Representation for Sketch ManipulationICCV 2021[code]Lattice graphSketch healing and image-to-sketch synthesis
SketchODE: Learning neural sketch representation in continuous timeICLR 2022[Project]Vector/Stroke-levelContinuous-time representation, reconstruction & interpolation
Linking Sketch Patches by Learning Synonymous Proximity for Graphic Sketch RepresentationAAAI 2023[code]Vector/Stroke-levelSketch synthesis and sketch healing
SketchKnitter: Vectorized Sketch Generation with Diffusion ModelsICLR 2023[code]Vector/Stroke-levelvectorized sketch generation
ChiroDiff: Modelling chirographic data with Diffusion ModelsICLR 2023[Project]Vector/Stroke-levelvectorization, de-noising/healing, abstraction
Enhance Sketch Recognition's Explainability via Semantic Component-Level ParsingAAAI 2024[code]Vector/Stroke-levelrecognition and segmentation
Modelling complex vector drawings with stroke-cloudsICLR 2024[code]Vector/Stroke-levelreconstruction and generation
Sketchformer++: A Hierarchical Transformer Architecture for Vector Sketch RepresentationCVM 2024Vector/Stroke-levelreconstruction, recognition and semantic segmentation
PaperSourceCode/Project LinkData MannerRelated Applications
A Learned Representation for Scalable Vector GraphicsICCV 2019[code]SVGFont design
DeepSVG: A Hierarchical Generative Network for Vector Graphics AnimationNeurIPS 2020[Code] [Project]SVGVector Graphics Animation, reconstruction and interpolation
SVGformer: Representation Learning for Continuous Vector Graphics using TransformersCVPR 2023SVGReconstruction, classification, interpolation, retrieval
StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesisarxiv 24.01SVGgeneration
PaperSourceCode/Project LinkData MannerRelated Applications
Sketch Fewer to Recognize More by Learning a Co-Regularized Sparse RepresentationTCSVT 2019few-shot classification
On Learning Semantic Representations for Large-Scale Abstract SketchesTCSVT 2020[Code]Bitmap and VectorRetrieval and recognition
Deep Self-Supervised Representation Learning for Free-Hand SketchTCSVT 2020[Code]Vector/stroke-levelRetrieval and recognition
Sketch-BERT: Learning Sketch Bidirectional Encoder Representation from Transformers by Self-supervised Learning of Sketch GestaltCVPR 2020[Code]Vector/stroke-levelSketch recognition, retrieval, and gestalt
Vectorization and Rasterization: Self-Supervised Learning for Sketch and HandwritingCVPR 2021[Code]Both Vector and Raster-levelRecognition
Self-Supervised Learning for Enhancing Spatial Awareness in Free-Hand SketchesIJCAI 2024[Code]Vector-levelSynthesis
PaperSourceCode/Project Link
SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and TextCVPR 2023
PaperSourceCode/Project Link
SketchINR: A First Look into Sketches as Implicit Neural RepresentationsCVPR 2024

16. Sketch and Visual Correspondence

NamePaperSourceCode/Project Link
CreativeFlow+Creative Flow+ DatasetCVPR 2019[code]
ATD-12KDeep animation video interpolation in the wildCVPR 2021[code]
AnimeRunAnimeRun: 2D Animation Correspondence from Open Source 3D MoviesNeurIPS 2022[code]
PaintBucket-CharacterLearning Inclusion Matching for Animation Paint Bucket ColorizationCVPR 2024[code]
Matching TypePaperSourceCode/Project Link
RegionGlobally optimal toon trackingSIGGRAPH 2016[project]
RegionThe Animation Transformer: Visual Correspondence via Segment MatchingICCV 2021
RegionLearning Inclusion Matching for Animation Paint Bucket ColorizationCVPR 2024[code] [project]
PointSketchDesc: Learning Local Sketch Descriptors for Multi-view CorrespondenceTCSVT 2020
PointSketchZooms: Deep Multi-view Descriptors for Matching Line DrawingsCGF 2021[project]
StrokeDifferential Operators on Sketches via Alpha ContoursSIGGRAPH 2023[code]

17. Sketch Animation/Inbetweening

Please refer to repository MarkMoHR/Awesome-2D-Animation for details.

18. Sketch and AR/VR

PaperSourceCode/Project Link
SweepCanvas: Sketch-based 3D Prototyping on an RGB-D ImageUIST 2017[video]
Model-Guided 3D SketchingTVCG 2018[video]
Mobi3DSketch: 3D Sketching in Mobile ARCHI 2019[video]
Interactive Liquid Splash Modeling by User SketchesSIGGRAPH Asia 2020[video]
Towards 3D VR-Sketch to 3D Shape Retrieval3DV 2020[code] [project]
3D Curve Creation on and around Physical Objects with Mobile ARTVCG 2021[video]
HandPainter - 3D Sketching in VR with Hand-based Physical ProxyCHI 2021[video]
Fine-Grained VR Sketching: Dataset and Insights3DV 2021[code]
Structure-Aware 3D VR Sketch to 3D Shape Retrieval3DV 2022[code]
GestureSurface: VR Sketching through Assembling Scaffold Surface with Non-Dominant HandTVCG 2023
3D VR Sketch Guided 3D Shape Prototyping and ExplorationICCV 2023[code]

19. Sketch Based Incremental Learning

PaperSourceCode/Project Link
Doodle It Yourself: Class Incremental Learning by Drawing a Few SketchesCVPR 2022[code]

20. Sketch Quality Measurement

PaperSourceCode/Project Link
Annotation-Free Human Sketch Quality AssessmentIJCV 2024[code]
Finding Badly Drawn BunniesCVPR 2022[code]

21. Cloud Augmentation with Sketches

PaperSourceCode/Project Link
Cloud2Sketch: Augmenting Clouds with Imaginary SketchesACM MM 2022[project]

22. Sketch and Re-identification

PaperSourceCode/Project Link
Cross-Compatible Embedding and Semantic Consistent Feature Construction for Sketch Re-identificationACM MM 2022[code]
SketchTrans: Disentangled Prototype Learning with Transformer for Sketch-Photo RecognitionTPAMI 2023[code]
Differentiable Auxiliary Learning for Sketch Re-IdentificationAAAI 2024

23. Sketch-based Salient Object Detection

PaperSourceCode/Project Link
Sketch2Saliency: Learning to Detect Salient Objects from Human DrawingsCVPR 2023

24. Study on Strokes

PaperSourceCode/Project Link
Neural Brushstroke Engine: Learning a Latent Style Space of Interactive Drawing ToolsSIGGRAPH Asia 2022[project]
Ciallo: GPU-Accelerated Rendering of Vector Brush StrokesSIGGRAPH 2024[code]