Sketch-to-Image¶
Image & 2D Art
Open Source¶
-
Component-Aware Sketch-to-Image Generation Using Self-Attention Encoding and Coordinate-Preserving Fusion Discovered by AI
2026
A component-aware, self-refining framework for sketch-to-image generation that utilizes self-attention encoding and coordinate-preserving fusion to reconstruct fine-grained details and maintain spatial alignment.
diffusionfusionself-attentionsketch -
MMFace-DiT Discovered by AI
2026
A unified dual-stream diffusion transformer for high-fidelity multimodal face generation that processes spatial conditions (such as sketches or masks) and semantic text tokens in parallel.
diffusion-transformerface-generationsketch-to-image -
SketchingReality: From Freehand Scene Sketches to Photorealistic Images Discovered by AI
2026
A sketch-to-image framework designed for human-drawn freehand scene sketches, utilizing a Stable Diffusion 2.1 backbone, a semantic sketch encoder, and a modulation network trained with attention supervision.
diffusionfreehand-sketchsketch-to-image -
d-Sketch Discovered by AI
2025
A training-free technique for sketch-to-image translation that leverages the feature generalization capabilities of pretrained latent diffusion models to improve visual fidelity under structural constraints.
diffusionsketch-to-imagetraining-free -
DreamOmni3 Discovered by AI
2025
A unified multimodal transformer-diffusion framework that supports scribble-based image editing and generation by combining user text, images, and freehand sketches.
editingmultimodalscribblesketch -
LOTS of Fashion! Multi-Conditioning for Image Generation via Sketch-Text Pairing Discovered by AI
2025
A multi-conditioning framework for fashion image generation that pairs global descriptions with localized sketches and text to enable precise control over design elements like necklines, pockets, and styles.
controlnetdiffusionfashionsketch -
Block and Detail: Scaffolding Sketch-to-Image Generation Discovered by AI
2024
A sketch-to-image tool that aligns with the iterative refinement process of artists by allowing users to sketch coarse blocking strokes for placement and form, and detail strokes to refine shapes and silhouettes. It uses a two-pass algorithm with ControlNet to generate high-fidelity images from these layered sketches.
controlnetdiffusioninteractivesketch -
ControlNet++: Improving Conditional Controls with Efficient Consistency Feedback Discovered by AI
2024
Enhances controllable image generation by optimizing pixel-level cycle consistency between generated images and conditional controls (such as sketches or scribbles), using a discriminative model to provide feedback.
conditioningcontrolnetscribblesketch -
It's All About Your Sketch: Democratising Sketch Control in Diffusion Models Discovered by AI
2024
An abstraction-aware sketch-to-image framework that enables amateur, rough sketches to guide text-to-image diffusion models effectively, resolving spatial deformities and alignment issues common in standard control adapters.
controlnetcvprdiffusionsketch -
One-Step Image Translation with Text-to-Image Models (Pix2Pix-Turbo) Discovered by AI
2024
Adapts single-step diffusion models (like SDXL Turbo) to conditional image-to-image tasks such as sketch-to-image in a single forward pass, combining adversarial learning with pre-trained text-to-image models.
diffusionone-stepreal-timesketch -
Scribble-Guided Diffusion Discovered by AI
2024
A training-free approach for text-to-image generation that utilizes simple user-provided scribbles to provide precise spatial guidance and alignment without requiring fine-tuning or paired datasets.
diffusionscribblespatial-controltraining-free -
Sketch-guided Image Inpainting with Partial Discrete Diffusion Process Discovered by AI
2024
A novel sketch-guided image inpainting method that introduces a partial discrete diffusion process (PDDP) to corrupt and reconstruct only the regions of interest, allowing precise shape and pose control.
diffusioninpaintingsketch -
Sketch-Guided Scene Image Generation Discovered by AI
2024
A framework that decomposes the task of generating complex scene images from hand-drawn sketches into object-level cross-domain generation and scene-level image construction, ensuring high fidelity and natural composition.
diffusionscene-generationsketch -
Training-Free Sketch-Guided Diffusion with Latent Optimization Discovered by AI
2024
A training-free pipeline that extends pre-trained text-to-image diffusion models to incorporate sketch conditioning. It uses latent optimization at each intermediate step to align the cross-attention maps of the generation process with the reference sketch structure.
diffusionsketchtraining-free -
U-Sketch Discovered by AI
2024
A sketch-to-image framework featuring a U-Net latent edge predictor that efficiently captures both local and global features as well as spatial correlations between pixels. It also introduces a sketch simplification network to preprocess and simplify input sketches for enhanced outputs.
controlnetdiffusionsketch -
DiffSketching Discovered by AI
2023
A sketch-guided image synthesis framework based on diffusion models that matches abstract and distorted sketches through cross-domain constraints and uses classifier guidance to steer the generation process.
classifier-guidancediffusionsketch -
flowty-realtime-lcm-canvas Discovered by AI
2023
A real-time sketch-to-image interactive canvas interface built with Gradio and powered by Latent Consistency Models (LCM) and LCM-LoRA.
canvaslcmrealtimesketch -
Inversion-by-Inversion: Exemplar-based Sketch-to-Photo Synthesis via Stochastic Differential Equations without Training Discovered by AI
2023
A training-free method for exemplar-based sketch-to-photo synthesis that leverages stochastic differential equations and diffusion inversion to transfer color and texture from a reference image to a sketch.
diffusionexemplarinversionsketch -
Krita AI Diffusion Discovered by AI
2023
An open-source plugin for the digital painting application Krita that integrates real-time sketch-to-image generation, inpainting, and outpainting using Stable Diffusion and LCM.
krita-pluginreal-timesketch-to-image -
Scribble Diffusion Discovered by AI
2023
An open-source web application that converts rough hand-drawn sketches into refined, detailed images using ControlNet and Stable Diffusion via the Replicate API.
controlnetdiffusionscribblesketch -
Sketch-a-Sketch: Controlling diffusion-based image generation with just a few strokes Discovered by AI
2023
A framework designed to control diffusion-based image generation using sparse, freehand sketches with only a few strokes, enabling intuitive and interactive visual drafting.
diffusioninteractivesketchstrokes -
T2I-Adapter Discovered by AI
2023
A lightweight adapter that aligns external control signals, such as sketches, scribbles, and depth maps, with the internal features of text-to-image diffusion models without fine-tuning the base model.
adaptercontrolnetsketch-to-image -
Sketch-Guided Text-to-Image Diffusion Models Discovered by AI
2022
Introduces a training-free, inference-time optimization method to guide pre-trained text-to-image diffusion models using an edge map or sketch, ensuring spatial alignment with the input drawing.
diffusionguidancesketchtraining-free
Proprietary / Industry Tools¶
-
Fotor Sketch to Image AI Discovered by AI
2024
A commercial web-based tool that converts hand-drawn sketches into high-quality digital artwork, supporting various styles such as realistic, 3D, anime, and oil painting.
animecommercialsketch -
OpenArt Sketch to Image Discovered by AI
2024
An online AI tool that transforms sketches, interior design concepts, or product ideas into detailed, realistic images with optional style reference guidance.
commercialrenderingsketch -
Samsung Galaxy AI Sketch to Image Discovered by AI
2024
An on-device generative AI feature integrated into Samsung Galaxy devices that transforms rough hand-drawn doodles or sketches on top of photos into detailed, photorealistic objects matching the style of the image.
drawing-assistmobileon-devicesketch -
Freepik Pikaso Discovered by AI
2023
A free, browser-based real-time sketch-to-image generator that transforms rough drawings, shapes, and icons into detailed visual artwork instantly.
browser-basedproprietaryreal-timesketch-to-image -
Krea AI (Realtime Sketch to Image) Discovered by AI
2023
A popular commercial AI design platform featuring a real-time sketch-to-image canvas that instantly renders polished concept art and product designs as the user draws.
canvasconcept-artproprietaryreal-time -
NewArc.ai Discovered by AI
2023
A commercial AI sketch-to-image generator tailored for fashion, footwear, and product designers to transform rough outlines and drawings into high-fidelity photos and renders.
fashionproduct-designrendersketch-to-image -
Stable Doodle Discovered by AI
2023
A user-friendly sketch-to-image tool by Stability AI that converts simple hand-drawn doodles into high-quality, stylized images using Stable Diffusion XL and T2I-Adapter.
doodlesdxlsketcht2i-adapter -
Vizcom Discovered by AI
2023
An AI-powered design platform widely used by industrial and concept designers to instantly render hand-drawn or digital sketches into photorealistic 3D-like product visuals.
concept-artindustrial-designrendersketch-to-image
