ControlNet-style Conditioning¶
Image & 2D Art
Open Source¶
-
Condition Preference Optimization Discovered by AI
2025
A preference learning method for controllable image generation that directly contrasts input control conditions instead of raw images, eliminating confounding factors and significantly improving controllability across multiple control types.
controllabilitycontrolnetdiffusionpreference-optimization -
DC-ControlNet Discovered by AI
2025
A highly flexible and precisely controllable framework for multi-condition image generation that decouples global control conditions into a hierarchical system of distinct elements, contents, and layouts.
conditioningcontrolnetdiffusionmulti-condition -
EasyControl Discovered by AI
2025
A highly efficient and flexible framework for adding spatial control (like Canny, Depth, Pose) to Diffusion Transformers (DiTs) such as FLUX.1. It introduces Condition Injection LoRA (CI-LoRA) to reduce parameter overhead and training costs while maintaining high-quality control.
conditioningcontrolnetditfluxProject · GitHub · arXiv · Hugging Face
-
FlexControl: Computation-Aware ControlNet with Differentiable Router for Text-to-Image Generation Discovered by AI
2025
A computation-aware ControlNet framework that dynamically selects which diffusion blocks to activate at each denoising step using a trainable gating mechanism, optimizing both control quality and computational efficiency.
controlnetdiffusionefficiency -
Heeding the Inner Voice: Aligning ControlNet Training via Intermediate Features Feedback Discovered by AI
2025
A training strategy that enforces spatial consistency throughout intermediate generation stages of ControlNet, improving alignment between the generated images and conditional controls.
alignmentcontrolnetdiffusion -
Minimal Impact ControlNet Discovered by AI
2025
A novel framework designed to refine the integration of multiple control signals within diffusion models by mitigating conflicts between different ControlNets. It addresses the issue where silent or smooth control signals suppress texture generation in unrelated areas, improving the fidelity and quality of multi-conditioned image generation.
conditioningcontrolnetdiffusionmulti-control -
NanoControl: A Lightweight Framework for Precise and Efficient Control in Diffusion Transformer Discovered by AI
2025
A lightweight framework designed for precise and efficient spatial conditioning in Diffusion Transformers (DiTs) like Flux, significantly reducing parameter overhead and computational costs compared to traditional ControlNet architectures.
controlnetditfluxlightweight -
RelaCtrl: Relevance-Guided Efficient Control for Diffusion Transformers Discovered by AI
2025
An efficient controllable generation framework designed specifically for Diffusion Transformers (DiTs). It introduces relevance-guided control to dynamically allocate computational resources across different transformer layers based on the importance of the control information, significantly reducing parameter and computational overhead.
controlnetditefficient -
SemanticControl Discovered by AI
2025
A training-free approach for handling loosely aligned visual conditions in ControlNet, enabling robust image generation even when spatial conditions do not perfectly match the text prompt.
conditioningcontrolnettraining-free -
ComfyUI-Anima-LLLite Discovered by AI
2024
ComfyUI-Anima-LLLite is a custom node implementation of ControlNet-LLLite tailored for the DiT-based Anima model. It allows users to apply lightweight, low-rank spatial conditioning to Diffusion Transformer architectures at inference time.
conditioningcontrolnetditlightweight -
ControlNet++ (ControlNet-Union SDXL) Discovered by AI
2024
An all-in-one ControlNet architecture for Stable Diffusion XL (SDXL) that supports over 10 different control conditions (such as canny, openpose, depth, and lineart) within a single unified model without performance degradation.
conditioningcontrolnetsdxlunion -
ControlNet-LLLite (ComfyUI) Discovered by AI
2024
ControlNet-LLLite is a lightweight, LoRA-like alternative to traditional ControlNet models, designed to inject low-rank spatial conditioning directly into the attention layers of the UNet. It significantly reduces VRAM and disk storage requirements (typically under 30MB per checkpoint) while maintaining high-quality control for tasks like Canny, depth, and lineart.
conditioningcontrolnetlightweightsdxl -
ControlNeXt: Powerful and Efficient Control for Image and Video Generation Discovered by AI
2024
ControlNeXt is a highly efficient alternative to ControlNet that replaces heavy additional branches with a lightweight adapter, reducing training and inference overhead while maintaining strong spatial control for image and video generation.
adapterconditioningcontrolnetefficient -
Ctrl-Adapter Discovered by AI
2024
An efficient and versatile framework designed to adapt existing spatial control models (like ControlNet) to any new text-to-image or text-to-video diffusion backbones without retraining. It resolves feature space mismatches through lightweight adapter layers, enabling rapid integration of diverse conditioning signals across different model architectures.
adapterconditioningcontrolnetmulti-modal -
Ctrl-X: Controlling Structure and Appearance for Text-To-Image Generation Without Guidance Discovered by AI
2024
A training-free and guidance-free framework for text-to-image diffusion models that enables precise control over both structure and appearance by designing feed-forward structure control and semantic-aware appearance transfer.
appearance-transfercontrolnetstructure-controltraining-free -
InstantID: Zero-shot Identity-Preserving Generation in Seconds Discovered by AI
2024
A tuning-free, identity-preserving generation method that utilizes a specialized ControlNet to condition on facial landmarks and identity features, enabling high-fidelity character generation from a single reference image.
controlnetface-conditioningidentity-preservingsdxl -
MistoControlNet-Flux-dev Discovered by AI
2024
A collection of high-precision ControlNet models trained for the FLUX.1-dev model, including MistoLine for advanced line art and edge-based conditioning.
conditioningcontrolnetfluxlineart -
MistoLine Discovered by AI
2024
A versatile and robust SDXL-ControlNet model designed for adaptable line art conditioning, utilizing the Anyline preprocessor to handle various sketch and line art styles with high precision.
anylinecontrolnetlineartsdxl -
OminiControl: Minimal and Universal Control for Diffusion Transformer Discovered by AI
2024
A parameter-efficient and versatile framework that integrates spatially-aligned and subject-driven image conditions into pre-trained Diffusion Transformer (DiT) architectures like FLUX.1.
controlnetditfluxparameter-efficientGitHub · arXiv · Hugging Face
-
RepControlNet: ControlNet Reparameterization Discovered by AI
2024
RepControlNet introduces a structural reparameterization method for ControlNet, allowing the control branch to be fused back into the main diffusion backbone during inference. This eliminates the extra computational overhead and parameter count of the control network while maintaining high-quality controllable generation.
controlnetdiffusionreparameterizationsdxl -
SmartControl: Enhancing ControlNet for Handling Rough Visual Conditions Discovered by AI
2024
SmartControl is an extension of ControlNet designed to handle rough or unaligned visual conditions, allowing the model to adaptively interpret inaccurate spatial prompts without producing visual artifacts.
conditioningcontrolnetrobustnesssketch -
Stable Diffusion 3.5 Large ControlNets Discovered by AI
2024
Official ControlNet models (including Blur, Canny, and Depth) released by Stability AI for Stable Diffusion 3.5 Large, enabling precise structural and spatial conditioning.
conditioningcontrolnetsd3.5stability-ai -
VideoX-Fun Discovered by AI
2024
A flexible framework for controllable image and video generation that hosts the official implementation of multi-functional ControlNet Union models, supporting Canny, depth, pose, and MLSD controls.
conditioningcontrolnetfluxvideo-generation -
When ControlNet Meets Inexplicit Masks: A Case Study of ControlNet on its Contour-following Ability Discovered by AI
2024
This paper analyzes the contour-following ability of ControlNet when dealing with noisy or inexplicit user-provided masks, proposing a Shape-aware ControlNet to enhance spatial controllability and reduce unwanted artifacts under degraded conditioning signals.
conditioningcontrolnetmask-conditioning -
XLabs-AI FLUX ControlNet Discovered by AI
2024
A suite of ControlNet models (including Canny, HED, and Depth) trained specifically for the FLUX.1-dev model, enabling precise spatial guidance and structure control.
conditioningcontrolnetdiffusionflux -
2023
Adds spatial conditioning (edges, depth, pose, scribbles) to diffusion models, enabling layout- and sketch-controlled generation central to production art workflows.
conditioningdepthpose -
ControlNet-XS Discovered by AI
2023
A highly efficient alternative to the standard ControlNet architecture that treats controllable image generation as a feedback-control system. By reducing the parameter size of the control branch and establishing direct feedback connections, it achieves faster inference and lower computational overhead while maintaining control fidelity.
conditioningcontrolnetdiffusionefficient -
FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Condition Discovered by AI
2023
A training-free approach for controllable text-to-image generation that supports various spatial conditions and works across different model architectures and checkpoints. It enables structural control without requiring the training of auxiliary conditioning modules like traditional ControlNet.
controlnetspatial-controltraining-free -
2023
A lightweight (~100MB) adapter that adds image-prompt conditioning to diffusion models, letting a reference image drive style or subject; handy for consistent character and style transfer.
conditioningimage-promptreference -
LooseControl: Lifting ControlNet for Generalized Depth Conditioning Discovered by AI
2023
LooseControl generalizes depth conditioning for diffusion models, allowing users to control scene boundaries and edit 3D layouts using rough or loose depth maps rather than highly detailed ones.
conditioningcontrolnetdepthlayout -
Meta ControlNet: Enhancing Task Adaptation via Meta Learning Discovered by AI
2023
A meta-learning framework designed to enhance the task adaptation speed of ControlNet, reducing the required training steps for new conditional control tasks.
adaptationcontrolnetmeta-learning -
Uni-ControlNet Discovered by AI
2023
An all-in-one control framework that integrates multiple spatial conditioning signals into a single, unified ControlNet model. It allows simultaneous or individual control using various inputs (such as canny edges, depth maps, and human poses) without requiring separate model checkpoints for each task, significantly reducing storage and deployment costs.
conditioningcontrolnetdiffusionmulti-control -
UniControl Discovered by AI
2023
A unified controllable text-to-image diffusion model that integrates multiple visual conditions, such as edge maps, depth maps, segmentation masks, and poses, into a single framework using a task-aware HyperNet.
conditioningcontrolnetmulti-taskGitHub · arXiv · Hugging Face
-
X-Adapter: Adding Universal Compatibility of Plugins for Upgraded Diffusion Model Discovered by AI
2023
A universal adapter framework that enables pre-trained plug-and-play modules (such as SD 1.5 ControlNets and LoRAs) to work directly with upgraded text-to-image diffusion models like SDXL without retraining.
adaptercompatibilitycontrolnetsdxl
