← See All Custom Node Packs

ComfyUI-Advanced-ControlNet

972
Updated 2 months ago
View on GitHub →See Common Issues →

Run ComfyUI Easily with InstaSD

Skip the complex setup. InstaSD helps creative professionals build workflows and deploy them to the world:

  • One-click deployment
  • Any model, any node
  • Powerful GPUs for rapid iteration
Get Started

Available Nodes

Documentation

ComfyUI Advanced ControlNet Custom Nodes

Overview

The ComfyUI-Advanced-ControlNet is a repository that provides advanced nodes for ComfyUI, facilitating complex tasks involving ControlNets. These nodes are designed to enable scheduling of ControlNet strength across timesteps, manage batched latents, and apply custom weights and attention masks. The repository builds on the functionalities offered by other tools like Auto1111's sd-webui ControlNet extension and extends them in various ways.

Installation

To use the nodes provided in this repository, follow these steps to install:

  1. Ensure you have ComfyUI installed and properly set up on your machine.
  2. Clone the repository or download it as a ZIP file from the URL: https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet
  3. Place the repository contents in the appropriate custom nodes folder within your ComfyUI installation.
  4. Restart ComfyUI to load the new nodes.

Purpose of the Repository

This repository provides advanced node functionalities for managing and scheduling ControlNet operations in ComfyUI. It caters to both standard and more complex needs by allowing detailed control over how ControlNets interact with image latents and through time. It enhances the ComfyUI experience by:

  • Providing nodes that support a broad range of ControlNet operations.
  • Enabling precise control over ControlNet effects with timestep and latent scheduling.
  • Facilitating integration with other ControlNet tools and preprocessors.

Types of Nodes Provided

This repository features several types of nodes, each serving a unique function in managing ControlNets within ComfyUI:

  1. Timestep Keyframes: Nodes like TimestepKeyframe, ACN_TimestepKeyframeInterpolation, and ACN_TimestepKeyframeFromStrengthList manage the scheduling of ControlNets over time.

  2. Latent Keyframes: Nodes such as LatentKeyframe, LatentKeyframeTiming, and LatentKeyframeGroup provide functionality for controlling the effect of ControlNets on specific latents in a batch.

  3. Advanced ControlNet Application: Nodes like ACN_AdvancedControlNetApply_v2 allow for applying ControlNet adjustments with advanced features over traditional nodes.

  4. ControlNet Model Loaders: Nodes such as ACN_ControlNetLoaderAdvanced and ACN_DiffControlNetLoaderAdvanced focus on loading ControlNet models with advanced capability support.

  5. Custom Weights and Masks: Special nodes to define weights and masks for various ControlNet applications, such as ACN_ScaledSoftControlNetWeights.

  6. SparseCtrl and SVD-ControlNet: These nodes, including ACN_SparseCtrlLoaderAdvanced, offer advanced functionality for working with SparseCtrls and SVD-ControlNet implementations.

Special Features and Capabilities

  • Scheduling ControlNet Strength: With timestep and latent keyframes, users can schedule how ControlNet effects are applied over time and across different latent images.
  • Custom Weights and Attention Masks: Influence the importance of the ControlNet vs. the user prompt and apply attention masks for refined results.
  • Support for Various ControlNet Variants: The nodes are compatible with a range of ControlNet implementations, including T2IAdapters, ControlLoRAs, CtrLoRA, and more.
  • Advanced Model Loading: Loads and converts ControlNets into advanced versions that integrate seamlessly into the workflow.
  • Reference Modes and Customization: Supports modes like reference_attn and reference_adain for reference-based operations.

Usefulness in ComfyUI Workflows

In the context of ComfyUI, this repository significantly enhances the capabilities when working with image generation:

  • Precision Control: By allowing timestep and latent-based controls, users can finely tune when and how ControlNets apply their effects.
  • Flexibility in Customization: The range of nodes supports extensive integration and customization, ideal for complex image processing pipelines.
  • Integration with Other Projects: The nodes can be used alongside other tools like ComfyUI-AnimateDiff-Evolved for more dynamic image results.
  • Streamlined User Experience: By replicating and improving upon features from other extensions, the user workflow in ComfyUI becomes more robust and versatile.

By integrating and using the nodes in this repository, users can achieve a higher level of control and sophistication in their ComfyUI image processing tasks.