ACN_ReferencePreprocessor Node Documentation

Overview

The ACN_ReferencePreprocessor node is a part of the ComfyUI-Advanced-ControlNet project. This extension enhances the capabilities of ControlNet by introducing advanced features such as scheduling ControlNet strength across timesteps and applying custom weights and attention masks. Specifically, the ACN_ReferencePreprocessor node is designed to handle reference inputs and preprocess them for use with Advanced ControlNet models. It supports modes like reference_attn, reference_adain, and reference_adain+attn.

Node Functionality

The ACN_ReferencePreprocessor node processes reference images to be used with Advanced ControlNet. This is particularly useful when using ControlNet with references to guide the generation process more accurately or with specific stylistic inputs.

Inputs

• Reference Image: The primary input for this node is a reference image or set of images that the Advanced ControlNet will use to deduce stylistic cues or specific features that need to be imitated or maintained in the generation process.

• Preprocessing Mode (optional) [reference_attn, reference_adain, reference_adain+attn]: Determines how the reference should be processed and what aspects of it should influence the output.

  • reference_attn utilizes attention mechanisms to consider the reference's features.
  • reference_adain uses adaptive instance normalization techniques to transfer style from reference images.
  • reference_adain+attn combines both the aforementioned methods for a more complex style incorporation.

• Style Fidelity (optional): Adjustable parameter translating to how closely the output should adhere to the style of the reference image. Higher values mean more fidelity to the reference style.

• Control Weight (optional): Determines the strength of reference influence compared to other guidance aspects, equivalent to adjusting how much the output should be influenced by the control inputs derived from the reference.

Outputs

• Preprocessed Reference: The output is a form of processed data that can be directly used by subsequent nodes within the Advanced ControlNet pipeline, which incorporate reference effects into the image generation process.

Usage in ComfyUI Workflows

In a typical ComfyUI workflow involving Advanced ControlNet nodes, the ACN_ReferencePreprocessor can be used in the following manner:

1. Insert the ACN_ReferencePreprocessor node and connect it to the path containing your reference image(s).

2. Adjust the preprocessing mode to fit your stylistic goals. Choose between attention-focused, adaptive normalization, or a combination of both.

3. Set optional parameters like style fidelity and control weight if you need specific control over the stylistic influence of your reference.

4. Feed the output of this node into an Advanced ControlNet node, such as "Apply Advanced ControlNet," where the processed reference influences the overall generation output according to the set weights and parameters.

This node plays a crucial role in using reference images to guide ControlNet, enabling more controlled and stylistically relevant outputs.

Special Features or Considerations

• Mode Versatility: The node's capability to switch between multiple preprocessing modes gives users flexibility, depending on whether the style transfer or feature-focused influence is needed.

• Granular Control: Users can fine-tune style fidelity and control weight to vary the degree of reference influence, enabling customized outputs.

• Compatibility: The node complements other Advanced ControlNet nodes, making it a flexible option for users wishing to integrate reference images in complex workflows.

In conclusion, the ACN_ReferencePreprocessor enhances the functionality of ControlNet by enabling reference images within the framework of ComfyUI-Advanced-ControlNet. With its ability to preprocess reference images and support a versatile range of modes, it allows users to extract and apply stylistic elements to generated images effectively.