ComfyUI-IC-Light-Native

Overview

The ComfyUI-IC-Light-Native repository is a native implementation of IC-Light for ComfyUI. It provides custom nodes and functionality for working with IC-Light models in a ComfyUI environment, enhancing capabilities for image relighting and manipulation within workflows.

Installation

Clone the Repository

To install ComfyUI-IC-Light-Native, you can either download and unzip the repository into the custom_nodes folder within your ComfyUI installation directory or clone it via Git.

To clone via Git, navigate to your ComfyUI installation directory and execute the following commands:

cd custom_nodes
git clone [email protected]:huchenlei/ComfyUI-IC-Light-Native.git

Download Models

The IC-Light implementation is based on diffusers. To use it within ComfyUI, the state_dict keys need conversion to ldm format. You can download models with compatible keys from the following location: IC-Light ldm keys.

There are two models available:

• iclight_sd15_fc_unet_ldm: Suitable for FG workflows.
• iclight_sd15_fbc_unet_ldm: Suitable for BG workflows.

After downloading these models, place them in the ComfyUI/models/unet directory and load them with the UNETLoader node.

Recommended Nodes

• ComfyUI-KJNodes: Offers various mask nodes for creating light maps.
• ComfyUI-Easy-Use: A comprehensive node pack including nodes for background removal.
• ComfyUI_essentials: Provides many useful tooling nodes, including image resizing.
• ComfyUI-IC-Light: Contains the DetailTransfer node to help preserve high-frequency details from input images.

Node Descriptions

ICLight

This node is implemented in ic_light_nodes.py as ICLight. It is essential for applying the IC-Light effects within workflows tailored for relighting and compositing.

ICLightApplyMaskGrey

Implemented in ic_light_nodes.py as ICLightApplyMaskGrey, this node ensures the masked areas of an image are correctly greyed out. This is crucial for maintaining image integrity in certain workflows.

VAEEncodeArgMax

The VAEEncodeArgMax node, also in ic_light_nodes.py, facilitates specific encoding processes that are central to utilizing IC-Light models effectively.

Workflows

Given FG, Generate BG and Relight

This workflow involves generating background (BG) and relighting, given a foreground (FG). To maintain the original colors of the FG object, it can be placed in the latent space to guide generation. Workflow Example

Given FG and Light Map, Generate BG, and Relight

This workflow uses an FG image along with a light map to generate a BG and relight the composed image based on directional lighting.

Given FG and BG, Combine and Relight

Here, an FG image is combined with a BG image and then relit according to the desired output.

Recover High-Frequency Detail

This technique allows for the transfer of high-frequency details like text from the original input image to the generated image for better clarity and quality. Detailed Example

Special Features and Capabilities

• Custom Node Implementations: ComfyUI-IC-Light-Native provides custom implementations for nodes like ICLight, ICLightApplyMaskGrey, and VAEEncodeArgMax for optimized performance and integration into ComfyUI.
• Advanced Relighting Techniques: Utilize IC-Light's capabilities to perform sophisticated image relighting and composition.
• Error Handling: Instructions and common error messages are provided, allowing users to troubleshoot model loading and usage issues effectively.

Use in ComfyUI Workflows

The nodes and functionalities provided by ComfyUI-IC-Light-Native enhance ComfyUI workflows by allowing users to implement complex image generation tasks involving light manipulation, background and foreground integration, and detail recovery. This makes the repository invaluable for users looking to perform advanced image editing and synthesis operations.