ComfyUI-KwaiKolorsWrapper

Overview

The ComfyUI-KwaiKolorsWrapper is a custom repository that provides a rudimentary wrapper to run the Kwai-Kolors text2image pipeline using the diffusers library within the ComfyUI framework. It enables users to integrate the Kwai-Kolors model and the ChatGLM3 model into their ComfyUI workflows for image synthesis and related tasks.

Installation

To set up the ComfyUI-KwaiKolorsWrapper, follow these steps:

1. Clone the Repository:

   • Navigate to the ComfyUI/custom_nodes folder within your ComfyUI installation directory.
   • Clone this repository into the custom_nodes folder.

   git clone https://github.com/kijai/ComfyUI-KwaiKolorsWrapper.git
   

2. Install Dependencies:

   • Ensure you have at least version 4.38.0 of the transformers library.
   • From the root of your ComfyUI installation, run the following to install dependencies:

   pip install -r ComfyUI/custom_nodes/ComfyUI-KwaiKolorsWrapper/requirements.txt
   

   • Alternatively, if you are using the portable version, execute:

   python_embeded\python.exe -m pip install -r ComfyUI\custom_nodes\ComfyUI-KwaiKolorsWrapper\requirements.txt
   

3. Model Download:

   • The FP16 models will be automatically downloaded from Hugging Face to the directory: ComfyUI/models/diffusers/Kolors
   • Ensure the directory structure matches the described folder hierarchy, including the scheduler, text_encoder, and unet subdirectories.

Purpose

This repository aims to provide a convenient way to integrate and use the Kwai-Kolors text2image pipeline within the ComfyUI framework, thereby expanding the capabilities of image generation and manipulation through customizable nodes.

Node Offerings

The repository offers the following nodes:

• DownloadAndLoadKolorsModel: Handles downloading and loading of the Kolors model.
• DownloadAndLoadChatGLM3: Handles downloading and loading of the ChatGLM3 model.
• KolorsSampler: A node used for sampling functionalities.
• KolorsTextEncode: Encodes text inputs for use in the pipeline.
• LoadChatGLM3: Specifically loads the ChatGLM3 model from available configurations.

Special Features and Capabilities

• Safetensors Support: The repository provides an alternative way to load the ChatGLM3 model using a single safetensors file, ensuring efficient memory utilization and compatibility.
• Quantization Options: The implementation supports model quantization, reducing VRAM requirements:
  • FP16 size is approximately 13 GB
  • Quant8 size reduces to around 8 GB
  • Quant4 size further reduces to around 4 GB

Utility in ComfyUI Workflows

The ComfyUI-KwaiKolorsWrapper can enhance ComfyUI workflows by:

• Providing additional model options for text2image generation, enriched by the specialized Kwai-Kolors model.
• Allowing seamless integration of advanced language models such as ChatGLM3.
• Offering flexibility in memory usage and model execution through quantization, making it accessible for users with varying hardware capabilities.
• Simplifying the setup of sophisticated image synthesis pipelines by abstracting model loading and configuration management into easy-to-use nodes.

Overall, users can leverage this repository to enrich their creative production pipelines with advanced AI-driven image generation capabilities.