ComfyUI TensorRT Custom Nodes

Overview

The ComfyUI_TensorRT repository is designed to enhance the performance of Stable Diffusion workflows on NVIDIA RTX™ Graphics Cards by leveraging NVIDIA TensorRT. This allows for optimized AI model execution, enabling superior performance, especially for graphics-intensive tasks. The repository provides custom nodes, which integrate TensorRT into ComfyUI, a user interface for managing and interacting with machine learning models.

What the Repository is For

This repository is specifically aimed at users who want to optimize the performance of Stable Diffusion models on NVIDIA RTX GPUs. By using TensorRT, users can create TensorRT engines tailored to their specific hardware, unlocking high-performance processing for tasks such as image generation and video diffusion.

Installation

Recommended Installation via ComfyUI Manager

1. Use the ComfyUI Manager to easily install the custom nodes into your existing ComfyUI instance.

Manual Installation

1. Navigate to your ComfyUI/custom_nodes directory.
2. Clone the repository:

   git clone https://github.com/comfyanonymous/ComfyUI_TensorRT
   

3. Navigate to the cloned directory:

   cd ComfyUI_TensorRT
   

4. Install the required Python packages:

   pip install -r requirements.txt
   

Provided Nodes

The repository includes the following nodes:

• DYNAMIC_TRT_MODEL_CONVERSION: Allows for the creation of TensorRT engines that support a range of resolutions and batch sizes. Optimal settings should be specified for best performance.

• STATIC_TRT_MODEL_CONVERSION: Converts models into TensorRT engines that support specific resolutions and batch sizes, having a consistent performance boost equivalent to dynamic engines at optimal settings. Requires less VRAM compared to dynamic engines.

• TensorRTLoader: Loads TensorRT engines into the ComfyUI workflow for accelerated image generation.

Special Features and Capabilities

• Dynamic and Static Engines: Users can choose between dynamic engines that support multiple resolutions and batch sizes, or static engines optimized for a specific setup. Static engines use less VRAM.

• Wide Model Support: Supports various models including Stable Diffusion 1.5, 2.1, 3.0, SDXL, SDXL Turbo, Stable Video Diffusion, Stable Video Diffusion-XT, AuraFlow, and Flux.

• High-Performance Optimization: By generating TensorRT engines specific to a user's GPU, the repository facilitates the highest performance possible for NVIDIA RTX graphics cards.

Usage in ComfyUI Workflows

Building a TensorRT Engine

1. Load Checkpoint Node: Begin by adding a Load Checkpoint Node.
2. Add Conversion Node: Insert either a DYNAMIC_TRT_MODEL_CONVERSION or STATIC_TRT_MODEL_CONVERSION node.
3. Connect Nodes: Link the output of the Load Checkpoint Node to the model input of the conversion node.
4. Specify Filename: To identify the converted model, add a meaningful filename as a prefix.
5. Queue Conversion: Start the conversion process by queuing the prompt.

Accelerated Image Generation

1. Load TensorRT Engine: Use a TensorRTLoader node to load the generated TensorRT engines.
2. Select Engine: Ensure that the model type matches the TensorRT engine being used.
3. Complete Workflow: Utilize the original model checkpoint for CLIP and VAE selection.

Limitations

• ComfyUI TensorRT engines currently do not support ControlNets or LoRAs, but future updates will address these limitations.

This repository is an excellent choice for users who need to take full advantage of their NVIDIA RTX hardware in ComfyUI, especially when working with Stable Diffusion models that are resource-intensive.