TensorRTLoader Node Documentation

The TensorRTLoader node is a component of the ComfyUI ecosystem, specifically designed to optimize and accelerate image generation tasks using NVIDIA's TensorRT engines. This node enables leveraging the power of NVIDIA RTX GPUs for improved performance in running AI models like Stable Diffusion.

Overview

The TensorRTLoader node is responsible for loading pre-compiled TensorRT engines into the ComfyUI environment. By utilizing these engines, users can experience enhanced performance when generating images using the Stable Diffusion family of models and other supported AI models.

Functionality

What This Node Does

• The TensorRTLoader node loads TensorRT engine files, which are optimized versions of AI models created using NVIDIA's TensorRT framework.
• These engines are designed to run on NVIDIA RTX GPUs, providing significant performance improvements over traditional model execution.
• Once loaded, the TensorRT engine can be used in place of standard models to accelerate image generation and processing tasks.

Node Inputs

The TensorRTLoader node accepts the following inputs:

• unet_name: A selection dropdown that allows the user to choose from a list of available TensorRT engine files.
• model_type: A selection dropdown that specifies the model type associated with the selected TensorRT engine. Options include various Stable Diffusion iterations and other AI models like AuraFlow and Flux.

Node Outputs

The node produces a single output:

• MODEL: This is the loaded and optimized model, which can be directly used in the ComfyUI workflows for image generation tasks.

Usage in ComfyUI Workflows

1. Adding the Node: Add the TensorRTLoader node to your ComfyUI workflow to start using TensorRT engines.
2. Selecting an Engine: Use the unet_name dropdown to select the desired TensorRT engine file.
3. Setting the Model Type: Specify the compatible model type using the model_type dropdown to ensure proper integration into the workflow.
4. Connecting to Workflows: Once configured, connect the MODEL output to other workflow components, such as image samplers, to process images using the accelerated model.

Special Features and Considerations

• Engine File Availability: To use the TensorRTLoader node, TensorRT engine files must be pre-built and accessible to the ComfyUI instance. Follow the instructions in the repository's README to generate these engine files.
• Optimized Performance: The primary advantage of using this node is the significant performance boost in image generation tasks, thanks to the optimized execution provided by TensorRT.
• Session Refresh Requirement: Newly created TensorRT engines may not immediately appear in the unet_name dropdown; refreshing the ComfyUI interface (e.g., F5 in your browser) may be necessary for the changes to take effect.
• Compatibility: TensorRT engines loaded through this node are specific to certain model types and GPU capabilities. Ensure that your system meets these requirements to achieve the best performance.

By integrating the TensorRTLoader node into your ComfyUI workflows, you can take full advantage of NVIDIA TensorRT's capabilities, optimizing your image generation tasks for speed and efficiency.