DYNAMIC_TRT_MODEL_CONVERSION Node Documentation

1. Overview

The DYNAMIC_TRT_MODEL_CONVERSION node is designed to optimize AI models for execution on NVIDIA RTX GPUs by leveraging NVIDIA TensorRT. This optimization aims to unlock the highest performance capabilities of your GPU when working with models like Stable Diffusion and its variants. The node specializes in creating dynamic TensorRT engines that support a range of input resolutions and batch sizes, providing flexibility for different use cases while maintaining optimal performance.

2. Inputs

The DYNAMIC_TRT_MODEL_CONVERSION node requires the following input to function:

• Model Checkpoint: This is the primary input where you connect a model you wish to convert into a TensorRT engine. The model checkpoint is usually provided by adding a "Load Checkpoint" node in ComfyUI and connecting it to the conversion node.

3. Outputs

The output of the DYNAMIC_TRT_MODEL_CONVERSION node is a TensorRT model engine that is specifically optimized for the resolutions and batch sizes you define. This engine can be saved with a meaningful filename prefix and is used later for accelerated inference.

4. Usage in ComfyUI Workflows

The DYNAMIC_TRT_MODEL_CONVERSION node is a crucial component in the optimization workflow for using AI models effectively with NVIDIA GPUs. Here's a general guide on how it is integrated into workflows:

1. Add a Load Checkpoint Node: First, incorporate a Load Checkpoint Node to bring your desired model into ComfyUI.

2. Attach the Conversion Node: Connect the output from the Load Checkpoint Node to the input of the DYNAMIC_TRT_MODEL_CONVERSION node. This sets the model for conversion to a TensorRT format.

3. Specify File Naming: Assign a helpful filename prefix which will be used to identify the generated TensorRT engine within the directory.

4. Initiate the Conversion: Activate the conversion process by queuing the operation, which will begin the construction of the TensorRT engine tailored to your GPU.

5. Engine Usage: Once the model has been converted, it can be loaded into the workflow using a TensorRT Loader Node for accelerated performance.

5. Special Features and Considerations

• Dynamic Optimization: Unlike static conversions that are limited to a single resolution and batch size, dynamic conversion supports a range of settings, allowing greater flexibility while still optimizing for a specific, user-defined 'optimal' configuration.

• Performance Gains: By using the dynamic mode, the TensorRT engine can adapt to various operational requirements while maintaining the best performance when operating at the optimal settings.

• VRAM Usage: Dynamic engines can consume more VRAM due to the range of supported resolutions and batch sizes. Therefore, ensure your GPU has adequate memory, especially when working with complex models such as SDXL or Stable Video Diffusion-XT.

• Session Refresh Requirement: After generating a TensorRT engine, refresh the ComfyUI session (using F5 refresh on a browser) to load and use the newly created engine with the TensorRT Loader node.

• Compatibility Limitations: Currently, the TensorRT engines generated are not compatible with ControlNets or LoRAs, though future updates are expected to address this limitation.

This node is instrumental for users seeking to achieve high-performance model inference using NVIDIA RTX GPUs and is best suited for advanced workflows where multiple resolutions and batch sizes are in play. Proper configuration and utilization of the node within ComfyUI can lead to significant performance improvements in model processing times.