ComfyUI-Marigold Custom Nodes Documentation

Overview

The ComfyUI-Marigold repository provides a set of custom nodes designed for depth estimation using the Marigold depth estimation model within the ComfyUI environment. These nodes allow for depth map generation and manipulation, suitable for use in visual effects and 3D modeling software. The repository offers a powerful toolset for users interested in enhancing their images with depth-related features, particularly when working with ControlNets and similar functionalities.

Installation

Recommended Installation

1. Use the ComfyUI manager to search for "marigold" and follow the installation prompts.

Manual Installation

1. Clone the repository to your ComfyUI/custom_nodes directory:

   git clone https://github.com/kijai/ComfyUI-Marigold.git
   

2. Install the required Python packages by running:

   pip install -r requirements.txt
   

3. Model Acquisition:

   • The required model will automatically download if it is not already present using huggingface_hub.
   • Alternatively, manually download the model from Hugging Face and place it in one of the following directories:
     • ComfyUI/custom_nodes/ComfyUI-Marigold/checkpoints
     • ComfyUI/models/diffusers

Purpose of the Repository

The ComfyUI-Marigold repository serves as a wrapper around the Marigold depth estimation model, enabling users to integrate depth estimation capabilities into their ComfyUI workflows. The primary goal is to provide accurate depth maps for a range of imaging applications, specifically for use with control and enhancement networks requiring depth information.

Nodes Provided

This repository contains the following custom nodes:

• MarigoldModelLoader: Implements the loading mechanism for the Marigold model.
• MarigoldDepthEstimation_v2: An enhanced version for estimating depth on static images.
• MarigoldDepthEstimation_v2_video: A node for estimating depth from video data.
• MarigoldDepthEstimation: Original node for depth map generation.
• MarigoldDepthEstimationVideo: Node for video-based depth estimation.
• ColorizeDepthmap: Applies colorization to depth maps for better visual interpretation.
• SaveImageOpenEXR: Allows saving depth maps in the OpenEXR format ideal for use in VFX/3D modeling.
• RemapDepth: Adjusts depth map ranges for better visualization.

Special Features and Capabilities

• Depth Map Inversion: The repository includes an option to invert the depth map, making it compatible with various control networks that may require a specific depth orientation.
• Memory Efficiency: Implements a feature that uses fp16 to reduce memory usage, making it feasible to work with large resolutions around 768p.
• Batch Processing: Provides parameters such as n_repeat_batch_size to handle large batch sizes if VRAM allows, speeding up processing.
• OpenEXR Support: Offers better dynamic range storage through the OpenEXR format, which surpasses typical PNG formats in VFX and 3D contexts.

Usage in ComfyUI Workflows

Integrating these nodes into ComfyUI workflows empowers users to:

• Generate and manipulate depth maps with customizable accuracy and processing speed.
• Seamlessly incorporate depth information into post-processing pipelines or use them directly with ControlNets for added dimensionality in image editing.
• Leverage advanced features for depth map visualization and storage, improving integration with professional VFX/3D software.

By extending ComfyUI with these capabilities, the repository significantly enhances the creative possibilities and accuracy in image depth estimation tasks.