SegmentV2 Node Documentation

Overview

The SegmentV2 node is part of the ComfyUI-RMBG repository and is designed for advanced image segmentation tasks. This node allows users to segment objects or scenes from images using tag-style or natural language prompts. It utilizes state-of-the-art image segmentation models such as SAM (Segment Anything Model) and GroundingDINO. The segmentation results can be optionally processed to adjust the mask, apply background colors, and more, making it an essential tool for background elimination and object separation workflows.

Inputs

The SegmentV2 node takes the following inputs:

1. Image: The primary image to be segmented. This input supports multiple images or batch processing if the input image tensor has four dimensions.

2. Prompt: A text string describing the object or scene you want to segment. The prompt can be written in a tag-style (e.g., "cat, dog") or natural language (e.g., "a person wearing a red jacket").

3. SAM Model: A selection from the available SAM models which are pre-loaded and optimized for different specifications such as model size and precision.

4. DINO Model: A selection from the available GroundingDINO models, optimized for zero-shot object detection.

5. Threshold: A numerical value ranging from 0.05 to 0.95 that adjusts the mask detection strength, with higher thresholds meaning more strict detection.

6. Mask Blur: An integer value from 0 to 64 that determines the amount of Gaussian blur applied to the mask edges.

7. Mask Offset: An integer value from -64 to 64 that decides whether the mask boundary is expanded (positive values) or shrunk (negative values).

8. Invert Output: A boolean option that inverts the mask output if set to true.

9. Background: Selection between "Alpha" (transparent background) and "Color" background options.

10. Background Color: A color value specified in hex format to be used when the background option is set to "Color".

Outputs

The SegmentV2 node produces the following outputs:

1. Image: The processed image with the specified segmentation applied, outputting as an RGBA image if a transparent background is chosen, otherwise an RGB image with the selected background color.

2. Mask: A binary mask of the segmented objects or scene, returned as a tensor.

3. Mask Image: A visualization of the mask in RGB format, useful for checking the segmentation result.

Usage in ComfyUI Workflows

The SegmentV2 node can be integrated into ComfyUI workflows for a variety of purposes, such as:

• Background Removal: Use the segmentation capabilities to isolate objects, allowing for the removal of backgrounds in user-defined colors or transparency.
• Object Detection: Employ textual prompts to detect and segment specific objects within an image, which is useful for tasks that require identifying and extracting particular elements.
• Image Editing: After obtaining the segmentation results, the outputs can be further processed for image retouching, compositing, or other editing tasks as needed.
• Batch Processing: Leverage the node's ability to handle batch inputs for processing large sets of images efficiently.

Special Features and Considerations

• Model Selection: The node offers a range of SAM and GroundingDINO models catering to users' needs for balance between performance and precision.
• Processing Customization: The node provides extensive options to refine the segmentation process, including the ability to blur edges and alter mask boundaries.
• Background Versatility: Users can select between maintaining alpha channels for transparency or applying solid color backgrounds.
• Prompt Flexibility: With support for both tag-style and natural language prompts, users can comfortably describe objects or scenes in a manner that best suits their needs.
• Batch Compatibility: Designed to handle batches of images, the node supports efficient processing suited for high-volume workflows.
• Ease of Integration: When included in ComfyUI workflows, this node facilitates seamless connectivity with subsequent image processing nodes to build comprehensive and complex image processing pipelines.

In conclusion, the SegmentV2 node in the ComfyUI-RMBG project provides robust segmentation features with customizable processing parameters, making it an essential tool for precise object and background segmentation in various image processing applications.