ComfyUI-IF_AI_tools

Overview

ComfyUI-IF_AI_tools provides a collection of custom nodes used to integrate Local and API-based Large Language Models (LLMs) and Language Multimodal Models (LMMs) within the ComfyUI ecosystem. The repository supports advanced features such as OCR-RAG, nanoGraphRAG, Supervision Object Detection, and integration with numerous APIs including OpenAI, Anthropic, Google Gemini, Mistral, xAI, and more. This toolset allows users to create custom character assistants with tailored prompts and presets.

Installation

Prerequisite Installation

To utilize PDF-related operations, the installation of poppler is necessary:

Windows:

1. Install scoop if not already installed by running the following in PowerShell:

   iwr -useb get.scoop.sh | iex
   

2. Install poppler with:

   scoop install poppler
   

Debian/Ubuntu:

sudo apt-get install poppler-utils

MacOS:

brew install poppler

Verify the installation by checking the version:

pdftotext -v

Installing Ollama

For optimal performance, it is recommended to install Ollama:

1. To install Ollama models, open Command Prompt or Terminal and run:

   ollama run llama3.2-vision
   

Additional models:

• Omost: ollama run impactframes/dolphin_llama3_omost
• Small Model: ollama run llama3.2

Installing ComfyUI-IF_AI_tools

Using the Manager

1. Within your ComfyUI manager, search for IF_AI_tools and install.

Manual Installation

1. Navigate to the ComfyUI custom_nodes directory.
2. Open Command Prompt from the address bar.
3. Clone the repository with:

   git clone https://github.com/if-ai/ComfyUI-IF_AI_tools.git
   

For Portable Version

1. Double-click embedded_install.bat or run:

   H:\ComfyUI_windows_portable\python_embeded\python.exe -m pip install -r requirements.txt
   

Replace H:\ with your drive letter where ComfyUI_windows_portable is located.

On Custom Environment

1. Activate your environment.
2. Navigate to the newly created ComfyUI-IF_AI_tools directory.
3. Install the required dependencies:

   python -m pip install -r requirements.txt
   

Node Overview

The repository provides the following nodes with their specified functionalities:

• IF_DisplayText
• IF_ImagePrompt
• IF_JoinText
• IF_LoadImagesS
• IF_PromptMkr
• IF_SaveText
• IF_StepCounter
• IF_TextTyper
• IF_VisualizeGraph
• IF_tools_LoadImagesS
• IF_ChatPrompt
• IF_DisplayTextWildcard
• IF_DisplayOmni

Features and Capabilities

• Advanced Interactions: Integration of numerous APIs for LLM and LMM capabilities.
• Custom Character Assistants: Create assistant characters with custom prompts and presets.
• Enhanced OCR and Object Detection: Leveraging OCR-RAG (Bialdy) and Supervision Object Detection for advanced multimedia tasks.
• New Tools:
  • nanoGraphRAG: Enhances graph-based visualizations and processing.
  • IF_Assistants: Offers templates for reasoning, reflection, and reward-based prompting.
• Vision Models: Incorporates models like Haiku and Florence2 for better visualization and perception.

Usage in ComfyUI Workflows

ComfyUI-IF_AI_tools extends the functionality of ComfyUI, allowing seamless integration of advanced LLM and LMM features into workflows. Its diverse set of nodes enable custom text, image processing, and API interaction, making it an invaluable resource for developers looking to embed complex AI functionalities into their ComfyUI applications. Whether for character creation, object detection, or interactive displays, these custom nodes provide a wealth of opportunity for enhancing AI-driven interactions and UI elements.