SV3D_BatchSchedule Node Documentation

Overview

The SV3D_BatchSchedule node is part of the ComfyUI-KJNodes repository. It is designed to facilitate the scheduling of azimuth and elevation conditions for SV3D models, particularly for video generation. This node allows you to control the rotational and elevational movements of a camera in a 3D scene for a specified number of frames. SV3D models, such as those hosted on Hugging Face (https://huggingface.co/stabilityai/sv3d), can harness this functionality to generate sequences where the camera view dynamically changes over time.

Inputs

1. clip_vision: A CLIP vision model used for encoding initial images.
2. init_image: The initial input image that will be used as a reference for generating encoded conditions.
3. vae: A Variational Autoencoder (VAE) model to encode the image.
4. width: The width of the images to be produced. It is measured in pixels and must be a multiple of 8.
5. height: The height of the images to be produced. It is measured in pixels and must be a multiple of 8.
6. batch_size: The total number of frames you want to generate in the sequence.
7. interpolation: The method of interpolation between the scheduled points. Available options are:
   • linear
   • ease_in
   • ease_out
   • ease_in_out
8. azimuth_points_string: A string representing the keyframes and their corresponding azimuth angles (in degrees) over time.
9. elevation_points_string: A string representing the keyframes and their corresponding elevation angles (in degrees) over time.

Outputs

1. positive: The positive conditioning tensor that combines image encodes and interpolated camera parameters.
2. negative: The negative conditioning tensor, similar to the positive output, but initialized to zeros and designed for operations like differential processing.
3. latent: An empty structure with defined dimensions to store latent variables for the specified number of frames.

Usage

In ComfyUI workflows, the SV3D_BatchSchedule node can be used to generate videos where the camera's viewpoint changes smoothly across a sequence of frames, adding an element of motion and depth. This node is particularly useful for projects where a transformative perspective is key, such as 3D animation, VR experiences, or simulated environments.

Here's how you could use this node in a typical workflow:

1. Initialize the Node: Drag and drop SV3D_BatchSchedule into your ComfyUI project.
2. Configuring Inputs: Link the required models and images to the input fields. Define the interpolation technique and the keyframes for azimuth and elevation.
3. Execute Workflow: After configuring the node with the desired settings, execute the workflow to generate a series of frames where the camera's position shifts according to the schedule.
4. Render Output: The outputs from the node are typically processed using subsequent nodes to produce a finalized video product.

Special Features and Considerations

• Ease Interpolations: The node supports multiple ease functions, allowing for smoother transitions in camera movements, essential for creating realistic animation effects.
• Rotational Continuity: Ensure that azimuth and elevation angles are carefully mapped to maintain smooth transitions and avoid sudden jerks in camera movements.
• Frame Forwarding Constraint: For SV3D video generation, ensure that camera schedules always progress forward to align with the video model requirements.
• Dimension Constraints: Remember that width and height must be multiples of 8 to prevent errors in latent generation and processing phases.

By leveraging this node, creators can achieve nuanced and dynamic camera effects in their video projects, giving rise to more engaging visual narratives.