Home

Awesome

[!NOTE] Officially implemented here: https://github.com/TencentQQGYLab/ComfyUI-ELLA

ComfyUI_ELLA

ComfyUI Implementaion of ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment

image

[!NOTE]
As per the ELLA developers / researchers, only the SD 1.5 checkpoint is released.

Installation

To install, simply navigate to custom_nodes, and inside that directory, do git clone https://github.com/ExponentialML/ComfyUI_ELLA.git

Models

These models must be placed in the corresponding directories under models.

Example: ComfyUI/models/ella/model_file

The directories should be created automatically. If they are not, please create them manually.

  1. Place the ELLA Model under a new folder ella: https://huggingface.co/QQGYLab/ELLA/blob/main/ella-sd1.5-tsc-t5xl.safetensors

  2. Create a folder called t5_model. Navigate to that folder (you must be in that directory), and git clone https://huggingface.co/google/flan-t5-xl to download the t5 model.

  3. If you don't wish to use git, you can dowload each indvididually file manually by creating a folder t5_model/flan-t5-xl, then download every file from here, although I recommend git as it's easier.

In summary, you should have the following model directory structure:

Usage

Workflow

To get started quickly, a workflow is provided in the workflow directory.

ELLA Loader

image

ELLA Text Encode

image

Support

All conditioning nodes should be supported, as well as prompt weighting and ControlNet.

image

Attribution

Thanks to the following for open sourcing. Please follow their respective licensing.