Home

Awesome

Face Editor

Face Editor for Stable Diffusion. This Extension is useful for the following purposes:

example

This is a extension of AUTOMATIC1111's Stable Diffusion Web UI. If you are using SD.Next, use the sd.next branch.

Setup

  1. Open the "Extensions" tab then the "Install from URL" tab.
  2. Enter "https://github.com/ototadana/sd-face-editor.git" in the "URL of the extension's git repository" field. Install from URL
  3. Click the "Install" button and wait for the "Installed into /home/ototadana/stable-diffusion-webui/extensions/sd-face-editor. Use Installed tab to restart." message to appear.
  4. Go to "Installed" tab and click "Apply and restart UI".

Usage

  1. Click "Face Editor" and check "Enabled". Check Enabled
  2. Then enter the prompts as usual and click the "Generate" button to modify the faces in the generated images. Result
  3. If you are not satisfied with the results, adjust the parameters and rerun. see Tips.

Tips

Contour discomfort

If you feel uncomfortable with the facial contours, try increasing the "Mask size" value. This discomfort often occurs when the face is not facing straight ahead.

Mask size

If the forelock interferes with rendering the face properly, generally, selecting "Hair" from "Affected areas" results in a more natural image.

Affected ares - UI

This setting modifies the mask area as illustrated below:

Affected ares - Mask images


When multiple faces are close together

When multiple faces are close together, one face may collapse under the influence of the other. In such cases, enable "Use minimal area (for close faces)".

Use minimal area for close faces


Change facial expression

Use "Prompt for face" option if you want to change the facial expression.

Prompt for face

Individual instructions for multiple faces

Individual instructions for multiple faces

Faces can be individually directed with prompts separated by || (two vertical lines).

Individual instructions for multiple faces - screen shot


Fixing images that already exist

If you wish to modify the face of an already existing image instead of creating a new one, follow these steps:

  1. Open the image to be edited in the img2img tab It is recommended that you use the same settings (prompt, sampling steps and method, seed, etc.) as for the original image. So, it is a good idea to start with the PNG Info tab.
    1. Click PNG Info tab.
    2. Upload the image to be edited.
    3. Click Send to img2img button.
  2. Set the value of "Denoising strength" of img2img to 0. This setting is good for preventing changes to areas other than the faces and for reducing processing time.
  3. Click "Face Editor" and check "Enabled".
  4. Then, set the desired parameters and click the Generate button.

How it works

This script performs the following steps:

Step 0

First, image(s) are generated as usual according to prompts and other settings. This script acts as a post-processor for those images.

Step 1. Face Detection

<details> <summary>Detects faces on the image.</summary>

step-1

</details>

Step 2. Crop and Resize the Faces

<details> <summary>Crop the detected face image and resize it to 512x512.</summary>

step-2

</details>

Step 3. Recreate the Faces

<details> <summary>Run <b>img2img</b> with the image to create a new face image.</summary>

step-3

</details>

Step 4. Paste the Faces

<details> <summary>Resize the new face image and paste it at the original image location.</summary>

step-4

</details>

Step 5. Blend the entire image

<details> <summary>To remove the borders generated when pasting the image, mask all but the face and run <b>inpaint</b>.</summary>

step-5

</details>

Completed

<details> <summary>Show sample image</summary>

step-6

</details>

Parameters

Basic Options

Workflow

Select a workflow. "Search workflows in subdirectories" can be enabled in the Face Editor section of the "Settings" tab to try some experimental workflows. You can also add your own workflows.

For more detailed information, please refer to the Workflow Editor section.

Use minimal area (for close faces)

When pasting the generated image to its original location, the rectangle of the detected face area is used. If this option is not enabled, the generated image itself is pasted. In other words, enabling this option applies a smaller face image, while disabling it applies a larger face image.

Save original image

This option allows you to save the original, unmodified image.

Show original image

This option allows you to display the original, unmodified image.

Show intermediate steps

This option enables the display of images that depict detected faces and masks. If the generated image is unnatural, enabling it may reveal the cause.

Prompt for face

Prompt for generating a new face. If this parameter is not specified, the prompt entered at the top of the screen is used.

For more information, please see: here.

Mask size (0-64)

Size of the mask area when inpainting to blend the new face with the whole image.

<details> <summary>Show sample images</summary>

size: 0 mask size 0

size: 10 mask size 10

size: 20 mask size 20

</details>
Mask blur (0-64)

Size of the blur area when inpainting to blend the new face with the whole image.


Advanced Options

Step 1. Face Detection

Maximum number of faces to detect (1-20)

Use this parameter when you want to reduce the number of faces to be detected. If more faces are found than the number set here, the smaller faces will be ignored.

Face detection confidence (0.7-1.0)

Confidence threshold for face detection. Set a lower value if you want to detect more faces.

Step 2. Crop and Resize the Faces

Face margin (1.0-2.0)

Specify the size of the margin for face cropping by magnification.

If other parameters are exactly the same but this value is different, the atmosphere of the new face created will be different.

<details> <summary>Show sample images</summary>

face margin

</details>
Size of the face when recreating

Specifies one side of the image size when creating a face image. If you are using the SDXL model, we recommend changing to 1024. For other models, there is usually no need to change from the default value (512), but you may see interesting changes if you do.

Ignore faces larger than specified size

Ignore if the size of the detected face is larger than the size specified in "Size of the face when recreating".

For more information, please see: here.

Upscaler

Select the upscaler to be used to scale the face image.

Step 3. Recreate the Faces

Denoising strength for face images (0.1-0.8)

Denoising strength for generating a new face. If the value is too small, facial collapse cannot be corrected, but if it is too large, it is difficult to blend with the entire image.

<details> <summary>Show sample images</summary>

strength: 0.4 strength 0.4

strength: 0.6 strength 0.6

strength: 0.8 strength 0.8

</details>
Tilt adjustment threshold (0-180)

This option defines the angle, in degrees, above which tilt correction will be automatically applied to detected faces. For instance, if set to 20, any face detected with a tilt greater than 20 degrees will be adjusted. However, if the "Adjust tilt for detected faces" option in the Face Editor section of the "Settings" tab is enabled, tilt correction will always be applied, regardless of the tilt adjustment threshold value.

Step 4. Paste the Faces

Apply inside mask only

Paste an image cut out in the shape of a face instead of a square image.

For more information, please see: here.

Step 5. Blend the entire image

Denoising strength for the entire image (0.0-1.0)

Denoising strength when inpainting to blend the new face with the whole image. If the border lines are too prominent, increase this value.


API

If you want to use this script as an extension (alwayson_scripts) in the API, specify "face editor ex" as the script name as follows:

   "alwayson_scripts": {
      "face editor ex": {
         "args": [{"prompt_for_face": "smile"}]
      },

By specifying an object as the first argument of args as above, parameters can be specified by keywords. We recommend this approach as it can minimize the impact of modifications to the software. If you use a script instead of an extension, you can also specify parameters in the same way as follows:

   "script_name": "face editor",
   "script_args": [{"prompt_for_face": "smile"}],

Workflow Editor

Workflow Editor is where you can customize and experiment with various options beyond just the standard settings.

Workflow Editor

Using the Workflow Editor UI

Workflow list and Refresh button

Workflow list and Refresh button

File name and Save button

Workflow list and Refresh button

Workflow definition editor and Validation button

Workflow definition editor and Validation button

Example Workflows

This project includes several example workflows to help you get started. Each example provides a JSON definition for a specific use case, which can be used as is or customized to suit your needs. To access these example workflows from the Workflow Editor, you need to enable the "Search workflows in subdirectories" option located in the Face Editor section of the "Settings" tab.

Settings

For more details about these example workflows and how to use them, please visit the workflows/examples/README.md.

Workflow Components (Inferencers)

In this project, the components used in the workflow are also referred to as "inferencers". These inferencers are part of the process that modifies the faces in the generated images:

  1. Face Detectors: These components are used to identify and locate faces within an image. They provide the coordinates of the detected faces, which will be used in the following steps.
  2. Face Processors: Once the faces are detected and cropped, these components modify or enhance the faces.
  3. Mask Generators: After the faces have been processed, these components are used to create a mask. The mask defines the area of the image where the modifications made by the Face Processors will be applied.

The "General components" provide the basic functionalities for these categories, and they can be used without the need for additional software installations. On the other hand, each functionality can also be achieved by different technologies or methods, which are categorized here as "Additional components". These "Additional components" provide more advanced or specialized ways to perform the tasks of face detection, face processing, and mask generation.

In this project, the components used in the workflow are also referred to as "inferencers". These inferencers fall into three functional categories: Face Detectors, Face Processors, and Mask Generators.

Note: When using "Additional components", ensure that the features you want to use are enabled in the "Additional Components" section of the "Settings" tab under "Face Editor". For detailed descriptions and usage of each component, please refer to the corresponding README.

General components

Additional components

Workflow JSON Reference

Rules are processed in the order they are specified. Once a face is processed by a rule, it will not be processed by subsequent rules. The last rule can be specified with then only (i.e., without when), which will process all faces that have not been processed by previous rules.


Settings

In the "Face Editor" section of the "Settings" tab, the following settings can be configured.

"Search workflows in subdirectories"

Overview
"Search workflows in subdirectories" is a setting option that controls whether Face Editor includes subdirectories in its workflow search.

Value and Impact
The value is a boolean, with either True or False to be specified. The default value is False, which indicates that the search does not include subdirectories. When set to True, the workflow search extends into subdirectories, allowing for the reference of sample workflows.


"Additional components"

Overview
"Additional components" is a setting option that specifies the additional components available for use in Face Editor.

Value and Impact
This setting is a series of checkboxes labeled with component names. Checking a box (setting to Enabled) activates the corresponding component in Face Editor.


"Save original image if face detection fails"

Overview
"Save original image if face detection fails" is a setting option that specifies whether to save the original image if face detection fails.

Value and Impact
The value is a boolean, with either True or False to be specified. The default value is True, which means that the original image will be saved if face detection fails.


"Adjust tilt for detected faces"

Overview
"Adjust tilt for detected faces" is a setting option that specifies whether to adjust the tilt for detected faces.

Value and Impact
The value is a boolean, with either True or False to be specified. The default value is False, indicating that no tilt correction will be applied when a face is detected. Even when "Adjust tilt for detected faces" is not enabled, the tilt correction may still be applied based on the "Tilt adjustment threshold" setting.


"Auto face size adjustment by model"

Overview
"Auto face size adjustment by model" is a setting option that determines whether the Face Editor automatically adjusts the size of the face based on the selected model.

Value and Impact
The setting is a checkbox. When checked (enabled):


"The position in postprocess at which this script will be executed"

Overview
"The position in postprocess at which this script will be executed" is a setting option that specifies the position at which this script will be executed during postprocessing.

Value and Impact
The value is an integer, with a value from 0 to 99 to be specified. A smaller value means that the script will be executed earlier. The default value is 99, which indicates that this script will likely be executed last during postprocessing.


Contribution

We warmly welcome contributions to this project! If you're someone who is interested in machine learning, face processing, or just passionate about open-source, we'd love for you to contribute.

What we are looking for:

Before starting your contribution, please make sure to check out our existing code base and follow the general structure. If you have any questions, don't hesitate to open an issue. We appreciate your understanding and cooperation.

We're excited to see your contributions and are always here to help or provide guidance if needed. Happy coding!