Reference images (ControlNet)

Reference images (ControlNet) control the composition of what you generate: the result follows the reference’s pose, depth (what is near and far), line art or edges, while clothing and style follow the prompt.

Supported models

Reference images only work with SDXL-family models (Illustrious, NoobAI, Pony and so on). When the current model is not SDXL-family, reference images are not applied.

Opening the reference window

Click the “Reference” slot on the left of the generate panel to open the “Reference images (ControlNet)” window. Press “+ Add reference image” to add one; up to 3 can be used at once. Each one can be switched on or off, or removed.

Ways to choose a reference image:

• “From library” or “From computer”, or drag an image into the window.
• While viewing an image in the library or in Explore, press “Use as reference”.

Kinds

Kind	Use
Pose	Follows the person’s pose (body, hands, face direction)
Depth	Follows what is near and far; composition and outlines stay close to the reference
Line art	Redraws and colours the line art; the lines barely change
Canny	Follows edges; finer than line art and includes background edges
Soft edge	Follows softer edges; looser than line art
Tile	Follows the reference’s colours and composition; often used to redraw small or blurry images more clearly

“Reference type” tells CoCo what kind of image the reference is:

• A normal image: CoCo first extracts the pose, estimates depth or finds the lines.
• Already a pose map / depth map / line art: used as is. Choose this for your own line art or sketches (black lines on white).

Press “Preview” to check what was extracted; a pose map, for example, shows how many people were found. Very dynamic poses are sometimes extracted incompletely, so checking first is recommended.

Models

The “Models” area at the bottom left of the window downloads the models you need:

Model	Use	Size
union promax	One model for every kind; also used by “Smoother seams” in Refine	about 2.5 GB
DWPose	Extracts poses from normal images	about 350 MB
Depth Anything V2	Estimates depth from normal images	about 100 MB

ControlNet models already in your model folders are recognised automatically and listed under “Models in your model folders”. ControlNet models for SD 1.5 can’t be used and are marked as such.

When the pose isn’t applied

The union model’s “Pose” often has no effect on anime models such as Illustrious and NoobAI (depth and edges are fine). In that case use a dedicated pose model, for example NoobAI’s openpose ControlNet. When you choose a kind, CoCo prefers a model dedicated to that kind.

Settings

Item	Description
Strength	How strongly the reference applies; higher follows it more closely
Start (steps), End (steps)	The range of steps in which the reference applies. Applying it only early keeps the composition and leaves details to the model
When shapes differ	When the reference and output have different aspect ratios: “Crop to fill” or “Show whole”
Low threshold, High threshold	Canny only. Lower thresholds pick up more lines

Sending to Generate

When you press “Send to Generate” on an image made with reference images, the references and their model, kind, strength and step range come back too. Sending an image made without references turns off any references that are on in the generate panel (they aren’t deleted).

Memory

When no reference is on, ControlNet models aren’t loaded at all. They are loaded when used and released after generating; the pose and depth models unload automatically after 2 minutes idle.