Skip to main content

ControlNet Main Endpoint

Overview​

You can now control Stable Diffusion with ControlNet. All the 8 ControlNet models are available in this API.

tip

You can also use this endpoint to inpaint images with ControlNet. Just make sure to pass the link to the mask_image in the request body.

info

Read our detailed blog article about the ControlNet advantages, before you dive in.

Request​

--request POST 'https://stablediffusionapi.com/api/v5/controlnet' \

Send a POST request to https://stablediffusionapi.com/api/v5/controlnet endpoint.

Body Attributes​

ParameterDescription
keyYour API Key used for request authorization
model_idThe ID of the model to be used. It can be public or your trained model.
controlnet_modelControlNet model ID. It can be from the models list or user trained.
controlnet_typeControlNet model type. It can be from the models list.
auto_hintAuto hint image;options: yes/no
guess_modeSet this to yes if you don't pass any prompt. The model will try to guess what's in the init_image and create best variations on its own. Options: yes/no
promptText prompt with description of required image modifications. Make it as detailed as possible for best results.
negative_promptItems you don't want in the image
init_imageLink to the Initial Image
mask_imageLink to the mask image for inpainting
widthMax Height: Width: 1024x1024
heightMax Height: Width: 1024x1024
samplesNumber of images to be returned in response. The maximum value is 4.
schedulerUse it to set a scheduler.
tomesdEnable tomesd to generate images: gives really fast results, default: yes, options: yes/no
use_karras_sigmasUse keras sigmas to generate images. gives nice results, default: yes, options: yes/no
vaeuse custom vae in generating images default: null
lora_strengthuse different lora strengths default: null
lora_modelmulti lora is supported, pass comma saparated values (lora_model_id). Example contrast-fix,yae-miko-genshin
num_inference_stepsNumber of denoising steps (minimum: 1; maximum: 50)
safety_checkerA checker for NSFW images. If such an image is detected, it will be replaced by a blank image.
enhance_promptEnhance prompts for better results; default: yes, options: yes/no
guidance_scaleScale for classifier-free guidance (minimum: 1; maximum: 20)
strengthPrompt strength when using init_image. 1.0 corresponds to full destruction of information in the init image.
seedSeed is used to reproduce results, same seed will give you same image in return again. Pass null for a random number.
webhookSet an URL to get a POST API call once the image generation is complete.
track_idThis ID is returned in the response to the webhook API call. This will be used to identify the webhook request.

Models​

ControlNet available models:

  • canny
  • depth
  • hed
  • mlsd
  • normal
  • openpose
  • scribble
  • segmentation

Schedulers​

This endpoint also supports schedulers. Use the "scheduler" parameter in the request body to pass a specific scheduler from the list below:

  • DDPMScheduler
  • DDIMScheduler
  • PNDMScheduler
  • LMSDiscreteScheduler
  • EulerDiscreteScheduler
  • EulerAncestralDiscreteScheduler
  • DPMSolverMultistepScheduler
  • HeunDiscreteScheduler
  • KDPM2DiscreteScheduler
  • DPMSolverSinglestepScheduler
  • KDPM2AncestralDiscreteScheduler
  • UniPCMultistepScheduler
  • DDIMInverseScheduler
  • DEISMultistepScheduler
  • IPNDMScheduler
  • KarrasVeScheduler
  • ScoreSdeVeScheduler

Example​

Body​

Body Raw
{
  "key": "",
  "controlnet_model": "controlnet_model_id",
  "controlnet_type" :"canny",
  "model_id": "midjourney",
  "auto_hint": "yes",
  "guess_mode" : "no",
  "prompt": "a model doing photoshoot, ultra high resolution, 4K image",
  "negative_prompt": null,
  "init_image": "https://huggingface.co/takuma104/controlnet_dev/resolve/main/gen_compare/control_images/converted/control_human_openpose.png",
  "mask_image": null,
  "width": "512",
  "height": "512",
  "samples": "1",
  "scheduler": "UniPCMultistepScheduler",
  "num_inference_steps": "30",
  "safety_checker": "no",
  "enhance_prompt": "yes",
  "guidance_scale": 7.5,
  "strength": 0.55,
  "lora_model": "lora_model_id",
  "tomesd": "yes",
  "use_karras_sigmas": "yes",
  "vae": null,
  "lora_strength": null,
  "embeddings_model": null,
  "seed": null,
  "webhook": null,
  "track_id": null
}

Request​

var myHeaders = new Headers();
myHeaders.append("Content-Type", "application/json");

var raw = JSON.stringify({
  "key": "",
  "controlnet_model": "controlnet_model_id",
  "controlnet_type" :"canny",
  "model_id": "midjourney",
  "auto_hint": "yes",
  "guess_mode" : "no",
  "prompt": "a model doing photoshoot, ultra high resolution, 4K image",
  "negative_prompt": null,
  "init_image": "https://huggingface.co/takuma104/controlnet_dev/resolve/main/gen_compare/control_images/converted/control_human_openpose.png",
  "mask_image": null,
  "width": "512",
  "height": "512",
  "samples": "1",
  "scheduler": "UniPCMultistepScheduler",
  "num_inference_steps": "30",
  "safety_checker": "no",
  "enhance_prompt": "yes",
  "guidance_scale": 7.5,
  "strength": 0.55,
  "lora_model": "lora_model_id",
  "tomesd": "yes",
  "use_karras_sigmas": "yes",
  "vae": null,
  "lora_strength": null,
  "embeddings_model": null,
  "seed": null,
  "webhook": null,
  "track_id": null
});

var requestOptions = {
  method: 'POST',
  headers: myHeaders,
  body: raw,
  redirect: 'follow'
};

fetch("https://stablediffusionapi.com/api/v5/controlnet", requestOptions)
  .then(response => response.text())
  .then(result => console.log(result))
  .catch(error => console.log('error', error));

Response​

{
  "status": "processing",
  "tip": "for faster speed, keep resolution upto 512x512",
  "eta": 146.5279869184,
  "messege": "Try to fetch request after given estimated time",
  "fetch_result": "https://stablediffusionapi.com/api/v3/fetch/13902970",
  "id": 13902970,
  "output": "",
  "meta": {
    "prompt": "mdjrny-v4 style a model doing photoshoot, ultra high resolution, 4K image",
    "model_id": "midjourney",
    "controlnet_model": "canny",
    "controlnet_type": "canny",
    "negative_prompt": "",
    "scheduler": "UniPCMultistepScheduler",
    "safetychecker": "no",
    "auto_hint": "yes",
    "guess_mode": "no",
    "strength": 0.55,
    "W": 512,
    "H": 512,
    "guidance_scale": 3,
    "seed": 4016593698,
    "multi_lingual": "no",
    "init_image": "https://huggingface.co/takuma104/controlnet_dev/resolve/main/gen_compare/control_images/converted/control_human_openpose.png",
    "mask_image": null,
    "steps": 20,
    "full_url": "no",
    "upscale": "no",
    "n_samples": 1,
    "embeddings": null,
    "lora": null,
    "outdir": "out",
    "file_prefix": "c8bb8efe-b437-4e94-b508-a6b4705f366a"
  }
}