Skip to main content

ControlNet Main Endpoint

Overview​

You can now control Stable Diffusion with ControlNet. The ControlNet models are available in this API.

tip

You can also use this endpoint to inpaint images with ControlNet. Just make sure to pass the link to the mask_image in the request body and use the controlnet_model parameter with "inpaint" value.

info

Read our detailed blog article about the ControlNet advantages, before you dive in.

Request​

--request POST 'https://stablediffusionapi.com/api/v5/controlnet' \

Send a POST request to https://stablediffusionapi.com/api/v5/controlnet endpoint.

Body Attributes​

ParameterDescription
keyYour API Key used for request authorization
model_idThe ID of the model to be used. It can be public or your trained model.
controlnet_modelControlNet model ID. It can be from the models list or user trained.
controlnet_typeControlNet model type. It can be from the models list.
auto_hintAuto hint image;options: yes/no
guess_modeSet this to yes if you don't pass any prompt. The model will try to guess what's in the init_image and create best variations on its own. Options: yes/no
promptText prompt with description of required image modifications. Make it as detailed as possible for best results.
negative_promptItems you don't want in the image
init_imageLink to the Initial Image
control_imageLink to the Controlnet Image
mask_imageLink to the mask image for inpainting
widthMax Height: Width: 1024x1024
heightMax Height: Width: 1024x1024
samplesNumber of images to be returned in response. The maximum value is 4.
schedulerUse it to set a scheduler.
tomesdEnable tomesd to generate images: gives really fast results, default: yes, options: yes/no
use_karras_sigmasUse keras sigmas to generate images. gives nice results, default: yes, options: yes/no
algorithm_typeUsed in DPMSolverMultistepScheduler scheduler, default: none, options: dpmsolver+++
vaeuse custom vae in generating images default: null
lora_strengthuse different lora strengths default: null
lora_modelmulti lora is supported, pass comma saparated values. Example contrast-fix,yae-miko-genshin
num_inference_stepsNumber of denoising steps (minimum: 1; maximum: 50)
safety_checkerA checker for NSFW images. If such an image is detected, it will be replaced by a blank image.
embeddings_modelUse it to pass an embeddings model.
enhance_promptEnhance prompts for better results; default: yes, options: yes/no
multi_lingualUse different language then english; default: yes, options: yes/no
controlnet_conditioning_scaleBasically "guidance_scale". Scale for controlnet guidance. Accepts floating values from 0.1 to 5 (e.g. 0.5)
strengthPrompt strength when using init_image. 1.0 corresponds to full destruction of information in the init image.
seedSeed is used to reproduce results, same seed will give you same image in return again. Pass null for a random number.
webhookSet an URL to get a POST API call once the image generation is complete.
track_idThis ID is returned in the response to the webhook API call. This will be used to identify the webhook request.
upscaleSet this parameter to "yes" if you want to upscale the given image resolution two times (2x). If the requested resolution is 512 x 512 px, the generated image will be 1024 x 1024 px.
clip_skipClip Skip (minimum: 1; maximum: 8)
base64Get response as base64 string, pass init_image, mask_image , control_image as base64 string, to get base64 response. default: "no", options: yes/no
tempCreate temp image link. This link is valid for 24 hours. temp: yes, options: yes/no
tip

You can also use multi ControlNet. Just make sure to pass comma saparated controlnet models to the controlnet_model as "canny,depth" and init_image in the request body.

Models​

ControlNet API using Controlnet 1.1 as default: Suported controlnet_model:

  • canny
  • depth
  • hed
  • mlsd
  • normal
  • openpose
  • scribble
  • segmentation
  • aesthetic-controlnet
  • inpaint
  • softedge
  • lineart
  • shuffle
  • tile
  • face_detector
  • qrcode

Schedulers​

This endpoint also supports schedulers. Use the "scheduler" parameter in the request body to pass a specific scheduler from the list below:

  • DDPMScheduler
  • DDIMScheduler
  • PNDMScheduler
  • LMSDiscreteScheduler
  • EulerDiscreteScheduler
  • EulerAncestralDiscreteScheduler
  • DPMSolverMultistepScheduler
  • HeunDiscreteScheduler
  • KDPM2DiscreteScheduler
  • DPMSolverSinglestepScheduler
  • KDPM2AncestralDiscreteScheduler
  • UniPCMultistepScheduler
  • DDIMInverseScheduler
  • DEISMultistepScheduler
  • IPNDMScheduler
  • KarrasVeScheduler
  • ScoreSdeVeScheduler
  • LCMScheduler

Example​

Body​

Body Raw
{
  "key": "",
  "controlnet_model": "canny",
  "controlnet_type" :"canny",
  "model_id": "midjourney",
  "auto_hint": "yes",
  "guess_mode" : "no",
  "prompt": "a model doing photoshoot, ultra high resolution, 4K image",
  "negative_prompt": null,
  "init_image": "https://huggingface.co/takuma104/controlnet_dev/resolve/main/gen_compare/control_images/converted/control_human_openpose.png",
  "mask_image": null,
  "width": "512",
  "height": "512",
  "samples": "1",
  "scheduler": "UniPCMultistepScheduler",
  "num_inference_steps": "30",
  "safety_checker": "no",
  "enhance_prompt": "yes",
  "guidance_scale": 7.5,
  "controlnet_conditioning_scale": 0.5,
  "strength": 0.55,
  "lora_model": "more_details",
  "clip_skip": "2",
  "tomesd": "yes",
  "use_karras_sigmas": "yes",
  "vae": null,
  "lora_strength": null,
  "embeddings_model": null,
  "seed": null,
  "webhook": null,
  "track_id": null
}

Request​

var myHeaders = new Headers();
myHeaders.append("Content-Type", "application/json");

var raw = JSON.stringify({
  "key": "",
  "controlnet_model": "canny",
  "controlnet_type" :"canny",
  "model_id": "midjourney",
  "auto_hint": "yes",
  "guess_mode" : "no",
  "prompt": "a model doing photoshoot, ultra high resolution, 4K image",
  "negative_prompt": null,
  "init_image": "https://huggingface.co/takuma104/controlnet_dev/resolve/main/gen_compare/control_images/converted/control_human_openpose.png",
  "mask_image": null,
  "width": "512",
  "height": "512",
  "samples": "1",
  "scheduler": "UniPCMultistepScheduler",
  "num_inference_steps": "30",
  "safety_checker": "no",
  "enhance_prompt": "yes",
  "guidance_scale": 7.5,
  "strength": 0.55,
  "lora_model": "more_details",
  "tomesd": "yes",
  "use_karras_sigmas": "yes",
  "vae": null,
  "lora_strength": null,
  "embeddings_model": null,
  "seed": null,
  "webhook": null,
  "track_id": null
});

var requestOptions = {
  method: 'POST',
  headers: myHeaders,
  body: raw,
  redirect: 'follow'
};

fetch("https://stablediffusionapi.com/api/v5/controlnet", requestOptions)
  .then(response => response.text())
  .then(result => console.log(result))
  .catch(error => console.log('error', error));

Response​

{
  "status": "processing",
  "tip": "for faster speed, keep resolution upto 512x512",
  "eta": 146.5279869184,
  "messege": "Try to fetch request after given estimated time",
  "fetch_result": "https://stablediffusionapi.com/api/v3/fetch/13902970",
  "id": 13902970,
  "output": "",
  "meta": {
    "prompt": "mdjrny-v4 style a model doing photoshoot, ultra high resolution, 4K image",
    "model_id": "midjourney",
    "controlnet_model": "canny",
    "controlnet_type": "canny",
    "negative_prompt": "",
    "scheduler": "UniPCMultistepScheduler",
    "safetychecker": "no",
    "auto_hint": "yes",
    "guess_mode": "no",
    "strength": 0.55,
    "W": 512,
    "H": 512,
    "guidance_scale": 3,
    "seed": 4016593698,
    "multi_lingual": "no",
    "init_image": "https://huggingface.co/takuma104/controlnet_dev/resolve/main/gen_compare/control_images/converted/control_human_openpose.png",
    "mask_image": null,
    "steps": 20,
    "full_url": "no",
    "upscale": "no",
    "n_samples": 1,
    "embeddings": null,
    "lora": null,
    "outdir": "out",
    "file_prefix": "c8bb8efe-b437-4e94-b508-a6b4705f366a"
  }
}