How to generate AI Avatars with Stable Diffusion API?

Artificial intelligence (AI) has revolutionized the way we interact with technology, and one of the most exciting developments in recent years is the use of AI to create personalized avatars. Avatars are digital representations of individuals, used in gaming, social media, and virtual reality. With AI, we can create avatars that resemble us and mimic our facial expressions and movements in real-time. This technology has the potential to transform the way we communicate and connect with each other online. In this article, we will explore the process of creating your own avatars with AI.

Generating your AI avatars

To generate your own AI avatars you need to have 7-10 images of yourself. You also need to have an image-generation model to finetune it with your images. Once you finetune the model you can get your images with a prompt. 

To perform all the above steps we will use the Stable Diffusion API to finetune a model and we will use the images of Elon musk, which are publicly available, to show this demo. To know more about the fine-tuning process refer to the documentation. 

You need to have 7-10 images that do not contain any background objects except the face. Each image must be cropped to 512x512 resolution.  I have created 8 images of Elon musk that have a resolution of 512x512. You can find those images here at this link.

Now, all we need to do is use these images and fine-tune an image generation model using the Stable Diffusion API. For this, you will need the API key which you can get by signing up for free. The following is the python code for sending a fine-tune request to the API endpoint. 

import requests

url = "https://stablediffusionapi.com/api/v3/fine_tune"

payload = {"key": "Your API key","instance_prompt": "photo of elon person","class_prompt": "photo of person","base_model_id" : "portraitplus-diffusion","images": ["https://i.imgur.com/nlh8rkE.png","https://i.imgur.com/APVCHAT.png","https://i.imgur.com/tzYiWkm.png","https://i.imgur.com/CXOXsDV.png","https://i.imgur.com/nuGg51i.png","https://i.imgur.com/egaCDD9.png","https://i.imgur.com/JBD4e22.png","https://i.imgur.com/tDeAZMi.png"],"seed": "0","training_type": "men","max_train_steps": "900"}
headers = {}
response = requests.request("POST", url, headers=headers, data=payload)
print(response.text)

In the above code, we imported the requests module and gave the URL of the API endpoint. In the next step, we gave the input parameters to the API. Here you need to define the class prompt and instance prompt. 

The class prompt is used to generate the regularization images. If you are training with images of a person, then the class prompt would be ‘photo of a person’. The instance prompt is what you will use for the pics of yourself. So the instance prompt will always be ‘photo of xyz person’. In the current demo of generating Elon musk avatars, the instance prompt is ‘photo of elon person’. After fine-tuning the model, to generate the images, we will use the prompts that start with ‘photo of elon person…’. In this way, our AI avatars are generated. 

After this, you have to choose the base model id. The base model is the model that we will finetune. There are many models available for fine-tuning on Stable Diffusion API. You can choose one model. In this case, I chose potraitplus-diffusion. After this, we give the training type as men since we are training for male images. Also, we give the training steps as 900. In the next step, we send a response request to the API to start the fine-tuning process. The output of the response request looks like below.

{
    "status": "success",
    "messege": "deploying_gpu",
    "data": "it will take upto 45 minutes.",
    "training_id": "your training id",
    "model_id": "your model id"
}

You can see the model_id and training_id in the above output. You can use this two to generate images and check the training status of your model. The following is the code for checking the model training status. 

import requests

url = "https://stablediffusionapi.com/api/v3/fine_tune_status/your_training_id"

payload = {"key": "Your API key}
headers = {}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

Once your model training is completed, you can use this trained model to generate your avatars using the dreambooth API provided by the Stable Diffusion API. The following is the code to use the dreambooth API for generating your avatars with your trained model. 

import requests

url = "https://stablediffusionapi.com/api/v3/dreambooth"

payload = {"key": "Your API key", "model_id": "Your_model_Id", "prompt": "a photo of elon person, blue eyes, shaved side haircut, hyper detail, cinematic lighting, magic neon, dark red city --ar 9:16 --upbeta, Canon EOS R3, nikon, f/1.4, ISO 200, 1/160s, 8K, RAW, unedited, symmetrical balance, in-frame, 8 k",  "negative_prompt": "((out of frame)), ((extra fingers)), mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), (((tiling))), ((naked)), ((tile)), ((fleshpile)), ((ugly)), (((abstract))), blurry, ((bad anatomy)), ((bad proportions)), ((extra limbs)), cloned face, (((skinny))), glitchy, ((extra breasts)), ((double torso)), ((extra arms)), ((extra hands)), ((mangled fingers)), ((missing breasts)), (missing lips), ((ugly face)), ((fat)), ((extra legs)), anime","width": "512","height": "512",   "samples": "1","num_inference_steps": "30","guidance_scale": 7.5}
headers = {}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

In this case, we generated an avatar of Elon musk which looks like the one below.

In this way, you can create your own avatars using the API endpoints offered by Stable Diffusion API.

Conclusion

In conclusion, AI-powered avatar creation is a rapidly evolving field with enormous potential for revolutionizing the way we interact online. From video games to virtual meetings, personalized avatars can offer a more immersive and engaging experience that feels more natural and authentic than traditional interfaces. As AI algorithms continue to improve and become more widely available, we can expect to see an increasing number of applications for this technology in various fields. Whether it's for social media, education, healthcare, or entertainment, AI-powered avatars are sure to become an essential part of our digital lives in the years to come.