A dream tool and a menace to the very existence of visual artists
Image created by Stable Diffusion
A new sector of digital art is pushing the frontiers of creativity and upending the way art is created. Artists create and give data to algorithms in order to train them to create innovative visual creations. They use computer systems that replicate the human mind to create an endless stream of one-of-a-kind artworks. As a desired partner in artistic production.
What is stable Diffusion ?
Image created by Stable Diffusion
Stable Diffusion is a machine learning model created by StabilityAI, EleutherAI, and LAION to produce digital pictures from natural language descriptions. The technique may also be used to generate image-to-image translations prompted by a text prompt. Yes, you read that correctly. You can make whatever form of ART you want with a simple a text prompt. The biggest attraction is that it is completely opensource and free to use.
How Stable Diffusion work ?
Image created by Stable Diffusion
The model was trained using the LAION Aesthetics dataset, a subset of the LAION 5B dataset that contained 120 million image-text pairings out of the whole set of almost 6 billion image-text pairs. LAION datasets are created to be publicly available in order to promote a democratised AI development environment. Stable Diffusion is said to operate on less than 10 GB of VRAM at inference time, creating 512x512 pictures in a matter of seconds, implying that it may be performed on consumer GPUs.
Essentially, the model learns to detect recognisable structures in a field of pure noise, then eventually focuses on those parts if they match the words in the prompt. To begin with, a person or group training the model collects photographs with information and creates a big data set. According to a recent review of the data set, many of the photographs are from sites like Pinterest, DeviantArt, and even Getty Images. As a result, Stable Diffusion has absorbed the styles of many contemporary artists, and some of them have strongly opposed the approach.
Image created by Stable Diffusion
The model is then trained on the image data set using a bank of hundreds of high-end GPUs. During the training phase, the model links words with images using a technique known as CLIP (Contrastive Language-Image Pre-training), which was devised and announced by OpenAI only last year.
An ISM utilising latent diffusion learns statistical connections regarding where specific coloured pixels normally belong in relation to one other for each subject during training. So it may not "understand" their relationship at a high level, but the outcomes can be astonishing and surprising, drawing inferences and style combinations that appear highly sophisticated. After training, the model never repeats any pictures from the original collection. Instead, it may build unique combinations of styles depending on what it has learnt.
How to get started with Stable Diffusion?
Stable Diffusion, as you might know, is open source and free to use. However, it requires a GPU with at least 10GB of VRAM. We can discuss how to set up and run Stable Diffusion locally, and how to run it on systems with limited VRAM shortly . For the time being, we will utilise Stable Diffusion from the Dream Studio API for free. There is no need to download any huge files because it can be launched inside a web browser. One disadvantage of using their cloud API is that there is a free use limit per account (because GPUs are costly). You may increase your use credits by purchasing a membership.
Step 1 : SIgn UP
Open the Dreamstudio api from here, and create an account. Intially you will get some free credits to use the api for free.
Step 2 : Set parameters.
When you verify and login using your email address, you will see the window displayed below.
On the right side of the screen, there are various menu bars with sliders that can be adjusted. You may change the width and height of the resulting picture, as well as the CFG scale (which determines how much the image will resemble your prompt. Higher values bring your image closer to your prompt), STEPS (How many steps you want to take to create your image). More steps, better image), NUMBER OF IMAGES, SAMPLER (Try them all and see which one fits you best), MODEL, and SEED
Please keep in mind that the slider values are directly proportional to the credit/picture, so spend wisely. You can view your current credit per image at the right top of the screen.
Step 3 : Start Dreamming
Now we come to the meat of the matter! You put your prompt in the text box at the bottom of the screen and then hit dream to make the image from your prompt. This will bring up the loading screen, and your image will gently diffuse into existence. The amount of time it takes to load the picture is determined by the parameters you provide in the menu bar. The higher the value, the longer it takes to load. You may save the image to your computer or hit dream again to generate a new one.
CONCLUSION
Image created by Stable Diffusion
While it's remarkable to be able to transform text into a picture, everyone should be aware that this model may create content that reinforces or exaggerates social prejudices. The model was trained using an unfiltered version of the LAION-400M dataset, which collected non-curated image-text-pairs from the internet (of course, the researchers performed some effort to remove unlawful information from the dataset), and is intended for research purposes. Keep this in mind while we work with these new and powerful tools.
I hope you found the article interesting. Please express your thoughts in the comments section below. Thank you.