You type "a lighthouse on a cliff at sunset, watercolour style" and a few seconds later you get an image that did not exist before. Behind the most popular image generators there is almost always a family of techniques called diffusion models. The principle is counter-intuitive but elegant: the model learns to create images by first learning to clean noise out of them.
The central idea: starting from noise
Imagine taking a photograph and adding a little digital "snow", like on old television sets. Then a bit more, and more again, until nothing of the original image remains: just random dots. This is the forward diffusion process, and it is easy to carry out because it requires no intelligence at all.
The interesting part is the reverse path. During training, a neural network is shown an image with a certain amount of noise and asked: which part of this is noise? By repeating the exercise on millions of images and at every level of disturbance, the network becomes very good at estimating the noise to remove.
Once trained, it can be used to generate: you start from pure noise and ask the network to remove a little disturbance, then a little more, for dozens of steps. Gradually shapes, colours and details emerge. The result is a new image, consistent with what the network has learned about images in general.
How text comes into play
If the process started from noise alone, you would get a plausible but random image. To control it you need a link between words and visual content.
- Text encoding. The description you write is turned into a numerical representation by a model trained to associate texts and images.
- Conditioning. At every denoising step, the network also receives this representation and uses it to steer the result: if the text mentions a lighthouse, the details that emerge will tend to form a lighthouse.
- Guidance. A parameter, often called the guidance scale, controls how closely the image must follow the text. High values follow the description more rigidly, sometimes at the expense of naturalness; low values leave more freedom.
Latent space: why it is so fast
Working directly on millions of pixels would be very heavy. Many systems therefore use a trick: they first compress images into a more compact representation, called latent space, using a component known as an autoencoder. The diffusion process takes place in this reduced space and only at the end is the result "decompressed" into a full-resolution image. This approach is known as latent diffusion, and it is why some models can even run on a home computer with a good graphics card.
The parameters you can control
| Parameter | What it does | Practical effect |
|---|---|---|
| Prompt | Describes the content | Subject, style, lighting, composition |
| Negative prompt | States what to avoid | Reduces unwanted elements |
| Number of steps | How many times noise is removed | More steps, more detail up to a point |
| Guidance | Adherence to the text | More rigid or more free |
| Seed | Starting point of the noise | Same seed and settings give similar images |
| Resolution | Image dimensions | Formats very different from training can cause defects |
The seed deserves a note: since everything starts from random noise, fixing the number that generates that noise lets you reproduce a result and change one element at a time, for example by altering a single word in the prompt.
Derived functions
The same mechanism makes it possible to do much more than generate from scratch:
- image to image: you start from an existing image with some noise added, instead of pure noise, to obtain a variation;
- inpainting: only a selected area is regenerated, for example to remove an object from a product photo;
- outpainting: the image is extended beyond its original borders;
- upscaling: plausible detail is added to a small image.
Why they get hands and lettering wrong
For a long time generators struggled with hands, text within images and exact numbers of objects. The reason is that the model learns visual statistical regularities, not rules: it knows hands have fingers but does not "count" to five; it knows signs contain letters but does not always compose correct words. Recent models have improved a great deal on these points thanks to better data and more powerful text encoders, but structured details remain a typical weak spot.
Points to bear in mind
Diffusion models learn from large collections of images, and the relationship between training and copyright is under discussion. For professional use it is advisable to read the service's terms on usage rights for generated images, avoid deliberately imitating the recognisable style of living artists and be transparent when an image is generated, as the AI Act also requires for synthetic content.
In short
- Diffusion models generate images by progressively removing noise, starting from pure noise.
- Text guides the process through its numerical representation at every step.
- Latent space makes the process lighter and faster.
- Prompt, steps, guidance and seed are the key parameters for controlling the result.
- Hands, lettering and counts remain the details where errors are most common.
