Stable Diffusion Checkpoint Merger: What It Is and How to Use It

Since its release in 2022, Stable Diffusion has proved to be a reliable and effective deep learning and text-to-image generation model. It helps artists, designers, and even amateurs to generate original images using simple text descriptions. Over time, the Stable Diffusion artificial intelligence (AI) art generator has significantly advanced, introducing new and progressive checkpoints that can be merged to generate better AI images.

To effectively use the new Stable Diffusion model to generate high-quality images for your work, you must understand what the Stable Diffusion checkpoint merger is and how it’s used. This article explains important aspects of the Stable Diffusion checkpoint merger.

What Is Stable Diffusion?

Stable Diffusion is an AI art generation model developed in 2022 by CompVis Group LMU Munich, Runway, and Stability AI to enable artists to generate photorealistic images using text prompts. This latent diffusion model is a deep-generative neural network that runs on most consumer hardware fitted with a standard GPU with a minimum of 8 GB VRAM. You don’t have to use the previous proprietary text-to-image models like DALL-E and Midjourney, which were only accessible through the cloud.

The latent diffusion model (LDM) used by this art generator is trained for the purpose of removing successive applications of Gaussian noise on training images, visualized as a series of denoising autoencoders. This AI art generator has three main parts: the U-Net, a variational autoencoder, and an elective text encoder.

The variational autoencoder compresses the images from pixels to smaller dimensional latent spaces that capture each image's most fundamental semantic implications. Recently, Stable Diffusion introduced the checkpoint merger function to its models to help you merge two checkpoints for better image generation.

The new addition allows you to merge different models to generate images that match your desired results. You don’t have to rely on the integrated Stable Diffusion models to generate a merger; you can now use your own trained models like the Stable Diffusion SDXL (Stable Diffusion extra-large) model from NightCafe. Although SDXL isn’t available publicly, it has proved to be more effective than previous models.

What Is a Stable Diffusion Checkpoint Merger?

Stable Diffusion checkpoint merger is a fairly new function introduced by Stable Diffusion to allow you to generate multiple mergers using different models to refine your AI images. With this function, you can merge up to three models, including your own trained models. Once you have merged your preferred checkpoints, the final merger will be generated in the checkpoint directory.

Stable Diffusion offers several unique checkpoints that you can merge with this function. You need to be aware of these checkpoints so that you can pick the right ones for your AI image generation needs. Here are the most popular Stable Diffusion checkpoints you should be familiar with.

Stable-Diffusion-V1-1

This is the first version of the four model checkpoints released by Stable Diffusion. Stable Diffusion developers are still developing other higher versions of checkpoints to improve your model training and image generation experience. Stable-diffusion-v1-1 checkpoint is prepared randomly and has been trained on over 237,000 steps at a resolution of 256x256 on laion2B-en.

Stable-Diffusion-V1-2

This checkpoint continued training from the previous version. So far, it has been trained on over 515,000 steps at a resolution of 512x512 on laion-improved-aesthetics—a subset of laion2B-en.

Stable-Diffusion-V1-3

This checkpoint continued training from the stable-diffusion-v1-2 version. It has been trained on 195,000 steps at a resolution of 512x512 on laion-improved-aesthetics. Another notable thing about this checkpoint is the 10% reduction of text-conditioning to improve classifier-free guidance sampling.

Stable-Diffusion-V1-4

This checkpoint continued training from stable-diffusion-v1-2 and so far it has been trained on 195,000 steps at a resolution of 512x512 on laion-improved-aesthetics. This checkpoint has also reduced text-conditioning by 10% to enhance classifier-free guidance sampling.

Stable-Diffusion-V1-4

This checkpoint started training from stable-diffusion-v1-2 and it has been trained on 225,000 steps at a resolution of 512x512 on laion-aesthetics v2 5+. It has also dropped 10% of text-conditioning to advance classifier-free guidance sampling.

How the Stable Diffusion Checkpoint Merger Works

Like the Stable Diffusion prompt matrix, the Stable Diffusion checkpoint merger allows you to generate photo-realistic AI images that suit your artistry needs by combining different checkpoints to create the exact images you need. Although Stable Diffusion models are trained on almost every aspect of image generation, they aren’t perfect. This is why you must merge different models—including your own trained models—to generate the desired images.

The most straightforward way to merge two Stable Diffusion checkpoints (and work on your own model training) is to use one of the the third-party interfaces that are available. Stability AI doesn’t endorse or promote any of these interfaces or the custom models resulting from them, so it’s going to be a matter of trial and error. One of the most popular at the time of this writing is AUTOMATIC1111. 

Closing Thoughts

In short, the Stable Diffusion checkpoint merger allows you to utilize two or more of your preferred models to generate high-quality and personalized images. Each Stable Diffusion model has its own strengths and weaknesses, and merging them can help to mitigate their limitations and enhance their strengths.