Stable Diffusion Prompt Guide

The Stable Diffusion AI generator is a free, open-source text-to-image conversion tool that instantly creates stunning graphics. The model extracts images from the LAION-5B dataset and is created by CompVis, Stability Al, and RunwayML.

When creating AI images, it is important to know the best prompts to use to help AI models produce expected results. AI art generator sites can vary, so you may need to experiment. 

Here are the basics on how to use Stable Diffusion to help you get started:

What Are Prompts?

Prompt engineering is the process of structuring words so that a text-to-image model can interpret and understand them. Think of it as the language you need to teach your AI model what to draw.

Prompts help machine-learning models understand your ideas. Prompts can be as simple as one-word descriptions or as complex as a specific image using multiple phrases. Every AI generator is pre-trained using a dataset composed of millions of text-image pairs. It uses the caption-matching method to develop an image based on specific instructions.

If you want to create great artwork, you need to provide great prompts with the right keywords. The right words can mean the difference between a picture that looks like something a toddler drew and a perfect rendering of the picture that you had in mind.

The more specific your prompts, the better the chances are that AI will depict an accurate image. For instance, asking AI to generate a cat will produce a lot of generic images. If you have something specific in mind, you might indicate a Persian cat. 

What Should You Include in a Prompt?

Here are some ideas to use in your prompts, but your options are nearly endless:

  • Subject: Person, animal, landscape
  • Verb: What the subject is doing, such as standing, sitting, eating
  • Adjectives: Beautiful, realistic, big, colourful
  • Environment/Context: Outdoor, underwater, in the sky, at night
  • Lighting: Soft, ambient, neon, foggy
  • Emotions: Cosy, energetic, romantic, grim, loneliness, fear
  • Artist inspiration: Pablo Picasso, Van Gogh, Da Vinci, Hokusai
  • Art medium: Oil on canvas, watercolour, sketch, photography
  • Photography style: Polaroid, long exposure, monochrome, GoPro, fisheye, bokeh
  • Art style: Manga, fantasy, minimalism, abstract, graffiti
  • Material: Fabric, wood, clay
  • Colour scheme: Pastel, vibrant, dynamic lighting
  • Computer graphics: 3D, octane, cycles
  • Illustrations: Isometric, pixar, scientific, comic
  • Quality: High definition, 4K, 8K, 64K

Sample Prompt

Prompts for AI imaging are usually created in a specific structure: (Subject), (Action, Context, Environment), (Artist), (Media Type/Filter). For example, a prompt might look something like this: “An oil painting of a dalmatian wearing a tuxedo, outdoor, natural light, Da Vinci.”

The order in which the words are written is essential to getting the desired output–the words at the beginning of your prompt are weighted more than the others. You should list your description and concepts explicitly and separate each with a comma rather than create one long sentence.  

For instance, these prompts will have different outcomes: 

  • “A dog sitting on a Martian chair.”
  • “A dog sitting on a chair on Mars.”
  • “A dog sitting on a chair. The chair is on Mars.”

Negative Prompts 

Negative prompts can be added to tell the Stable Diffusion AI what it should not show. This feature often removes elements from the initially created image that the user does not want to see.

Other Important Settings

Finding the right prompt is the hardest part of using Stable Diffusion, but a few other parameters can make a big difference to your results.

Context-free Grammar (CFG) determines how carefully Stable Diffusion will follow your prompt. A higher number means the image generator will follow suggestions more closely, while a lower number allows for more freedom in how your instructions are interpreted. Adjust this setting as necessary.

The sampling method reduces noise from an image into more recognizable shapes. Popular choices of sampling methods include PLMS, k_LMS, and Euler_a.

Note that Stable Diffusion might sometimes run into memory issues, and you may encounter the CUDA out-of-memory error message. If this happens, you can restart your computer or reduce the image resolution.

In Summary

AI is one of the most exciting new technologies and is transforming the creative industry. It can build unique artwork solely based on text input. The potential for AI art to enhance aesthetic art is remarkable. It lets you discover new perspectives on how you see yourself and the world around you.

Ready to create some AI magic? Get started with a free account from NightCafe and let your imagination run wild.