Get Started With Disco Diffusion to Create AI Generated Art

Mechanical arm with a paint brush and a canvas by Li Shuxing and Tyler Edlin
Mechanical arm with a paint brush and a canvas by Li Shuxing and Tyler Edlin

Disco Diffusion is a free tool that you can use to create AI generated art. You can create machine learning generated images and videos with it.

It’s a software written in Python, and meant to be run in a Google Colab notebook.

If this things like Google Colab are unfamiliar to you, that’s ok. You’ll quickly get the hang of it. We don’t need to touch most buttons or code. All we want now is to get you to generate your first image with minimal work.

This tutorial is meant to give you a very beginner-friendly quick start into using Disco Diffusion to generate images.

Additionally, to quickly get familiar with it you can check out our beginner Google Colab tutorial.

Note: There is a similar software available, called Stable Diffusion, that can generate amazing AI generated art very fast, and has a similar setup. You can check out our basic guide on Stable Diffusion and our guide to using Stable Diffusion with a user interface. Another article you might find useful is our ranking of the 4 best AI art generators.

tl;dr AI News Section Preview
tl;dr AI News Section Preview

Sidenote: AI art tools are developing so fast it’s hard to keep up.

We set up a newsletter called tl;dr AI News.

In this newsletter we distill the information that’s most valuable to you into a quick read to save you time. We cover the latest news and tutorials in the AI art world on a daily basis, so that you can stay up-to-date with the latest developments.

Check tl;dr AI News

Getting Started with Disco Diffusion

In this guide we’ll generate AI art with Disco Diffusion, specifically images, as fast and easy as possible.

We won’t tweak the available settings for this run – you can learn about the various settings later. This way you can see how to run it, and see what it does, as fast as possible. That should break the ice in case you’re intimidated at first.

You can also generate videos with Disco Diffusion. However in this tutorial we’ll just generate images.

Quick Video Demo

Here’s a quick video demo of using Disco Diffusion for the first time, in which we’re going from start to finish as quickly as we can.

Step 1: Open & Copy the Disco Diffusion Colab Notebook

Go here to open the Disco Diffusion 5.61 Colab Notebook: https://colab.research.google.com/github/alembics/disco-diffusion/blob/main/Disco_Diffusion.ipynb

Next go in the Google Colab menu to File > Save a copy in Drive to save the notebook into your Google Drive.

Saving Disco Diffusion Google Colab in Drive
Google Colab menu to File > Save a copy in Drive

Your copy of the Disco Diffusion notebook will open in a new tab, with the name Copy of Disco Diffusion [..]. This is the one we’ll be using. You can close the old tab.

At some point you’ll probably see a popup notification saying Notebook requires high RAM. This is just a notification and it’s nothing to worry about. Click OK when you see it.

Make Sure Google Colab is Using GPU

To run Disco Diffusion in Google Colab we need Colab to use a GPU. To make sure we’re using one, in the menu click on Runtime > Change runtime type.

click on runtime > change runtime type
Runtime > Change Runtime type

A little popup will appear. In the Hardware accelerator dropdown make sure GPU is selected.

Change Runtime Type Modal Window
Hardware Accelerator set to GPU

Click Save. Now we’re ready.

Step 2: Run Check GPU Status

Running this cell will check the GPU we’re allocated. Google Colab usually allocates GPUs randomly, depending which is available, and some GPUs are better than others.

Ideally we want a Tesla P100 or Tesla T4. Along with the default settings for Disco Diffusion and a T4, you should get an image in ~15 minutes.

If you use a Tesla K80 it will take much longer to generate an image.

How to Get a Tesla P100 or T4 in Google Colab

To change your assigned GPU you’ll can restart your Google Colab session until you get the one you like.

Video Demo

Steps:

  1. To do this, towards the upper right corner of Colab, where you can see RAM and Disk, click the little downward arrow > Manage Sessions > Terminate. Alternatively you can do the same thing by clicking in the Colab menu, in the upper left, on Runtime > Manage Sessions > Terminate. This terminated your current session.
  2. Next click to run the 1.1 Check GPU Status cell. This should start a new session.
  3. Do this until you get a Tesla P100 or T4.

Step 3: Connect to Google Drive

The next cell is called 1.2 Prepare Folders.

Run it, and it will create a popup window asking you to select a Google account whose Google Drive you’d like to use, because Disco Diffusion needs to setup some folders and download some files.

You’ll have to select your Google account that you’d like to use, and then click to allow certain permissions for Google Colab.

Step 4: Run Everything Else Until “Prompts”

Next we can run every other cell until we get to the Prompts cell.

It should take about 1-2 minutes for Disco Diffusion to do its thing, and download the files it needs.

Step 5: Write our Text-to-Image Prompt

Next we can write our prompt that Disco Diffusion will use to generate the image.

By default you’ll see these prompts. You can learn more about how to write prompts later.

text_prompts = {
    0: ["A beautiful painting of a singular lighthouse, shining its light across a tumultuous sea of blood by greg rutkowski and thomas kinkade, Trending on artstation.", "yellow color scheme"],
    100: ["This set of prompts start at frame 100","This prompt has weight five:5"],
}

If you’d like an example of a prompt likely to result in a nice image, you can use something like this:

text_prompts = {
    0: ["A beautiful fantasy artwork of a celestial cyberpunk castle by li shuxing, trending on artstation"],
}
Google Colab Prompts Cell
Example Prompt: “A beautiful fantasy artwork of a celestial cyberpunk castle by li shuxing, trending on artstation”

Some Prompt Examples with Results

Here are a few other examples of prompts and resulting images. You can use those prompts as inspiration. We’ll mention more resources for inspiration at the end of this article.

When you’ve finished writing your prompt you can run the cell.

Step 6: Do the Run! – Generate Your Image

We will explain two settings before finally running the cell to create the image.

display_rate: this is so you can see the progress on your image. By default it’s set to 25, which means you’ll see the image progress every 25 steps (you’ll learn about steps later).

You can just leave it to 25 or set it to something less if you’d like. If you’ve got a Tesla T4 GPU and default other settings, with 25 you’ll probably see the image progressing every 1-2 minutes, as you can see in the video demo from earlier.

n_batches: These are the number of images you want Disco Diffusion to create. If you leave 50, it will continue generating variations of your prompt.

As you can see I’ve changed it to 1, so it generates one image and it stops, so I can change the prompt and run it again, and so on. I recommend you change it to 1 for your first run.

Do the Run
“Do the Run!” Cell

Step 7: Wait For Image to Generate

Now all we have to do is wait until our image is finished. You can do something else in the meantime.

Step 8: Image is Finished Generating

This is my first result, as you can see in the video demo as well.

A beautiful house in the woods by Thomas Kinkade
A beautiful house in the woods by Thomas Kinkade

You can right click to save it to your computer, but you can also see it in your Google Drive.

Where Are Images Located

They’re located in your Google Drive. When you connected Google Drive earlier, the Disco Diffusion Colab notebook set up some folders in your Drive.

You should a folder called AI in your Drive. In it you have everything Disco Diffusion set up.

Our images, in my case, are located in AI > Disco_Diffusion > images_out > TimeToDisco. TimeToDisco is our batch_name that we didn’t configure. You can change that later, as it helps with categorizing experiments.

Access Google Drive from Google Colab

You can access Google Drive from Google Colab.

Google Drive in Google Colab Sidebar
Google Drive in Google Colab Sidebar

The TimeToDisco(0)_0.pngTimeToDisco(0)_8.png are the images I generated. I let it generate multiple images using the same prompt and stopped it after TimeToDisco(0)_8.png. You can double click any of them and Google Colab will display them.

The TimeToDisco(0)_settings.txt are your entire configuration that you used. You can double click it to see the text file to see its contents. Disco Diffusion conveniently saves settings for every run you make so you can share them, or come back to them if you had a good result and replicate or improve future experiments.

To download any of the files just right click them and click on Download.

Access Images Directly from Google Drive Interface

You can also easily access images in your Google Drive interface:

timetodisco gdrive

Very Useful Resources

The community is still growing and experimenting with machine learning generated images. These are some excellent resources that I have learned from so far and I recommend you check out.

  • Image Generation with CLIP + Diffusion models (Disco Diffusion 4.1) [Youtube] – This is the first tutorial I ever saw and it gave me the basic overview of the Disco Diffusion settings. I highly recommend it. Even if they use version 4.1, it’s still an excellent resource.
  • Get started quickly with Disco Diffusion v5 ~ Generative AI Art – The first tutorial I ever read on Discord Diffusion. It presents something similar to this article but maybe you’ll find it explains some things better than I did.
  • /r/DiscoDiffusion Subreddit – Other Disco Diffusion users post here all the time, and you can ask them their prompts and settings. You can also just ask for help in general. They’re very helpful.
  • Zippy’s Disco Diffusion Cheatsheet v0.3 – This Google Doc is epic. It’s beautifully written,. It presents every setting for Disco Diffusion in an easy to understand manner. Don’t be intimidated by the length of the document. I initially expected it to be a dry read but it isn’t. The writer is humble in mentioning they’re still trying to understand Disco Diffusion. This will make everything clearer for you.
  • Disco Diffusion 70+ Artist Studies by weirdwonderfulai.art – How mentioning an artist in your prompt will impact your generated art. This person did an amazing job centralizing samples of generated art for 560+ Artists, and how they can affect the outcome of a generated image. These are contributions made by many others experimenting with generating art and submitting their finds. You can do that too.
  • Disco Diffusion Modifiers by weirdwonderfulai.art – Modifiers, like artist names, are keywords that guide the image generation in a certain direction. This is another amazing job at centralizing a valuable resource in a single place for all of us to use and experiment.
  • Getting Started with Disco Diffusion by weirdwonderfulai.art – another great tutorial on getting started with Disco Diffusion
  • Anything Punk Modifiers for AI Art by weirdwonderfulai.art – Steampunk, Cyberpunk, Biopunk, and others. These are all easy wins to write prompts for beautiful results.
  • Twitter and Instagram. You can search for #aiart, #aiartcommunity or #generatedart and you’ll most likely find others posting Disco Diffusion generated art. You can ask them their prompts and settings, and they’ll probably ask you yours.
  • CLIP Prompt Engineering for Generative Art – an excellent and very insightful article to help understand how to create prompts. I can’t recommend it enough.
  • Tools and Resources for AI ArtConstantly updated, extremely useful and fun list of tools and resources to create AI art. Maintained by @pharmapsychotic, they regularly test AI art tools, post amazing results on Twitter and update the list with a link to the tool and a short description. Highly recommend it!
  • A Traveler’s Guide to the Latent Space – Another fantastic guide, and one of the best guides, that tries to give you a beginner friendly explanation on Disco Diffusion, so as to give you an intuition on how to generate beautiful AI generated art. It gives you lots of additional useful tips that other guides don’t cover. I know I say this often, but I highly recommend it!
  • Disco Diffusion on Replicate.com – Replicate.com is a website that allows you to run software like Disco Diffusion for free right now. It requires no set up and it’s a great way to try out the latest machine learning models in the cloud. After some time it requires you to sign up, however they are very generous with the amount of free usage you get, in my opinion.
  • Want to learn A.I. Art? – Here’s THE ULTIMATE Disco Diffusion Tutorial (Video) – A very detailed and beginner friendly tutorial on Disco Diffusion. They explain the every feature extremely well. It’s 40 minutes long and not tedious at all. I highly recommend it!

Troubleshooting

There are some well known issues when using certain configurations in Disco Diffusion.

RuntimeError: CUDA error: misaligned address

Full Error Message
RuntimeError: CUDA error: misaligned address CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

This is likely happening because you’re trying to run Disco Diffusion with a Tesla T4 GPU while using the ViTL14 model.

ViTL14 can significantly improve your images under certain configurations, but T4 doesn’t work with ViTL14 in Colab for some reason, and many others have this issue.

If you want to use ViTL14 make sure to use Tesla P100.

Conclusion

Hopefully this article helped you get started with Disco Diffusion to create “AI” generated images. If you have any feedback please feel free to leave a comment and I’ll answer as fast as I can.

  • 1
    A Jupyter Notebook is a web-based environment for interactive computing.

    The name, Jupyter, comes from the core supported programming languages that it supports: Julia, Python, and R.

    Jupyter Notebooks have become very popular in the last few years, and for good reason. They allow you to create and share documents that contain live code, equations, visualizations and narrative text.

    Use of Jupyter Notebooks has exploded in recent years. They are now an essential tool in many different fields, including data science, machine learning, and artificial intelligence.
3 Shares:
Subscribe
Notify of
guest
Receive notifications when your comment receives a reply. (Optional)
Your username will link to your website. (Optional)

32 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
B Secret
B Secret
1 year ago

Thanks for this!

zero
zero
1 year ago

what if i dont have a gpu?

eNFThusiast
1 year ago

Hey, a silly question maybe.
I want to create a unique collection of, let’s say, 2K items.
Is there any easier way to accomplish this task?
I mean like I just enter the total supply amount like 2K and then I enter my prompt for that, get my settings ready for resolution and etc. and when I run it, I get 2K images, all generated using the same single prompt but all unique.
You know a link or any other ideas how I can accomplish this?

colin brown
colin brown
1 year ago

Can it add some reference pictures to generate image? Just want to give it some reference in order not to let it deviate too much from my expectations

Stef
Stef
1 year ago

Hi, thanks for the tutorial. It worked fine. I saved a bookmark to my notebook.

Unfortunately, I get an error, when I try to use Disco Diffusion again.

Screenshot 2022-08-02 072738.png
Stef
Stef
1 year ago
Reply to  EdXD

Thanks for the quick response.

This is the error message I get.

Screenshot 2022-08-02 094152.png
David
1 year ago

Thanks for this!! The only thing is that I’m getting a RuntimeError. Can you help me with it?

Captura de Pantalla 2022-08-13 a las 12.23.42.png
David
1 year ago
Reply to  EdXD

Hi! Thanks for the quick reply. I just run everything again and got the same error… 🙁

Max
Max
1 year ago

Diffusion and CLIP model settings loads indefinitely. Anyone any suggestions?

Max
Max
1 year ago
Reply to  EdXD

To provide some screenshots, I did run it again, and it seems to work now, but I got the same problem as David from 5 days ago. Checked and tried everything you said below, nothing worked for me, too.

Tilopa
Tilopa
1 year ago

Thanks for the tutorial. However, I am getting this error

ds.JPG
Pablo Ciarlante
1 year ago

Hey there! thanks for the useful information. Any tips on how to generate better figures such as bodies or faces?
Do the clips settings affect this somehow?

Pablo Ciarlante
1 year ago
Reply to  EdXD

I hadn’t seen that option! thank you very much for the quick reply, I’ll try it out!
I’m having an issue unrelated to this. When trying to create images in Full HD (1920×1080), there appears to be an error that says I ran out of Memory for that run. Is it because using the unpaid version of Google Collab limits the amount of resources available?

Pablo Ciarlante
1 year ago
Reply to  EdXD

Thank you so much for all the information, I’ll check out the links you shared!

Blaze
Blaze
1 year ago

Hi, I am running Disco Diffusion for the first time and I am running into this issue.

Please help!

Thank you!

dd.PNG
love you
love you
1 year ago

Hi, can you tell me how to use these parameters

捕获.PNG
Insanity
Insanity
1 year ago

First of All! Amazing work! Thank you very much. I am getting started in this was very very helpful. Can you tell me if Disco Diffusion is able to train through my artpieces? Like images I already have drawn for example? And do you have a Reference where i can find this for animation. I know you mentioned it in the beginning that this is a “only image” tutorial. But this would also be very helpful thanks 🙂

You May Also Like