Get Started With Disco Diffusion to Create AI Generated Art

Disco Diffusion is a free tool that you can use to create AI generated art. You can create machine learning generated images and videos with it.

It’s a software written in Python, and meant to be run in a Google Colab notebook.

[powerkit_alert type=”info” dismissible=”false” multiline=”false”]
Google Colab is a free cloud service hosted by Google to encourage Machine Learning and Artificial Intelligence research, where often the barrier to learning and success is the cost of hardware.

Google Colab provides a Jupyter Notebook¹

A Jupyter Notebook is a web-based environment for interactive computing.

The name, Jupyter, comes from the core supported programming languages that it supports: Julia, Python, and R.

Jupyter Notebooks have become very popular in the last few years, and for good reason. They allow you to create and share documents that contain live code, equations, visualizations and narrative text.

Use of Jupyter Notebooks has exploded in recent years. They are now an essential tool in many different fields, including data science, machine learning, and artificial intelligence. environment that requires no setup to use, runs entirely in the cloud, and is completely free.
[/powerkit_alert]

Note: There is a similar software available, called Stable Diffusion, that can generate amazing AI generated art very fast, and has a similar setup. You can check out our basic guide on Stable Diffusion and our guide to using Stable Diffusion with a user interface. Another article you might find useful is our ranking of the 4 best AI art generators.

Sidenote: AI art tools are developing so fast it’s hard to keep up.

We set up a newsletter called tl;dr AI News.

In this newsletter we distill the information that’s most valuable to you into a quick read to save you time. We cover the latest news and tutorials in the AI art world on a daily basis, so that you can stay up-to-date with the latest developments.

Check tl;dr AI News

Getting Started with Disco Diffusion

In this guide we’ll generate AI art with Disco Diffusion, specifically images, as fast and easy as possible.

We won’t tweak the available settings for this run – you can learn about the various settings later. This way you can see how to run it, and see what it does, as fast as possible. That should break the ice in case you’re intimidated at first.

You can also generate videos with Disco Diffusion. However in this tutorial we’ll just generate images.

Quick Video Demo

Here’s a quick video demo of using Disco Diffusion for the first time, in which we’re going from start to finish as quickly as we can.

Step 1: Open & Copy the Disco Diffusion Colab Notebook

Go here to open the Disco Diffusion 5.61 Colab Notebook: https://colab.research.google.com/github/alembics/disco-diffusion/blob/main/Disco_Diffusion.ipynb

Next go in the Google Colab menu to File > Save a copy in Drive to save the notebook into your Google Drive.

Your copy of the Disco Diffusion notebook will open in a new tab, with the name Copy of Disco Diffusion [..]. This is the one we’ll be using. You can close the old tab.

At some point you’ll probably see a popup notification saying Notebook requires high RAM. This is just a notification and it’s nothing to worry about. Click OK when you see it.

Make Sure Google Colab is Using GPU

To run Disco Diffusion in Google Colab we need Colab to use a GPU. To make sure we’re using one, in the menu click on Runtime > Change runtime type.

A little popup will appear. In the Hardware accelerator dropdown make sure GPU is selected.

Click Save. Now we’re ready.

Step 2: Run Check GPU Status

Running this cell will check the GPU we’re allocated. Google Colab usually allocates GPUs randomly, depending which is available, and some GPUs are better than others.

Ideally we want a Tesla P100 or Tesla T4. Along with the default settings for Disco Diffusion and a T4, you should get an image in ~15 minutes.

If you use a Tesla K80 it will take much longer to generate an image.

How to Get a Tesla P100 or T4 in Google Colab

To change your assigned GPU you’ll can restart your Google Colab session until you get the one you like.

Video Demo

Steps:

1. To do this, towards the upper right corner of Colab, where you can see RAM and Disk, click the little downward arrow > Manage Sessions > Terminate. Alternatively you can do the same thing by clicking in the Colab menu, in the upper left, on Runtime > Manage Sessions > Terminate. This terminated your current session.
2. Next click to run the 1.1 Check GPU Status cell. This should start a new session.
3. Do this until you get a Tesla P100 or T4.

[powerkit_alert type=”info” dismissible=”false” multiline=”false”]
Note: If you want to avoid the hassle, you can get Google Colab Pro for $9.99, or Google Colab Pro+ for $49.99. With Colab Pro you won’t be assigned Tesla K80 anymore, and you’ll mainly get either Tesla T4 or Tesla P100. You can check out the plans here https://colab.research.google.com/signup. I like using Google Colab Pro so I recommend it.
[/powerkit_alert]

Step 3: Connect to Google Drive

The next cell is called 1.2 Prepare Folders.

Run it, and it will create a popup window asking you to select a Google account whose Google Drive you’d like to use, because Disco Diffusion needs to setup some folders and download some files.

You’ll have to select your Google account that you’d like to use, and then click to allow certain permissions for Google Colab.

Step 4: Run Everything Else Until “Prompts”

Next we can run every other cell until we get to the Prompts cell.

It should take about 1-2 minutes for Disco Diffusion to do its thing, and download the files it needs.

Step 5: Write our Text-to-Image Prompt

Next we can write our prompt that Disco Diffusion will use to generate the image.

By default you’ll see these prompts. You can learn more about how to write prompts later.

text_prompts = {
    0: ["A beautiful painting of a singular lighthouse, shining its light across a tumultuous sea of blood by greg rutkowski and thomas kinkade, Trending on artstation.", "yellow color scheme"],
    100: ["This set of prompts start at frame 100","This prompt has weight five:5"],
}

If you’d like an example of a prompt likely to result in a nice image, you can use something like this:

text_prompts = {
    0: ["A beautiful fantasy artwork of a celestial cyberpunk castle by li shuxing, trending on artstation"],
}

Some Prompt Examples with Results

Here are a few other examples of prompts and resulting images. You can use those prompts as inspiration. We’ll mention more resources for inspiration at the end of this article.

When you’ve finished writing your prompt you can run the cell.

Step 6: Do the Run! – Generate Your Image

We will explain two settings before finally running the cell to create the image.

display_rate: this is so you can see the progress on your image. By default it’s set to 25, which means you’ll see the image progress every 25 steps (you’ll learn about steps later).

You can just leave it to 25 or set it to something less if you’d like. If you’ve got a Tesla T4 GPU and default other settings, with 25 you’ll probably see the image progressing every 1-2 minutes, as you can see in the video demo from earlier.

n_batches: These are the number of images you want Disco Diffusion to create. If you leave 50, it will continue generating variations of your prompt.

As you can see I’ve changed it to 1, so it generates one image and it stops, so I can change the prompt and run it again, and so on. I recommend you change it to 1 for your first run.

Step 7: Wait For Image to Generate

Now all we have to do is wait until our image is finished. You can do something else in the meantime.

Step 8: Image is Finished Generating

This is my first result, as you can see in the video demo as well.

You can right click to save it to your computer, but you can also see it in your Google Drive.

Where Are Images Located

They’re located in your Google Drive. When you connected Google Drive earlier, the Disco Diffusion Colab notebook set up some folders in your Drive.

You should a folder called AI in your Drive. In it you have everything Disco Diffusion set up.

Our images, in my case, are located in AI > Disco_Diffusion > images_out > TimeToDisco. TimeToDisco is our batch_name that we didn’t configure. You can change that later, as it helps with categorizing experiments.

Access Google Drive from Google Colab

You can access Google Drive from Google Colab.

The TimeToDisco(0)_0.png … TimeToDisco(0)_8.png are the images I generated. I let it generate multiple images using the same prompt and stopped it after TimeToDisco(0)_8.png. You can double click any of them and Google Colab will display them.

The TimeToDisco(0)_settings.txt are your entire configuration that you used. You can double click it to see the text file to see its contents. Disco Diffusion conveniently saves settings for every run you make so you can share them, or come back to them if you had a good result and replicate or improve future experiments.

To download any of the files just right click them and click on Download.

Access Images Directly from Google Drive Interface

You can also easily access images in your Google Drive interface:

Very Useful Resources

The community is still growing and experimenting with machine learning generated images. These are some excellent resources that I have learned from so far and I recommend you check out.

• Image Generation with CLIP + Diffusion models (Disco Diffusion 4.1) [Youtube] – This is the first tutorial I ever saw and it gave me the basic overview of the Disco Diffusion settings. I highly recommend it. Even if they use version 4.1, it’s still an excellent resource.
• Get started quickly with Disco Diffusion v5 ~ Generative AI Art – The first tutorial I ever read on Discord Diffusion. It presents something similar to this article but maybe you’ll find it explains some things better than I did.
• /r/DiscoDiffusion Subreddit – Other Disco Diffusion users post here all the time, and you can ask them their prompts and settings. You can also just ask for help in general. They’re very helpful.
• Zippy’s Disco Diffusion Cheatsheet v0.3 – This Google Doc is epic. It’s beautifully written,. It presents every setting for Disco Diffusion in an easy to understand manner. Don’t be intimidated by the length of the document. I initially expected it to be a dry read but it isn’t. The writer is humble in mentioning they’re still trying to understand Disco Diffusion. This will make everything clearer for you.
• Disco Diffusion 70+ Artist Studies by weirdwonderfulai.art – How mentioning an artist in your prompt will impact your generated art. This person did an amazing job centralizing samples of generated art for 560+ Artists, and how they can affect the outcome of a generated image. These are contributions made by many others experimenting with generating art and submitting their finds. You can do that too.
• Disco Diffusion Modifiers by weirdwonderfulai.art – Modifiers, like artist names, are keywords that guide the image generation in a certain direction. This is another amazing job at centralizing a valuable resource in a single place for all of us to use and experiment.
• Getting Started with Disco Diffusion by weirdwonderfulai.art – another great tutorial on getting started with Disco Diffusion
• Anything Punk Modifiers for AI Art by weirdwonderfulai.art – Steampunk, Cyberpunk, Biopunk, and others. These are all easy wins to write prompts for beautiful results.
• Twitter and Instagram. You can search for #aiart, #aiartcommunity or #generatedart and you’ll most likely find others posting Disco Diffusion generated art. You can ask them their prompts and settings, and they’ll probably ask you yours.
• CLIP Prompt Engineering for Generative Art – an excellent and very insightful article to help understand how to create prompts. I can’t recommend it enough.
• Tools and Resources for AI Art – Constantly updated, extremely useful and fun list of tools and resources to create AI art. Maintained by @pharmapsychotic, they regularly test AI art tools, post amazing results on Twitter and update the list with a link to the tool and a short description. Highly recommend it!
• A Traveler’s Guide to the Latent Space – Another fantastic guide, and one of the best guides, that tries to give you a beginner friendly explanation on Disco Diffusion, so as to give you an intuition on how to generate beautiful AI generated art. It gives you lots of additional useful tips that other guides don’t cover. I know I say this often, but I highly recommend it!
• Disco Diffusion on Replicate.com – Replicate.com is a website that allows you to run software like Disco Diffusion for free right now. It requires no set up and it’s a great way to try out the latest machine learning models in the cloud. After some time it requires you to sign up, however they are very generous with the amount of free usage you get, in my opinion.
• Want to learn A.I. Art? – Here’s THE ULTIMATE Disco Diffusion Tutorial (Video) – A very detailed and beginner friendly tutorial on Disco Diffusion. They explain the every feature extremely well. It’s 40 minutes long and not tedious at all. I highly recommend it!

Troubleshooting

There are some well known issues when using certain configurations in Disco Diffusion.

RuntimeError: CUDA error: misaligned address

RuntimeError: CUDA error: misaligned address CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

This is likely happening because you’re trying to run Disco Diffusion with a Tesla T4 GPU while using the ViTL14 model.

ViTL14 can significantly improve your images under certain configurations, but T4 doesn’t work with ViTL14 in Colab for some reason, and many others have this issue.

If you want to use ViTL14 make sure to use Tesla P100.

Conclusion

Hopefully this article helped you get started with Disco Diffusion to create “AI” generated images. If you have any feedback please feel free to leave a comment and I’ll answer as fast as I can.

• 1
  
  
  A Jupyter Notebook is a web-based environment for interactive computing.
  
  The name, Jupyter, comes from the core supported programming languages that it supports: Julia, Python, and R.
  
  Jupyter Notebooks have become very popular in the last few years, and for good reason. They allow you to create and share documents that contain live code, equations, visualizations and narrative text.
  
  Use of Jupyter Notebooks has exploded in recent years. They are now an essential tool in many different fields, including data science, machine learning, and artificial intelligence.