How to Use Whisper: A Free Speech-to-Text AI Tool by OpenAI

How to Use Whisper A Free Speech-to-Text AI Tool by OpenAI

Whisper is automatic speech recognition (ASR) system that can understand multiple languages. It has been trained on 680,000 hours of supervised data collected from the web.

Whisper is developed by OpenAI, it’s free and open source, and p

Speech processing is a critical component of many modern applications, from voice-activated assistants to automated customer service systems. This tool will make it easier than ever to transcribe and translate speeches, making them more accessible to a wider audience. OpenAI hopes that by open-sourcing their models and code, others will be able to build upon their work to create even more powerful applications.

Whisper can handle transcription in multiple languages, and it can also translate those languages into English.

I’m not very knowledgeable in speech recognition, but given how well this tool performs, and considering the fact that it’s free and open-source, I think it is fantastic. This will probably be used by a lot of people who don’t have the time or money to invest in a commercial speech recognition tool.

It will also be used by commercial software developers who want to add speech recognition capabilities to their products. This will help them save a lot of money, since they won’t have to pay for a commercial speech recognition tool.

I think this tool is going to be very popular, and I think it has a lot of potential.

In this tutorial we’ll get started using Whisper in Google Colab. We’ll quickly install it, and then we’ll run it with one line to transcribe an mp3 file. We won’t go in-depth, and we want to just test it out to see what it can do.

You can also immediately test out how Whisper transcribes speech to text on HuggingFace spaces here. Just make sure you can use your microphone.

Quick Video Demo

This is a short demo showing how we’ll use Whisper in this tutorial.


tl;dr AI News Section Preview
tl;dr AI News Section Preview

Sidenote: AI art tools are developing so fast it’s hard to keep up.

We set up a newsletter called tl;dr AI News.

In this newsletter we distill the information that’s most valuable to you into a quick read to save you time. We cover the latest news and tutorials in the AI art world on a daily basis, so that you can stay up-to-date with the latest developments.

Check tl;dr AI News

Using Whisper For Speech Recognition Using Google Colab

If you don’t have a powerful computer or don’t have experience with Python, using Whisper on Google Colab will be much faster and hassle free. For example, on my computer (CPU I7-7700k/GPU 1660 SUPER) I’m transcribing 30s in a few minutes, whereas on Google Colab it’s a few seconds.

Open a Google Colab Notebook

First we’ll need to open a Colab Notebook. To do that you can just visit this link https://colab.research.google.com/#create=true and Google will generate a new Colab notebook for you.

Alternatively you can go anywhere in your Google Drive > Right Click (in an empty space like you want to create a new file) > More > Google Colaboratory. A new tab will open with your new notebook. It’s called Untitled.ipynb but you can rename it anything you want.

Create a Google Colab Notebook in Google Drive
Create a Google Colab Notebook in Google Drive

Enable GPU

Next we want to make sure our notebook is using a GPU. Google often allocates us a GPU by default, but not always.

To do this, in our Google Colab menu go to Runtime > Change runtime type.

Click on Runtime > Change Runtime Type in Google Colab
Runtime > Change Runtime Type

Next a small window will pop up. Under Hardware accelerator there’s a dropdown. Make sure GPU is selected and click Save.

Set GPU as Hardware Accelerator in Google Colab
Set GPU as Hardware Accelerator

Install Whisper

Now we can install Whisper. (You can also check install instructions in the official Github repository).

To install it just paste the following lines in a cell. To run the commands click the play button at the left of the cell or press Ctrl + Enter. The install process should take 1-2 minutes.

!pip install git+https://github.com/openai/whisper.git 
!sudo apt update && sudo apt install ffmpeg
Install Whisper in Google Colab
Install Whisper

Upload an Audio File

Now we can upload a file to transcribe it. To do this open the File Browser at the left of the notebook, by pressing the folder icon.

Open File Browser in Colab
Open File Browser in Colab

Now you can press the upload file button at the top of the file browser, or just drag and drop a file from your computer and wait for it to finish uploading.

Drag & Drop Upload in File Browser in Google COlab
Drag & Drop Upload in File Browser in Google Colab

Run Whisper to Transcribe Speech to Text

Next we can simply run Whisper to transcribe the audio file using the following command. If this is the first time you’re running Whisper, it will first download some dependencies.

!whisper "Rick Astley - Never Gonna Give You Up Official Music Video.mp3"

In less than a minute it should start transcribing.

Whisper Transcribing Never Gonna Give You Up by Rick Astley
Whisper Transcribing Never Gonna Give You Up by Rick Astley

When it’s finished you can find the transcription files in the same directory, in the file browser:

Whisper Transcription Files
Transcription Files

Using Whisper Models

Whisper comes with multiple models. You can read more about Whisper’s models here.

Whisper's Models
Whisper’s Models
A model is a statistical representation of the speech to text engine. The model is trained to recognize speech and convert it to text for the user. There are many different types of models, each designed for a specific purpose.

By default it it uses the small model. It’s faster, but not as accurate as a larger model. For example let’s use the medium model.

We can do this by running the command:

!whisper AUDI_FILE – model medium

In my case:

!whisper "Rick Astley - Never Gonna Give You Up Official Music Video.mp3" – model medium
Results of Using the Whisper Medium Model
Using the Whisper Medium Model

The result is more accurate when using the medium model than the small one.

Whisper Command-Line Options

You can check out all the options you can use in the command-line for Whisper by running !whisper -h in Google Colab:

whisper -h
whisper -h

Conclusion

In this tutorial we covered the basic usage of Whisper by running it via the command-line in Google Colab. This tutorial was meant for us to just to get started and see how OpenAI’s Whisper performs.

You can easily use Whisper from the command-line or in Python, as you’ve probably seen from the Github repository. We’ll most likely see some amazing apps pop up that use Whisper under the hood in the near future.

Useful Resources & Acknowledgements

7 Shares:
Subscribe
Notify of
guest
Receive notifications when your comment receives a reply. (Optional)
Your username will link to your website. (Optional)

8 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
adam s
adam s
2 months ago

worked great – THANK YOU !!

I’m using this to transcribe voice audio files from clients… super helpful.

David
David
2 months ago

Great tip to use it on Colab instead of locally. WAY faster.

Abra
Abra
2 months ago

What’s the best way to use it for long transcriptions? Say 1-2 hours?

Dakuo
Dakuo
1 month ago

Thank you!! Very helpful for my 8-mins talk.

K Bill
K Bill
1 month ago

I tried several files and they kept erroring out and follow this to a t.
channel element 0.0 is not allocated

You May Also Like