Use Speech-to-Text with ChatGPT (Whisper) and Make / Integromat

"I want a voice note-taking application on my phone and then send it to ChatGPT for a speech-to-text breakdown, send the list of tasks to Trello, and notify my assistant on Facebook."

This was an Upwork ad for a job that the customer ultimately removed.

Ok, I'm clear: when they refer to AI, they say ChatGPT, but we know they're referring to AI — so let's continue.

Anyway, it piqued my curiosity, and with some time to spare, I decided to do this by myself, just for fun, and for the sake of this article. Instead of Facebook and Trello, I will record the results in a Google Sheet.

As usual, clients' requests are in some way "zipped" — there's more than just two sentences to get a result. That's the beauty of challenges.

To go for this, I developed a simple voice-note recording app that sends the recorded file to a Make (formerly Integromat) webhook.

Then, using a couple of modules in Make, I was able to connect the rest of the applications and let ChatGPT do the main task: converting audio into text and breaking down the received audio into text tasks that will then be sent to Trello and to Facebook Messenger as a notification.

It sounds simple, and it is when using Make/Integromat. This is exactly the kind of workflow automation we build every day. Let's dive into the process.

Audio Recorder application

This is a simple audio recorder, nothing fancy — but if you want to improve it and use it for yourself, here is the code.

Before continuing with details, let's see the final Make/Integromat scenario.

Webhook

When the "stop recording" button is pressed in our Audio Recorder App, this function is invoked and the audio file is sent to the Make/Integromat webhook. If you need the scenario to reply back to the caller, see how to handle Make.com webhook responses.

function sendAudioData(data, fileName) {
  const serverUrl = "https://hook.us1.make.com/YOURWEBHOOKID";
  const formData = new FormData();
  formData.set('file', data, fileName);
  fetch(serverUrl, { method: "POST", body: formData })
    .then(response => console.log(`Audio data (${fileName}) sent to server.`))
    .catch(error => console.error("Error sending audio data to server: ", error));
}

Google Drive modules

This simple part is important: we send the webhook file to Google Drive with an upload module, and then get the share link with a second Google Drive module.

HTTP module — Get a file

In this case, we just pass the "Web Content Link" field from the previous Google Drive module to the HTTP module to download the file from the specified URL.

HTTP module — Make an OAuth 2.0 request

To make an HTTPS request to servers that require OAuth 2.0 authorization, you need to create an OAuth connection first. I'll take the time to explain this module, as for many of you who may not be developers, this could be the tricky part.

Use these parameters:

• Method: POST
• In the headers section, add the key Authorization with the value Bearer YOUR_CHATGPT_API_KEY. Note: the word Bearer is important — that's how you authenticate to ChatGPT.
• In Body type use: multipart/form-data
• Add a field of type File, key file, and map the file that came from the previous module.
• Add another field of type text, key model, value whisper-1 — this sends the audio note to ChatGPT's whisper-1 model to process speech into text.

ChatGPT module

We're almost done. With our Voice Note Recorder app sending the audio file to our webhook, the webhook sending the file to Google Drive for storage, and the shared link of the audio file going to the ChatGPT API, we're just one step away from converting the audio into text. This is the prompt we will use:

"Act as an experienced project manager and break down the following text into tasks:" and then provide the data from the previous module, as shown in the picture. For more advanced control over the model, you can also try OpenAI Assistants function calling in Make.

And voilà — we have a formatted list of tasks that we can send to our favorite PM application, via SMS, to Facebook, or wherever we want. You could even generate PDFs in Make from the result. The possibilities are limited only by our imagination.

Voice Note: "I need all leads taken from the Contact Us form on our website during the last week to be contacted directly, and then send them an offer of our free air conditioner installation if purchased using our online store."

ChatGPT speech-to-text result:

1. Collect all leads captured from the website's Contact Us form for the past week.
2. Prepare a list of the obtained leads.
3. Contact each lead on the list directly.
4. Deliver an offer of a free air conditioner installation to every contacted lead.
5. Ensure the offer is contingent on the lead making a purchase using the online store.
6. Clearly explain the terms and conditions of the proposal to the contacted leads.