How to run Whisper locally on a Mac

If you want a transcript of a meeting, lecture, interview or voice note without uploading the audio to a web service, you can run the transcription on your Mac. The model most tools use for this is Whisper. This guide explains what Whisper is, which model size to pick, how to run it yourself from Terminal, and where it falls short.

What Whisper is

Whisper is a speech recognition model that OpenAI released in September 2022. It was trained on 680,000 hours of audio in many languages, as described in the Whisper paper. It can transcribe speech in the language it was spoken, and translate speech into English.

What makes it useful on a Mac is that its model weights are public. Anyone can download the model and run it on their own hardware, which is why so many transcription apps are built on it. When you run it locally, the audio stays on your computer, it works offline, and there is no per-minute fee.

Model sizes and trade-offs

Whisper comes in several sizes. Bigger models are more accurate, especially for accents, noisy audio and languages other than English, but they are slower and need more memory. The table uses figures from the Whisper repository (parameters) and the whisper.cpp README (download size and memory):

Model	Parameters	Download (whisper.cpp)	Memory (whisper.cpp)	Good for
tiny	39 M	75 MB	~273 MB	Quick tests, clear English
base	74 M	142 MB	~388 MB	Short, clear recordings
small	244 M	466 MB	~852 MB	A reasonable everyday default
medium	769 M	1.5 GB	~2.1 GB	Better accuracy, slower
large	1,550 M	2.9 GB	~3.9 GB	Accuracy over speed
large-v3-turbo	809 M	about 1.6 GB	lower than large	Close to large, much faster

A few things worth knowing:

• The .en models (such as base.en) are English-only and do a little better on English at the small sizes. For any other language, use the multilingual model.
• Turbo is a version of large-v3 with a much smaller decoder, so it is several times faster than large with a small loss in accuracy. The Whisper repository notes that it is not trained for translation.
• Quantised files (names ending in -q5_0 or -q8_0) are compressed versions that take less disk space and memory, at a small cost in accuracy. They are useful on Macs with 8 GB of memory.

If you are unsure, start with large-v3-turbo on an Apple silicon Mac, or small on an older one, and move up or down depending on the results.

whisper.cpp and Metal

OpenAI’s own Whisper code is written in Python and is usually run on NVIDIA graphics cards. whisper.cpp is a separate implementation of Whisper in C and C++ by Georgi Gerganov and contributors, built to run efficiently on ordinary computers.

On Apple silicon Macs (M1 and later), its README states that inference runs fully on the GPU via Metal, Apple’s graphics framework. It also uses Apple’s Accelerate framework, and it can optionally run part of the model on the Neural Engine through Core ML. In practice, this makes large models fast enough to be practical on a laptop. Intel Macs can run it too, on the CPU, but expect it to be much slower.

Run it from Terminal

This follows the quick start in the whisper.cpp README. You need Apple’s command line developer tools (run xcode-select --install if you do not have them), git and cmake. If you use Homebrew, brew install cmake installs cmake.

1. Download and build whisper.cpp.

git clone https://github.com/ggml-org/whisper.cpp.git
cd whisper.cpp
cmake -B build
cmake --build build -j --config Release

On Apple silicon, Metal support is built in by default.

2. Download a model. The repository includes a script that fetches converted models:

sh ./models/download-ggml-model.sh large-v3-turbo

Replace large-v3-turbo with base, small, medium or another name from the table. The file is saved in the models folder as ggml-<name>.bin.

3. Try the sample file.

./build/bin/whisper-cli -m models/ggml-large-v3-turbo.bin -f samples/jfk.wav

You should see a short transcript of a speech by John F. Kennedy, with timestamps.

4. Transcribe your own recording. whisper.cpp works most reliably with 16 kHz, 16-bit WAV files. Convert your recording with ffmpeg (for example brew install ffmpeg), using the command from the README:

ffmpeg -i lecture.m4a -ar 16000 -ac 1 -c:a pcm_s16le lecture.wav

Then transcribe it, setting the language and choosing the output formats you want:

./build/bin/whisper-cli -m models/ggml-large-v3-turbo.bin -l en -otxt -osrt -f lecture.wav

• -l sets the spoken language (en, de, is and so on, or auto to detect it). Setting it yourself avoids wrong guesses at the start of a recording.
• -otxt writes a plain text file, -osrt a subtitle file with timestamps, and -ovtt a WebVTT file. The files are saved next to the audio.

Run ./build/bin/whisper-cli --help for all options.

If you would rather not use Terminal

Several Mac apps put Whisper behind a normal window: you drag in an audio file and get a transcript, with the model running on your Mac. They differ in price, which models they offer, whether they can record calls, and what they do with the transcript afterwards. When comparing, check whether transcription really happens on the device, since some apps send audio to a cloud service for some features. Tools come and go, so search for “Whisper Mac app” and read the current details on each developer’s own site.

Accuracy: what to expect

Whisper is good, but it is not a human transcriber, and its accuracy depends on a few things.

• Language. Whisper was trained on far more English than anything else. The Whisper repository publishes error rates by language, and they vary widely. Languages with less training data tend to have noticeably higher error rates, so test with your own recordings before relying on it. Some languages have community fine-tuned models that do better.
• Audio quality. Distance from the microphone, echo, background noise and people talking over each other cost more accuracy than moving up a model size.
• Names and jargon. Uncommon names, product names and technical terms are often misspelled. whisper-cli’s --prompt option lets you give a short text with the words you expect.
• Silence and music. Whisper can sometimes produce text that was never said, such as a repeated phrase, during long silences or music. Trimming silence helps, and whisper.cpp has an optional voice activity detection setting for this.
• Speakers. Plain Whisper does not tell you who said what. Speaker labels need a separate step, usually called diarization.

How Hearkeeper builds on whisper.cpp

Hearkeeper is a Mac app that is not released yet, and it is built on whisper.cpp with Metal. It records your microphone and your computer’s audio directly on the Mac, in any call app or in the room, with no bot and no ffmpeg step. Every recording starts with a consent step, and the recording pill stays visible.

• Live captions appear about 2–4 seconds after the words are spoken.
• A final pass with Whisper large-v3 turbo runs after you stop, with speaker separation, at roughly 10 times faster than real time on Apple silicon.
• 99 languages, with Icelandic treated as a first-class language.
• A local language model then writes a summary, decisions and to-dos, and lets you ask questions across meetings, with every line linked to the second it was said. No API key is needed.

Everything is stored in one SQLite file on your Mac and exports to Markdown. Nothing is uploaded unless you connect a team server or your own API key. It runs on macOS 14.2 or later, on Apple silicon and Intel. See local AI meeting notes for more.

If you like the idea of local Whisper without the Terminal, join the waitlist.