The Opus Codec: One Format for Voice Calls, Music and Speech Uploads

Where Opus came from

Before Opus, you usually had to choose: speech codecs that sounded good for voice at low bitrates but poor on music, or music codecs that needed more bits and added more delay. Opus, published by the IETF as RFC 6716, set out to remove that choice. It merged two existing designs: SILK, a speech codec developed at Skype, and CELT, a low-delay general-purpose codec developed under the Xiph.Org Foundation. The reference implementation, libopus, is open source, and the format is free to implement.

Opus quickly became the standard audio codec for real-time communication on the web: WebRTC, the technology behind browser-based calls, requires implementations to support it. It is also widely used by voice chat, conferencing and messaging services, and inside WebM video.

How Opus handles speech and music

Opus has three modes and moves between them as the audio and bitrate demand:

SILK mode

Linear-prediction coding modelled on how speech is produced. Efficient for voice at low bitrates and narrower bandwidths

CELT mode

Transform coding, similar in spirit to music codecs, with very low delay. Suited to music and higher bandwidths

Hybrid mode

SILK covers the lower frequencies of speech while CELT adds the higher ones, giving full-bandwidth voice at moderate bitrates

An encoder can be told what kind of signal to expect through an application setting, such as voice-over-IP or general audio, which biases its choices. The format also supports variable or constant bitrate, built-in loss concealment for dropped packets, optional forward error correction and discontinuous transmission, which sends almost nothing during silence on a call.

Bandwidths, bitrates and frame sizes

Opus describes audio bandwidth in named steps, each corresponding to an effective sampling rate:

Narrowband

8 kHz sampling, audio up to 4 kHz, telephone quality

Wideband

16 kHz sampling, audio up to 8 kHz, the band speech recognition uses

Super-wideband

24 kHz sampling, audio up to 12 kHz

Fullband

48 kHz sampling, audio up to 20 kHz, the full range of hearing

Bitrate range

From 6 kb/s up to 510 kb/s

Frame sizes

From 2.5 to 60 milliseconds, with 20 ms a common default

The low frame sizes are why Opus suits live calls: with default settings its algorithmic delay is in the tens of milliseconds, far below many music codecs. For stored files, latency matters less, but the efficiency at low bitrates matters a lot. Opus documentation and long community experience suggest that wideband speech sounds clear at modest bitrates in the tens of kilobits per second, while music in stereo benefits from considerably more. Treat any single figure as a starting point and listen, because the right bitrate depends on the material.

Public listening tests over the years have generally rated Opus favorably against older codecs at low and medium bitrates. As always with codec comparisons, results depend on the encoder version and settings, so judge with your own material rather than a headline.

Containers and file extensions

Opus is a codec, so it needs a container to become a file:

• Ogg, defined for Opus in RFC 7845, is the classic choice for audio-only files. The recommended extension is .opus, but files with .ogg are common, and an .ogg file may contain Vorbis or FLAC instead of Opus.
• WebM and Matroska (MKV) carry Opus alongside video, and many browser-made recordings use WebM with Opus audio.
• MP4 can carry Opus according to its specification, but support across tools and players is less consistent than for AAC.
• Apple's CAF format can also hold Opus.

Because the extension is not a reliable guide, a media inspector is the quickest way to confirm that a file really contains Opus. For the general difference between codecs and containers, see what an audio codec is.

Opus in the browser

Browsers have long been able to decode Opus for playback and calls, although support arrived at different times in different browsers and, for some, varied by container and operating system version. Encoding is a separate question. Pages have been able to record Opus through the MediaRecorder API in several browsers, and the newer WebCodecs API exposes an AudioEncoder that can produce raw Opus packets directly from audio samples, using the browser's own native encoder.

WebCodecs support has spread across browsers over several releases rather than all at once, so a careful page asks first. The API includes an isConfigSupported check that reports whether a given codec, sample rate, channel count and bitrate can be encoded before any work starts. The encoder produces packets, not files, so the page must also wrap the packets in a container such as Ogg. Check current browser compatibility tables before relying on any of these features. The wider set of browser media technologies is covered in browser-based video processing.

Why mydubly uploads speech as Opus

mydubly uses Opus for the one thing it sends over the network: speech. In your browser, the soundtrack is decoded to 16 kHz mono, wideband in Opus terms, and split into chunks of about 30 seconds cut at quiet moments. Each chunk is encoded to Opus at about 32 kb/s with the browser's native WebCodecs encoder and sent over HTTPS for recognition with Whisper; the video file never leaves your device. Where the browser cannot encode Opus, the chunks are sent as uncompressed WAV instead.

The reasons follow from the sections above. Opus was designed for voice at exactly this kind of bandwidth and bitrate. A native encoder is fast and does not need a large library to be downloaded. The files are small, so uploads are quick on slow connections, a point worked through with numbers in reducing upload size. And because the recognizer only uses audio up to 8 kHz, a wideband encode loses nothing it would have used. Opus is used only for this upload step: the translated audio mydubly returns is AAC in an M4A file, which is more broadly compatible with editors and players.

Encoding Opus yourself

1. Start from the best source you have, ideally lossless audio, rather than another lossy file.
2. With ffmpeg, a speech-oriented encode looks like this: ffmpeg -i interview.wav -c:a libopus -b:a 32k -ac 1 -application voip interview.opus
3. For music or mixed content, drop the voip application setting and raise the bitrate, and keep stereo if the source is stereo.
4. With the opusenc command-line tool from the Opus project, opusenc --bitrate 32 interview.wav interview.opus does a similar job.
5. Listen to a demanding section, such as fast speech or sibilant consonants, before converting a whole archive.
6. Confirm that the players, editors or platforms you rely on accept Opus in your chosen container.

Where Opus falls short

• Compatibility gaps. Some video editors, older devices and hardware players do not import or play Opus, especially in MP4 or on older Apple software.
• It is lossy. Opus is excellent for delivery and transfer, not for masters you might edit and re-encode later.
• Container confusion. An .ogg file is not necessarily Opus, and some tools only recognize one of the extensions.
• Platform requirements. Upload forms for some services expect MP3, AAC or WAV and may reject Opus files.
• No rescue for poor recordings. Opus compresses what it receives; it cannot remove noise or reverb.

Next step

If you have .opus, .ogg or WebM recordings, check their contents with a media inspector and keep them as they are; re-encoding to another lossy format adds loss without benefit. To see Opus-based uploading in action, run a short clip through private video translation and watch how little data is sent, or transcribe an audio file with audio to text; the OGG to text page covers Ogg recordings specifically.