Benchmarks

Introducing V4-Pro: Our most accurate transcription yet

Compare Cockatoo V4-Pro across 36 languages on a public benchmark and on internal real-world recordings.

Try Cockatoo free See the results

Updated October 27, 2026

FLEURS25 of 36

languages where Cockatoo V4-Pro has the
lowest error rate, ties included

Internal real-world benchmark25 of 36

languages where Cockatoo V4-Pro has the
lowest error rate, ties included

Human-labeled recordings with accents and noisy audio, testing robustness in challenging real-world conditions.

FLEURS · Below 5% WER12

languages with less than 5% word error rate

Bulgarian, Catalan, Czech, Danish, English, German, Indonesian, Norwegian, Slovak, Spanish, Turkish, Ukrainian

Built for the way people actually speak

Meet Cockatoo V4-Pro. Our most advanced transcription experience combines greater accuracy, clear speaker labels, and polished formatting to help you spend less time correcting transcripts and more time using them.

We’ve deliberately focused Cockatoo V4-Pro on 36 languages, choosing a narrower scope than services advertising support for 90 or 100 languages. Our priority is raising the quality bar within each language.

Why we measure beyond read speech

Academic benchmarks are valuable, but they don’t always reflect the recordings people need transcribed. FLEURS, for example, focuses on people reading prepared sentences aloud. That makes it useful for consistent comparisons, but read speech cannot capture the full complexity of spontaneous conversation. A low word error rate on those recordings can create an overly optimistic impression of performance on interviews, meetings, or imperfect everyday audio. FLEURS dataset documentation

Testing everyday audio and difficult conditions

To go further, we built our Internal Realworld Benchmark using real recordings drawn from different geographic regions and transcribed by human annotators. We combined randomly sampled recordings with deliberately challenging segments to test both everyday performance and difficult failure cases.

We examine results language by language and recording by recording, including the weakest results, rather than allowing one attractive average to hide inconsistent performance. The goal is to understand where transcription breaks down—and improve the experience where it matters most.

Cockatoo V4-Pro’s 36-language focus

Arabic, Bulgarian, Catalan, Chinese, Croatian, Czech, Danish, Dutch, English, Filipino/Tagalog, Finnish, French, German, Greek, Hebrew, Indonesian, Italian, Japanese, Khmer, Korean, Lithuanian, Malay, Norwegian, Persian/Farsi, Polish, Portuguese, Romanian, Russian, Slovak, Slovenian, Spanish, Swahili, Thai, Turkish, Ukrainian, and Vietnamese.

Better transcription should prove itself on the recording you actually have. That’s the standard we’re building Cockatoo V4-Pro to meet.

Head to head

How Cockatoo V4-Pro compares

Per-language pooled error rates, compared on the same clips. Averages give each language equal weight.

Compared with

FLEURS

Public benchmark, read speech

Real-world recordings

Calls, meetings, interviews and lectures

Scribe V2

ElevenLabs

6%fewer errors

6.6% vs 7.1% average

Ahead in 19 of 36 languages, 9 tied, 8 behind

6%fewer errors

21.3% vs 22.8% average

Ahead in 26 of 36 languages, 0 tied, 10 behind

Universal-3.5 Pro

AssemblyAI

71%fewer errors

6.0% vs 20.8% average

Ahead in 13 of 16 languages, 0 tied, 3 behind

52%fewer errors

19.3% vs 40.0% average

Ahead in 16 of 16 languages, 0 tied, 0 behind

Nova-3

Deepgram

55%fewer errors

6.0% vs 13.4% average

Ahead in 34 of 34 languages, 0 tied, 0 behind

41%fewer errors

20.7% vs 35.2% average

Ahead in 33 of 34 languages, 0 tied, 1 behind

Whisper Large V2

OpenAI

64%fewer errors

6.6% vs 18.4% average

Ahead in 36 of 36 languages, 0 tied, 0 behind

65%fewer errors

19.2% vs 55.4% average

Ahead in 31 of 31 languages, 0 tied, 0 behind

Each square is one language, A to Z

Cockatoo V4-Pro aheadTiedCockatoo V4-Pro behindNo result

By language

Error rates in 36 languages

Pooled word error rate (WER), or character error rate (CER) for languages marked *. Lower is better.

Cockatoo V4-ProCockatoo V4-ProScribe V2Scribe V2AssemblyAI Universal-3.5 ProAssemblyAIDeepgram Nova-3DeepgramWhisper Large V2Whisper

Sort languages

Language

FLEURSPublic benchmark, read speechCockatoo V4-Pro

Real-world recordingsInternal recordings, in-sample diagnosticCockatoo V4-Pro

Language

0%5%10%15%20%25%

Cockatoo V4-Pro

0%10%20%30%40%50%

Cockatoo V4-Pro

Arabic

15.2%

36.3%

Bulgarian

2.9%

22.8%

Catalan

4.0%

12.6%

Chinese *

2.8%

31.9%

Croatian

5.4%

19.1%

Czech

4.6%

14.5%

Danish

4.3%

16.8%

Dutch

5.6%

13.9%

English

3.9%

—

English (global)

—

10.4%

English (US)

—

10.9%

Filipino (Tagalog)

7.8%

21.6%

Finnish

9.2%

24.9%

French

7.8%

21.0%

German

2.4%

11.1%

Greek

7.4%

16.4%

Hebrew

6.0%

22.0%

Indonesian

4.9%

27.7%

Italian

6.2%

17.4%

Japanese *

6.5%

19.4%

Khmer *

19.7%

22.6%

Korean

7.6%

31.9%

Lithuanian

5.8%

11.0%

Malay

6.6%

37.4%

Norwegian

4.9%

28.7%

Persian (Farsi)

9.6%

21.9%

Polish

5.7%

18.9%

Portuguese

11.1%

14.8%

Romanian

8.0%

24.3%

Russian

7.8%

22.3%

Slovak

4.7%

27.6%

Slovenian

5.4%

25.2%

Spanish

1.0%

14.5%

Swahili

14.9%

42.6%

Thai *

6.9%

23.2%

Turkish

2.6%

16.4%

Ukrainian

2.8%

16.1%

Vietnamese

6.9%

8.5%

* CER for Chinese, Japanese, Khmer and Thai.› Above the chart limit; exact values are in the tooltip and CSV.No marker: no result. English US/global are separate real-world groups.

Samples

Listen for yourself

Read the Cockatoo V4-Pro transcript while you listen. Choose a language to explore more examples.

FLEURS · Read speech

0:26

Expertos de la Universidad de Princeton, de Estados Unidos, y de la Uppsala, en Suecia, dieron a conocer que la nueva especie logró evolucionar en tan solo dos generaciones, aun cuando se estimaba que demoraría mucho más, a causa de la reproducción entre un pinzón de Darwin endémico, *Geospiza fortis*, y un pinzón de cactus migrante, *Geospiza conirostris*.

FLEURS · Read speech

0:29

イタリアのリグーリア州にあるチンクエ・テッレは五つの土地を意味し、オマッジョーレ、マナローラ、コルニリア、ヴェルナッツァ、モンテロッソという五つの小さな村が寄り添うようにできた沿岸沿いの地域です。

FLEURS · Read speech

0:27

على الرغم من تكهّن بعض العلماء بأنه نظراً لوجودها أيضاً في أحواض نهر سرسواتي الجاف الآن، فيجب أن تسمى "حضارة السند وسرسواتي" بدلاً من "حضارة نهر السند"، بينما يطلق عليها البعض "الحضارة الهارابية" نسبة لقرية هارابا حيث تم التنقيب عن أول مواقع تلك الحضارة في العشرينات.

Methodology

How we measured

The same audio and the same scoring rules for every model.

Read the full methodology

Datasets: FLEURS and internal recordings across 36 languages each. Internal recordings informed provider selection and are an in-sample diagnostic, not an independent holdout. Real-world English US and global groups are shown separately.
Metric: Pooled word error rate: substituted, missing and extra words across a language, divided by the words in its reference transcripts. Character error rate for Chinese, Japanese, Khmer and Thai. Comparisons use only clips completed by both models, then average each language equally; missing results are not counted as zero.
Scoring: Both raw references and hypotheses are normalized with pinned OpenAI Whisper normalizers: EnglishTextNormalizer for English and default BasicTextNormalizer for other languages. CER removes whitespace after normalization. Ties use unrounded scores (tolerance 0.000000001 percentage points).
Models: Cockatoo V4-Pro, Scribe V2, AssemblyAI Universal-3.5 Pro, Deepgram Nova-3 and Whisper Large V2. Unsupported and unavailable results remain missing. The table shows Cockatoo V4-Pro error rates by language.
Tested: Updated October 27, 2026

Benchmark questions

What the numbers mean and how we got them.

The share of words a model gets wrong compared with a human reference transcript, counting substituted, missing and extra words. A 5% word error rate means about one word in twenty needs fixing.