Vous m'avez trouvé sur Upwork ? Si vous m'avez trouvé via Upwork et souhaitez faire appel à mes services, merci de me contacter via Upwork. Les messages de clients Upwork envoyés en dehors d'Upwork ne recevront pas de réponse.
Comparatif🎬 Sous-titrage

Auto-captions vs human captioning: ASR failures and word error rate

ASR fails on proper nouns, overlapping speech and sound cues. WER spot-check: 2 name errors in 12 words = 16.7%, not broadcast-compliant without a human QC pass.

Dao Huy (Lucas)8 min read8 sections · tableau comparatif
Auto-captions vs human captioning: ASR failures and word error rate

💡 ASR tools (Whisper, YouTube) produce a usable first draft - especially for clean, single-speaker English audio - but fail in predictable, measurable ways: proper nouns guessed phonetically, overlapping speakers dropped, sound cues omitted, speaker IDs absent. Word Error Rate (WER) = (S + D + I) / N × 100%. Two name errors in a 12-word segment = WER 16.7%, which does not pass the FCC accuracy benchmark without a human correction pass.

Key takeaways

  • WER formula: WER = (Substitutions + Deletions + Insertions) / Reference word count × 100%. A substitution on a proper noun scores the same as any word error but carries a higher meaning penalty.
  • ASR fails most on: proper nouns (phonetic guesses), overlapping speech (one speaker dropped), non-speech audio (sound cues omitted), speaker IDs (absent by default), and low-resource languages.
  • The FCC accuracy benchmark (47 CFR § 79.1) requires captions to match spoken words verbatim, in order, without paraphrasing, including speaker identification - none of which Whisper provides out of the box.
  • Auto-captions are a time-saver, not a finished deliverable. The break-even is a human correction pass in Subtitle Edit: 1-2 minutes of correction per minute of clean-audio video.
  • Broadcast delivery (SCC for US TV, SMPTE-TT for streaming) requires a human QC pass regardless - FCC accuracy includes speaker IDs and sound cues, neither of which ASR provides.

What does "auto-caption" actually produce, technically?

Auto-captioning is Automatic Speech Recognition (ASR): the tool converts the acoustic signal to a word lattice, picks the most probable word sequence, and writes timed text segments. YouTube uses Google's proprietary ASR. OpenAI Whisper is the leading open-source alternative, trained on 680,000 hours of multilingual audio across 99 languages, including Vietnamese. Neither tool adds speaker identification, sound effect notation, or music cues - those are human decisions.

The output is a raw SRT or VTT file: dialogue only, no [MUSIC], no [LAUGHTER], no NARRATOR: labels. For a documentary or a panel discussion, that is already a significant gap from what broadcast captioning requires. The primary keyword for every frame of this workflow is auto captions vs human captioning, and the distinction starts right here: ASR gives you words, human captioning gives you a complete, standards-compliant record.

Where does ASR fail? The error-type table

I keep this table in my workflow notes because it tells me how much correction time to budget before I open the file:

Error typeASR behaviorExampleFCC benchmark hit
Proper nounsPhonetic guess from the vocabulary"Dr. Sarah Chen" → "Dr. Sara Chin"Accuracy
Overlapping speechDominant speaker wins; second speaker dropped or garbledInterviewer's interjection missingCompleteness
Non-speech audioOmitted entirely by default[EXPLOSION] absent from transcriptAccuracy (IP standard requires sound IDs)
Speaker identificationNo labels; all dialogue presented as one speakerPanel discussion reads as monologueAccuracy (IP: "must identify speakers")
Numbers and datesInconsistent numeral vs. word form"Q4 2026" → "queue for 2026"Accuracy
PunctuationComma and period placement varies; question marks often missingRun-on sentences across cue boundariesAccuracy (required verbatim by FCC)
Accented or low-resource audioHigher substitution rate; words skipped on difficult segmentsNon-native speaker in English interviewAccuracy + Completeness
Technical terms / acronymsSpelled phonetically or dropped"CEA-708" → "seye 708"Accuracy

How do I calculate WER on a sample?

Word Error Rate is the standard metric. The formula:

WER = (S + D + I) / N × 100%

S = Substitutions  (wrong word)
D = Deletions      (reference word missing from hypothesis)
I = Insertions     (extra word added by ASR)
N = total words in the reference transcript

Here is a worked example I run on the first 30 words of any new project to gauge correction effort:

Reference (what was actually said, 12 words):
  "Dr. Sarah Chen confirmed the Q4 results will drop Thursday."

ASR hypothesis (Whisper output, 12 words):
  "Dr. Sara Chin confirmed the Q4 results will drop Thursday."

Alignment:
  Dr.   Sarah  Chen   confirmed the Q4 results will drop Thursday.
  Dr.   Sara   Chin   confirmed the Q4 results will drop Thursday.
        [S]    [S]

S = 2, D = 0, I = 0, N = 12
WER = (2 + 0 + 0) / 12 x 100% = 16.7%

A 16.7% WER on a 12-word segment means two wrong words, both of them names. The FCC accuracy criterion specifically requires "correct proper names" - that 16.7% would not pass a QC check for broadcast. I run this calculation on a representative 30-50 word sample, covering a name-dense segment, to decide how deep a correction pass the file needs.

Whisper's research paper (Radford et al., 2022) reports approximately 2.7% WER on LibriSpeech test-clean using the Large V2 model - that is clean, studio-quality read speech. Real conversational audio, panel discussions, or accented speakers perform notably worse. The paper explicitly notes performance scales with training data volume, so lower-resource languages have higher error rates. For broadcast QC, Whisper is my starting point, never my endpoint.

How does ASR accuracy map to the FCC's four quality benchmarks?

The FCC adopted four closed caption quality benchmarks in 2014 under 47 CFR § 79.1(j)(2). No specific numeric threshold is mandated - the standard uses "to the fullest extent possible". But the IP captioning standard adds precision: captions "must match the spoken words verbatim, in order, without paraphrasing" and "must identify the speakers."

Here is how raw ASR output maps to each benchmark:

FCC benchmarkDefinition (§ 79.1)Raw ASR statusWhat a human pass fixes
AccuracyMatches spoken words verbatim, in order, including proper names and punctuation; identifies speakersPartial: dialogue mostly correct on clean audio; proper nouns, punctuation and speaker IDs routinely wrongName corrections, punctuation, speaker labels, sound cues
SynchronicityCaptions coincide with speech; displayed at a readable speedUsually acceptable for Whisper/YouTube; minor drift on rapid speechAdjust cue boundaries on rapid-fire speech; check CPS
CompletenessRuns beginning to end; no dropsUsually complete except during overlapping speech or very low-SNR audioFill dropped segments; add sound cue events
PlacementDoes not cover important visual contentNot controlled by ASR (depends on player/renderer)Human sets placement in SCC PAC row or SRT positioning

When is auto-captioning enough, and when do I call in a human?

This is the question I get most from first-time clients. Here is how I think through it:

  • Auto-captions alone for: informal internal communications, social-media content where you explicitly disclose "auto-generated captions", or any video where you review and correct the output yourself before publishing.
  • Auto-captions as a first draft (with a human correction pass) for: most online video where the audio is clean and single-speaker. The correction pass in Subtitle Edit takes 1-2 minutes of editor time per minute of video for good-quality audio. That is the break-even where ASR saves meaningful time.
  • Human captioning from scratch for: multi-speaker panel recordings, heavily accented audio, any content with significant technical terminology or proper-noun density, and any broadcast or streaming deliverable where FCC quality benchmarks apply. A correction pass on bad ASR can take longer than writing from scratch.
  • Broadcast delivery (SCC for US TV) requires a human QC pass regardless: FCC's accuracy benchmark includes speaker identification and sound effects, neither of which ASR provides. For SCC files, I always run a caption-inspector decode to verify the byte-level encoding and PAC row assignments.

My caption QC service includes an accuracy review against the four FCC benchmarks, speaker ID and sound-cue additions, and a caption-inspector decode for SCC files. If you have a Whisper draft and need it broadcast-ready, that is the step that gets it there.

FAQ

Can I use Whisper output directly for a broadcast SCC delivery?

No. Whisper produces SRT or VTT - neither is an SCC file. Beyond the format conversion (which SeConv handles: seconv input.srt ScenaristClosedCaptionsDropFrame --fps:29.97 --output-folder out --overwrite), the content needs a human accuracy pass: speaker IDs, sound cues, proper-noun corrections, and punctuation review must all be done before an SCC file is broadcast-compliant. The FCC accuracy benchmark requires verbatim dialogue including correct proper names and speaker identification.

What is a realistic WER for Whisper on a typical talking-head interview?

For a clean, single-speaker interview in English in a quiet environment, Whisper Large typically reaches a WER in the range of 3-8% on straightforward segments. Segments with technical jargon, proper nouns, or crosstalk will be worse. I always do a quick manual WER spot-check on the first dense-name segment of a new project: if it is above 10-15%, I budget time for a thorough correction pass rather than a quick skim.

Does Whisper produce speaker identification?

No. Standard Whisper produces a single undifferentiated transcript. Speaker diarization is a separate step; open-source tools like pyannote.audio can be combined with Whisper, but results are imperfect. For broadcast captioning where the FCC IP standard requires speaker identification, a human editor must add speaker labels after the ASR pass.

How does WER differ from caption accuracy in an FCC compliance sense?

WER is a numerical metric: it counts word-level substitutions, deletions, and insertions relative to a reference transcript. FCC accuracy is a legal standard that uses qualitative language and adds requirements WER does not capture: speaker identification, correct punctuation, non-speech audio, and no paraphrasing. A file can have a low WER and still fail the FCC benchmark if it omits speaker labels and sound effects.

Is YouTube's automatic caption generation good enough for accessibility compliance?

Not without review. WCAG 1.2.2 (the standard incorporated by Section 508 for federal agencies) requires captions for prerecorded synchronized media. The W3C guidance notes that captions must include dialogue and important background sounds - which auto-captions do not reliably provide. Section508.gov guidance explicitly states that auto-captioning alone is insufficient. After a correction pass, export the cleaned SRT to WebVTT with SeConv (seconv input.srt WebVTT --output-folder out --overwrite) and attach it via an HTML5 <track kind="captions"> element.

Official Sources

Dao Huy (Lucas)
Dao Huy (Lucas)
Diplômé de l'Université des langues étrangères, Université de Da Nang. Signe une attestation d'exactitude sur chaque traduction certifiée (USCIS, IRCC).

Écrit par Đào Huy (Lucas), traducteur vietnamien et spécialiste de la localisation (anglais, chinois et français vers le vietnamien). See translation services →

DevisWhatsApp