Word-by-Word Karaoke Captions: ASS Tags, Whisper, and WebVTT

💡 Word-by-word karaoke captions highlight each spoken word in turn. For a quick SRT draft: whisper audio.mp3 --word_timestamps True --highlight_words True --output_format srt --max_words_per_line 4 --max_line_count 1. For real color sweeps, use ASS karaoke tag {\kf60}word (duration in centiseconds) with PrimaryColour &H0000FFFF (yellow) and SecondaryColour &H00FFFFFF (white pre-highlight).

What makes word-by-word captions different from regular subtitles?

A standard subtitle cue holds a full sentence on screen for 1-3 seconds. A word-by-word karaoke caption does the opposite: it highlights each word as it is spoken, synchronizing the visual reveal to the audio. The primary keyword here is "word-by-word karaoke captions," and the use case spans YouTube lyric videos, TikTok lip-sync clips, and any creator video where the caption is the visual hook rather than a compliance track.

I work with three implementations depending on the target. Whisper's --highlight_words flag gives an SRT with underline markup in under two minutes. ASS karaoke tags (\kf, \k) bake real color transitions into a .ass file that burns into the video cleanly with ffmpeg. WebVTT inline timestamps handle the web-player case without re-encoding, using CSS pseudo-classes.

Generating word timestamps with Whisper

Whisper's word-level timestamp feature aligns each recognized word to a specific time window in the audio. The command I run first on any new file:

whisper audio.mp3 \
  --model base \
  --language en \
  --word_timestamps True \
  --highlight_words True \
  --output_format srt \
  --max_line_width 42 \
  --max_line_count 1 \
  --max_words_per_line 4 \
  --output_dir captions/

• --model base: fast enough for a timing draft; switch to medium or large-v3 for accented speech or dense content.
• --word_timestamps True: enables per-word timing. Without this, --highlight_words has no effect.
• --highlight_words True: wraps the currently spoken word in <u>...</u> in the SRT. Players that support HTML inline tags render this as an underline, not a color change. For actual color you need ASS or WebVTT.
• --max_words_per_line 4: chunks the transcript into groups of four words per cue, short enough to read at phone-screen viewing distance.
• --max_line_count 1: one line per cue, so the display never stacks and loses the word-by-word focus.

The result is a first draft, not a production file. Whisper misplaces word boundaries on fast speech, overlapping voices, and proper nouns. I always do a manual timing pass in Subtitle Edit before delivering any file that uses this timing.

ASS karaoke tags: \kf and \k, colors, and the complete style block

ASS karaoke tags appear before each word in the dialogue event text, carrying a duration in centiseconds (100 = 1 second). Three variants exist:

• \k<dur>: instant fill switch from SecondaryColour to PrimaryColour when the word starts.
• \kf<dur> (identical to \K<dur>): left-to-right color sweep from SecondaryColour to PrimaryColour over the word's duration. This is the karaoke sweep effect.
• \ko<dur>: the outline appears instantly at word start. Useful for transparent-fill lyric styles.

ASS colors use &HAABBGGRR notation: AA = alpha (00 = fully opaque), BB = blue, GG = green, RR = red. Yellow (highlight) in this format is &H0000FFFF (R=FF, G=FF, B=00). White is &H00FFFFFF. Here is the full .ass style block I use for vertical 1080x1920 karaoke:

[Script Info]
ScriptType: v4.00+
PlayResX: 1080
PlayResY: 1920

[V4+ Styles]
Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding
Style: Kara,Montserrat,72,&H0000FFFF,&H00FFFFFF,&H00000000,&H80000000,1,0,0,0,100,100,0,0,1,3,1,2,30,30,160,1

[Events]
Format: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text
Dialogue: 0,0:00:01.20,0:00:04.50,Kara,,0,0,0,,{\kf30}This {\kf45}is {\kf60}ka{\kf55}ra{\kf70}oke

• PrimaryColour &H0000FFFF: yellow, the active highlight color.
• SecondaryColour &H00FFFFFF: white, the pre-highlight color before the sweep arrives.
• Fontsize 72 / Alignment 2 / MarginV 160: bottom-center at 160 px margin, keeping text above the iOS home indicator and TikTok's scrub bar on a 1920-px canvas.
• Outline 3: a 3 pt black outline on yellow and white text is non-negotiable for readability over any background.
• Bold 1: bold weight improves legibility at speed for karaoke.

To convert a Whisper SRT to ASS as a starting structure: seconv.exe captions/audio.srt AdvancedSubStationAlpha --output-folder captions/ --overwrite. Then open the .ass in Aegisub, switch to karaoke timing mode (the musical note icon), and tap each syllable to the audio. Aegisub inserts \kf tags at the right centisecond values automatically. To burn the .ass into a vertical video: ffmpeg -i input.mp4 -vf "ass=captions/audio.ass" -c:v libx264 -crf 18 -preset slow -pix_fmt yuv420p -c:a copy output_karaoke.mp4.

What word count and contrast settings keep karaoke readable?

Karaoke captions have tighter readability constraints than regular subtitles because the viewer is simultaneously tracking the highlight position and reading the full line.

• 3-4 words per cue maximum. More than four words means scanning a wide span while following the sweep. On a portrait phone at 72 pt, four short words fill the screen width comfortably.
• Minimum sweep duration 25 centiseconds per word. Any \kf tag below about 20 centiseconds runs faster than the eye can follow and the animation becomes noise. I flag and merge anything below 25 cs.
• Minimum cue duration 0.5 seconds. Very short cues disappear before the eye registers them. I run Subtitle Edit's "Fix duration too short" pass with a 500 ms floor.
• Contrast ratio 4.5:1 or higher, per WCAG 2.1 SC 1.4.3, between the highlight color and the background behind it. Yellow on a black outline over a dark background exceeds this easily. Yellow on white fails - always pair the yellow highlight with a dark outline.

The most common mistake I see is animating too fast: the sweep runs faster than the eye tracks and the caption becomes visual noise. The fix is not a bigger font - it is fewer words per cue or a slower delivery at record time.

WebVTT inline timestamps for HTML5 video

If the target is an HTML5 <video> element rather than a burn-in, WebVTT inline timestamps give karaoke behavior without re-encoding. Each word is preceded by a timestamp in angle brackets, monotonically increasing within the cue's start-end window:

WEBVTT

STYLE
::cue {
  color: white;
  font-size: 1.6em;
  background-color: rgba(0, 0, 0, 0.65);
}
::cue(:past) {
  color: #ffff00;
}
::cue(:future) {
  color: rgba(255, 255, 255, 0.5);
}

00:00:01.200 --> 00:00:04.500
<00:00:01.200>This <00:00:01.700>is <00:00:02.200>ka<00:00:02.700>ra<00:00:03.100>oke

• Inline timestamp format: <HH:MM:SS.mmm> or <MM:SS.mmm>, always increasing and within the cue's time window.
• ::cue(:past): styles words whose timestamp has already passed. I set yellow (#ffff00) to show what has been spoken.
• ::cue(:future): dims words not yet reached to 50% opacity so they are visible but not competing with the active word.
• Browser support note: as of 2026, ::cue(:past) and ::cue(:future) work well in Chrome but have inconsistent support in Firefox. Test in your target browser before deploying.

Attach the file with: <track kind="captions" src="captions/audio.vtt" srclang="en" default>. For vertical video captions embedded in a web page, set the container to aspect-ratio: 9/16.

When should I do this myself, and when is a specialist worth it?

A creator working in English on social video can run this entire pipeline with free tools. Whisper handles the transcript, Aegisub handles the karaoke timing, and ffmpeg handles the burn-in. Once the workflow is set up, the time cost is roughly one hour of editing per ten minutes of audio. For content you post yourself, that is a reasonable investment.

Where I recommend outside help: when the karaoke burn also needs a parallel accessibility track (not just the visual effect), that track must meet platform specs and, for US content, closed-caption compliance rules. A burned karaoke file is decorative; a compliant caption deliverable is a separate technical product. If you also need the caption in another language, that is the point where a native-language specialist saves both time and errors - a translated caption track needs the same timing precision with language-specific CPS constraints and line-break rules that auto-tools rarely get right.

FAQ

Does Whisper's --highlight_words generate real karaoke color or just an underline?

Just an underline. Whisper wraps the active word in <u>...</u> HTML tags in the SRT output. Most players render that as an underline, not a color change. For real color sweeps, use ASS \kf tags or WebVTT inline timestamps with ::cue(:past) CSS.

What is the difference between \k and \kf in ASS?

\k flips the fill color from SecondaryColour to PrimaryColour instantly when the word starts. \kf sweeps that color change from left to right over the word's duration, creating the karaoke wipe effect. For the visual sweep look, always use \kf. Use \k only when you want a hard cut with no animation.

Why is my karaoke caption unreadable even at large font sizes?

Almost always: too many words per cue combined with a sweep duration below 20 centiseconds per word. The animation runs faster than the eye can track and becomes visual noise. Reduce to 3 words per cue maximum and verify no single \kf tag has a duration below 25 centiseconds. Also confirm you have Outline 3 or higher and a contrasting highlight color, not yellow on white without an outline.

Can I use ASS karaoke as a soft track in MKV without burning in?

Yes. MKV supports ASS/SSA as a subtitle track codec, and players like VLC, mpv, and MPC-HC render \kf animation from a soft ASS track. Mux with ffmpeg: ffmpeg -i video.mp4 -i captions/audio.ass -map 0 -map 1 -c copy -c:s ass out.mkv. MP4 does not support ASS tracks; for MP4 you must burn in or use a WebVTT/SRT track without color animation.

What Fontsize should I use in a .ass file for 1080x1920 vertical video?

At PlayResY 1920, I use Fontsize 72 as a baseline for karaoke text. That maps to roughly 3.75% of the frame height, readable on a 6-inch phone at arm's length. Increase to 80-96 for a dominant lyric style. Do not go below 60 or viewers on small screens will struggle at normal watch distance.

Official Sources

• W3C WebVTT Living Standard - inline timestamps, cue settings, STYLE block, ::cue/:past/:future. Verified Oct 2026.
• Aegisub ASS Override Tags Reference - \k, \kf, \ko karaoke tags, centisecond units, &HAABBGGRR color format. Verified Oct 2026.
• OpenAI Whisper GitHub README - --word_timestamps, --highlight_words, --max_words_per_line, --max_line_count CLI flags (version 20250625). Verified Oct 2026.
• WCAG 2.1 SC 1.4.3 Contrast (Minimum) - 4.5:1 contrast ratio requirement. Verified Oct 2026.