How beat sync actually works in a lyric video
In short
Lyric timing and beat timing are two independent clocks. Word timing comes from forced alignment against the isolated vocal, and is never moved onto the beat grid. The beat grid drives clip cuts, section markers and animation pulses. Sync feels right only once you add three small leads: fire beat-driven motion about 100ms early, start the word highlight 50ms before the vocal onset, and bring a line in up to 80ms ahead of its first word. The clock itself is never shifted; until September 2026 it was, and the shift was cancelling the lead.
If you have ever built a lyric video by hand, you know the specific frustration: the word looks like it lands on the beat, you play it back, and it is a frame off. You nudge it. Now it is a frame off the other way. Sometimes it is exactly on the beat and it still feels wrong.
That last case is the interesting one, and it is not a timing bug. It is a sign that you are treating one clock where there are two.
Beats and words are two different clocks
A song gives you two independent timelines, and they are produced by completely different machinery.
The beat grid is a property of the whole mix. A beat detector listens to the full track and returns a list of beat times, which of them are downbeats, and the tempo. Ours is beat_this, a neural detector from ISMIR 2024, with a multi-band librosa detector as fallback when the model cannot load. It knows nothing about lyrics. It would return the same grid for an instrumental.
Word timing is a property of the vocal performance. It comes from forced alignment: you hand an aligner the audio and the text, and it returns a start time for every word. Ours runs wav2vec2 over a vocal stem that demucs has separated from the mix, because aligning against a full mix means aligning against a drum kit. It knows nothing about tempo. It would return the same timings if you halved the BPM of the backing track.
These two lists do not line up, and they are not supposed to. A singer who lands every syllable exactly on the grid sounds like a metronome. The push and drag against the beat is the performance.
What we do not do: move words onto the beat
This is the part most tools get wrong, and it is worth being blunt about because we shipped the wrong version of it for six months.
Every word we produce carries an annotation naming its nearest beat: nearest_beat_ms, the beat number, whether that beat is a downbeat, and a confidence score. That annotation is used to draw markers, to colour the timeline, and to decide where a clip edge should snap.
It is never used to move the word.
We did once run a snapping stage, though it snapped to detected vocal onsets rather than to beats. It was described in our own architecture doc as the load-bearing accuracy lever. When we finally measured it against human-labelled ground truth it was making timing worse, and deleting it dropped median word error from 68ms to 48ms. The full numbers are in the benchmark writeup.
The general lesson survives the specific case. A correction step that cannot verify its own target will move things away from truth about as often as toward it.
Where the beat grid is genuinely useful
Having said all that, the grid is not decoration. It carries four jobs:
Clip edges snap to it, so cuts land on musical boundaries instead of arbitrary timestamps. Section boundaries are labelled from it, so a chorus can look different from a verse. Beat-driven motion, the pulse or scale bump that makes a video feel like it is moving with the song, is triggered from it. And it gives you the visible ruler that makes manual editing tractable at all.
None of those require touching a single word timestamp.
Why exactly on the beat reads as late
Here is the effect that catches everyone.
A visual event and an audio event register as simultaneous only when the visual arrives slightly first. Live performance has physical latency baked into it. A drummer's stick is visibly moving before the snare sounds. A guitarist's pick crosses the string before the note. Watching music has trained everyone's expectations around that lead.
So an animation fired at the exact beat timestamp reads as lagging, even though the arithmetic is perfect. The fix is to fire early:
const { fps } = useVideoConfig(); // never hardcode 30
const BEAT_ANTICIPATION_FRAMES = 3; // ~100ms at 30fps
const beatFrame = Math.round((beatTimeMs / 1000) * fps);
const triggerFrame = beatFrame - BEAT_ANTICIPATION_FRAMES;
Three frames is the value we ship. Below two the lead stops being perceptible. Past five the motion detaches and starts reading as early, which is a different kind of wrong.
The lead that used to be cancelled out
The anticipation offset is well known. This one is a correction of our own, and it is worth writing down because the mistake is easy to make.
Alignment pins a word to its acoustic onset: the consonant attack, or the start of an energy rise. That is the correct answer to the question the aligner was asked. A common theory says it is not where a listener hears the word, because word recognition anchors on the vowel, which for a consonant-initial word lands 50 to 100ms after the acoustic start. Until September 2026 we acted on that theory: the highlight led the onset by 70ms, and the whole karaoke clock was then shifted 80ms the other way to land on the vowel.
Measured, the two nearly cancelled. Worse, the aligner's onsets at the time were themselves about 27ms late on average, so the net result was a highlight roughly 10ms behind an onset that was already behind. The clock shift was removed. The aligner now takes a second opinion from other timing estimators on every word, its median signed error is 0ms on full songs, and there is one visual lead: the highlight fires 50ms before the onset, capped at half the gap to the previous word so a fast run stays in order.
Reading lags listening, and visual-early is the forgiving direction. Visual-late reads as off almost immediately. That asymmetry is the whole reason the lead exists and the reason it is small.
The three leads, in one place
| Lead | Value | What it applies to | Why |
|---|---|---|---|
| Beat anticipation | 3 frames, ~100ms at 30fps | Beat-driven pulses and motion | Visual must lead audio to read as simultaneous |
| Word highlight | 50ms, capped at half the gap to the previous word | The active-word highlight | Reading lags listening; early is the forgiving direction |
| Line entrance | Up to 80ms, scaled to the gap | A line appearing before its first word | The eye needs to find the line before it needs to read it |
All three are measured against a musical event, and every one is capped relative to the surrounding gap, so none can overrun the previous word during fast passages. A fixed 80ms lead is fine at 90 BPM and catastrophic in a triplet run. There is also a release rule: the highlight holds to the next word unless the next word is more than 250ms away, in which case it lets go at the word's end instead of lingering across the silence.
If the result still reads early or late to you, the Sync card in the Style panel has a per-project offset from 2,000ms early to 2,000ms late. The four different reasons lyrics can look off, and which tool fixes each, are in why the words look early or late.
What this adds up to
Beat sync sounds like one feature and is really a stack of small decisions, most of which are about perception rather than arithmetic. You detect the beats, separate the vocal, align the words against the vocal and leave them there, use the grid for cuts and motion, then apply three small leads, each bounded so it cannot run into its neighbour.
None of that is visible in the finished video. It shows up only as words landing where you expect them to.
Questions people ask about this
- Should lyrics be snapped to the beat in a lyric video?
- No. Words should be timed to the vocal performance, not moved onto the beat grid. Singers push and drag against the beat deliberately, and forcing each word onto the nearest beat destroys that phrasing. Use the beat grid for clip cuts, section changes and animation pulses, and let word timing follow the voice.
- Why do my lyrics look late even when the timing is correct?
- Because a visual event reads as simultaneous with a sound only when the visual arrives slightly first. Animation fired exactly on the beat timestamp reads as late, and a word highlight fired at the exact millisecond of the vocal onset reads as late too. The fix is a small lead: the highlight fires 50ms before the onset, capped at half the gap to the previous word, and beat motion fires three frames early.
- How far ahead of the beat should a lyric animation fire?
- Around 100ms, which is 3 frames at 30fps. Below about 2 frames the anticipation stops being perceptible; past about 5 frames the motion detaches from the beat and reads as early rather than tight. Denser genres want the lower end of that range and sparse ones tolerate the higher end.
- What is the difference between beat detection and forced alignment?
- Beat detection finds the rhythmic pulse of the whole mix and returns beat times, downbeats and tempo. Forced alignment takes a known lyric text plus the audio and returns a start time for each individual word. They answer different questions and run on different inputs, which is why a tool that only does beat detection cannot place words.