How accurate is AI lyric transcription, really
In short
Measured against a human-annotated benchmark with correct lyrics supplied, our alignment reaches 97ms mean absolute error, 47ms median, and 95% of words within the 300ms tolerance the field treats as correct. That is the bulk of a song landing tight. It is also the optimistic figure, because it assumes the words themselves are right. On real uploads the transcription is the bottleneck, not the timing: a wrong word is a timing error by construction. Five things break it reliably, which are dense layered harmony, heavy vocal processing, ad-libs over the lead, quiet or sparse passages, and languages outside the dominant training data. The workable product is tight bulk plus a cheap manual fix, not perfection.
Every tool in this category implies that you upload a song and get correct, timed lyrics. It is worth stating plainly what the actual ceiling is, because the gap between the claim and the reality is where people lose an afternoon.
The numbers
Measured on JamendoLyrics, which is 20 English songs with word onsets labelled by hand and released under MIT:
| Metric | Our aligner | What it means |
|---|---|---|
| Mean absolute error | 97ms | Average distance from the human-labelled word start |
| Median error | 47ms | Half of all words land inside this |
| Within 50ms | 52% | Roughly a frame and a half at 30fps |
| Within 300ms | 95% | The tolerance the field generally treats as correct |
For context, published systems as of 2024 sat around 200ms mean error. The bulk of a song lands tight.
Now the caveat that matters more than any of those numbers: the benchmark hands the aligner the correct lyrics. It measures alignment in isolation, deliberately, because that is the only way to test an alignment change without transcription noise swamping the result. It proves the aligner. It does not prove the product.
On a real upload, the text comes from speech recognition, and a wrong word is a timing error by construction. There is no timestamp for a word that was never sung.
Why sung vocals are hard
Speech recognition is built on assumptions that singing deliberately violates.
Pitch is supposed to be incidental to meaning. In singing it is the point, and it moves across a range speech never uses. Vowels are supposed to last a certain duration. In singing they are held for whole bars. Consonants are supposed to be crisp boundary markers. Singers soften them constantly because a hard consonant interrupts tone.
On top of that, the vocal shares frequency space with everything else in the mix. We separate the vocal stem before doing anything else, which helps a great deal and is not free: separation itself introduces artefacts, and it is imperfect on dense material.
The five things that actually break it
Layered harmony. Three vocal takes stacked in thirds are three simultaneously valid transcriptions of the same line, at slightly different times. The model has to pick one, and there is no principled basis for the choice.
Heavy processing. Autotune, doubling, heavy reverb and delay all smear the acoustic boundaries the aligner reads. A long reverb tail in particular makes a word appear to end considerably later than it was sung.
Ad-libs over the lead. A background vocal saying something different underneath the main line is, acoustically, another voice in the same stem. Ad-libs get interleaved into the main transcript in the wrong place more often than any other single failure.
Quiet and sparse passages. An intro sung against near-silence gives energy-based detection almost nothing to work with. Counter-intuitively these are often harder than dense sections rather than easier.
Languages outside the dominant training data. Accuracy tracks training data availability closely. Code-switching mid-line is the worst case, because the switch has to be detected before it can be transcribed across.
The tail
Averages hide the shape of the failure, and the shape matters more than the average.
Our remaining error is not spread evenly. Most songs sit near the median. A small number fail badly, mis-seating entire phrases rather than drifting by a few tens of milliseconds. On our benchmark the worst track has a mean error of 466ms and a 90th percentile over 1.3 seconds while the median song is fine.
This is why an average is a misleading way to describe a system like this. Ninety-five percent of words within 300ms and one song in twenty that needs real intervention are both true at the same time, and the second one is what a user experiences on the day it happens to them.
What honest looks like in a product
Given all of the above, the design question is not how to reach perfection. It is what to do about the cases that will not reach it.
Our answers, in order of how much work they save:
Show the transcript before anything is built on it. The review step exists because a correction is cheap before styling and expensive after. Every downstream decision, from line breaks to layout, sits on the text.
Accept a pasted lyric sheet. If you have the real lyrics, giving them to the system removes the transcription problem entirely and leaves only the alignment problem, which is the one with the 97ms number attached to it. It is the single most useful thing a user can do here, and it is why the feature exists.
Make re-matching one action. Sometimes the wrong song was matched. Correcting that line by line is absurd, so it is one button.
Keep manual editing free and unlimited. Retiming a line, nudging a word, or shifting the whole track with a global offset costs nothing and is not rationed. A system with a known error tail cannot also put its correction tools behind a paywall.
What we would tell you before you upload
If your track is a single lead vocal, moderately processed, in a well-supported language, expect the transcript to need a couple of corrections and the timing to be publishable as-is.
If it is dense, layered, heavily processed, or in a smaller language, expect to paste your lyrics and expect to nudge a section or two.
Either way, listen to it once before exporting. We keep an internal rule that the objective benchmark is for iterating and the human ear is the final gate, and it applies to your track for the same reason it applies to ours: the metric cannot hear.
Questions people ask about this
- How accurate is automatic lyric transcription and timing?
- On a human-annotated benchmark with correct lyric text supplied, our aligner reaches 97ms mean absolute onset error, 47ms median, and places 95% of words within 300ms of the labelled onset, which is the tolerance the field generally treats as correct. Published systems as of 2024 sat around 200ms mean error. Those figures measure alignment only. End-to-end accuracy on a real upload is lower, because it also depends on the transcription being right.
- Why does AI get song lyrics wrong?
- Sung vocals violate most assumptions speech recognition is built on. Pitch is deliberately varied, vowels are held far beyond speech duration, consonants are softened for tone, and the vocal is mixed with instruments that occupy the same frequency range. Layered harmonies present several simultaneous versions of the same line, and heavy processing such as autotune, doubling and reverb smears the acoustic boundaries the model relies on.
- Can AI transcribe lyrics in languages other than English?
- Yes, though accuracy drops noticeably outside the languages with the most training data. English, Spanish and the other major European languages perform best. Smaller languages, heavy regional accents and code-switching within a single line are all harder, and code-switching is the hardest of the three because the model has to detect the switch before it can transcribe across it.
- What should I do when the transcription is wrong?
- Correct it before styling anything, because everything downstream is built on the text. For scattered errors, edit the individual words. If the transcript is broadly wrong, paste your own lyric sheet and let the system align that text instead of its own guess, which removes the transcription problem entirely and leaves only the alignment one. If the wrong song was matched, re-match it rather than correcting line by line.