How a song becomes lyrics

A finished song is like a baked cake — the eggs, flour and sugar are all combined, and you can't simply pluck one back out. A stem separator is a neural network trained to do exactly that: it listens to the finished mix and rebuilds the individual ingredients, or stems. We use it to pull the singing voice away from everything else.

Why bother? Because drums and guitars are noise as far as reading lyrics is concerned. Working from the clean voice track makes every later step easier.

The clean voice now goes to a speech-recognition model (the technical term is ASR — automatic speech recognition). It's the same family of technology that powers voice assistants and meeting transcripts: it listens to audio and writes down the words it hears, along with when each line was sung.

But singing is much harder than speech — words are stretched, pitched and layered with harmonies. So the raw output is good, not perfect. Press play below and watch the model's actual output light up as the song plays. The dotted words are its real mistakes — click them to see what was actually sung.

One model mishears; three models rarely mishear the same way. So we transcribe the song with three different speech models, then hand all three drafts to a large language model — think of it as a referee — that compares them line by line and asks: which version is actually a plausible lyric?

Below are three real moments from our song. Click through them — one shows the referee fixing a mistake all three models made, and one shows a line that stayed wrong. Honesty matters: this step helps a lot, but it isn't magic.

So… how good is the result? The industry grades transcripts with a wonderfully blunt metric: word error rate (WER) — out of every 100 words in the true lyrics, how many did the system get wrong? Lower is better. A human paying close attention scores a few percent; raw machine output on music is often 20–50%.