Mapping sound patterns to words
At a conceptual level, an automatic speech recognition (ASR) model takes the audio signal, usually already converted into a spectrogram, and learns to map patterns of sound to the words and phonemes that produced them. A phoneme is a distinct unit of sound in a language, roughly the smallest building block of pronunciation, and there are only a few dozen of them in most languages. The model is effectively learning a giant lookup of the form 'this pattern of frequencies changing over time corresponds to this sequence of phonemes, which corresponds to this sequence of words,' trained on enormous amounts of paired audio and transcript data.
Modern ASR systems are trained end to end, meaning a single model learns the whole path from spectrogram to text, rather than using separate hand-built stages for phoneme detection, pronunciation dictionaries, and language modeling the way older systems did. This lets the model learn subtle correlations, like how a speaker's accent shifts certain sounds, directly from data instead of needing those rules programmed in by hand.
