HomeLearnCoursesHackathonsAccount
Speech Recognition & Synthesis
Why Audio Is a Different Kind of Input · 1/2

Sound is continuous, text is discrete

Text arrives pre-chopped into a finite set of tokens, letters, words, subwords, that a model can look up in a vocabulary. Audio has no such structure. A spoken sentence is a continuous sound wave, a single unbroken signal of air pressure changing over time, with no built-in boundaries between words, let alone between letters. Before any model can do anything useful with speech, that continuous wave has to be turned into a digital representation a computer can actually work with.

The first step is sampling: measuring the amplitude of the wave many thousands of times per second, commonly 16,000 or 44,100 samples per second. This turns the wave into a long sequence of numbers. That sequence is already usable in principle, but it is extremely long and dense, a few seconds of speech can be tens of thousands of numbers, which makes it hard for a model to find patterns directly in the raw waveform.