A modality is a fundamentally different kind of data
A modality is a type of data or signal, text, images, audio, video, each with its own native structure. Text is a sequence of discrete symbols pulled from a fixed vocabulary. An image is a 2D grid of continuous pixel values. Audio is a continuous waveform sampled over time. These aren't just different file formats, they're different mathematical objects, and a model built to process one of them cannot directly process another without some kind of translation step first.
This is why 'multimodal AI' isn't just about hooking a text model and an image model together and letting them pass messages. The real challenge is finding a shared representation where information from genuinely different kinds of data can be compared and combined meaningfully. If a system can't put 'the word dog' and 'a photo of a dog' on the same footing internally, it can't reason about how they relate, it can only handle them as two separate, disconnected problems.