The brute-force wall
The most obvious way to find the nearest vectors to a query is exact nearest-neighbor search: compute the similarity between the query vector and every single vector in the collection, then sort and take the top results. This is exact, it will always find the true closest matches, and it's also the reason nobody runs it at real scale. Comparing against every vector means the query cost grows linearly with the size of the collection. At a few thousand vectors this is instant. At tens of millions of high-dimensional vectors, computing millions of similarity scores for every single query becomes far too slow for any interactive application, and the cost only gets worse as the collection keeps growing.
Approximate nearest neighbor, ANN, search is the answer: instead of guaranteeing the exact top results, ANN algorithms build an index structure ahead of time that lets a query skip the vast majority of comparisons, in exchange for a small, tunable chance of missing the true best match or returning a very close substitute instead. This trade, sacrificing a small amount of accuracy for a large amount of speed, is what makes vector search viable at real-world scale. Every ANN index makes this trade somewhere, the question is only how it's tuned.