Built around similarity, not exact match
A traditional relational database is architected around exact-match and range lookups: B-tree indexes let you jump straight to rows where a column equals, or is greater or less than, some value. That structure is a poor fit for the question a vector workload asks, which is not 'find rows where this column equals X' but 'find the vectors whose position in high-dimensional space is closest to this other vector'. A vector database's core indexing structure is built specifically to answer that question efficiently, typically using an ANN structure like HNSW or IVF as its primary index rather than as an add-on.
This changes more than just the index type, it changes the storage layout and query planner too. Vectors are often kept in memory or in memory-mapped structures because ANN graph traversal benefits enormously from low-latency random access, something spinning disk or even ordinary paged storage handles poorly. Write paths also look different: inserting a new vector may mean updating graph connections or reassigning it to an IVF cluster, which is a fundamentally different operation from appending a row to a table. Many vector databases are built as a layer purpose-fit for this workload rather than as a feature bolted onto an existing general-purpose database, though general-purpose databases increasingly offer vector extensions as well.