Vector databases emerged as a must-have technology foundation at the beginning of the modern gen AI era. 

What has changed over the last year, however, is that vectors, the numerical representations of data used by LLMs, have increasingly become just another data type in all manner of different databases. Now, Amazon Web Services (AWS) is taking the next leap forward in the ubiquity of vectors with the general availability of Amazon S3 Vectors. 

Amazon S3 is the AWS cloud object storage service widely used by organizations of all sizes to store any and all types of data. More often than not, S3 is also used as a foundational component for data lake and lakehouse deployments. Amazon S3 Vectors now adds native vector storage and similarity search capabilities directly to S3 object storage. Instead of requiring a separate vector database, organizations can store vector embeddings in S3 and query them for semantic search, retrieval-augmented generation (RAG) applications and AI agent workflows without moving data to specialized infrastructure

The service was first previewed in July with an initial capacity of 50 million vectors in a single index. With the GA release, AWS has scaled that up dramatically to 2 billion vectors in a single index and up to 20 trillion vectors per S3 storage bucket. 

According to AWS, customers created more than 250,000 vector indexes and ingested more than 40 billion vectors in the four months since the preview launch. The scale increase with the GA launch now allows organizations to consolidate entire vector datasets into single indexes rather than fragmenting them across infrastructure. The GA launch also shakes up the enterprise data landscape by providing a new production-ready approach for vectors that could potentially disrupt the market for purpose-built vector databases.

Adding fuel to the competitive fires, AWS claims that the S3 Vector service can help organizations to "reduce the total cost of storing and querying vectors by up to 90% when compared to specialized vector database solutions."

AWS positions S3 Vectors as complementary, not competitive to vector databases

While Amazon S3 vectors provide a powerful set of vector capabilities, the answer to whether or not it replaces the need for a dedicated vector database is somewhat nuanced — and depends on who you ask.

Despite the aggressive cost claims and dramati...