Postgres and vector storage - The rise of pgvector in RAG world | S02 EP14
Jonathan Katz, AWS Principal PM is going to join us on the hottest topic in GenAI town - vector storage in RAG (retrieval augmented generation) domain. We will dive deep into pgvector extension capabilities. John will share his perspective on pgvector improvements and how it brings GenAI workloads closer to postgres customers.
Tony Mullen
Amazon Employee
Published Apr 10, 2024
Todays show focused we discussed included the rise of vector databases, choosing a vector database engine, improvements in PG Vector over the past year, integrating AWS AI/ML services with Postgres and PG Vector, and the PG Vector roadmap.
Key highlights:
- Vector math has existed since the early 1900s but using vectors for database similarity search is relatively new.
- There has been a lot of innovation recently across different vector database options with improvements in performance, relevance, and scalability.
- PG Vector now supports indexing methods like HNSW which can be over 20x faster than previous methods and makes it easy for developers to use.
- New PG Vector features like quantization will help workloads scale to billions of vectors by reducing index size.
- Role-based security in Postgres can be combined with PG Vector and retrieval augmented generation to control which users can access which vectorized documents.
Check out the recording here:
Loading...
Tony Mullen - Senior RDS Specialist Solutions Architect @ AWS
Jonathan Katz - Principal Product Manager - Technical @ AWS & PostgreSQL Core Team Member
Jay Sampath - Principal Solution Architect @ AWS
Raj Jayakrishnan - Senior Database Specialist Solutions Architect @ AWS
- Leverage pgvector and Amazon Aurora PostgreSQL for Natural Language Processing, Chatbots and Sentiment Analysis - https://aws.amazon.com/blogs/database/leverage-pgvector-and-amazon-aurora-postgresql-for-natural-language-processing-chatbots-and-sentiment-analysis/
- AWS re:Invent 2023 - Best practices for querying vector data for gen AI apps in PostgreSQL (DAT407) - https://www.youtube.com/watch?v=PhIC4JlYg7A
- Accelerate HNSW indexing and searching with pgvector on Amazon Aurora PostgreSQL-compatible edition and Amazon RDS for PostgreSQL - https://aws.amazon.com/blogs/database/accelerate-hnsw-indexing-and-searching-with-pgvector-on-amazon-aurora-postgresql-compatible-edition-and-amazon-rds-for-postgresql/
- Jonathan Katz blog - https://jkatz05.com/
- Generative AI Use Cases with Aurora PostgreSQL and pgvector - https://catalog.workshops.aws/pgvector/en-US & /github.com/aws-samples/aurora-postgresql-pgvector
- Building AI-powered search in PostgreSQL using Amazon SageMaker and pgvector - https://github.com/aws-samples/rds-postgresql-pgvector
You can check out our past shows from out community page -
Any opinions in this post are those of the individual author and may not reflect the opinions of AWS.