What's Semantic Cache?
Cache Methods: 1. In-Memory Cache 2. DB Cache 3. Semantic Cache
Published Jan 29, 2024
Cache Methods:
1. In-Memory Cache often causes cost increases.
2. DB Cache usually data formats mismatch will cause the cache miss.
3. Semantic Cache stores prompts and responses, and evaluate hits based on semantic similarity.
1. In-Memory Cache often causes cost increases.
2. DB Cache usually data formats mismatch will cause the cache miss.
3. Semantic Cache stores prompts and responses, and evaluate hits based on semantic similarity.