# Semantic caching

> Cut My AI Spend — rank #4 of 10. Typical savings: 30–70% of redundant calls eliminated. Effort: Medium — embedding store + similarity threshold.

Exact-match caches miss paraphrases. Semantic caching embeds incoming queries and serves a stored answer when a new query is similar enough to a previous one — eliminating 30–70% of redundant API calls in workloads where users ask the same things in different words (support, search, FAQ-style traffic).
It is the natural next step after prompt caching: prompt caching discounts repeated prefixes, semantic caching skips the model call entirely.

## How to do it

1. Embed each query; store (embedding, response) pairs in a vector store.
2. Serve cached responses above a tuned similarity threshold; start conservative (~0.95) and loosen with monitoring.
3. Scope caches per-user or per-tenant when answers depend on private context.
4. Set TTLs matched to how fast the underlying facts change.

## Frequently asked questions

### When is semantic caching a bad idea?

When answers are personalized, time-sensitive, or high-stakes. A stale or subtly-wrong cached answer costs more than the tokens it saved. Scope and TTL carefully.


---
Canonical: https://cutmyaispend.com/methods/semantic-caching
All methods: https://cutmyaispend.com/ | JSON: https://cutmyaispend.com/api/methods.json