What Is CAG (Cache-Augmented Generation)? RAG's Faster Cousin

30 June 2026 · 2 min read

If you’ve read What Is RAG?, you know it’s the technique that lets AI answer questions using your own documents by retrieving relevant chunks at the moment someone asks. CAG (Cache-Augmented Generation) is a simpler, faster alternative for a specific — and common — situation.

The idea behind CAG

Instead of searching a document index fresh for every question, CAG loads a fixed, relatively small knowledge base directly into the model’s context ahead of time, and caches that processed state. When a question comes in, the model already “has” the material loaded — there’s no retrieval step to run first.

Think of RAG as looking something up in a filing cabinet every time you’re asked a question. CAG is more like having already memorised the one binder you need, so you just answer.

Why that’s faster

Skipping the retrieval step removes a chunk of latency and one point of failure (a bad search that pulls the wrong passages). For a knowledge base that doesn’t change constantly — your product catalogue, your standard policies, your onboarding manual — that’s a meaningful speed and reliability win, especially in a live conversation where every second of lag is noticeable.

When CAG makes sense

CAG is a good fit when:

The knowledge base is small and stable enough to fit comfortably in the model’s context window (this varies by model, but think “one well-organised handbook,” not “every email your company has ever sent”).
The content doesn’t change every day — policies, FAQs, product specs updated on a predictable schedule rather than in real time.
You want the fastest possible response in something like a live chat widget or voice assistant.

When RAG is still the better choice

RAG wins once the knowledge base is large, spans many documents, or changes frequently — support tickets, a growing document library, live inventory data. Retrieval scales to that in a way that stuffing everything into one cached context doesn’t.

In practice, a lot of businesses end up using both: CAG for the stable core of “here’s how we operate,” RAG for the larger, shifting body of documents and records behind it.

Not sure which fits your situation?

That decision depends entirely on how big and how volatile your knowledge base actually is — which is exactly the kind of assessment worth doing before building anything. Get in touch and we’ll help you figure out which approach (or combination) makes sense.