Your RAG System Thinks It's Always Today
An HR assistant confidently quoted a parental leave policy that had been replaced months earlier. The current policy was live on the intranet the whole time. It had simply never been re-indexed, and nothing in the retrieval path had an opinion about which of the two documents was true.
That failure is usually told as a data hygiene story — someone forgot to run the pipeline. I think it is more interesting than that, because the same failure happens on a perfectly fresh index. Vector similarity does not know that one document superseded another. It knows they are both about parental leave.
Most RAG failures are retrieval failures
Between 70% and 80% of enterprise RAG projects never reach production, and the failures cluster before the model ever runs. When retrieval hands over incomplete, irrelevant, or badly ranked evidence, a stronger model does not save you. It produces a more fluent wrong answer.
A published taxonomy of retrieval failures puts numbers on the modes:
| Failure mode | Share | What it looks like |
|---|---|---|
| Granularity mismatch | 11.4% | Retrieved a whole handbook when the answer was one clause |
| Relevance miss | 9.4% | Topically right, missing the specific fact asked for |
| Recency lag | 7.4% | Correct document, wrong version of it |
Recency lag is the smallest of the three and by far the most damaging, because the other two produce answers that are visibly unhelpful. Recency lag produces an answer that is specific, well-sourced, plausible, and wrong. Users act on it.
Cosine similarity has no opinion about time
Most embedding-based retrieval treats every document as existing in an eternal present. There is no notion of recency, supersession, or version currency anywhere in the scoring function.
Worse, the bias frequently runs the wrong way. Superseded documents are often the more canonical text — the original policy written carefully from scratch, versus the amendment written as a diff against it. The old version reads like the definitive answer to the question, because when it was written, it was. Embedding search rewards exactly that quality.
So the failure is not that recency is weighted too lightly. It is that recency is not in the function at all, and the thing that is in the function has a mild preference for the past.
Recency is not a timestamp sort
The obvious fix — sort by date, take the newest — is worse than the problem. Plenty of content does not decay. An architecture decision record from 2021 explaining why the system is sharded the way it is remains exactly as true as the day it was written. Down-rank it for age and you have traded a stale-answer problem for a no-answer problem.
What you need is a decay that knows what kind of question it is answering. Which is why the classification step earns its place in the pipeline:
query → classify → vector search → recency re-rank → LLM streamClassification decides what temporal profile the query has. A question about current policy, pricing, or configuration gets aggressive decay. A question about rationale, history, or design intent gets almost none. A question about an incident gets a window rather than a slope — documents from around that time, not documents from recently.
The re-rank step then applies that profile to the candidate set, which is the right place for it: after retrieval has done recall, before the model has done anything at all. Re-ranking a shortlist of forty candidates is cheap. Filtering during vector search is not, and it wrecks recall.
Supersession is a different signal from age
Age is a proxy. What you actually want to know is whether a document has been replaced, and by what.
Where that relationship is knowable, encode it explicitly rather than inferring it from timestamps. A document that has a successor should carry a pointer to it, and retrieval should either follow the pointer or drop the ancestor. This is the difference between “this is old, so probably less relevant” and “this is wrong, and here is the thing that replaced it.”
Three sources of supersession worth capturing at ingest:
- Explicit versioning. Same document ID, new revision. Cheap and reliable when the source system has it.
- Structural replacement. A page at the same URL whose content changed materially. Requires content hashing at ingest, but catches most CMS-backed corpora.
- Semantic supersession. A new document that contradicts an old one without any structural link. This one is hard, and mostly worth handling by flagging conflicts for a human rather than resolving them automatically.
Most teams can get the first two for a day of work at ingest time, and they cover the majority of real supersession.
Freshness and ranking are separate problems
It is worth being precise about this, because conflating them causes teams to fix the wrong thing.
Index freshness is whether your corpus reflects the current state of the source. It is an ingestion problem, solved with change detection, incremental re-indexing, and a queue — this is the least glamorous part of the system and the part most likely to be silently broken. Instrument the age of the newest document per source, and alert when a source goes quiet.
Recency ranking is whether retrieval prefers current documents over superseded ones. It is a scoring problem, and it exists even on a perfectly fresh index, because a fresh index contains both versions.
The HR chatbot had a freshness bug. But a team that fixes only freshness still has a system that will happily return the 2019 policy alongside the 2026 one and let the model pick.
You cannot fix what you are not measuring
Around 70% of teams running RAG in production have no systematic evaluation of retrieval quality. Those teams cannot tell a retrieval failure from a generation failure, which means every bug report gets triaged as “the model is bad” and every fix is a prompt change.
The minimum viable eval is smaller than people think. Take fifty real queries. For each, record which source documents actually contain the answer. Then measure one thing: did those documents appear in the retrieved context at all.
That single number separates the two halves of the system. If the right document was in context and the answer was still wrong, you have a generation problem — prompt, model, or context window. If it was not in context, no amount of prompt engineering will help, and you have been debugging the wrong half.
Add the temporal dimension once that works: for queries whose correct answer changed at some point, does retrieval return the current version or the superseded one. That is the number that would have caught the parental leave policy months before a user did.
Sources: Why RAG systems fail in production, enterprise RAG failure rates, Beyond Similarity Search: a unified data layer for production RAG.
Common questions
- Why do RAG systems return outdated information?
- Because vector similarity has no concept of time. An embedding of a document superseded two years ago is just as close to the query as the document that replaced it, and often closer, since older documents are frequently written in more canonical language. Nothing in the retrieval step encodes that one version won and the other lost.
- What is recency re-ranking in RAG?
- Recency re-ranking is a second scoring pass applied after vector retrieval that adjusts document ranking based on age, version, and supersession before the results reach the model. It is not a sort by timestamp — it is a per-query-type decay applied only to content whose truth actually changes over time.
- Is recency the same as freshness of the index?
- No, and conflating them causes real outages. Index freshness is whether the corpus reflects the current state of the source. Recency ranking is whether retrieval prefers current documents over superseded ones. A perfectly fresh index still returns stale answers if ranking treats all versions equally.
- How do you evaluate RAG retrieval quality?
- Evaluate retrieval separately from generation. Build a set of queries with known correct source documents, then measure whether those documents appear in the retrieved context at all. Roughly 70% of teams running RAG in production have no systematic retrieval eval, which means they cannot distinguish a retrieval failure from a model failure.