Retrieval Needs Admission Control

Retrieval systems are usually evaluated on whether they found something relevant.

Production AI systems fail because they found too many things that were relevant in incompatible ways.

A troubleshooting guide from last year matches the query. So does the current runbook. A support discussion contains the exact error message but describes an abandoned workaround. A policy summary is easier to read than the signed policy it paraphrases. Semantic search ranks them all highly because they share the right concepts.

The model receives a persuasive pile and is expected to discover which source deserves authority.

Retrieval is not complete when documents are found. The system still has to decide what is allowed into the working context.

Similarity Is Not Authority

Vector similarity answers a useful question: which content resembles this query?

It does not answer whether the content is current, approved, complete, scoped to this tenant, or permitted to control the decision. A postmortem and a runbook may be equally similar while serving opposite roles. One explains what went wrong. The other defines what to do now.

Every retrievable source needs metadata that supports authority decisions: owner, content type, effective period, version, scope, approval state, sensitivity, and supersession relationships.

The retrieval layer can then rank with more than semantic proximity. It can reject expired instructions, prefer primary sources, and label historical material as context rather than policy.

Authority should not be inferred from confident prose.

Admission Is a Separate Stage

Search and admission have different jobs.

Search should gather plausible candidates with enough breadth to avoid missing important evidence. Admission should assemble the smallest defensible context for the current step.

That second stage checks whether a candidate is authorized for the user and operation, fresh enough for the decision, within the correct domain, and compatible with higher-priority sources. It may include a document as background while preventing it from being treated as instruction.

Separating the stages makes the system easier to reason about. A team can improve recall without automatically flooding the model. It can inspect why an apparently relevant source was excluded. It can evaluate discovery and context quality independently.

The context window should be curated, not merely filled.

Build a Source Hierarchy

When sources conflict, the model needs a rule stronger than rhetorical confidence.

A source hierarchy might prefer signed policy over a wiki summary, the current service definition over an old ticket, a system-of-record query over a copied spreadsheet, and tenant-specific configuration over a global example.

The hierarchy will differ by domain, but it should be explicit. Attach authority class and scope to each admitted item. When two sources at the same level disagree, preserve the conflict rather than letting retrieval order choose a winner accidentally.

The model can then say, “The approved policy and current account configuration disagree,” which is an actionable state. Silently blending both into a plausible answer destroys the evidence needed to resolve it.

Freshness Is a Decision Requirement

Freshness is not one global time-to-live.

A product manual may remain useful for months. Inventory can change in seconds. A deployment status should be read immediately before an action. A legal policy may remain authoritative until a specific effective date even when a newer draft exists.

Define freshness per source type and decision. Record retrieval time and source version in the context manifest. Recheck volatile facts at the boundary where they become consequential.

Caching is still valuable. Cache the source response with its validity conditions. Do not let a performance optimization silently outlive the meaning of the data.

“Retrieved recently” and “valid for this decision” are different claims.

Treat Retrieved Content as Untrusted Data

Documents can contain instructions to the reader. That does not make them instructions to the agent.

A web page can say to ignore prior policy. A ticket can request secrets. A repository file can contain a malicious comment. Retrieval moves these tokens close to the model’s operating instructions, where their visual similarity becomes a security problem.

Preserve source boundaries in the assembled context. Label content as evidence with provenance and trust level. Do not concatenate it into the same undifferentiated block as system policy. Constrain tools and authority so a successful prompt injection still cannot create an unacceptable effect.

Admission control can also exclude sources that are unnecessary for the current action, reducing both attack surface and privacy exposure.

Diversify Evidence Deliberately

Top-k retrieval often returns several fragments from the same document or several documents that copied one another.

The context looks well supported while containing one underlying claim repeated five times.

Group candidates by origin and dependency. Prefer complementary evidence: the governing policy, the current system state, and the relevant case record. Deduplicate near-identical passages. When independent corroboration matters, require sources with genuinely separate provenance.

This is not about maximizing citation count. It is about assembling evidence that can challenge itself.

A model is less likely to notice a missing perspective when every retrieved item speaks in the same inherited voice.

Retrieve for the Current Step

One giant retrieval at the start of a long workflow becomes stale and noisy.

Planning needs architecture and requirements. Implementation needs specific files and interfaces. Approval needs the proposed artifact and consequence summary. Verification needs authoritative state after the action.

Retrieve again at these boundaries with a query shaped by the current decision. Carry forward stable artifacts and source identities, not the entire opening pile.

This step-level approach reduces context cost and makes provenance clearer. It also prevents a document that was useful for planning from continuing to influence execution after its role is over.

Retrieval is part of the loop, not a preface to it.

Make Exclusion Observable

When a system produces a bad answer, teams inspect what it retrieved. They should also inspect what it rejected.

Record candidate identity, score, authority, freshness, access result, admission decision, and reason for exclusion. Sensitive content need not be copied into every trace; references and governed snapshots can preserve the evidence.

This reveals failures that raw relevance metrics miss. The correct runbook may have been discovered but rejected because its metadata was incomplete. An old document may have been admitted because no supersession link existed. Tenant filtering may have removed the only applicable source.

Admission traces turn mysterious context behavior into an improvable system.

Evaluate the Context Set, Not Only the Retriever

Traditional retrieval evaluation measures whether relevant passages appear in the candidate list. Agent systems need end-to-end cases that test the admitted set.

Include stale but similar documents, conflicting authorities, duplicated claims, hostile instructions, missing metadata, and facts that expire during a long run. Ask whether the system produces the right answer, exposes uncertainty, requests a fresh read, or stops when authority is unresolved.

Measure unnecessary context too. A system that finds the answer while admitting fifty irrelevant documents may be fragile, expensive, and vulnerable even when the final response is correct.

The quality target is sufficient, authoritative context with minimal avoidable exposure.

Good Retrieval Knows What to Leave Outside

More context feels safer because omission is visible and clutter is not.

But every admitted source competes for attention, consumes budget, carries trust assumptions, and may outlive its freshness. Reliable retrieval therefore needs a gate.

Search broadly. Admit narrowly. Preserve source roles. Resolve or expose conflicts. Recheck volatile facts. Keep untrusted content inside its provenance boundary. Observe exclusion decisions.

The goal is not to give the model everything that might be relevant.

It is to give the current decision the evidence it is entitled to trust.