Agentic Retrieval, When the Agent Decides What to Fetch

Pull-quote: “The moment retrieval becomes a decision the agent makes, honesty stops being a default you inherit and becomes a property you engineer.”

Why this matters

A fixed RAG pipeline runs the same retrieval for every question: embed, search, stuff, generate. Agentic retrieval changes the contract. The retriever becomes a tool, and the agent decides whether to fetch at all, how to phrase the query, which source to ask, and when to stop digging. For hard questions this is strictly better: the agent decomposes the question, routes each sub-question to the right source, and iterates when the first pass comes back thin. But every one of those new freedoms is also a new way to be wrong, because fetching is now optional and the agent’s incentives are fluency, not diligence.

What the agent now decides

Four decisions move from your pipeline config into the model’s judgment: whether to retrieve or answer from priors, how to formulate and reformulate the query, which source fits which sub-question, and when the evidence is sufficient. Exposing the sources as tools behind a stable contract is what keeps this manageable. With retrieval sources published over MCP, each declaring its scope and usage rules, adding a new corpus is a server registration rather than an agent rewrite.

The failure modes

Retrieval laziness is the quiet one: the model answers from its priors and skips the fetch, and the answer often looks fine. Citation laundering is the dangerous one: the trace looks rigorous, the citations resolve, and the cited text does not say what the answer claims it says.

The honest loop

   Question ──► plan sub-questions
        │            budget: max calls, max depth, max tokens
        ▼
   Retrieval tools (MCP): vector index │ knowledge graph │ SQL
        │            every call logged: query, source, results
        ▼
   Draft answer with citations
        │
        ▼
   Verifier: does each cited span support its claim?
        │ no ──► fetch again, or answer "insufficient evidence"
        ▼ yes
   Final answer + the full retrieval trace

Three mechanisms do the work. Budgets bound cost and force prioritization, and they should bind per question class, not globally. Traces record every query issued, every source consulted, and every result returned, which turns “why did it say this” from speculation into a reading exercise. Verification closes the loop: an independent check that each cited span actually supports the claim attached to it, with “insufficient evidence” as a first-class answer rather than a failure state. Evaluate the retrieval behavior itself, not just final answers: fetch rates, source mix, rewrite depth. A falling fetch rate at steady accuracy usually means the judge got friendlier, not that the agent got smarter.

Where this shows up in the field

The baseline discipline is universal: retrieval and analytics tools exposed to agents over a stable contract, with every tool call logged as part of the answer’s record. Intelligence work earns an extra layer, because the domain rewards motivated reasoning: retrieval evidence passes a critic before a judgment ships, and the dissent is preserved alongside the conclusion.

Closing

Agentic retrieval is the right architecture for questions that need decomposition and source judgment. It stays honest only under engineering: budgets that bound the search, traces that make every fetch auditable, and verification that ties each claim to a span that actually supports it. Give the agent the freedom to decide what to fetch, and pair it with instruments that notice when it decides not to.