AGENTIC REASONING: WHEN AGENTS HELP, WHEN THEY MAKE THINGS WORSE

Agentic reasoning is the loop a language model runs when it works through a problem in stages — thinking, taking an action against an external tool, observing the result, and reasoning again — rather than producing a single answer in one pass. Where the AI agents concept describes what an agent is and how organizations deploy it, this page is about the reasoning patterns inside the loop: how they are structured, when the extra steps earn their cost, and how they fail.

The distinction that matters is not whether a system "has an agent" but whether the problem actually needs multi-step reasoning at all. A model asked "what's our inventory?" can answer in one shot. A model asked to find current inventory, check a supplier API, and compute a reorder point has to reason about the dependency between those steps and act on the world between each one. The second is agentic reasoning; the first only looks like it if you over-engineer it. Most of the value in this topic comes from telling the two apart before you build anything.

The ReAct Loop

The foundational pattern is ReAct — reasoning interleaved with acting. The model emits a thought, chooses an action, reads back an observation, and repeats until it has enough to answer. A typical trace looks like this:

Thought: I need to find the order for customer ID 123
Action: query_database('orders', 'customer_id=123')
Observation: Found 3 orders. Let me get the most recent one's details.
Action: query_database('orders', 'id=789')
Observation: Order 789 has status "shipped"
Thought: The answer is ready. The order was shipped on March 3.
Answer: Your order was shipped on March 3.

Every arc of that loop is a fresh inference, so the cost scales with the number of steps: three iterations at roughly thirty cents each is about ninety cents, against thirty cents for the one-shot answer — a three-fold premium for the same question. That multiplier is the through-line of everything below, and it compounds directly with the inference cost of the underlying model.

Beyond ReAct: Tree-Search and Hierarchies

ReAct commits to one path at a time. When a problem has several viable solutions and one is far cheaper or more correct than the others, a tree-search pattern lets the model explore branches in parallel — querying customers one way, joining tables another, or reaching for a pre-built view — then evaluate which branch is right and which is fastest. The payoff is finding the optimal path rather than the first path that happens to work, but it typically burns five to ten iterations doing so, and it only makes sense when the search space genuinely contains meaningfully different answers.

Hierarchical patterns go the other direction, decomposing a hard problem rather than searching within it. A high-level agent handling "can this user upgrade their plan?" delegates to sub-agents that check the current plan, the available upgrades, and the payment method, then aggregates their results into a single answer. Each sub-agent is its own three-to-five-step loop, so a modest-looking hierarchy can easily consume twenty to thirty iterations end to end. The structure buys clarity and separation of concerns; it does not buy them cheaply.

When the Loop Helps and When It Hurts

Agentic reasoning earns its cost on multi-step problems where each step needs an external tool call and the result of one step genuinely determines the next — checking a balance before evaluating a purchase, filtering a customer list before acting on it, or running code, reading the error, and iterating. These are cases where no single inference could have produced the answer because the model needed to see the world change between steps.

It actively hurts on everything else. Simple lookups, text generation, classification, and summarization are all single-inference problems, and wrapping them in a loop adds latency, cost, and new failure surface for no benefit. The strongest default is skepticism: agents should be routed to deliberately when a problem is detected to be multi-step and conditional, not reached for by reflex. A useful test is to walk the dependencies — does the problem need multiple steps, does each step need a tool, and does step one's result decide step two? Only when all three hold does the loop pay off; if the steps are independent you can run them in parallel, and if there are no tool calls you can simply chain the logic.

What the Loop Costs

The economics are stark enough to drive the design decision on their own. A simple query is a single inference at roughly thirty cents; a two-step agentic query runs sixty to ninety cents; a five-step query lands somewhere between $1.50 and $2.50. So a five-step agent had better be solving a problem worth $2.50, and whether it is comes down to measuring the outcome rather than assuming it. When most of an application's traffic is simple — as it usually is — routing that traffic through agents multiplies the bill without moving the product. This is also why the agent is best understood as one layer inside a larger system rather than the system itself.

How Agentic Reasoning Fails

The loop introduces failure modes that a single inference never faces, and each has a cheap, well-understood guardrail. The most expensive is the infinite loop, where the model keeps deciding it needs one more action, observes nothing new, and decides again until fifty iterations have quietly cost fifteen dollars; a hard cap of five to ten iterations, with escalation on exhaustion, contains it. Wrong tool selection is subtler — the model reaches for the database when it needed the API, gathers partial information, switches, and stalls — and it is as much a prompt clarity problem as a tooling one, fixed with sharp tool descriptions and examples of when each applies.

Two more failures are structural. The model may hallucinate a tool call to a function that does not exist, which is why calls should be validated before execution, with the error fed back so the model can correct itself. And over a long enough loop the context window fills with intermediate observations until the model can no longer reason over them — context explosion — which is why observations should be summarized down to what the next step actually needs. None of these are exotic; they are the standing tax on running a loop instead of a single call.

Open Questions

• As per-step reasoning and tool reliability improve, how deep can autonomous loops safely run before compounding error, rather than cost, becomes the binding constraint?
• Can models learn to select the right reasoning pattern — one-shot, ReAct, tree-search, or hierarchical — for a given problem, or will that routing decision remain something engineers hard-code around cost?
• When a hierarchy of agents produces a wrong answer, how do you attribute the error to the specific sub-loop that caused it without replaying the entire tree?

Part of the knowledge graph at The Best Blog Ever — reference definitions for ideas that matter.