How Much Does an AI SRE Investigation Cost in LLM Tokens?
An AI SRE investigation costs the total input and output tokens of every reasoning turn the agent takes, multiplied by the model's rate. The rate explains little of the bill. The quantity of tokens does, and the agent's architecture sets that quantity: how much telemetry enters each prompt, and whether it is re-sent on every later turn.
Teams approve an AI SRE budget expecting faster resolution and then meet an invoice that climbs with telemetry volume and incident frequency. The per-token rate on the model's price sheet explains little of that invoice. The bill is set by how much telemetry the agent loads into context on each step and how many times it re-sends that context while it reasons toward a root cause. Two agents running the same model on the same incident can differ many times over in token cost. One pastes raw telemetry into the prompt. The other queries data where it lives and feeds the model only what a step needs. Token cost is an architecture decision, made before the first prompt is sent, and it is the first thing to evaluate in an AI SRE.
What drives the token bill for a single investigation
An investigation's token cost is the sum of input and output tokens across every reasoning turn the agent takes. A turn is one round trip. The agent sends the model a prompt made of instructions, the conversation so far, and any data it has fetched. The model returns a response that usually decides the next query. An investigation that takes a dozen turns is billed for a dozen prompts and a dozen responses, and each prompt carries whatever the agent has accumulated up to that point.
Input tokens usually dominate the bill. A model is stateless between calls. To keep what it found on turn one, the agent has to include that material in the prompt for turn two and every turn after. Total input tokens therefore grow faster than the number of turns: doubling the turns more than doubles the input bill. Output tokens cost more per token than input tokens across major providers, but the model's reasoning runs to a few paragraphs per turn. A single log query can return a payload many times that size, and that payload is then re-sent on every later turn.
Four drivers set the total, and the fourth decides whether the other three compound:
| Driver | What it measures | Agent that dumps raw context | Agent that serves tailored context |
|---|---|---|---|
| Context size per turn | Tokens of instructions, history, and fetched data in each prompt | Grows every turn | Stays roughly flat |
| Number of reasoning turns | Query-and-reason cycles before the agent concludes | Each extra turn re-sends everything so far | Each extra turn adds only a short conclusion |
| Tool-call payload size | Volume each query to logs, metrics, or traces returns into context | Raw results, unfiltered | Results filtered and aggregated on the backend first |
| Re-sending context each turn | Whether earlier payloads ride along on every later turn | Every payload, every turn | Payloads dropped, conclusions kept |
An agent that appends every raw payload pays for the first turn's data on every subsequent turn. A modest payload fetched early becomes a large recurring charge by the end.
Per-token price versus investigation cost
Model price per token is a rate. Investigation cost is a quantity of tokens multiplied by that rate, and the quantity varies far more than the rate does. A lower-priced model driving a context-heavy agent can produce a larger bill than a higher-priced model driving a disciplined one.
| Per-token price | Tokens per investigation | |
|---|---|---|
| Set by | The model the tool licenses | The tool's architecture |
| Spread between tools | A few multiples between major models | Many times over, on the same model and incident |
| What it predicts | The cost of one token | The bill |
Comparing AI SRE tools on per-token price measures the variable that moves least. The useful evaluation questions are how much context a typical investigation pulls in, how many turns it takes, and whether payloads fetched on early turns are still being sent on late ones.
Architecture carries this much weight because an AI SRE is an agent loop. The loop is what separates an AI agent from a bare LLM for SRE work. A single LLM call takes one prompt and returns one answer, so its cost is bounded by that exchange. An agent queries a data source, decides what to query next, and repeats until it has isolated a cause. The loop lets the agent investigate, and each repetition carries the loop's history, so the loop also multiplies token volume.
Why token cost scales with telemetry volume and incident frequency
The production estate sets how large each investigation's context can get. 83% of organizations run four or more observability tools, according to NeuBird's 2026 State of Production Reliability and AI Adoption Report. Each tool is another source the agent can query, and in a naive design each result lands in context and stays there. An investigation that pulls logs from one tool and metrics from a second carries two payload streams, and every later turn re-sends both.
Incident frequency multiplies that per-investigation cost. Engineers spend 40% of their time on incident management, per the same report, so investigations run constantly. A per-investigation cost that looks tolerable becomes a line item that tracks incident count. Teams cannot investigate less, which leaves one lever on the bill: how many tokens each investigation consumes.
An ingest-first design makes the scaling worse. It replicates production telemetry into a second store that the agent reads from. That design bills twice for the same data: once to ingest and hold the copy, and again to reason over it in context. Both bills track telemetry volume, because a larger store gives the agent more to pull in.
A token bill that tracks the size of the estate rather than the size of each incident means the agent is reading more than the incident requires. Cloud spend behaves the same way: spend that climbs faster than the workload it serves usually points to something reading more than the service requires. That is why cloud cost is a reliability signal worth reviewing alongside error rates and latency.
Naive context dumping versus tailored context
A naive agent treats the prompt as its working memory. When it needs logs, it queries the log store and pastes the result into context. When it needs metrics, it pastes those in too. By the final turn the prompt holds every raw payload fetched, most of it irrelevant to the conclusion, and the model has read all of it on every turn since it arrived.
An architected agent keeps data where it lives. It queries the log store and the metrics backend in place, scoped to the time window and services the current step concerns, and it filters on the backend before anything is sent to the model. The model receives the handful of log lines or the one anomalous series that the step turns on. Earlier findings travel forward as a few sentences stating what was ruled in or out, and the raw payloads behind them are dropped.
"A good AI agent has to be four things at once: cost-efficient, fast, accurate, and secure. Teams routinely underestimate cost. The fix is doing the heavy lifting on the backend: pre-filtering data so the agent only processes the context it actually needs. Token efficiency is the new engineering efficiency."
Vinod Jayaraman, CTO & Co-founder, NeuBird
A vendor's answers to these questions predict its token bill far better than its model's rate does:
| Question to ask the vendor | Answer that keeps spend bounded | Answer that inflates it |
|---|---|---|
| Does raw telemetry enter the prompt? | No: the backend filters and aggregates first | Yes: query results are pasted in as returned |
| Is a payload fetched on turn two still sent on turn ten? | No: it is replaced by a short conclusion | Yes: context accumulates for the whole investigation |
| Does a recurring incident start from the last verified fix? | Yes: remembered conclusions are reused | No: every recurrence is investigated from scratch |
How NeuBird keeps investigation token cost bounded
NeuBird is the AI SRE, and it answers each of those questions the bounded way.
| Problem | What NeuBird does |
|---|---|
| A second store whose size and bill track the estate | Queries 50+ integrations in place, with no bulk telemetry stored and no second log lake |
| Raw telemetry re-sent on every turn | Filters on the backend and serves the model tailored context for each step |
| Recurring incidents investigated at full cost | Remembers every verified fix, with no bulk telemetry stored, so a recurrence starts from the prior root cause |
| Token spend nobody can attribute | Attributes model and cloud costs back to specific teams |
The result is roughly 90% lower token spend and roughly 10% of the cost of alternatives. The savings come from the quantity side of the bill, the number of tokens each investigation consumes. The filtered context still carries the signals the diagnosis turns on: NeuBird analyzes 15+ sources in parallel, read where they live, and isolates root cause in under 5 minutes at 94% accuracy.
Memory compounds the saving over time. NeuBird's memory holds conclusions, never raw logs, metrics, or traces. A recurring incident consumes far fewer tokens than a fresh investigation, and between incidents the same remembered fixes feed prevention, so fewer incidents need investigating at all.
Evaluate AI SRE tools on how many tokens each investigation consumes, not on the per-token rate. To see how NeuBird keeps that number bounded on your own stack, see the AI SRE.
