NeuBird
LoginDemo

How Much Does an AI SRE Investigation Cost in LLM Tokens?

An AI SRE investigation costs the total input and output tokens of every reasoning turn the agent takes, multiplied by the model's rate. The rate explains little of the bill. The quantity of tokens does, and the agent's architecture sets that quantity: how much telemetry enters each prompt, and whether it is re-sent on every later turn.

Teams approve an AI SRE budget expecting faster resolution and then meet an invoice that climbs with telemetry volume and incident frequency. The per-token rate on the model's price sheet explains little of that invoice. The bill is set by how much telemetry the agent loads into context on each step and how many times it re-sends that context while it reasons toward a root cause. Two agents running the same model on the same incident can differ many times over in token cost. One pastes raw telemetry into the prompt. The other queries data where it lives and feeds the model only what a step needs. Token cost is an architecture decision, made before the first prompt is sent, and it is the first thing to evaluate in an AI SRE.

What drives the token bill for a single investigation

An investigation's token cost is the sum of input and output tokens across every reasoning turn the agent takes. A turn is one round trip. The agent sends the model a prompt made of instructions, the conversation so far, and any data it has fetched. The model returns a response that usually decides the next query. An investigation that takes a dozen turns is billed for a dozen prompts and a dozen responses, and each prompt carries whatever the agent has accumulated up to that point.

Input tokens usually dominate the bill. A model is stateless between calls. To keep what it found on turn one, the agent has to include that material in the prompt for turn two and every turn after. Total input tokens therefore grow faster than the number of turns: doubling the turns more than doubles the input bill. Output tokens cost more per token than input tokens across major providers, but the model's reasoning runs to a few paragraphs per turn. A single log query can return a payload many times that size, and that payload is then re-sent on every later turn.

Four drivers set the total, and the fourth decides whether the other three compound:

DriverWhat it measuresAgent that dumps raw contextAgent that serves tailored context
Context size per turnTokens of instructions, history, and fetched data in each promptGrows every turnStays roughly flat
Number of reasoning turnsQuery-and-reason cycles before the agent concludesEach extra turn re-sends everything so farEach extra turn adds only a short conclusion
Tool-call payload sizeVolume each query to logs, metrics, or traces returns into contextRaw results, unfilteredResults filtered and aggregated on the backend first
Re-sending context each turnWhether earlier payloads ride along on every later turnEvery payload, every turnPayloads dropped, conclusions kept

An agent that appends every raw payload pays for the first turn's data on every subsequent turn. A modest payload fetched early becomes a large recurring charge by the end.

Context sent to the model, turn by turn
Raw telemetry in the prompt
TURN 1
TURN 2
TURN 3
TURN 4
TURN 5
TURN 6
Every earlier payload is re-sent, so each prompt is larger than the last.
Tailored context
TURN 1
TURN 2
TURN 3
TURN 4
TURN 5
TURN 6
Earlier findings travel as short conclusions, so each prompt stays close to the first.

Per-token price versus investigation cost

Model price per token is a rate. Investigation cost is a quantity of tokens multiplied by that rate, and the quantity varies far more than the rate does. A lower-priced model driving a context-heavy agent can produce a larger bill than a higher-priced model driving a disciplined one.

Per-token priceTokens per investigation
Set byThe model the tool licensesThe tool's architecture
Spread between toolsA few multiples between major modelsMany times over, on the same model and incident
What it predictsThe cost of one tokenThe bill

Comparing AI SRE tools on per-token price measures the variable that moves least. The useful evaluation questions are how much context a typical investigation pulls in, how many turns it takes, and whether payloads fetched on early turns are still being sent on late ones.

Architecture carries this much weight because an AI SRE is an agent loop. The loop is what separates an AI agent from a bare LLM for SRE work. A single LLM call takes one prompt and returns one answer, so its cost is bounded by that exchange. An agent queries a data source, decides what to query next, and repeats until it has isolated a cause. The loop lets the agent investigate, and each repetition carries the loop's history, so the loop also multiplies token volume.

Why token cost scales with telemetry volume and incident frequency

The production estate sets how large each investigation's context can get. 83% of organizations run four or more observability tools, according to NeuBird's 2026 State of Production Reliability and AI Adoption Report. Each tool is another source the agent can query, and in a naive design each result lands in context and stays there. An investigation that pulls logs from one tool and metrics from a second carries two payload streams, and every later turn re-sends both.

Incident frequency multiplies that per-investigation cost. Engineers spend 40% of their time on incident management, per the same report, so investigations run constantly. A per-investigation cost that looks tolerable becomes a line item that tracks incident count. Teams cannot investigate less, which leaves one lever on the bill: how many tokens each investigation consumes.

An ingest-first design makes the scaling worse. It replicates production telemetry into a second store that the agent reads from. That design bills twice for the same data: once to ingest and hold the copy, and again to reason over it in context. Both bills track telemetry volume, because a larger store gives the agent more to pull in.

Where telemetry goes before the model reads it
Ingest-first
Production telemetry
↓ copied
Second store, billed to hold the copy
↓ loaded
Model context, billed again for the same data
Query in place
Production telemetry, where it lives
↓ scoped query
Filtered and aggregated on the backend
↓ served
Model context: the signals this step needs

A token bill that tracks the size of the estate rather than the size of each incident means the agent is reading more than the incident requires. Cloud spend behaves the same way: spend that climbs faster than the workload it serves usually points to something reading more than the service requires. That is why cloud cost is a reliability signal worth reviewing alongside error rates and latency.

Naive context dumping versus tailored context

A naive agent treats the prompt as its working memory. When it needs logs, it queries the log store and pastes the result into context. When it needs metrics, it pastes those in too. By the final turn the prompt holds every raw payload fetched, most of it irrelevant to the conclusion, and the model has read all of it on every turn since it arrived.

An architected agent keeps data where it lives. It queries the log store and the metrics backend in place, scoped to the time window and services the current step concerns, and it filters on the backend before anything is sent to the model. The model receives the handful of log lines or the one anomalous series that the step turns on. Earlier findings travel forward as a few sentences stating what was ruled in or out, and the raw payloads behind them are dropped.

"A good AI agent has to be four things at once: cost-efficient, fast, accurate, and secure. Teams routinely underestimate cost. The fix is doing the heavy lifting on the backend: pre-filtering data so the agent only processes the context it actually needs. Token efficiency is the new engineering efficiency."

Vinod Jayaraman, CTO & Co-founder, NeuBird

A vendor's answers to these questions predict its token bill far better than its model's rate does:

Question to ask the vendorAnswer that keeps spend boundedAnswer that inflates it
Does raw telemetry enter the prompt?No: the backend filters and aggregates firstYes: query results are pasted in as returned
Is a payload fetched on turn two still sent on turn ten?No: it is replaced by a short conclusionYes: context accumulates for the whole investigation
Does a recurring incident start from the last verified fix?Yes: remembered conclusions are reusedNo: every recurrence is investigated from scratch

How NeuBird keeps investigation token cost bounded

NeuBird is the AI SRE, and it answers each of those questions the bounded way.

ProblemWhat NeuBird does
A second store whose size and bill track the estateQueries 50+ integrations in place, with no bulk telemetry stored and no second log lake
Raw telemetry re-sent on every turnFilters on the backend and serves the model tailored context for each step
Recurring incidents investigated at full costRemembers every verified fix, with no bulk telemetry stored, so a recurrence starts from the prior root cause
Token spend nobody can attributeAttributes model and cloud costs back to specific teams

The result is roughly 90% lower token spend and roughly 10% of the cost of alternatives. The savings come from the quantity side of the bill, the number of tokens each investigation consumes. The filtered context still carries the signals the diagnosis turns on: NeuBird analyzes 15+ sources in parallel, read where they live, and isolates root cause in under 5 minutes at 94% accuracy.

Memory compounds the saving over time. NeuBird's memory holds conclusions, never raw logs, metrics, or traces. A recurring incident consumes far fewer tokens than a fresh investigation, and between incidents the same remembered fixes feed prevention, so fewer incidents need investigating at all.

Evaluate AI SRE tools on how many tokens each investigation consumes, not on the per-token rate. To see how NeuBird keeps that number bounded on your own stack, see the AI SRE.

FAQ

Frequently asked questions

How much does one AI SRE investigation cost in LLM tokens?

The cost is the total tokens consumed across every reasoning turn, so it varies with the agent's architecture more than with the model. Input tokens dominate because accumulated context is re-sent on each turn. An agent that queries data in place and serves tailored context consumes far fewer tokens than one that pastes raw telemetry into the prompt.

Why is per-token price a poor way to compare AI SRE tools?

Per-token price is a rate, and investigation cost is a quantity of tokens multiplied by that rate. The same model produces very different bills depending on how the agent assembles and re-sends context. Evaluate tools on the tokens each investigation consumes, including context size per turn and the number of turns.

What makes an AI SRE investigation burn more tokens than expected?

Three habits cause most overruns. Loading raw logs, metrics, and traces into the prompt inflates every turn. Running many reasoning turns that each re-send the accumulated context makes input tokens grow faster than the turn count. Keeping large tool-call payloads in context, where they ride along on every later turn, compounds both.

Does querying telemetry in place actually reduce token cost?

Yes. Querying in place lets the backend filter, aggregate, and narrow telemetry before anything reaches the model, so each step receives only the context it needs. Nothing is copied into a second store. NeuBird queries 50+ integrations in place with no bulk telemetry stored, for roughly 90% lower token spend.

How does institutional memory affect token cost over time?

Remembered conclusions let an agent start a recurring incident from the prior verified root cause instead of rebuilding the same context from raw data, so it consumes far fewer tokens. NeuBird remembers every verified fix with no bulk telemetry stored: memory holds conclusions, never a copy of logs, metrics, or traces.

See NeuBird in action

Root cause in minutes, not war rooms.

Request a Demo →