Just-in-Time vs Point-in-Time: When Should an AI Agent Read Your Telemetry?
Point-in-time agents read your telemetry before an incident. Just-in-time agents read it during one. That choice decides if the root cause is right.
Every AI agent that investigates production answers one design question: when does it read your telemetry? A point-in-time agent reads it before anything goes wrong and builds a model of your environment. A just-in-time agent reads it at the moment of investigation.
The choice decides whether the agent reasons over what is happening now or over what was true when it last looked. That difference determines whether the root cause it hands you is correct, and it is why just-in-time is the right design for production: incidents are usually caused by what changed since the last look.
Why did the CMDB fail as a map of production?
Operations teams have tried the up-front model before. The configuration management database (CMDB) promised a complete map of every service, host, and dependency. Teams populated it through discovery scans and manual entry.
The map drifted the day after each scan. Deployments, autoscaling, config changes, and new dependencies outpaced every refresh cycle. Few engineers trust their CMDB during an incident. They open the live system instead.
How do point-in-time AI agents work?
Some AI SRE products repeat the CMDB pattern. They crawl your observability tools continuously, before any incident. They ingest logs, metrics, and traces line by line and build an internal model of the environment. They query that model when an investigation starts.
The vendor's pitch is efficiency: the system makes few calls during the investigation because the reading already happened. That efficiency is paid for in staleness, noise, duplication, and load:
- Staleness. The model reflects production as of the last crawl. Incidents are usually caused by what changed since then.
- Noise. Most ingested data has no diagnostic value. The system pays to read and store it anyway.
- Duplication. Your observability platform already stores this telemetry. A second copy adds cost and a second source of truth.
- Load. Continuous crawling puts steady query pressure on the tools you rely on during incidents.
Counting API calls during an investigation hides all four costs.
Why is pre-indexing no longer necessary?
The pre-indexed design made sense when it was conceived. Agents had no standard way to query observability tools on demand, and building a local model up front was the practical workaround.
The Model Context Protocol (MCP) removed that constraint. An agent can now query Datadog, Splunk, Prometheus, Grafana, cloud APIs, and Kubernetes directly, scoped to the question at hand. Architectures designed before MCP still carry the workaround. Architectures designed for MCP do not need it.
How does just-in-time investigation work?
NeuBird's Production Ops Agent, powered by the Agentic Reliability Center (ARC), reads telemetry at the moment it is needed, in five steps:
- An alert, a question, or a predictive signal defines the problem.
- The agent identifies which sources and signals bear on that problem.
- It queries those sources live, over a trailing window sized to the problem.
- It narrows or widens the window based on what each query returns.
- It stops reading once the evidence supports a conclusion.
The goal is the least telemetry needed to solve the problem. The same approach applies to a live incident and to a predicted one. The data is always current because it is read at the moment of use, which is what real-time production context means in practice. The agent queries 50+ tools in place with zero telemetry storage, so there is no second copy to pay for or to drift out of date.
How do point-in-time and just-in-time compare?
| Dimension | Point-in-time (pre-indexed) | Just-in-time |
|---|---|---|
| When telemetry is read | Continuously, before any incident | At the moment of investigation |
| Data freshness | As of the last crawl | Current |
| Volume read | Everything the crawler reaches | Only what the problem requires |
| Copy of telemetry | Second copy held by the vendor | None; queries the source of record |
| Load on observability tools | Constant | Proportional to investigations |
| Calls during an investigation | Few | More, each scoped |
Pre-indexed systems make fewer calls during an investigation. They do so by reading far more data in total, and that data is older.
What should you ask a vendor about how it reads telemetry?
- When does your system read our telemetry: before incidents, during them, or both?
- How old is the data your agent reasons over when an investigation starts?
- How much telemetry do you ingest per day when nothing is wrong?
- Do you store a copy of our telemetry? Where, and for how long?
- What query load does your system place on our observability tools outside investigations?
- How does your system account for a change deployed five minutes before an incident?
The answers show which model a vendor runs, whatever its marketing says.
