The Model Is Not the Moat
BloombergGPT lost to GPT-4 within months, and model prices fall 10x a year. The moat is the knowledge layer around the model, built from your estate.
A belief is taking hold in agentic operations: that the way to win is to train your own model. Take an open-weight base, reinforcement-train it on incident data, publish a benchmark, call it a moat.
I understand the instinct. Training a model is real engineering, and it makes a great announcement. But as a strategy, it misreads where value lives in this business: the model is a depreciating asset. The knowledge layer around it is the appreciating one. If you're betting a platform, or a purchase decision, on a proprietary model, history and arithmetic are both against you.
We have run this experiment before
In 2023, Bloomberg built BloombergGPT: a purpose-built finance model, trained on 363 billion tokens of proprietary financial data, about as strong a case for a domain model as anyone will ever have. Within months, independent evaluation found general-purpose GPT-4, with zero fine-tuning and zero access to Bloomberg's data, beating it on most financial tasks, and not narrowly. On conversational financial Q&A, the purpose-built model scored 43.4 to GPT-4's 76.5.¹
The purpose-built model lost on its home turf
Score out of 100 · higher is better
GPT-4 never saw Bloomberg's data. On conversational financial Q&A it scored 76.5 to the domain model's 43.4.
The lesson wasn't that Bloomberg executed badly. It's that a domain model is a snapshot: it freezes one moment of general capability and adds domain flavor, and then the frontier moves. Every purpose-built model since has faced the same actuarial table: lapped not by a competitor's domain model, but by the next general release plus good engineering around it.
The arithmetic is worse than the history
Even when a custom model is good on day one, its economics decay on a schedule. The price of constant LLM capability has been falling roughly 10× per year: GPT-3-class capability cost ~$60 per million tokens in late 2021 and ~$0.06 three years later, a thousandfold collapse. GPT-4-class capability has followed the same curve.² A one-billion-parameter model today outperforms a 175-billion-parameter model from three years ago.
Same capability, 1,000x cheaper in three years
Price per million tokens · log scale
Whatever cost and accuracy a custom model offers at launch, the open market matches it for a tenth the price within a year.
Now apply that to a proprietary model's pitch. Whatever cost–accuracy point it occupies at launch, the open market delivers that same point for a tenth the price within a year, without the owner spending another GPU-hour. The custom model must be perpetually retrained just to stand still, while its general-purpose rivals improve by somebody else's billions. That is not a moat. It is a treadmill with a capital bill attached.
No model can know your estate ahead of time
Here is the deeper problem, and it holds no matter how good the model is: the knowledge that decides production investigations does not exist at training time.
Your dependency graph changed this morning, three deploys ago. The connection-pool fix that worked in September is wrong since the architecture review in October. The alert that matters today is the one your historian started emitting after last week's vendor upgrade. None of this is in any training corpus, anywhere, ever. It is private, perishable, and specific to one estate. A model can memorize the internet; it cannot memorize your Tuesday.
What answers incidents accurately is therefore not what the model knows but what it is shown: a live, confidence-weighted map of services and dependencies; institutional memory of every prior incident, versioned and allowed to decay; retrieval that assembles the minimum evidence for this step of this investigation; and an independent referee that re-checks every claim against live telemetry before a human sees it. That is context and knowledge engineering. It is why our platform holds 94% accuracy across thousands of incident types at under five minutes per verified root cause, numbers that have survived contact with Fortune 500 estates, not a self-authored benchmark. And unlike a trained model, this layer appreciates: every incident makes the map sharper and the memory deeper.
The architecture that actually wins the cost–accuracy curve
If models depreciate 10× a year and knowledge lives outside the weights, the winning design follows directly: be model-interchangeable, and route.
Our platform treats models (frontier, open-weight, small and specialized) as an array of interchangeable parts. Each step of an investigation goes to the smallest model that holds accuracy for that step: heavy causal synthesis to a frontier model, extraction and classification to small open-weight models, a seven-stage alert pipeline that spends exactly one model call. Roughly 90% of repeated context is served from cache at a tenth the rate. The result is about a tenth of raw-context token cost at equal or better accuracy, and the whole curve shifts in our favor every time any lab, anywhere, releases a better or cheaper model. Benchmark after benchmark, the leaderboard changes hands; a router inherits the new winner the day it ships, while a proprietary model is married to the base it was trained from.
To be clear: fine-tuned and custom small models absolutely have a place. We run them ourselves. Where the task is fixed, known and high-volume (alert correlation, log and ticket classification, entity extraction, dedup), a small tuned model is the right tool: faster and dramatically cheaper than a frontier call, on a distribution stable enough that training doesn't go stale overnight. That is precisely what routing is for: ancillary tasks go to cheap specialists to cut cost and latency. What a custom model should not be is the reasoning core, the open-ended causal judgment at the center of an investigation. That job belongs to the best general model alive this quarter, whichever lab it comes from, because frozen judgment in a moving world is where accuracy goes to die.
This is also why we can price on outcomes and run on the customer's own model account: Gemini on their Vertex commitment, Bedrock, Azure, or open-weight in their VPC. The platform's edge doesn't live in a model we must protect and amortize. It lives in the layer no one else has: the customer's own estate, mapped, remembered, and verified.
Questions to ask any "we trained our own model" vendor
- What happens when the frontier moves? If next quarter's general model beats yours at a tenth of your inference cost, as the last four years say it will, what, exactly, is left of the differentiation?
- Who re-trains when my estate changes? My dependencies change daily. Your weights change per training run. Where does my Tuesday live?
- Who graded the benchmark? A few dozen cases, authored and scored by the model's creator, measures the author's scaffold as much as the model. Ask for accuracy on your incidents, verified against your telemetry, with the evidence attached.
- What am I paying to amortize? A proprietary training pipeline is a cost the vendor must recover (from you) on an asset that loses an order of magnitude of value a year.
Models are fantastic, and getting cheaper faster than any technology we've ever bought. That is precisely why they are not the moat. The moat is everything the model sees, and that layer is built, continuously, from your estate. It cannot be trained in advance by anyone. Including us.
Frequently Asked Questions
Is a custom AI model a competitive moat?
A custom AI model is a weak moat because it depreciates. General-purpose models improve and get cheaper roughly 10× a year, so whatever cost and accuracy a custom model offers at launch, the open market matches within about a year. The durable advantage is the knowledge layer around the model: live dependency maps, incident memory and verification against telemetry.
Why did GPT-4 outperform BloombergGPT?
GPT-4 outperformed BloombergGPT because a domain model freezes one moment of general capability, and the frontier kept moving. Independent evaluation found GPT-4, with no fine-tuning and no Bloomberg data, beating BloombergGPT on most financial tasks, including 76.5 to 43.4 on ConvFinQA conversational Q&A.
How fast are LLM prices falling?
The price of constant LLM capability has fallen roughly 10× per year. GPT-3-class capability cost about $60 per million tokens in late 2021 and about $0.06 three years later, a thousandfold drop, according to a16z's "LLMflation" analysis. GPT-4-class capability has followed the same curve.
When does a fine-tuned small model make sense?
A fine-tuned small model makes sense when the task is fixed, known and high-volume, such as alert correlation, log and ticket classification, entity extraction and deduplication. On a stable distribution, a small tuned model is faster and far cheaper than a frontier call. It should not be the reasoning core of an investigation.
How does NeuBird choose which model to use?
NeuBird routes each step of an investigation to the smallest model that holds accuracy for that step: frontier models for causal synthesis, small open-weight models for extraction and classification. Caching serves about 90% of repeated context at a tenth the rate. NeuBird, the AI SRE, can also run on the customer's own model account, including Vertex, Bedrock, Azure or open-weight models in their VPC.
Sources
- Li et al., "Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics?" (arXiv:2305.05862), 2023. GPT-4 zero/few-shot vs. BloombergGPT: ConvFinQA 76.5 vs 43.4; FiQA sentiment 88.1 vs 75.1; Financial PhraseBank 86 vs 51 (F1×100).
- Guido Appenzeller, "Welcome to LLMflation: LLM inference cost is going down fast," a16z, November 2024. Inference price for constant MMLU capability falling ~10×/year; GPT-3-class from ~$60/M tokens (2021) to ~$0.06/M (2024).
Venkat Ramakrishnan is President and COO of NeuBird AI.





