How Decision Models Are Different From Large Language Models

I recently opened a chat with Verizon's AI agent to get an unwanted service taken off my bill. I fully expected the usual routine: explain the problem to the bot, watch it give up, and wait for a person to join the chat. That never happened. The agent asked me whether I wanted the service removed. I said yes. It removed the service on the spot and told me what my new, lower bill would be. Amazing!

That chat was the first time I saw an end-to-end agentic workflow in a consumer-facing chatbot: it took a real action with real consequences, in this case a smaller phone bill, with no person stepping in. I don't know how Verizon built their agent. Every action like that one still rests on a few judgment calls: is the service actually on the account, does removing it affect anything else, and is it safe for software to do this without a person? Decision models are one way to make those calls quick, economical, and decided by objective probabilities.

A decision model is a new kind of AI model that reads text like a large language model (LLM) but never writes an answer. You give it the situation, a question, and the options you'll accept, and in a single pass it returns the option it picks with a calibrated probability attached. An LLM writes its answer one token at a time and can't give you a trustworthy number for how likely it is to be right. So no, decision models and LLMs are not the same thing. If you build agents, use a decision model for the judgment calls where you can list the options, set a probability threshold for when software may act on its own, and keep the LLM for the open-ended work.

A decision model puts a probability on the judgment calls your code can't make and your LLM is overkill for.

Why are LLM calls getting more expensive?

The price per token keeps falling, but the bills don't, because the newest models think before they answer and every thinking token is billed. Artificial Analysis ran the same seven benchmarks on each model and tracked the cost, as TechCrunch reported. Claude 3.6 Sonnet cost $81.41 to evaluate. Claude 3.7 Sonnet, which reasons before it answers, cost $1,485.35, at exactly the same $3 in and $15 out per million tokens. OpenAI shows the same pattern: GPT-4o cost $108.85, while o1 generated over 44 million tokens, about eight times as many, and cost $2,767.05.

Thinking used to be a special mode. Today most frontier models think by default, and on Claude Opus 5.5 you can't turn it off: Anthropic's documentation says a request that disables thinking returns an error. Reasoning is great when a task needs it. Every call your agent makes now pays for it whether the call needs it or not.

Reasoning models charge for the thinking they write before the answer, so the same benchmark suite cost 18 times more on Claude 3.7 Sonnet than on Claude 3.6 Sonnet at an identical price per token.

Higher costs haven't slowed usage down. OpenAI's API was processing 6 billion tokens a minute at DevDay 2025, and more than 15 billion a minute by early 2026, with OpenAI pointing to record engagement across agentic workflows. Anthropic's Economic Index found that 77% of business uses of Claude on the API follow automation patterns, compared to about 50% for people using the Claude app. Most business use is already software handing work to a model, not a person chatting with one.

How are AI models getting more efficient?

Open models showed that you don't need a frontier model for every job. A smaller model tuned for one task can do it well at a fraction of the cost, and the open-source community has found several ways to get there:

• Distillation. A big model teaches a small one. DeepSeek distilled the reasoning of R1, a 671-billion-parameter model, into dense open models as small as 1.5 billion parameters.
• Adapters. The big model stays frozen and a small add-on learns the task. LoRA cut the trainable parameters for GPT-3 175B by 10,000 times and performed on par with full fine-tuning.
• Quantization. Each weight is stored in fewer bits. Going from 16-bit to 4-bit weights cuts memory to about a quarter, which is how people run capable models on a laptop.
• One-job models. A model is trained to answer one kind of question. Meta's Llama Guard is an LLM-based safeguard that classifies conversations as safe or unsafe.

Decision models take the one-job idea further. Instead of one fixed question, they answer any question you can write down, as long as you also write down the options.

The cheapest way to make an AI call faster is to ask a smaller model a narrower question.

How many agent calls are actually decisions?

Engineers use LLMs to generate code, scripts and CLI commands, and LLMs are great at it. Many of the calls an agent makes are not generation at all, though. Watch an agent work an incident. "Is this alert credible?" is a decision. The LLM then writes the log and metrics queries. "Which service owns it?" is another decision. The LLM writes the rollback command. "Is it safe to run?" and "Did it work?" are decisions too.

Every orange box in that loop has a handful of known answers, yet many agentic systems send each one to an LLM and pay for it in latency, tokens, parsing and confidence:

1. Latency. The LLM generates its answer token by token, so each decision takes seconds, and an agent makes many decisions in one task.
2. Output tokens. You pay for every generated token, including the hundreds of reasoning tokens written before the answer.
3. Format risk. You need parsers, validators and retries for the times the JSON comes back broken.
4. Unknown confidence. An LLM saying it is "pretty sure" is not a probability. Without a calibrated number you can't set a threshold and let software act, so a person stays in every loop.

When an agent asks an LLM to make a yes-or-no call, it pays seconds of latency and hundreds of output tokens for one bit of information, with no probability it can set a threshold on.

What is a decision model?

A decision model is a model that chooses among options you provide and returns a probability with the choice. It reads text the way an LLM does, so it understands the situation you describe. Its only output, though, is a decision. There's no paragraph to parse and no reasoning to pay for.

Decision models answer yes-or-no, pick-one and rate-it questions, and every answer comes back with a probability:

The options for a pick-one question can be fixed, or you can build them at run time from something like a service registry. For a rate-it question, you describe each level in words, lowest first, and the model numbers them by position: 0, 1, 2. The score is each level's number times its probability, added up, so 0.05 minor, 0.50 degraded and 0.45 outage gives 1.4, between degraded and outage. You get one number you can sort by. Jev, the first decision model on the market, calls these three primitives Noul, Choice and Score.

A decision model returns one of the options you supplied, plus a probability, in a single pass: yes or no, pick one, or rate it on a scale.

How does a decision model answer differently from an LLM?

An LLM writes its answer one word at a time. At every step it scores every token in its vocabulary, around a hundred thousand of them, turns those scores into a probability distribution, picks one, adds it to the sentence, and starts again. A few hundred words means a few hundred rounds. The probabilities stay inside the model, and what you get back is just the words.

A decision model skips all of that. You give it the options, here just yes and no, and in one pass it scores only those. The probability isn't hidden: the answer is yes, at 0.97.

An LLM scores about 100,000 possible next words hundreds of times and hides the probabilities; a decision model scores only your options, once, and hands you the probability.

How are decision models trained differently from LLMs?

The two models are rewarded for different things during training, and that is why one gives you a usable probability and the other doesn't.

Chat LLMs are trained with reinforcement learning from human feedback (RLHF). The model produces answers, say "Definitely a duplicate." and "Might be a duplicate." for a customer who may have been charged twice. A person picks the one they prefer, and that answer is rewarded. People tend to prefer the answer that sounds sure, so every loop makes the model better at sounding sure, whether or not it is right.

Decision models are trained for calibration instead. The model answers with a probability, say 0.97, and training checks it against what really happened. A correct answer earns a small reward. A wrong answer at 0.97 earns a large penalty. Being sure and wrong is the most expensive mistake, so over millions of loops the model learns to be sure only when it is likely to be right. TypeSafe, the company behind Jev, call their version Reinforcement Learning for Calibrated Decisions (RLCD).

Calibration means the numbers mean what they say: across all the answers a calibrated model gives at 0.9, about nine in ten are right. That property is what lets you put a threshold in code.

RLHF rewards an LLM for answers people prefer, which teaches it to sound sure; calibration training penalizes confident wrong answers, which teaches a decision model to be sure only when it is right.

Decision model vs LLM: which is faster, cheaper and more accurate?

Every model trades off speed, cost and accuracy. A frontier LLM reaches furthest toward accuracy and handles hard, ambiguous cases well, but it is measured in seconds and you pay for every token it generates. A decision model covers a very different shape.

Jev is the decision model with the most published measurements. In the head-to-head test behind the live demo at my FLOCK '26 talk, Jev answered in 153 milliseconds against 1.6 seconds for Claude Sonnet 5.5, about 10 times faster and 90 times cheaper per call. Independent tests point the same way with smaller gaps. BrillMark triaged 100 support tickets with Jev and three mid-tier LLMs, and found Jev 4 to 7 times faster and 31 to 65 times cheaper on that workload. TypeSafe's own claims run much higher, so test on your workload before you believe any multiplier.

Accuracy is the catch: on a public 200-item benchmark drawn from four public datasets, Jev 1.13 scored 72.5% against 84.0% for Claude Fable 5.1. Neither shape covers the whole triangle, which is why you use both.

On a public 200-item benchmark, the decision model Jev scored 72.5% against 84% for Claude Fable 5.1, with a median latency of 0.43 seconds against 4.27.

What would the chatbot's decisions look like with a decision model?

I don't know what Verizon's agent runs on, but here is how the same flow could work with a decision model making the calls. I told the agent I didn't want the device protection plan, and the agent then has three questions to settle.

First, the agent reads the account and finds device protection on line 2, then asks the decision model whether the customer actually has the service: yes, 0.99. Second, it checks what depends on the service, such as bundle discounts, open claims and device payment plans, and asks whether removing it would affect anything else: no, 0.96. The third call is the one that surprised me: is it safe for the agent to do this itself? The account holder is verified, there's no fee and the service can be added back: yes, 0.94.

All three clear a threshold of 0.8, so when the customer confirms, the agent removes the service. If any had come back low, the agent would hand the chat to a person. The business decides exactly where a human has to step in, by setting a number.

A calibrated probability turns "should a person handle this?" into a threshold the business sets, instead of a judgment buried in generated text.

How do decision models fit into incident response?

Several incidents are often open at once, and the on-call team needs to know which one to respond to first. A decision model can make one call per open incident, all in parallel, and answer three questions in each pass: how severe it is, whether customers are affected, and whether it is getting worse.

// one call per open incident, in parallelawait Promise.all(incidents.map((incident) =>  decisionModel.decide({    state: incident.details,    questions: {      severity: pickOne('How severe is it?',        [sev1, sev2, sev3]),      customers: yesNo(        'Are customers affected?'),      worsening: yesNo(        'Is it getting worse?'),    },  })));

Rank the results by the probability of sev1. Anything at 0.8 or above gets a response right away, so the checkout errors and the duplicate charges go straight to the top, and the slow cache warm-up waits in the queue. The expired certificate is the interesting row: 0.55 on severity. The model is telling you it isn't sure, so a person or the LLM takes a closer look before the incident is ranked, instead of it being pushed up or left behind. The decision model makes the fast calls, and the uncertain ones get the expensive attention.

In incident triage, a low-confidence score is as useful as a high one: it routes exactly the ambiguous incidents to an LLM or a person.

Does context engineering still matter for decision models?

Context matters as much to a decision model as it does to an LLM. The riskiest decision in an investigation is whether a command is safe to run in production. The LLM proposes the fix, because a decision model can't write one, and the decision model scores it before it runs, here with an automatic-run threshold of 0.90.

The honest answer for the command alone, kubectl delete pod checkout-7f9c, is "it depends". Deleting a pod with nothing behind it is an outage; deleting one that gets recreated is routine. The model lands around half, and the command goes to a person, which is the right behavior. Add the cluster state straight from the Kubernetes API, that the pod belongs to a deployment with two of three pods healthy, and the same model gives the same question 0.97. We hit exactly this while building the demo: a perfectly valid input scored 0.61 until we added one sentence explaining the situation, and then it scored 0.99.

Context engineering principles carry straight over. Give the model the context a careful engineer would need to make the call, and find the right framing with evals on your own data rather than by guessing. Our post on the Fable 5 context window covers why a bigger window alone doesn't solve this.

The same decision model, asked the same question, moved from 0.52 to 0.97 once the state included one fact about the cluster; what you leave out of the input shapes the answer as much as what you put in.

Which decision models are available today?

• Jev, from TypeSafe AI, was first to market on September 15, 2026, as a hosted API. TypeSafe describe it as "a frontier-intelligence function call: unstructured state in, typed probabilistic decisions out." TypeSafe deserve credit: they brought typed decisions with probabilities to market at the right time and showed the idea works.
• OpenAI's Decisions API followed about three weeks later. OpenAI released it in beta on October 6, 2026, running on gpt-6-luna, the high-volume tier of their newest model family. It inherits a frontier lab's training, so in theory it should read situations better. Public head-to-head benchmarks don't exist yet.
• Laya, from Convai Innovations, is an open-weight decision model under the Apache 2.0 license, with about 421 million parameters and an ~808 MB download. Its model card reports about 33 milliseconds per call on a single T4 GPU, and notes that the base model is not a zero-shot decision engine, so expect to tune it on your own decisions.

Enterprises will end up choosing the model that is most accurate and reliable on their own data, and that may well come from the labs that already have frontier models. Test on your own data, and keep your decision interface model-agnostic, so switching models is a configuration change.

Decision models are an early market: Jev launched in September 2026, OpenAI's Decisions API entered beta weeks later, and Laya runs open-weight on a single GPU.

When should you use a decision model instead of an LLM?

Use a decision model when you can list the options, the input fits, and you want a probability to set a threshold on: alert credibility, ownership routing, severity, "is this safe to run?", "does this message contain PII?". Those calls happen at volume inside agent loops, where latency and token cost add up fastest. Use an LLM when the task needs generation, tool calls, an explanation, or careful reasoning through an ambiguous case. Most real agents need both: the LLM does the open-ended work, and the decision model makes the calls in between.

How does NeuBird make production decisions quickly and with confidence?

The problems this post raised, seconds of latency per decision, tokens spent on reasoning a call doesn't need, and confidence nobody can measure, are the problems we work on every day in production operations. NeuBird is the Agentic Reliability Center: one governed platform to access your telemetry and LLMs in place, record institutional operations memory, and audit every agentic action in production to build resilient systems. Our Production Ops Agent runs on it, and an investigation is full of exactly the judgment calls in this post: is this alert credible, which service owns it, is this fix safe to stage.

We are always researching the latest technologies at NeuBird, so that agentic production operations run at high speed and high accuracy while staying strategic and economical. Decision models are one of many mechanisms we study. No single mechanism carries the job; working in concert, they let crucial decisions be made quickly and with confidence. The Production Ops Agent already finds root cause in under 5 minutes at 94% accuracy, with roughly 90% less token waste than alternatives, and every action it takes goes through human approval.

Calibrated decisions can speed up the investigations we already run: once the agent has produced a root cause analysis, quick, calibrated calls can rank open incidents, route each one to the owning team, and send only the uncertain cases for a deeper look. They can also make prevention more strategic. A probability on "is this worth investigating?" makes it practical to weigh many weak signals and run the right proactive investigations before an issue turns into a page, building on how the Production Ops Agent already catches degradation 30 to 60 minutes early. A threshold also fits the way NeuBird already governs action, as a per-environment policy of Suggest, Recommend or Act, with a person approving every change. We've written more about what an agent has to prove before it touches production and why a model is not an agent.

Frequently Asked Questions

What is a decision model?

A decision model is an AI model that reads a situation in natural language and chooses among options you provide, returning the chosen option with a calibrated probability. It never writes free-form text. It answers yes-or-no, pick-one and rate-on-a-scale questions in a single pass, which makes it much faster than an LLM for judgment calls.

Are decision models and large language models the same thing?

No, they are different kinds of model. Both read and understand text, but an LLM generates its answer one token at a time and can write anything, while a decision model only picks from options you supply. A decision model is trained for calibration, so its probability is part of the answer. An LLM's probabilities stay inside the model.

What is the difference between a decision model and an LLM?

An LLM generates text, code and tool calls, reasons step by step and explains itself, but it is slow and bills every output token. A decision model returns one option and a calibrated probability in one pass, in milliseconds. LLMs are more accurate on hard cases; decision models are faster and less expensive for decisions with known options.

When should I use a decision model instead of an LLM?

Use a decision model when you can list the possible answers and want a probability to set a threshold on: alert triage, severity ratings, ownership routing, safety checks before running a command. Use an LLM when the task needs generation, tool use, an explanation, or reasoning through an ambiguous case. Most production agents benefit from both.

Are decision models cheaper than LLMs?

A decision model is usually far less expensive per decision. It produces no reasoning or output text, so a call costs a small fraction of an LLM call. In the test behind our FLOCK '26 demo, Jev was about 90 times cheaper than Claude Sonnet 5.5 per call, and BrillMark measured it 31 to 65 times cheaper on support-ticket triage.

What does a calibrated probability mean?

A calibrated probability matches how often the model is actually right. If a calibrated decision model gives a thousand answers at 0.9, about nine hundred of them should be correct. Calibration is what lets you write a rule like "act automatically above 0.9, ask a person below it" and trust the result.

Can a decision model replace an LLM in an AI agent?

A decision model can't replace an LLM, because it can't write queries, commands or fixes, call tools, or explain its reasoning. It replaces the LLM only for the decision steps inside an agent loop, such as whether an alert is credible or a command is safe. The LLM still does the open-ended work, and the two work best together.

What are some examples of decision models?

Jev from TypeSafe AI was the first decision model on the market, launched in September 2026 as a hosted API. OpenAI's Decisions API entered beta in October 2026, running on gpt-6-luna. Laya from Convai Innovations is an open-weight decision model with about 421 million parameters, released under the Apache 2.0 license.

Sources

• Artificial Analysis benchmark costs: TechCrunch, "The rise of AI 'reasoning' models is making benchmarking more expensive," April 10, 2025
• Model list prices: Anthropic, Claude 3.7 Sonnet; OpenAI model docs for GPT-4o and o1
• Thinking always on: Anthropic, What's new in Claude Opus 5.5
• API usage: OpenAI DevDay 2025; OpenAI, "Accelerating the next phase of AI"; OpenAI, "The next phase of enterprise AI"
• Automation share: Anthropic Economic Index, September 2025 report
• Efficiency techniques: DeepSeek-R1; Hu et al., "LoRA: Low-Rank Adaptation of Large Language Models"; Inan et al., "Llama Guard"
• Jev and RLCD: TypeSafe, "Introducing System One models and Jev"; TypeSafe docs; TypeSafe models
• Independent tests: jev-frontier-bench on GitHub; BrillMark, "TypeSafe claims Jev is 193x faster, 444x cheaper: is it really true?"
• OpenAI Decisions API: OpenAI API changelog; Decisions API guide
• Laya: convaiinnovations/laya on Hugging Face