unrealistsJev-style decision models
A lot of “classification in LLM clothing” is still chat completion: the model writes “Category: refund” or a JSON blob, you parse and pray. A Jev-style decision model fixes the output space up front (choice, yes/no, ordinal score) and returns a probability vector software can threshold, branch on, or send to a review queue. No tokenizer lottery on curly braces. The interesting split in 2026 is not “RAG vs fine-tune” but purpose-built decision stacks vs reading decisions out of a general LLM with a disciplined prompt and logit math.
Jev and System One
TypeSafe’s Jev is the reference product shape: unstructured state in, typed probabilistic decisions out, parallel sampling instead of autoregressive string generation, training framed as reinforcement learning for calibrated decisions (RLCD) rather than preference on chat quality. Their pitch is workflow automation: classify, route, score, guardrail, at tens to hundreds of milliseconds per call, with confidence that is supposed to track accuracy so you can automate the high-confidence slice and escalate the rest. Treat the marketing numbers as demos until you replay them on your schema and your holdout. We still read a model card and a DPA before any production path, same as an API LLM.
AnyJev on open weights
Nokia’s AnyJev (Apache-2.0,
technical report) implements the Jev-shaped
interface on causal LMs you already run: one prefill (or a small bounded number) and a readout
restricted to option tokens. Raw next-token logits lie: answers flip when you reorder the option
list, and some labels win because the model likes the letter, not because the state fits. L0 fixes
that without labels: cyclic rotations of the option list plus a running label-prior correction from
unlabeled traffic. Their published BANKING77-style table is the number we care about in a PoC: share
of decisions confident enough to auto-route at a stated error budget moved from single digits to
~40%+ with L0 alone. Optional L1 adds temperature scaling on a few hundred labeled examples per
question. Tacit checkpoints add self-distillation so one forward pass is enough on a tuned backbone;
adaptive=True can send only the ambiguous cases to generated reasoning, with a cap so
latency does not explode. Fits the
local vs API conversation when the job is thousands of
identical route decisions per hour on a GPU you already own.
LLM-as-Jev: when training is optional
Microsoft’s
LLM-as-Jev paper asks a blunt question: how much
of this capability is already in the base model? Their recipe numbers options [1]…[K],
prefills Best answer: [, and scores each candidate suffix from next-token
log-probabilities (same family of trick as letter readouts and AnyJev, different bias fixes). On
public JevBench items they report a frozen Qwen3.5-4B matching community fine-tunes on the same
backbone, beating naive option-letter logits, and scaling to large K without a custom head per
backbone. Fine-tuning helps when the backbone is weak or the task is high-cardinality intent
routing; on strong models the gains are uneven and KL anchors matter so you do not wreck normal
chat generation. For an eight-week PoC that reads like
exploration if you cannot name the option set and the
deferral rule yet.
Where we use this shape
Pick this pattern when downstream code needs if probability > τ then branch, not a paragraph: ticket routing, doc-type gates before OCR, compliance triage, procurement eligibility checks next to register and form pulls. Success on PoC scorecards is still the business metric (precision on the auto-routed slice, cost per decision, time in queue), not leaderboard accuracy on someone else’s workflow eval. Start training-free (AnyJev L0 or a bracketed readout you can reproduce); add labels only when calibration on your holdout is visibly wrong. If the “decision” is really ten fields from a PDF, you are back in extraction, not Jev.