The Deterministic Shell, Code Computes and Models Narrate
Pull-quote: “The rule is worth stating plainly: the model never authors a number. Every figure is computed by code and matched back to its source, and the model’s only job is the sentence around it.”
Why this matters
Language models are excellent at language and unreliable at arithmetic, and no amount of prompting changes which of those two things a transformer is. In an enterprise deliverable that distinction is not academic. A board deck, a quality report, a research note lives or dies on whether its numbers can be defended, and a number that emerged from a generative pass has no defense: no formula, no source, no lineage, just a plausible token sequence. The failure mode is insidious because the surrounding prose is fluent. The paragraph reads perfectly, and the figure in the middle of it is off by a factor nobody catches until an auditor does.
The fix is architectural, not rhetorical. Split the system into a deterministic shell and a generative core. Code computes everything computable and verifies everything verifiable. The model does the one thing it is genuinely best at, turning computed facts into clear language, and is structurally prevented from doing the thing it is worst at.
The pattern
governed data
│
▼
facts engine (code) ─┐
metrics, joins, comparisons, thresholds │
output: typed facts with provenance │
│ │ the deterministic
▼ │ shell
prompt assembly: facts in, numbers locked │
│ │
▼ │
model (generative core) │
selects, orders, explains, narrates │
│ │
▼ │
verifier (code) │
every number, name, and claim in the draft │
must match a fact record ─┘
│ unmatched ──► reject, retry with the violation
▼
sealed artifact: narration + fact table + lineageThe verifier is the half of the shell most teams skip, and it is the half that makes the rule structural instead of aspirational. It is unglamorous code: parse the draft, extract every numeric token and named entity, match each against the fact table, reject on any orphan. A prompt that says “do not invent numbers” reduces the frequency of fabrication. A matcher that refuses to release an artifact containing an unmatched figure eliminates its delivery.
The division of labor
| Responsibility | Owner | Why |
|---|---|---|
| Compute metrics, aggregations, comparisons | Code | Reproducible, testable, carries provenance |
| Select what matters, order it, phrase it | Model | Salience and language are its real strengths |
| Check the draft against the facts | Code | The check must not share the generator’s failure modes |
| Approve exceptions | Human | Breaking the rule is a policy decision, not a retry |
One rule, many domains
The rule works as a founding constraint for any system that renders deliverables: outputs assemble from a facts engine bound to governed sources, the model narrates around computed figures, and the sealed artifact carries its fact table with it. The pattern generalizes across domains. An anomaly-detection pipeline computes findings with detection models and statistics, and its language model narrates them for analysts; the detection never moves into the narration pass. A quantitative research workflow runs walk-forward backtests in code and lets the model draft the commentary; a return figure authored by a language model would be a firing offense in any quant shop, and the architecture makes it a non-event instead of a risk. The shell is what lets a regulated business enjoy model fluency without inheriting model arithmetic.
What it costs, and when to pay
The pattern demands a facts engine, a fact schema with provenance fields, and a matcher, which is real engineering. It earns that cost wherever outputs face auditors, regulators, or operational decisions, and wherever a wrong number costs more than the narration is worth. For a brainstorming tool, skip it. For anything a signature lands on, it is the difference between an AI system you can defend and one you can only apologize for.
Closing
The division is old wisdom in new clothing: the calculator does not write, and the writer does not calculate. Give code the numbers, give the model the sentences, put a verifier between them and a human above the exceptions. Fluency where fluency helps, determinism where determinism is owed.