When we started building Myra, our financial analysis platform, we wrote one rule before any code: no number a model made up. Every figure a user sees, a tax position, a reconciliation, a capital gain, must be produced by deterministic code that can be tested, or it does not appear. The language model is welcome in the building, but it does not get to touch the numbers.
This was not caution for its own sake. Large language models are astonishing at reading, structuring and explaining, and confidently unreliable at arithmetic and recall of specifics. In casual settings that unreliability is a quirk. In financial analysis it is disqualifying, because a plausible wrong number is worse than no number: it travels, it gets relied on, and by the time it is questioned it has compounded.
Models narrate. Code calculates. The wall between those two jobs is the product.
What the wall looks like in practice
On one side: parsers that reconstruct bank statements and prove them against printed balances to the cent, double-entry ledgers, calculators for tax positions vectored against the regulator's own worked examples, and a reference library of verified rates and provisions that every computation cites. All of it deterministic, all of it under more than thirteen hundred automated tests, with evaluation gates that block any release that cannot reproduce known answers.
On the other side: the model. It categorises the ambiguous transaction and queues it for a human when unsure. It drafts the narrative that explains a position, arguing both sides the way an adviser would. And every figure in anything it writes is checked against the deterministic layer before a user sees it: if the model cites a number the calculators did not produce, the sentence does not ship. Citation or rejection, no third option.
Why this matters beyond us
The pattern generalises to any domain where numbers carry consequences: pricing, payroll, risk, engineering. The productive question is not "can AI do this job" but "which parts of this job are calculation and which parts are narration". Give the first to code you can test and the second to models you can gate, and you get software that is both genuinely intelligent and actually trustworthy. Blur the two and you get demos that impress and systems that quietly cannot be audited.
When the mandatory AI rules now being designed in Canberra ask organisations to show their systems are controlled, this architecture is what showing looks like: a test suite, an evidence trail, and a wall the model cannot cross.
What to do about it
- Separate calculation from narration in any AI system that touches consequential numbers.
- Demand a citation path: every figure in generated text traceable to a tested source.
- Treat evaluation gates as release infrastructure, not research niceties.
- Ask vendors where their wall is. If the answer is a pause, you have your answer.
Myra manages real family wealth on these rules, which is why we trust it, and why the discipline behind it now shapes everything we build and everything our AI governance practice recommends.