Architecture decisions to settle before approving AI at scale

Pull-quote: “You cannot back-fill a trace you never captured, and an evaluation set takes months of expert time. That is the only cost in this list that cannot be paid later.”

Most decisions in an AI programme are cheap to reverse, and six are not. Which model you call, which framework you start with, which vector index you use, and how you chunk documents are all changes you can make in a sprint. The six below change what you can do for years, because each one either writes state you cannot re-derive or places a dependency inside a boundary you cannot renegotiate later.

This is written for the person who signs the budget and is technical enough to want the actual reasons. It is not a maturity model and it contains no capability heat map. For each decision you get three things: what the decision is in plain terms, why reversing it costs more than making it, and the question that shows whether the team has thought about it. The questions matter more than the answers, because a team that has considered the reversal path answers in a paragraph and a team that has not changes the subject.

One framing to hold on to. The move from AI that answers to AI that acts creates no new governance categories, it removes the buffer a human reader used to provide. All six decisions are about where that buffer now sits.

What you will be able to do

• Apply a two-factor test separating decisions worth escalating from engineering choices.
• Ask six questions in an approval review that surface unexamined reversal costs.
• Distinguish a control enforced in code from an instruction written in a prompt.
• Set a human review boundary by consequence class rather than model confidence.
• Write an exit inventory that prices what would be rebuilt if you left a platform.
• Recognise the three artefacts that cannot be recovered retroactively: traces, evaluation sets, and audit attribution.

Which AI decisions are actually expensive to reverse?

A decision is expensive to reverse when undoing it requires re-deriving data you no longer have, or when the change propagates into systems whose owners you do not control. Those two factors are independent, and plotting a decision against both tells you whether it belongs in a design review or on your calendar.

Re-derivation cost is the harder of the two, because money cannot fully solve it. If you did not record a trace, no budget recovers it. If nobody labelled correct answers while the subject-matter experts were still on the project, more engineers do not produce a set. Blast radius is more familiar: how many teams, contracts, and integrations move when you change your mind.

The low-low corner is a healthy place to move fast. Swapping a model behind a stable interface, changing a retrieval strategy, or rewriting a prompt costs an afternoon plus a re-run of your evaluation set. The more you keep in that corner, the fewer real decisions you have to make.

Where should the data live, and who governs it?

The decision is which account, region, and catalog hold the data an AI system reads and writes, and which single team owns the permission model and the lineage. It is one decision, not two, because a governance model that does not follow the data is decoration.

Reversal is expensive for a reason that surprises people: moving bytes is the easy part. What does not move cheaply is the accumulated permission model, the lineage graph, and the consumers that hard-coded a path or a connection string. Every grant has to be re-derived and every consumer re-pointed and re-tested, and in regulated or government work the residency commitment sits in a contract, so a move reopens documents that took months to sign.

There is a second trap specific to AI systems. Retrieval indexes, embeddings, feature tables, and fine-tuning datasets are derived data, and they must remain re-derivable from the governed source. When the only usable copy of a document collection lives inside a vector store with no lineage back to the records it came from, your access controls have quietly stopped applying: a permission revoked at the source does not revoke the embedding. Treat every derived store as a cache, and test the rebuild.

The question to ask. “If we had to serve this workload from a different account or region within ninety days, list what breaks and who has to re-approve access.” A team that has designed for it answers in a page. A team that says “we would export the tables” has priced the copy and nothing else.

Should you build or buy the model layer?

The decision is not which model to use. It is where the reasoning loop runs and who owns the orchestration state. Model choice is cheap to reverse if you kept the loop. Handing over the loop is not.

Buying a model is a call to an interface. If your prompts, tool schemas, evaluation set, and traces stay on your side, changing providers is a configuration change plus a re-run of your evaluations. Buying a model layer means a vendor runtime holds the orchestration state: conversation history, memory representation, tool invocation records, retry and escalation behaviour. Those artefacts are rarely exportable in a form you can replay elsewhere, and replay is what you need to compare a new provider against last month’s real traffic.

The default that preserves the most optionality is to buy the model and own the loop, which is what an agent harness is for: a thin layer you control holding prompt assembly, the tool registry, state, and logging, with the model behind a swappable interface. Build deeper only where control is worth the engineering: a regulatory obligation, an unusual tool surface, or a latency and cost profile a general runtime cannot hit.

The question to ask. “What artefacts do we still own if we change vendors next year, and can we replay last month’s traffic against a different model without rewriting the application?” If replay is impossible, you have bought more than a model.

What evaluation and observability are you committing to?

The decision is what you record on every AI call and what you score it against, and it has to be made before launch because neither can be created retroactively. This is the only item on this list where delay makes the cost unpayable rather than merely higher.

Traces are the first half. Record the model your code requested and the model the provider actually served as separate fields, because aliases, deployment names, and automatic routing mean they differ more often than teams expect. Add input and output token counts, tool calls with their outcomes, latency, error type, and a hash of the prompt template so you can group results by prompt version without storing prompt text. Whether you capture message content at all is a separate, governed decision with its own retention rules.

On naming, use the OpenTelemetry generative AI semantic conventions but do not assume they are settled. As of mid-2026 the gen_ai surface remains in Development status and has moved to a dedicated semantic conventions repository, so attribute names can still change. Emit the conventional attributes, then build dashboards on a small internal set of names you control and map onto, so a convention change is a mapping edit rather than a rewrite.

Evaluation is the second half and the more expensive one. An evaluation set is a collection of realistic inputs with known-good outputs, built by people who can tell a correct answer from a plausible one. It takes weeks to months of expert attention, and that attention is available at the start of a project and gone by the end. Building it during the pilot costs a fraction of building it after a regression, and the artefact survives every model change afterwards. This is the discipline described in building an evaluation harness for agents.

The question to ask. “If the provider silently changes the model behind an alias, how many hours pass before we notice, and what do we compare against?” The answer names an alert and a dataset, or it names neither.

Where does human review sit, and who owns the decision?

The decision is which classes of action may never execute without a human, and in which layer that rule is enforced. Draw the boundary by consequence class, not by model confidence, because a confidence score is a property of the model under its training distribution and consequence is a property of the world.

Four classes cover most real systems. Reversible and low value can run unattended, with logging. Reversible and high value can run unattended given a notification, a working undo path, and someone who watches the notification. Irreversible actions require human approval before execution. Externally binding actions, anything creating an obligation to a customer, a regulator, or a counterparty, require a named human recorded as having approved. The classification is about the action, not the agent: one agent can be unattended for one tool and gated for another.

Enforcement location is where most designs fail review. An instruction in a system prompt saying the agent must ask before deleting is a request, and a model under adversarial input can be persuaded to ignore it. A tool that refuses to execute without a valid approval token is a control, and it holds regardless of what the model was persuaded to believe. If the enforcement disappears when the prompt is compromised, it was never a control. That is the reasoning behind the trust boundaries in prompt injection and LLM security.

Reversal is expensive here for an organisational reason rather than a technical one. Adding automation to a reviewed process is straightforward. Inserting review into an automated process means rebuilding a queue, a role, a rota, and a service level in a team that has already restaffed around not needing them. The regulatory ground is moving too: under the EU AI Act as amended by the Digital Omnibus on AI, Article 50 transparency obligations apply from 2 August 2026, while high-risk obligations for stand-alone Annex III systems shifted to 2 December 2027 and for AI embedded in Annex I regulated products to 2 August 2028, per the European Parliament’s legislative record. The dates moved. The design requirement did not.

The question to ask. “Name the action class that must never execute without a human, and show me where that is enforced in code.”

How do agents get identity and access?

The decision is whether an agent runs under its own identity with its own scopes, or borrows credentials from a human. Borrowing is faster to ship and produces an audit trail that can never be corrected.

Run one identity per agent deployment, with least-privilege scopes granted per tool rather than per agent, short-lived credentials, and an audit record naming both the acting agent and the human on whose behalf it acted. Revocation has to be independent in both directions: disable the agent without disabling the person, and remove a person’s authority without breaking every automation. If a single credential does both jobs, you have one switch for two decisions.

Reversal is expensive and partly impossible. Retrofitting identity means re-issuing every credential, re-scoping every integration, and re-auditing every log written during the impersonation period. The re-issuing is work. The re-auditing is not achievable: for writes already recorded under a human’s identity, no later change tells you which were the agent’s. In a regulated environment that is a finding.

The security case compounds the governance case. An agent that reads untrusted content while holding a human’s full credentials is the complete prompt injection attack in one sentence: the attacker supplies the instruction, and your access control supplies the authority. Narrow, per-tool scopes limit what a successful injection can reach.

The question to ask. “In the audit log, can you separate an agent’s write from the operator’s write, and can you revoke one without the other?”

What is the exit cost of this platform choice?

The decision is which artefacts become unreadable or unrunnable if you leave a platform, and the answer is almost never the data. Open table formats keep tables readable by other engines, and object storage keeps files where you put them. Exit cost lives in everything expressed in a vendor’s own language.

Write an exit inventory. List every artefact class the platform holds and sort each into one of three columns: files we own in an open format, things exportable in a usable form, and things that exist only inside the vendor’s system. Typical rows are table data, orchestration and job definitions, semantic models and metric definitions, access policies, feature definitions, notebooks, dashboards, evaluation harnesses, and agent configurations. Data lands in the first column. Orchestration, semantics, and access policy land in the third, and that third column is your exit cost.

What matters is not what the third column would cost to export, since by definition it cannot be. It is what rebuilding it elsewhere costs in engineer-months, and whether the business could tolerate that gap. Doing this before signing is a negotiation position. Doing it two years in is a discovery. It is also the honest version of the comparison in choosing a data platform by workload, where the tie-breaker between credible options is the shape of the third column rather than a feature matrix.

The question to ask. “Show me the exit inventory and the engineer-month estimate against the third column.”

Where this goes wrong

Governance written as policy, enforced nowhere. The symptom is a well-structured policy document and an agent that can still call any tool it can reach. The cause is treating governance as a review artefact instead of a runtime one. Require every stated control to name the code path enforcing it, and reject any control that names only a prompt.

Pilot success with no evaluation set. The symptom is a demo that impresses everyone and a production system nobody can defend when it starts failing. The cause is a pilot that proved the technology and never produced the measurement. Fund the evaluation set as part of the pilot, not as a later hardening task.

Agents sharing a service account. The symptom is an audit log where every action came from svc-automation. The cause is scoping identity to a system instead of a deployment. Split identities per agent, per tool scope, before the number of agents grows past what you can reconstruct by hand.

Derived stores that outlive their source permissions. The symptom is a retrieval index returning content the requesting user no longer has access to. The cause is a vector store populated once, with no lineage and no re-derivation path. Rebuild derived stores from the governed source on a schedule, and test the rebuild.

Reversal cost estimated as the cost of the copy. The symptom is a migration budgeted at three months that takes a year. The cause is pricing data movement and not the re-approval, re-pointing, and re-testing.

Common questions

How do I tell an architecture decision from an engineering choice?

Apply two tests. Ask how much data would have to be re-derived if the decision were reversed, and how many teams outside the one making it would have to change something. If both answers are small, it is an engineering choice and should be delegated. If either is large, it belongs in a design review with the affected owners present. If both are large, it belongs on your calendar.

Do these decisions change when we move from chatbots to agents?

The decisions are the same; the tolerance for getting them wrong drops. A system that produces text for a person to read has a human buffer between the model and the consequence. A system that takes actions does not. That is why the review boundary and agent identity move from good practice to prerequisites once tools can write.

Can we defer evaluation until after the pilot proves value?

You can defer scoring, but not collection. Traces and labelled examples cannot be created retroactively, and the experts who can label correct answers are available at the start of a project and busy by the end. Capture full traces from day one and build the labelled set incrementally during the pilot, so the artefact exists when the pilot is judged.

What is the difference between a guardrail and a control?

A guardrail shapes model behaviour through instructions, filters, or a classifier, and a sufficiently clever input can bypass it. A control is enforced outside the model, in the tool layer or the permission system, and holds regardless of what the model was persuaded to do. Both are useful. Only one is something you can tell an auditor about.

Should we standardise on one AI platform or keep several?

Standardise on the layer where fragmentation hurts, which is governance and identity, and stay flexible above it. Multiple model providers behind one interface is healthy, because it keeps switching cheap and gives you a fallback during an outage. Multiple governance models over the same data is not, because the seams between them are where access mistakes live.

Who should own the human review boundary decision?

The business owner of the process, with security and legal as required reviewers and engineering as the implementer. It is a decision about acceptable consequence, and engineering cannot make it because engineering does not carry the outcome. What engineering owes the conversation is an honest statement of what can be enforced and what can only be requested.

Next steps

Take one AI initiative currently seeking approval and run the six questions against it in a single meeting, writing the answers down. You are not looking for good answers. You are looking for which questions produce a page and which produce a change of subject, because the second group is the actual risk register.

Then read building an evaluation harness for agents, which turns the evaluation commitment into a running system, and prompt injection and LLM security, which explains why identity scope rather than model behaviour is the control that holds. If the platform question is live, choosing a data platform by workload covers how to compare credible options without a feature matrix.

Zorost runs these controls in production as a trusted Databricks partner, where approval gates on writes and per-agent identity are part of the build rather than a later hardening pass, and we publish what we test in AI Fieldwork.