The layers an organization needs in place before AI agents are allowed to touch systems of record, and the constraints that stop each layer absorbing work belonging somewhere else. Read it if you'll be accountable for the result in production.
Governance and telemetry are drawn as planes rather than layers because that's what they are. Enforcement embeds at every boundary; telemetry aggregates from every layer. Neither is a slab you can point at in the request path.
The negative constraints matter more than the positive ones. Most failures we're called in to fix are a layer quietly doing another layer's job.
OwnsThe surfaces where people meet agents: conversational UI, copilots inside the tools already open on the desk, the approval inbox, run status, and a catalog of who owns which agent. Feedback belongs here too, and a rating is useless unless it carries session ID, agent version and the output text.
Must notHold model credentials, call a provider directly, or decide what a user may see. A widget that acquires its own API key is a governance hole nobody finds until an audit.
Fails asAdoption. A standalone portal gets a fortnight of curiosity traffic and flatlines, and since nothing writes back to the system of record, someone re-keys the output by hand.
OwnsThe path from intent to completed work: intake with an idempotency key, durably committed before the caller is acknowledged; a decisioning step picking agent, model tier and permitted tools before any inference; state held outside the workers so workers stay disposable; a durable runtime with checkpoints, retries and saga compensation; the tool gateway and the agent sandbox.
Must notDefine policy, store the corpus, or call a provider directly. One rule matters more than the rest: when an agent delegates to a sub-agent, the delegation re-enters at intake and passes every gate again. An internal fast path there hands an attacker a tool-calling primitive with no authorization on it.
Fails silentlyMost in-house builds break here around month four, when the first two-day workflow meets the first pod eviction. With no side-effect ledger, a retry after a crash sends the same invoice twice. With no compensation, a workflow dying at step four leaves an account half-created across three systems and nobody knows which three.
OwnsA gateway in front of every provider, a registry of approved models and their costs, tier routing by task class and data classification, and the optimizations worth real money off the bill: prefix caching, semantic response caching, batched embedding runs. Tool execution is the other half, sandboxed and classified per call as read, write or irreversible, with a circuit breaker per tool so one sick downstream API isolates itself instead of taking out the worker pool.
Must notBe bypassable. If any layer can hold a provider SDK and its own key, safety filtering, cost attribution and the audit record become optional, and optional controls are absent controls.
Fails asLock-in you discover during a pricing change, and an inference bill nobody can attribute to a team.
OwnsArtifacts, and not a slab of the request path: policy bundles in Git, machine identities for agents, the secrets store, the evidence and lineage archive, a bill of materials per deployed agent, and the pipeline that threat-models and adversarially probes an agent before promotion. Identity covers three principal types, human, agent and service, with agent credentials revocable fleet-wide in seconds, which is what makes containment possible.
Must notBe drawn as a horizontal band between orchestration and data. Enforcement embeds at every boundary in every other layer; only authoring, identity and evidence live in one place. Every enforcement point fails closed, which teams find painful in week one and stop noticing by week six.
Fails asA policy document instead of a policy engine, at which point "show me every action this agent took against customer data in March" takes six weeks of log archaeology to answer badly.
OwnsIngestion, retrieval and memory. Connectors pull incrementally so the corpus tracks the business instead of a nightly snapshot. Retrieval is hybrid: vector for semantic similarity, a graph for the relational questions vector search cannot answer, keyword for the exact-term cases both miss.
Must notScrub PII after embedding. Once sensitive text is embedded it's mathematically present in vectors you cannot fully purge, so the DLP gate sits before chunking. Entitlements are enforced inside the data layer at query time, from ACL metadata attached at ingest. Filter in application code after retrieval and one application bug is one data breach.
Fails asAn entitlement sync lagging group membership changes, so a leaver keeps retrieving for hours.
OwnsInfrastructure as code, GitOps promotion, environment isolation, GPU pools that idle to zero outside production, and the option to run inside your residency boundary. What matters operationally is that rollback is a routing change: repointing takes seconds, and a container rebuild takes minutes you don't have during a quality incident.
Fails asAgent versions tracked in a spreadsheet.
Authored once, evaluated in many places, never all in one box. The decision points: intake, where a principal with no grant is refused before compute is spent; retrieval, where the ACL goes into the query instead of onto the results; routing, where a data classification forces a different or local model; tool invocation, where irreversible operations divert to a human; promotion, where a failed safety probe blocks release and no seniority overrides it; and configuration change, since a runtime parameter edit is itself a governed action.
The cost is a few hundred milliseconds. The benefit is that no single component failing open produces an uncontrolled action.
Every layer emits, one place aggregates, and collection is infrastructure-level instrumentation so no application code carries monitoring logic. Five signals matter and they're genuinely different: infrastructure metrics, execution traces, quality scores, cost events keyed to agent and tenant, and human feedback.
It can't be an afterthought, because AI failures are quiet. A degrading agent returns HTTP 200 at a zero error rate while its answers get worse, so the service-level indicator has to be successful structured completions and groundedness. Response codes tell you nothing.
One further discipline: attribute cause before remediating. Rolling back an agent version when the real problem was a stale index costs you the outage twice.
Fourteen hops. The interesting column is what can refuse it.
Resuming costs little in state and more than teams expect in time. The workflow checkpoints atomically before the pause and replays from exactly that point. Rehydration does have to re-validate that retrieved knowledge is still current, since a document may have changed during a two-day wait, and a long enough pause can invalidate the plan itself.
But the real cost is queue latency, and it's invisible to infrastructure monitoring. A backed-up review queue stalls production workflows while every dashboard stays green. Instrument queue depth, reviewer response time and SLA breach, or the design quietly degrades into a batch process on a weekly cadence.
The floor is a gateway, identity and policy enforcement, a vector store, a durable workflow engine, tracing and an evidence archive. License cost for that set is real and rarely the binding constraint. Three things dominate instead: inference spend, which caching and tier routing reduce substantially, though the range is wide and depends on how repetitive your traffic is; a GPU floor if residency rules force self-hosting, usually the largest single line; and two to four engineers keeping policy bundles, connectors and evaluation sets current, indefinitely. Integration cost is front-loaded. Operating cost is mostly salaries.
Two weeks, one value stream, a costed plan for your board. We measure your baseline first, and if the honest answer is that you're not ready, we say so then.
Book a readiness assessment
iNexCCo