Chapter 19 · Where Does the Agent Live? Centralised vs. Embedded Models
The question of where an agent sits in the organisation is not a technical question. It is a question about who is accountable for what it does.
The Question Organisations Ask Too Late
Most enterprises approach agentic deployment as a sequence of individual decisions: which use case to start with, which model to use, which platform to build on. The question of where the agent lives — which team owns it, which budget funds it, which governance structure covers it, and how it relates to agents owned by other teams — tends to arrive late, when the proliferation of independent deployments has already created the conditions that make it urgent.
By the time an organisation asks the structural question seriously, it typically has several agents already in production or active development: one built by the IT team for service desk automation, one built by the HR team for onboarding, one built by a product squad for user research synthesis, and two more in the pipeline from Finance and Marketing. Each was built with reasonable autonomy by a team that understood their own use case. None was built with visibility into what the others were doing. The result is a portfolio of agents that share no infrastructure, apply inconsistent governance standards, duplicate integrations to the same underlying systems, and have no coherent accountability structure when something goes wrong.
This is not a failure of any individual team. It is a failure of organisational design, and it is almost universal in enterprises that reached Stage 3 of the maturity framework from Chapter 16 without addressing the structural question first.
This chapter examines the two primary models — centralised and embedded — and the hybrid that most mature organisations converge on. It then addresses the questions that the model choice determines: where AI engineering talent sits, how standards travel across the organisation, what the governance implications are, and how the answer should change as the organisation matures.
The Two Archetypes
The centralised and embedded archetypes sit at opposite ends of a spectrum. Neither is the right answer in its pure form. Understanding each in its full expression makes the trade-offs clear.
The Centralised Model
In the centralised model, a dedicated AI team — variously titled Centre of Excellence, AI Platform team, or AI Enabling function — owns the infrastructure on which agents run, sets the standards agents must meet, and is either directly responsible for building and operating agents across the organisation or provides the tooling and oversight that business-unit builders must work within.
The centralised model looks like this in practice: the AI team operates a shared agent platform — logging infrastructure, evaluation tooling, model access, integration libraries, governance frameworks — and business units access it to build and deploy agents within its constraints. Alternatively, in a more tightly centralised variant, the AI team builds agents on behalf of business units and retains operational ownership after deployment.
What the centralised model does well:
Consistency is the primary benefit. When all agents run on the same infrastructure, the security posture, data handling standards, audit trail requirements, and governance procedures are consistent across the portfolio. A new data protection requirement can be applied once at the platform level and propagated to every agent simultaneously rather than requiring coordination across a dozen independent teams.
Reuse is the second benefit. Integrations built once — a connector to the HRIS system, a shared customer data retrieval tool, a standard authentication layer — are available to every agent without being rebuilt. The marginal cost of the second agent that uses an integration is close to zero; in the embedded model, it is the full cost of building it again.
Economies of scale in talent are the third. AI engineering is a scarce skill. A centralised team of eight strong AI engineers will deploy more capable, better-governed agents than eight individual engineers embedded across eight business units with varying levels of AI expertise and no shared learning.
What the centralised model does poorly:
Speed. A business unit that needs an agent to solve a specific, time-sensitive problem and must route the request through a central team's prioritisation process will be frustrated. Central teams accumulate backlogs. Business units with genuine urgency — and the technical capacity to build — will route around the central team if the process is slow enough.
Domain knowledge. Central AI teams are generalists by necessity. The HR team understands HR processes, data, edge cases, and user expectations in ways that a central team building on their behalf cannot fully replicate. This knowledge gap shows up in production: agents built without deep domain understanding produce outputs that are technically correct and practically wrong.
Ownership. When a central team builds and operates an agent on a business unit's behalf, the accountability for the agent's outputs is diffuse. The central team owns the technology; the business unit owns the process the agent is embedded in; neither has complete visibility into both. When something goes wrong, the accountability gap is immediately apparent.
The Embedded Model
In the embedded model, AI capability is distributed across business units. Each team owns its own agents: the HR team builds and operates the HR onboarding agent, the Finance team builds and operates the reconciliation agent, the Product team builds and operates the research synthesis tool. There is no central AI team, or the central team exists in an advisory capacity without operational authority.
What the embedded model does well:
Speed and iteration. A team that owns its own agent can change it, test it, and redeploy it in hours, not weeks. The iteration cycle matches the pace at which the team's needs evolve, not the pace of a central backlog.
Domain fit. Teams that build their own agents encode their own domain knowledge directly. The Finance team's reconciliation agent reflects how the Finance team actually does reconciliation, including the institutional knowledge that does not appear in any documentation.
Accountability. When the HR team owns the HR agent, there is no ambiguity about who is responsible for what it does. The accountability is direct, clear, and felt by the people closest to the consequences of the agent's outputs.
What the embedded model does poorly:
Governance fragmentation. Without a shared framework, different teams apply different standards to the same fundamental questions: what data can agents access, how are outputs logged, what human oversight is required before an agent acts, how are failures escalated? The answer to each question is different depending on which team built the agent. When a governance requirement arrives — a new regulation, a security audit, a board inquiry about AI risk — the organisation discovers that it has no coherent answer because it has no coherent structure.
Duplication. The integration built by the HR team to connect their agent to the HRIS system will be rebuilt, with different design decisions and different quality, by the Finance team when they need the same data. The testing infrastructure, the logging pipeline, the model access layer — all duplicated, all maintained separately, all consuming engineering capacity that could produce new capability instead.
Inconsistent quality. AI engineering expertise is not evenly distributed across business units. The team with a strong ML engineer produces a well-designed, well-governed agent. The team without one produces something that works in the demo and fails in production. The embedded model surfaces the organisation's engineering talent distribution in the quality of its agent portfolio, and that distribution is almost never flat.
The Federated Model
The organisation that has thought through both archetypes almost always converges on a hybrid — typically called a federated model, though the terminology varies. The federated model separates infrastructure from application, centralises the former and distributes the latter, and uses governance standards as the interface between them.
The logic of the federated model is that the things which benefit from centralisation — infrastructure, standards, shared integrations, economies of scale in expensive tooling — are genuinely different from the things which benefit from distribution — domain knowledge, iteration speed, local ownership, accountability. The federated model keeps each category in the layer where it belongs.
What the central platform team owns in the federated model:
The shared infrastructure on which agents run: logging and observability pipelines, evaluation tooling, model access credentials and rate-limit management, security scanning, and the deployment pipeline that agents use to go from development to production. Business units do not build this infrastructure — they use it.
The governance standards that agents must meet to deploy: what data access patterns are permitted, what logging fidelity is required, what human oversight must be in place before an agent is authorised to take consequential actions, what testing must be completed before deployment. These standards travel as requirements that business-unit builders must satisfy, not as constraints that the central team enforces by reviewing every agent build manually.
The shared integrations to common systems: the organisation's identity and access management layer, the major data platforms (CRM, HRIS, ERP, data warehouse), the communication and collaboration infrastructure. These integrations are built once, maintained by the platform team, and available to any agent that operates within the governance framework.
What business units own in the federated model:
The domain logic of their agents: what the agent is designed to do, what decisions it is authorised to make, how it routes edge cases, what its scope boundaries are. This is where the HR team's knowledge of HR processes and the Finance team's knowledge of financial controls live — it cannot be centralised without losing the domain depth that makes the agent useful.
The tools and integrations specific to their domain: the HR team's connection to their recruitment platform, the Finance team's connection to their trading system, the Product team's connection to their analytics infrastructure. Where these integrations are reusable across teams they graduate to the platform layer; where they are genuinely domain-specific they stay with the business unit.
The ongoing operation and improvement of their agents: responding to feedback, tuning outputs, adjusting scope as processes change, managing the user relationships that depend on the agent. The people closest to the agent's consequences are the ones best positioned to make the day-to-day decisions about how it should behave.
Key takeaway: The federated model separates two genuinely different problems — building shared infrastructure and enforcing consistent standards (centralised) versus encoding domain knowledge and owning local outcomes (distributed). The error in both pure archetypes is treating these as the same problem and choosing a single structure to solve both.
This structure has a precedent in IT governance research: a study of 256 organisations by Weill and Ross found that a federal arrangement — in which central and distributed authorities share decision-making, each layer retaining authority appropriate to its knowledge — was by far the dominant governance structure among high-performing firms for business-oriented technology decisions, used by over 80% of surveyed organisations, while purely centralised and purely distributed archetypes were associated with weaker portfolio governance outcomes; the same analysis explicitly cautions against the feudal model — each business unit deciding independently — for technology application decisions, on the grounds that it cannot produce the portfolio-level coherence that consistent governance requires.6
How the Model Relates to Maturity Stage
The appropriate organisational model shifts as the organisation moves through the maturity stages described in Chapter 16. A governance structure appropriate for Stage 2 will constrain progress at Stage 4; a governance structure designed for Stage 4 applied at Stage 2 will create bureaucratic overhead that kills the experimentation necessary to develop organisational AI intuition.
| Maturity Stage | Appropriate Model | Rationale |
|---|---|---|
| Stage 1 — Augmented Assistance | Embedded, informal | Individual productivity tools; governance overhead not yet warranted |
| Stage 2 — Assisted Automation | Centralised standards, distributed builds | First agents need a governance baseline; central team sets it, business units build within it |
| Stage 3 — Supervised Agency | Federated with shared infrastructure | Enough agent volume to justify shared infrastructure; domain complexity requires distributed ownership |
| Stage 4 — Delegated Operations | Federated with formal governance layer | Consequential autonomy requires consistent accountability structure across portfolio |
| Stage 5 — Orchestrated Intelligence | Federated with platform as product | Inter-agent dependencies require platform thinking; business units remain owners of domain agents |
The transition that most organisations find most difficult is the Stage 1 to Stage 2 shift. In Stage 1, individuals and teams experiment informally and governance adds cost without visible benefit. In Stage 2, agents begin taking consequential actions and the absence of governance creates visible risk. The central team or governance framework needs to be established before that transition, not after the first Stage 2 incident makes its absence obvious.
The transition from Stage 3 to Stage 4 is where the federated model needs to formalise. At Stage 3, informal governance coordination between a small number of agent-owning teams is workable. At Stage 4, the number of agents, the autonomy of their actions, and the regulatory exposure of their decisions require a governance layer that is explicit, documented, and applies consistently across the portfolio.
Where AI Engineering Talent Sits
The organisational model question and the talent question are inseparable. The choice of centralised, embedded, or federated model determines not just where agents are built but where the people who build them are located, how they develop their skills, and what career paths are available to them.
The talent question has three components:
Concentration vs distribution. Centralising AI engineering talent produces depth and shared learning — a team of engineers working on agents together develops a coherent body of institutional knowledge about what works. Distributing AI engineers across business units produces breadth and domain fit — each engineer develops deep knowledge of a specific domain. The trade-off is real. The federated model typically resolves it by concentrating platform engineering in the central team and accepting that domain-facing engineers (who may have lighter AI specialisation) embed in business units. The critical design question is whether the central platform team maintains a strong enough feedback loop with embedded teams to keep the platform genuinely useful.
The Centre of Excellence model and its failure mode. A common pattern in large enterprises is the creation of an AI Centre of Excellence (CoE) — a team of specialists tasked with building capability across the organisation. The CoE model works when it functions as an enabler: it builds shared infrastructure, develops standards, provides consulting to business-unit teams, and upskills practitioners across the organisation. It fails when it functions as a gatekeeper: it controls all AI activity, requires every agent to be approved through its backlog, and becomes the bottleneck that the embedded model was designed to avoid. The difference between enabling CoE and gatekeeping CoE is partly structural — the governance model it operates — and partly cultural: whether the team sees its role as building capability across the organisation or as owning AI capability on behalf of the organisation.
Upskilling the edges. One of the consistent findings from mature agentic deployments is that the most valuable AI practitioners in the medium term are not the specialists at the centre — they are the domain experts at the edges who develop enough AI fluency to build and maintain agents in their own field. A Finance professional who understands reconciliation processes deeply and can build and maintain an agent for them with support from the central platform team is more valuable than either a pure Finance professional or a pure AI engineer working on Finance problems from the outside. This is the talent model that federated organisations are building toward, and it requires both investment in upskilling domain practitioners and a platform that is accessible to practitioners who are not AI specialists.
How Standards Travel
In the federated model, governance standards are the connective tissue between the central platform and distributed business units. Getting this mechanism right is the difference between a federated model that works and one that fragments into the embedded model's worst characteristics while sacrificing its best ones.
Standards travel in three ways, each appropriate at a different stage and for a different class of requirement:
Hard constraints enforced by the platform. Some standards should be non-negotiable: logging requirements, data access controls, authentication, and the prohibition on certain categories of autonomous action without human review. These are not communicated as guidance — they are enforced by the platform. An agent that does not meet them cannot deploy. This is the right mechanism for the standards where non-compliance creates legal, security, or safety exposure that the organisation cannot accept at any business unit.
Soft standards communicated as templates and tooling. Many governance requirements are best communicated by making the compliant path the easy path: provide a pre-built evaluation framework, a standard audit log schema, a reference architecture for human oversight checkpoints. Business units that use the templates comply by default; the effort cost of non-compliance (building equivalent infrastructure from scratch) is higher than the effort cost of compliance. This is the right mechanism for the standards where there is a better and a worse implementation but where the organisation can tolerate some variation.
Guidance and community of practice. Some standards are genuinely contextual — the right approach to escalation design for an HR agent is different from the right approach for an IT service desk agent. These are best communicated through shared examples, a community of practice where agent-owning teams share what they have learned, and a central team that can consult on specific deployments. This is the right mechanism for standards that require domain judgement to apply correctly.
The failure mode in federated governance is attempting to communicate all standards through the first mechanism — hard enforcement — when many are better suited to the second or third. An organisation that responds to every governance gap by adding a deployment gate creates a bureaucratic load that generates workarounds, shadow deployments, and the exact fragmentation the governance model was designed to prevent.
The Shadow AI Problem
Any discussion of organisational models for agentic AI must address the reality that organisational models do not fully control where agents get built. When the central team's process is slow, when governance requirements are perceived as disproportionate, or when a business unit has the technical capacity and the urgency to act independently, agents get built outside whatever formal structure exists.
This is not a new problem — shadow IT has existed for as long as central IT teams have been slow. But shadow AI has characteristics that make it more consequential than shadow IT in most cases. A spreadsheet built outside IT governance can contain errors that affect a process. An agent built outside AI governance can take autonomous actions at scale, with data access that was not reviewed, under oversight arrangements that were not designed, producing audit trails that were not specified.
The response to shadow AI is not tighter enforcement — it rarely solves the underlying speed and relevance problem that generates the shadow activity in the first place. It is making the governed path faster and more useful than the shadow path. When the central platform provides genuine leverage — shared integrations, evaluation tooling, a deployment pipeline that is faster than building from scratch — business units choose it because it is better, not because they are required to. The governance is a side effect of adoption rather than a barrier to it.
Key takeaway: Shadow AI is a signal about the governed path, not about the business units that route around it. If the governed path is genuinely better — faster to use, more capable, lower maintenance — the shadow problem is largely self-correcting.
The Accountability Question
The centralised-vs-embedded debate is, at its root, a debate about accountability. When an agent causes harm — makes a wrong financial decision, handles a disciplinary matter incorrectly, triggers an irreversible action based on bad data — who is responsible?
Any effective governance structure for a technology portfolio must answer three questions: what decisions must be made to ensure safe and effective use, who has the authority to make each, and how those decisions will be monitored — the formulation Weill and Ross proposed for IT decision rights, and which applies without modification to an AI agent portfolio.6
In the pure centralised model, the accountability is with the central team. They built it, they own it. The business unit that uses it is a customer of the central service. This is a clean accountability structure, but it produces a central team that is responsible for outcomes in domains it does not fully understand, and business units that lack the ownership necessary to engage deeply with an agent they did not build.
In the pure embedded model, the accountability is with the business unit. They built it, they use it. The problem is that business units often do not have the governance expertise to know what accountability requires: what logging is needed for a regulator to reconstruct a decision, what oversight is required before an agent takes a consequential action, what disclosure is required when an agent communicates with an employee or customer.
In the federated model, accountability is layered. The platform team is accountable for the infrastructure's reliability, security, and compliance with the standards it enforces. The business unit is accountable for the domain logic, the appropriateness of the agent's scope, the human oversight arrangements, and the outcomes the agent produces. The governance standards specify what each layer owes to the other and what each owes to the organisation.
This layered accountability is more complex to design than either pure model. It is also the only version that holds at scale — because the pure centralised model does not scale to the number of agents a mature organisation operates, and the pure embedded model does not produce the governance consistency that consequential autonomy requires.
Deciding Which Model to Use
The decision is not primarily a function of organisation size, though size affects the economics. It is a function of three variables: the volume and diversity of agent deployments, the distribution of AI engineering talent across the organisation, and the regulatory and governance context in which the organisation operates.
Low volume, concentrated talent, low regulatory exposure — embedded model is appropriate. A small number of agents, a small number of capable teams, and limited governance obligation does not justify the overhead of a centralised platform. Build where the talent is, apply lightweight governance manually, and revisit when volume grows.
High volume, distributed talent, moderate regulatory exposure — federated model is required. When there are enough agents across enough teams that governance fragmentation creates visible risk, and when the talent is distributed enough that a purely central model creates a bottleneck, the federated model is the structural answer.
Regulated industry, high consequence deployments, any volume — federated with strong centralised governance layer. Financial services, healthcare, insurance, and other regulated sectors cannot tolerate the governance variability of a pure embedded model regardless of volume. The governance layer needs to be centralised and enforced, even if the building and operation of individual agents remains distributed. Bank of America's deployment illustrates this at scale: with over 90% of its 213,000 employees using AI tools governed by a 16-pillar governance framework, and adoption managed through an AI catalyst programme with embedded champions across business functions, the bank achieves broad distribution without sacrificing the governance consistency that financial regulation requires.9
Early-stage, experimental, single-team — centralised by default. If one team is responsible for all AI activity, the question of centralised vs embedded does not arise. It will arise when the second team starts building, which is exactly when the structural question needs to be answered.
The Structural Decision as a Strategic One
The question of where agents live is not an IT governance question dressed up as a strategic one. It is genuinely strategic because it determines the pace at which the organisation can deploy capable, governed agents at scale — and therefore the pace at which agentic capability becomes a source of competitive advantage or operational resilience.
An organisation that gets the structure right early — that builds shared infrastructure before it is needed, establishes governance standards before they are violated, and creates accountability structures before an incident makes their absence costly — will compound its agentic capability faster than one that reconstructs its structure reactively. The structural decisions are not glamorous, and they do not appear in product demos. But they are the decisions that determine whether the agents deployed in Chapters 17 and 18 produce durable value or require expensive reconstruction when the portfolio reaches a scale that informal coordination can no longer manage.
The chapters that follow turn to the operational challenge of taking agents from successful pilot into production at scale — beginning with the crossing of the valley that separates a demonstration that works from a system that works reliably, at volume, over time.
References
- McKinsey & Company (2025). The State of AI in 2025: Agents, Innovation, and Transformation. QuantumBlack, AI by McKinsey. November 2025.
- Gartner (2025). Hype Cycle for Artificial Intelligence, 2025 (ID: G00828523). Gartner, Inc. June 2025.
- Andreessen Horowitz (2025). How 100 Enterprise CIOs Are Building and Buying Gen AI in 2025. Andreessen Horowitz. June 2025.
- Wharton Human-AI Research & GBK Collective (2025). Accountable Acceleration: Gen AI Fast-Tracks Into the Enterprise. Wharton Human-AI Research & GBK Collective, University of Pennsylvania. October 2025.
- Deloitte (2025). State of Generative AI in the Enterprise. Deloitte AI Institute. Q1 2025.
- Weill, P. & Ross, J.W. (2004). IT Governance: How Top Performers Manage IT Decision Rights for Superior Results. Harvard Business School Press. (The governance rights framework developed here for IT decision-making applies directly to AI operating model design.)
- Menlo Ventures (2025). State of Generative AI in the Enterprise 2025. Menlo Ventures. December 2025.
- IBM Institute for Business Value (2025). The Enterprise Guide to Agentic AI. IBM Corporation. 2025.
- Bank of America (2025). AI Adoption by BofA's Global Workforce Improves Productivity, Client Service. Bank of America Newsroom. April 8, 2025. https://newsroom.bankofamerica.com/content/newsroom/press-releases/2025/04/ai-adoption-by-bofa-s-global-workforce-improves-productivity--cl.html
Building agentic AI and wondering why alignment is harder than the technology? Get in touch