1 · Applications
Where people work with AI: workplace assistants, AI inside business applications, dedicated agents for a process. This is where value is created and measured.
Insights · AI architecture
AI reference architecture: from the application to the multi-agent platform
Many companies now run a dozen AI tools, each with its own data access and its own rules. A reference architecture brings order: what belongs in which layer, and what is built once and used by everyone?
By Michele Mangieri · Updated
In short
An AI reference architecture is a frame that sets out what belongs in which layer and what is built once and used by everyone. It describes six layers: applications, orchestration, knowledge and meaning, data and systems, models, and cross-cutting governance and security. The layers describe capabilities, not products.
Six layers
What goes where.
The layers describe capabilities, not products. Which solution fills a layer is a later decision.
2 · Orchestration
The multi-agent platform assigns tasks to specialised agents, combines their results and obtains approvals. More in the next section.
3 · Knowledge and meaning
Searchable documents, a glossary, metrics with fixed definitions and the relationships between business objects. This layer makes answers correct, not the model.
4 · Data and systems
ERP, CRM, procurement, document storage, data platform. Connected through interfaces with clear agreements, not through clicks on the user interface.
5 · Models
Large and small language models, image and speech models, classic predictive models. Connected through one access layer and replaceable.
6 · Cross-cutting: governance and security
Identities and permissions, logging, cost control, quality measurement and classification under the EU AI Act. Applies to every layer and is built once.
Multi-agent platform (MAP)
The layer that turns single agents into a system.
As soon as more than two or three agents are in use, a shared platform is needed. Otherwise every team rebuilds access, approvals and logging. A MAP bundles six capabilities:
- Agent catalogue: which agents exist, what they can do, who owns them and which data they use.
- Task assignment: an orchestrator breaks a goal into subtasks and hands them to the right agents.
- Controlled access: agents reach systems and tools only through approved interfaces and with their own identity. Open standards keep this replaceable, such as MCP for connecting systems (explained simply) and A2A for coordination between agents.
- Collaboration: agents pass intermediate results through shared memory instead of copies.
- Human in the loop: approval points follow the risk, with clear responsibility and reasoning.
- Governance: every action is logged, cost and quality are measured per task, and any agent can be switched off immediately.
Start from business capabilities
Capability first, technology second.
We do not start with the tool but with the company’s capability map: which capability gains most from AI, and what does it need from the layers below?
Procurement
Bundle demand, assess suppliers, review contracts. A detailed example is in AI in procurement.
Sales and service
Classify enquiries, prepare quotes, answer customer questions from knowledge and contract data.
Finance
Check invoices, explain variances, prepare closing tasks.
Human resources
Answer policy questions, support onboarding, match profiles and requirements.
What several capabilities need belongs in the shared layers: the same permission check, the same glossary, the same platform for agents. Each new use case then builds on the foundations instead of creating another island.
What matters
A reference architecture is a frame, not a blueprint.
- Compare with your own landscape: what exists, what is missing, what is duplicated? That is the first work step, not a by-product.
- Fix the shared layers early: permissions, logging and access to models are expensive to retrofit.
- Aim for replaceability: models and tools change at short intervals. Clear interfaces between the layers protect the investment.
- Proportionate rules: the EU AI Act classifies AI systems by risk. The cross-cutting layer records which use case falls into which class.
From our work
We have worked on enterprise architecture since 2007 and use AI agents ourselves (AI in practice). We use the reference architecture as common ground between business and IT: the business brings the capability, IT brings the layers. What this requires on the data side is covered in Chat with your own data.
Want to bring your AI initiatives into one common frame? AI Decision Readiness →
Frequently asked questions
Questions about AI reference architecture
What is a multi-agent platform (MAP)?
The layer that turns single agents into a system. It assigns tasks to specialised agents, combines their results and obtains approvals. It is needed as soon as more than two or three agents are in use; otherwise every team rebuilds access, approvals and logging.
What role do MCP and A2A play in a multi-agent platform?
They are open standards that keep controlled access replaceable: MCP for connecting systems, A2A for coordination between agents. Agents reach systems and tools only through approved interfaces and with their own identity.
Where do you start with an AI reference architecture?
With the company's capability map, not with the tool: which capability gains most from AI, and what does it need from the layers below? The first work step is comparing the architecture with your own landscape: what exists, what is missing, what is duplicated?
How does an AI reference architecture deal with the EU AI Act?
In the cross-cutting layer for governance and security. The EU AI Act classifies AI systems by risk, and this layer records which use case falls into which class.
30 minutes, and you will know whether we can help.
You describe the decision ahead. We tell you which package fits, or that none does.