The wrong question, asked first
"Should we run AI in the cloud, or invest in our own infrastructure?"
This is usually the first question raised when planning enterprise AI, and it arrives too early. Before deciding where AI runs, the organisation needs to understand what it expects AI to do, what information it will touch, and which responsibilities it is prepared to own. The hosting decision follows those answers; it is not the starting point.
Cloud, local and hybrid are implementation choices. Which one fits depends on business criticality, information sensitivity, operational readiness, economics, and how much freedom the organisation wants to preserve for later. Choosing the technology first produces architecture in reverse.
In this article, local AI means an organisation-operated environment — on-premises, at the edge or within dedicated infrastructure under its direct operational control. Cloud AI refers to provider-hosted services, while hybrid AI deliberately distributes workloads across these environments.
Begin with the capability, not the platform
A few plain questions deserve honest answers before any hosting conversation starts: what business outcome requires AI — employee productivity, an experiment, or a capability meant for indefinite production use? How critical will it become if it succeeds, and what happens if it becomes unavailable for a day? An AI capability used occasionally by a handful of people carries a different profile from one embedded into a regulated or customer-facing workflow. The first question is not where this should run, but how much the organisation depends on it.
Information architecture comes before infrastructure
Where a model runs matters less than what it is allowed to read. Before choosing a deployment model, the organisation needs to know what information the AI will need, where it resides, who is accountable for it, how sensitive it is, and whether it can legally or contractually leave its boundary — plus a working answer for how access is granted, and revoked, as people and workloads change.
Physical locality does not resolve any of this by itself. Keeping a model on-premises does not make information ownership clearer or its quality better; it only shifts where the same questions get answered. Information governance is a precondition for a sound deployment decision, not a byproduct of any hosting choice. This connects to a broader readiness question covered in Before You Buy Another AI Tool: Is Your Enterprise Architecture Actually AI-Ready?
Decision rights and accountability
Every deployment choice should be shaped by what the AI is allowed to do: what it can support, what it can act on unattended, which outputs need human review, who is accountable when a result is wrong, who can pause or withdraw it, and how actions get traced back later.
An AI capability with broad decision authority and weak traceability is unsafe in any deployment model. Hosting changes how controls are distributed; it does not remove the underlying accountability problem. This is explored further in Your Enterprise Is Not Ready for AI Agents — and That's an Architecture Problem.
The operating-model question
Running AI on owned infrastructure creates responsibilities that are easy to underestimate: platform ownership, patching, lifecycle management, monitoring, capacity planning, backup and recovery, and incident response — often requiring skills the organisation does not already have.
Cloud AI reduces some of this burden, but the shift is partial and varies by service model. What does not move is the organisation's responsibility for the information the AI can access, who is allowed to use it, how it is configured and integrated, and what it may decide. Greater direct control over infrastructure also creates greater operational responsibility — worth choosing only if the organisation is prepared to operate what it demands.
Economics beyond the monthly bill
Cost comparisons between cloud and local AI often stop at a monthly figure, which is rarely where the real difference lies. A fuller comparison weighs predictable versus irregular demand, capital versus operating expenditure, how consistently the capability will be used once enthusiasm settles, the skills and integration cost of running either option, and how quickly it ages.
It should also account for the cost of changing direction later. The cheapest option at the point of decision is not always the one with the lowest cost across its working life — an irregular workload on owned infrastructure, or a steady one paid for continuously at cloud rates, can each become the more expensive choice.
Reversibility and architectural freedom
A deployment decision made under real uncertainty benefits from preserving options rather than closing them. Reversibility depends less on where a model runs than on how portable the model and its data are, how deeply the platform is woven into proprietary interfaces, and what an exit would cost under contract.
When uncertainty is high, preserving the ability to change direction may matter more than optimising the first implementation — a less efficient deployment that is easy to unwind is often safer than one that performs marginally better while locking the organisation in.
Comparing the three operating choices
None of the three models is universally superior. Each distributes cost, control and responsibility differently — no label below guarantees an outcome alone.
| Dimension | Cloud AI | Local AI | Hybrid AI |
|---|---|---|---|
| Initial investment | Often lower upfront; subscription, consumption or committed capacity | Usually higher upfront when infrastructure is purchased; may also be leased | Varies according to the split between environments |
| Operational responsibility | Shared — provider handles infrastructure; organisation owns information, identity, integration, outcomes | Largely in-house, infrastructure to application | Split; assigned at the boundary |
| Elasticity | High; scales with demand | Limited to owned capacity | Elastic for cloud portion; fixed for rest |
| Information control | Depends on contractual, architectural safeguards | Direct control, if architecture and skills support it | Matched to sensitivity, workload by workload |
| Integration complexity | Simpler where standard APIs exist | Depends on integration maturity | Depends on boundaries introduced |
| Specialist skills | Fewer infrastructure skills; governance skill required | Significant in-house depth required | Both operating models maintained |
| Reversibility | Depends on portability, formats, interfaces, contract terms — not the label alone | Depends on the same factors | Depends on coupling between environments |
| Best suited for | Variable demand, faster time to value | Direct control, where sustainable | Mixed sensitivity or criticality |
Sensitive information does not automatically require local infrastructure — a sovereign, dedicated or private-cloud arrangement can satisfy requirements public cloud cannot.
A simple framework
Capability → Information → Authority → Operations → Economics → Exit
Capability — the outcome this must enable, and how critical it becomes. Information — what it will access, how sensitive it is, and who owns it. Authority — what it may decide or only recommend, and who is accountable. Operations — who owns running it once the pilot ends. Economics — the cost across its working life, not the first invoice. Exit — how easily course can change if assumptions prove wrong.
Only once these six are answered does cloud, local or hybrid deployment become the sensible next question — the outcome, not the starting point.
Four scenarios
General employee AI productivity. A broad population uses AI for drafting, summarising and everyday research on low-sensitivity material, with unpredictable demand and moderate criticality. A cloud-hosted service with clear access boundaries and an approved list of sources is a reasonable starting direction — provided an owner is named and the assistant's limits are defined.
Sensitive document analysis. A smaller group works with confidential or regulated material restricted from leaving a defined boundary. Information control usually matters more than convenience here — but that does not automatically mean local infrastructure. A private or sovereign cloud arrangement, a dedicated tenancy, or an on-premises deployment can each satisfy the requirement, depending on which the organisation can operate to that standard.
High-volume repeatable AI workload. A steady, high-volume process — document classification across a known pipeline, for example — runs continuously. Sustained demand is exactly the condition under which owning the infrastructure can become more economical than paying continuously at cloud rates, provided the organisation builds the skill to run it well.
A mixed organisation requiring hybrid deployment. A larger organisation has both a broad low-sensitivity need and a narrower sensitive one. A hybrid split — cloud for the general case, a bounded environment for the sensitive case — matches control to the workload, at the cost of maintaining two operating models with a clear boundary.
Each rests on assumptions about information maturity, operating skill and risk appetite — a different organisation could reasonably reach a different answer.
What this asks of the architect
The Enterprise Architect should not select technology by trend, act as a catalogue, assume owning infrastructure automatically means security or sovereignty, assume cloud automatically means cheap, or let a pilot quietly become an unmanaged permanent platform.
Instead, the architect connects the deployment choice to the capability it serves, exposes the dependencies each option carries, clarifies accountability, weighs consequences across the enterprise, preserves future choices where uncertainty is high, and defines the guardrails for the decision. This is the same discipline behind When an Enterprise Architect Should Say No—and What Comes Next.
Choosing what the enterprise can sustain
Cloud, local and hybrid AI are not competing ideologies. They are different distributions of cost, control, responsibility and risk.
The right place to run AI is not decided by where the model performs best alone. It is decided by what the enterprise is prepared to trust, operate, govern and sustain.