The wrong question — cloud or on-prem?
Most infrastructure conversations begin with a platform debate. Cloud or on-premises? SaaS or build it ourselves? The framing is too broad to be useful.
The same organisation may need SaaS for one capability, public cloud for another, colocation for a specific workload, and on-premises infrastructure for a local operational system that cannot leave the building. Choosing a universal platform before understanding what each workload actually needs is how organisations end up with the wrong tools in the wrong places.
The better question — what responsibility should we own?
Every hosting model shifts responsibility differently. Some models put more in the hands of the organisation. Others shift most of it to the provider. Neither is automatically better.
Infrastructure architecture is not only about where something runs. It is about who owns the responsibility when it runs, fails, scales, needs securing or needs changing.
The useful question before choosing a platform: which responsibilities does the organisation genuinely want to retain, which can be safely delegated, and which should not be operated in-house at all?
The infrastructure spectrum — control and responsibility
The models below are not maturity levels. They are responsibility models. Each places a different combination of control and operational burden on the organisation and the provider.
In-house server room
This only makes sense when the organisation genuinely requires direct physical control — and has the capacity to manage everything that comes with it: power, cooling, physical security, hardware lifecycle, backup, monitoring, patching, disaster recovery, and operational staffing. Common fits include factory or site-specific operational systems, strict data locality requirements, and latency-sensitive local workloads. The honest test is whether the organisation can sustain this across staff changes and hardware failures — not just on a good day.
Colocation
Colocation suits organisations that want to retain hardware and network control but not the burden of operating the building. The provider owns the facility. The organisation owns and operates the equipment inside. It works well when specific network interconnections or appliances are required, or when regulatory constraints need physical hardware ownership but not facility management.
Dedicated and managed hosting
Dedicated hosting provides a predictable, single-tenant environment. Managed hosting shifts operational responsibility — patching, monitoring, backup — to the provider. These fit predictable web applications, legacy workloads, and teams that need infrastructure without the overhead of full cloud complexity.
Private cloud
Private cloud only makes architectural sense when there is genuine scale, standardisation, automation capability, and operating maturity. A virtualisation cluster is not automatically a private cloud. Private cloud requires platform thinking, service catalogues, automation, governance and lifecycle discipline. Without those, it is usually a more expensive way to achieve what managed hosting or public cloud IaaS could deliver more reliably.
Public cloud IaaS
IaaS gives flexible access to compute, storage, and networking at scale. The provider manages the physical infrastructure. The organisation still owns a great deal: architecture, OS patching, identity, network configuration, backup, monitoring, security posture, cost governance, and resilience design. IaaS fits variable workloads, development environments, disaster recovery, and organisations that benefit from genuine elasticity.
PaaS
PaaS allows teams to focus on application delivery without managing underlying infrastructure. The speed-to-delivery benefit is genuine. The trade-off is platform dependency — application design may become tightly coupled to provider-specific runtimes, data services, or APIs. This is sometimes acceptable. It becomes a problem when the dependency was never examined.
SaaS
SaaS is often the most sensible choice when the capability is standard, non-differentiating, and better consumed than operated — email, CRM, HR, payroll, collaboration, ticketing, and standard finance workflows. If operating the infrastructure does not create control, compliance advantage, or competitive differentiation, SaaS may be the more responsible architecture choice.
Where AI changes the hosting decision
AI workloads introduce additional questions beyond standard hosting. The model itself is a deployment choice. Where does it run? Where does the data go? Who operates it? How predictable is the cost? What happens if the service is unavailable? How will output quality, auditability, and governance be managed?
AI is not one hosting choice. It can be consumed as SaaS, called through an API, operated on a cloud AI platform, controlled through a private platform, self-hosted, deployed locally, or split across a hybrid architecture.
Public AI SaaS
Standard productivity and low-risk use cases — writing support, meeting summaries, basic productivity assistance. Fastest to access, least infrastructure responsibility, but lowest data control. Provider terms and configuration govern what is acceptable.
AI API / foundation model service
Adding AI capability into an application — document summarisation, semantic search, ticket classification, chat over internal knowledge — without operating models directly. Good capability and speed, but requires governance for data sharing, cost, access, logging, and fallback.
Cloud AI platform
Managed model services, vector search, AI pipelines, and integration with cloud data platforms. Fits enterprise AI applications, retrieval-augmented generation, and analytics workloads. More scalable and integrated, but cost governance and operational maturity become important.
Private AI platform
Stronger control over models, data, identity, and network boundaries — typically for regulated workloads or sensitive internal knowledge at scale. Only makes sense with scale, appropriate skills, and mature governance.
Self-hosted open-source model
Strong reason to control the model runtime, data path, or customisation — specialised internal models, sensitive data processing, offline or restricted environments. Very high control, but the team owns model serving, GPU infrastructure, scaling, patching, monitoring, and security.
Edge or local AI
When latency, offline operation, data locality, or physical operations matter — factory inspection, local video analytics, field devices, branch operations, real-time equipment monitoring. Strong local control, but distributed operations and lifecycle management across many sites or devices is demanding.
Hybrid AI architecture
Often the most pragmatic enterprise pattern. Some parts of the workflow use external AI services; sensitive or latency-dependent parts remain controlled. Requires clear boundaries between AI assistance, deterministic authority, and system of record.
AI deployment patterns compared
| AI pattern | Best fit | Cost behaviour | Security & data control | Operational burden | Scalability | Ease of access |
|---|---|---|---|---|---|---|
| Public AI SaaS | Standard productivity or low-risk use cases | Subscription or per-user cost | Lowest; depends on provider terms and configuration | Very low | Provider-managed | Very high |
| AI API / foundation model service | Adding AI into an application | Usage/token based; can grow quickly | Moderate; requires data filtering, logging and policy controls | Low to medium | High; provider-managed | High |
| Cloud AI platform | Enterprise AI apps, RAG, pipelines, cloud data workloads | Variable; needs active cost governance | Good if cloud controls are well designed | Medium | High | Medium |
| Private AI platform | Sensitive enterprise AI at scale | Higher fixed and platform cost | High | High | Medium to high if engineered well | Medium to low |
| Self-hosted open-source model | Custom, restricted or specialised model use cases | Infrastructure and GPU-heavy | Very high if operated correctly | Very high | Harder; must be engineered | Low |
| Edge / local AI | Low-latency, offline or site-specific AI | Device and lifecycle cost | High local control | High across many sites or devices | Operationally complex | Low to medium |
| Hybrid AI architecture | Enterprise workflows needing AI capability and control | Mixed; depends on workload split | Balanced if boundaries are clear | Medium to high | Good if designed modularly | Medium |
For AI systems, exit strategy should cover more than data export. Prompts, embeddings, evaluation sets, model-specific behaviours, retrieval pipelines, and human approval workflows may all become part of the operating model. If these become provider-specific, moving is harder than the initial integration appeared.
The enterprise architecture lens
Every hosting decision is worth assessing across a consistent set of factors. Not all will carry equal weight — but each deserves a considered answer before the decision is made.
- Cost model — is the cost structure predictable, and does it remain sustainable at the scale this workload will reach?
- Security and data control — what data handling and sovereignty requirements apply, and does the model meet them?
- Operational responsibility — what does the organisation commit to operating, and does it have the skills and staffing to do so?
- Scalability — how does this model behave when demand grows, shrinks, or changes unexpectedly?
- Resilience and recovery — what is the recovery path when something fails, and has it been tested?
- Integration complexity — does the hosting model complicate connections to other systems?
- Lock-in and exit path — what would it cost to move, and is that a known and acceptable risk?
- Governance and auditability — can the organisation demonstrate appropriate oversight and audit capability?
- Business criticality — does the hosting model's reliability match the importance of this workload?
A technically possible hosting model is not automatically an architecturally responsible one.
Exit strategy is part of the architecture
Every hosting choice creates a dependency. In-house environments create hardware and facility dependencies. Managed hosting creates provider dependencies. PaaS creates runtime and service dependencies. SaaS creates data, integration, and workflow dependencies. AI API deployments create model and data pathway dependencies.
Exit strategy does not mean avoiding every form of lock-in. It means knowing what would be difficult to change before the decision is made — not after the organisation has been running on a platform for three years and finds itself negotiating from a weak position.
Some lock-in is acceptable when the value is clear, the risk is understood, and the exit path is known. Unexamined lock-in is the problem.
A few questions worth asking before committing:
- In-house / colocation — can the workload move later? Is the hardware lifecycle documented? Are backups portable?
- Managed hosting — can the provider hand over configuration, backups, and operational documentation? Could another provider operate the same workload?
- Private cloud — is the platform built on portable standards or proprietary tooling? Can workloads move to public cloud or hosted infrastructure?
- Public cloud IaaS / PaaS — how much depends on cloud-specific services? Can the workload be rebuilt elsewhere from infrastructure-as-code? Are backups tested, not just configured?
- SaaS — can data be exported in usable formats? Are integrations documented? What happens if price, terms, or product direction change?
- AI deployment patterns — can prompts, embeddings, and evaluation data move? Are model-specific behaviours documented? Is there a fallback if the AI service is unavailable?
The exit path does not need to be easy. It should be known. If leaving a platform would be impossible, expensive, or operationally dangerous, that risk belongs in the original decision.
A comparison worth keeping
| Decision factor | In-house / colocation | Dedicated / managed hosting | Private cloud | Public cloud IaaS / PaaS | SaaS |
|---|---|---|---|---|---|
| Hardware control | Full | Owned or provider-owned | Owned or leased | Provider-owned | Provider-owned |
| Facility responsibility | Organisation (in-house) or provider (colo) | Provider | Organisation or provider | Provider | Provider |
| OS responsibility | Organisation | Shared or provider | Organisation | Organisation (IaaS) or provider (PaaS) | Provider |
| Scaling flexibility | Low; hardware-bound | Low to medium | Medium; capacity planning required | High | Provider-managed |
| Operational burden | Very high | Medium | High | Medium to high | Low |
| Cost predictability | Predictable but front-loaded | Predictable | Predictable but infrastructure-heavy | Variable; requires cost governance | Subscription; predictable |
| Exit complexity | Medium; hardware and network dependencies must be planned | Medium; depends on contracts and operational handover | Medium to high; depends on platform choices and automation | Medium to high; depends on managed services used | High if data, workflow and integrations are deeply embedded |
| Best fit | Local control, regulated, latency-sensitive, restricted environments | Predictable workloads, legacy, moderate custom apps | Scale, standardisation, automation maturity, strong governance | Variable demand, elastic workloads, dev and DR environments | Standard, non-differentiating capabilities |
Five principles that matter
- Do not choose a platform before understanding responsibility.
- Own only what creates control, compliance or advantage.
- Do not build what you cannot operate.
- Commodity capabilities usually belong higher up the stack.
- Design continuity, backup, exit and recovery before migration.
For AI-enabled workloads, also decide where the model runs, where the data goes, and who remains accountable for the output.
What this means in practice
For normal infrastructure, the question is where the workload should run. For AI-enabled infrastructure, the additional question is where the intelligence should run, where the data should go, and who remains accountable for the output.
A responsible hosting decision includes not only how the workload starts, but how it can be recovered, moved, reduced, replaced, or retired when conditions change.
Good infrastructure architecture is not about choosing one universal platform. It is about placing each workload where the balance of control, responsibility, cost, resilience, and business value makes sense — and knowing what it would take to change that placement if circumstances require it.