A proposal lands on your desk for an AI workstation or GPU server.
The specifications look impressive. Plenty of memory. Expensive GPUs. Enough compute to make the proposal feel serious.
So the discussion quickly turns to specifications.
Is this the right GPU? Is the memory enough? Should we buy something larger so we don't regret it later?
But there is a more basic question that should have come first:
What exactly is this machine going to spend its day doing?
"We want to run AI locally" is not a workload.
Neither is "we need private AI."
Before choosing a GPU, server or cloud platform, understand the work: what the system will do, what data it will touch, what model it needs, how many people will use it, how often they will use it, and who will operate it afterwards.
Hardware should come near the end of that conversation, not the beginning.
Workload Before Hardware: a way to ask the right questions
Before you compare a single product, run the decision through eight questions. Call it Workload Before Hardware. It isn't a framework you need to buy into or brand — it's simply the order in which the questions have to be asked if the answer is going to mean anything.
- UseWhat exactly is the workload — drafting, internal search, coding assistance, image generation, something else?
- DataWhat information does it need, and where is that data allowed to be processed?
- ModelWhat type and size of model does the task actually require?
- UsersHow many people will use it, and how much of that use overlaps at once?
- PerformanceWhat response time or throughput genuinely matters here?
- OperationsWho will run, patch, monitor and support this once it's live?
- CostWhat is the true cost, across utilisation, infrastructure and ongoing operations?
- ChangeWhat happens when the model, the demand, or the requirement itself shifts?
Answer these honestly, and you can usually eliminate several wrong architectures before you start comparing products.
Use and data: what are you actually running, and where can it run?
"We want AI" is not a workload. A workload is something you can name: summarising contracts, answering questions against internal documents, assisting developers, generating images for a catalogue. Until you can name it, no infrastructure conversation is worth having.
Once the task is named, the next question is where the data behind it is allowed to go. This is where most people jump straight to "it has to stay on our premises." That instinct deserves more care than it usually gets.
Privacy is a requirement. On-premises is only one possible architecture response to that requirement. What actually protects sensitive data is a combination of things: how the data is classified, what your contracts and regulators permit, how the system is architected, what a provider's terms actually commit them to, who has access, and how carefully that access is managed day to day. A well-controlled cloud or API arrangement can meet a genuine privacy requirement. A poorly managed on-premise server can fail one just as easily.
So the useful question isn't "does this need to be local?" It's: does this data need to stay on our premises, or does it need to stay under contractual and access controls we can actually verify — controls that some cloud and API providers are equally able to offer?
Model and users: sizing the actual job
Two things size an AI workload more than anything else: how large a model the task genuinely requires, and how many people will be using it at the same time.
A small, focused model handling one narrow task is a different piece of infrastructure from a general-purpose model expected to cover five loosely related jobs at once. It's worth checking which one you actually have. "AI" as a category hides a lot of variation underneath it. That variation matters because a proposal can only be as accurate as the workload assumptions behind it.
Model size alone is not enough either. Context length, response expectations, modality and concurrency can all change the compute and memory requirement substantially.
Concurrency matters just as much as scale. A workload used occasionally by three people is not the same architectural problem as the same workload used continuously by forty. The gap between those two situations is usually where a proposal has quietly oversized — or undersized — what's actually needed.
Before sizing anything, it's worth asking plainly: is this a small, focused model for one task, or does "AI" here quietly mean several different tools with several different footprints, bundled under one purchase?
Performance, operations and cost: the parts the quote leaves out
A quote covers a purchase price. It rarely covers the three things that determine whether the purchase was sound.
Performance is the first gap. What response time or throughput does this workload actually need? A tool used a handful of times a day has a very different performance bar from one embedded in a live customer interaction. Buying for the second case when you only have the first is a common way to overspend.
Operations is the second gap, and the one most easily forgotten in the excitement of a new capability. Someone has to patch this, monitor it, and support it when it misbehaves. And the GPU is only part of the system. Memory, storage, networking, operating system, model-serving software, access control, backup and monitoring still have to work together. A powerful accelerator inside a poorly designed environment does not become a good AI platform. If no one has been named for that role, the infrastructure doesn't have an owner — it has a liability with a delivery date.
Cost is the third, and the widest, gap. The real cost of an AI workload includes electricity, support time, software licensing, upgrade cycles and eventual obsolescence — not just the number on the invoice. Whether a local server or an ongoing subscription works out cheaper over its useful life is genuinely case-dependent. It should be worked out with real numbers for your situation, not assumed either way because one option feels more "serious" than the other.
The question worth sitting with is this: are we paying for capacity we actually use, or capacity we bought because we were unsure?
Change: what happens when the requirement moves?
AI requirements can change materially during the useful life of the hardware you buy. The task you're solving today may not be the task you're solving in eighteen months — the model you depend on may change, demand may grow or shrink unevenly, or the business priority behind the workload may simply move on.
This doesn't call for guessing the future. It calls for treating flexibility as a real requirement, alongside performance and cost, rather than something to worry about later. Build in a point where the decision gets reviewed on purpose, rather than by accident when something breaks.
The question to ask before committing: if usage triples, or the model we currently depend on is retired or changed by its provider, what does our next step actually look like — and can we take it without starting over?
The same questions can lead to different architectures
The same eight questions, applied honestly, can lead three different organisations to three different — and equally defensible — architectures. None of them is more advanced than the others.
A professional services firm wants AI for drafting and internal document search, used in short, concentrated bursts by a small group of people. The workload doesn't demand dedicated infrastructure. A managed or API-based approach, with careful attention to data handling, is likely proportionate to the actual use.
A manufacturer has proprietary process data that, after reviewing its contractual and security requirements, the company has decided should remain within a controlled internal environment. Local or hybrid becomes a reasonable architecture to evaluate — but it still needs to be sized against actual use and concurrency.
A small software team wants an internal coding assistant connected to private repositories and expects steady usage throughout the working day. That predictable workload makes local compute worth evaluating alongside managed services and API consumption. The numbers still have to decide.
Notice what we still haven't discussed: GPU brand, processor generation, workstation model or server configuration. That is deliberate. Those questions matter, but only after the workload has narrowed the architecture. Otherwise specifications become a substitute for requirements.
If you can't answer these yet, you're not ready to choose hardware
Before any proposal, quote or product comparison, it's worth returning to the eight questions plainly:
Use — what exactly is the workload? Data — what information does it need, and where may that be processed? Model — what type or size does the task really require? Users — how many, and how much concurrency? Performance — what response experience genuinely matters? Operations — who will run and support this? Cost — what is the true cost across utilisation, infrastructure and operations? Change — what happens when the requirement moves?
None of these questions is difficult on its own. Together, they take real effort to answer properly — which is exactly why they're worth answering before a hardware decision, not after one.
If you cannot answer these questions yet, you probably aren't ready to choose the hardware.