Somewhere right now, a leadership team is approving budget for a machine learning team, and nobody in the room can name the decision the models will make. The demos were dazzling, a competitor has just announced an “AI-first strategy”, and the board has started asking what ours is. The plan almost writes itself: hire a head of data science, recruit a few scientists, choose a platform. It feels less like a decision and more like a deadline. This post is the pause button, written for the people in that room: the honest signals that you need machine learning, the equally honest signals that you don’t yet, and the pitfalls waiting for the teams that skip the question.
One question before the org chart
Ordinary software runs on rules that people write down. Machine learning is software that works out the rules for itself, from examples. That single sentence carries the whole business case, because it tells you exactly when ML earns its keep: when the rules are too subtle for anyone to write down, and when you have enough examples for the software to learn from. Both halves have to hold. If a domain expert can sketch the rules on a whiteboard in an afternoon, you should pay an engineer to implement the whiteboard, not a team to approximate it statistically. Google’s own engineering guide to the subject opens with exactly this advice, and it is worth quoting because of who wrote it: “Rule #1: Don’t be afraid to launch a product without machine learning.”1
There is a second clue hiding in that sentence about examples. Learning from examples only pays off for decisions that recur: which product to show this shopper, which transaction to flag, which machine to service next, which ticket to route where. Thousands of small judgments, each slightly different, each cheap to get slightly wrong. If the decisions keeping you up at night are the big, rare ones (entering a market, pricing a contract, acquiring a company), those are questions for analysis and judgment. An analyst with clean data will serve you better there than any model, at a fraction of the cost.
So before anyone drafts an org chart, the question to answer is not “how do we get AI?” but “which decision, made over and over at scale, would we hand to software that learns?” If the room goes quiet, the room has its answer, at least for now.
The small box
Suppose the room does not go quiet. Someone names a real decision, the data plausibly exists, and the case starts to look genuine. The next thing to understand is what you are actually signing up to buy, because it is not a model.
In 2015, a group of Google researchers published a paper about the machine learning systems they had spent years running in production, and its most famous figure is a map of one.2 The machine learning code is a small box in the middle. Everything around it, and dwarfing it, is the machinery required to keep that small box fed, honest, and running.
Executives are routinely surprised by this picture, and the surprise is expensive. The pitch deck is about the ember-coloured box. The budget will be spent on the gray ones: pipelines that collect and verify data, infrastructure that serves predictions quickly and reliably, monitoring that notices when something silently degrades. A model that will influence real customer decisions needs all the robustness of ordinary production software, plus the extra complexity of data collection, data governance, and the model itself.
And unlike ordinary software, the small box rots. The world a model learned from keeps moving: customers change habits, competitors change prices, fraudsters change tactics. A model trained once and left alone will quietly drift out of date, which is why mature teams build retraining checkpoints and performance tracking from day one. It is more useful to think of the purchase as a small factory that manufactures decisions rather than a machine you buy once. Factories need maintenance crews, and the maintenance is an operating cost, not a one-off investment.
The pyramid underneath
Every organisation believes it has data. Somewhere there is a storage account that has been filling quietly for years, owned by a team that no longer quite remembers why, and its existence is offered in meetings as proof of readiness. It almost never is. Data that has been accumulating untouched has usually never been cleaned, never been reconciled with the systems around it, and never been asked a single question. Before it can teach a model anything, someone with domain knowledge has to work out what it actually contains, and someone with engineering skills has to make it trustworthy. Consistently, this takes longer than the modelling it enables.
In 2017, data science leader Monica Rogati gave this problem a picture that has aged very well: a hierarchy of needs, with AI at the top and everything it depends on stacked beneath.3
The pyramid is a self-diagnosis tool. If your analysts cannot yet answer routine questions about last quarter from numbers everyone trusts, your organisation lives in the bottom half, and the honest next hire is a data engineer, not a deep learning researcher. That is not a demotion of your ambitions; it is their sequencing. Teams that hire from the top of the pyramid while the base is missing end up with expensive scientists doing plumbing, badly and resentfully, or worse, with models trained on foundations nobody trusts. And if you are early enough that the data does not exist at all, the cheapest, highest-return AI investment available to you is instrumentation: start logging today, and the option to do ML opens up in the future.
A short field guide
Pulled together, the signals point in reasonably honest directions.
| You may genuinely need ML | You probably don’t, yet |
|---|---|
| A decision made thousands of times a day | A decision made a few times a quarter |
| Rules too subtle for an expert to write down | An expert can state the rules on a whiteboard |
| Outcomes already logged, or loggable now | Data collection would be starting from zero |
| Occasional wrong answers are survivable | A single wrong answer is a lawsuit |
| A simple heuristic was tried and hit its ceiling | Nobody has tried a heuristic |
The same signals, arranged as the path a decision should actually walk:
over and over,
at scale?"} -->|no| B["ML is the wrong tool.
Automate the workflow,
hire analysts"] A -->|yes| C{"Could a simple rule
get most of the way?"} C -->|yes| D["Ship the rule first.
It becomes the baseline
ML must beat"] D -.->|"when the rule
stops improving"| E C -->|no| E{"Do you log the data
the decision
depends on?"} E -->|no| F["Start collecting now.
Revisit in six to
twelve months"] E -->|yes| G{"Can you afford, and
detect, wrong answers?"} G -->|no| H["Keep people deciding.
Use models to assist,
not to decide"] G -->|yes| I["A real ML case.
Start with one use case
and a small team"] class A,C,E,G decision; class B,F,H guardrail; class D focus; class I terminal;
Notice how much of the diagram is exits. That is the point. “No, not yet” is not a failure of ambition; it is the finding that a cheaper tool solves your problem, which is what good engineering judgment looks like. The rule-based exit deserves particular respect, because the heuristic you ship today does double duty: it delivers value immediately, and it becomes the baseline any future model has to beat before it deserves your money.
Where funded teams still sink
Some organisations pass every gate above and still watch the investment underdeliver. Having seen a few of these up close, the failure modes are remarkably consistent, and almost none of them are about algorithms.
The value proposition is the algorithm. An initiative gets funded because someone heard about a technique, and the technique becomes the point. Your customers will never care. They care whether the recommendation is useful, whether the search finds the thing, whether the claim is settled quickly. Nobody has ever churned because your recommender was insufficiently sophisticated; plenty have churned because it recommended things they had already bought. Start from the customer’s unmet need and work backwards to the method, never the reverse.
The model optimises the wrong thing, sincerely. A model pursues exactly the target you give it, with complete dedication and no judgment. Optimise a search engine purely for engagement and it will learn to surface whatever keeps people clicking, including results that waste their time attractively. The gap between “what we measured” and “what we meant” is where ML projects quietly turn against their owners, and closing it is a leadership job, not a technical one: deciding what the business actually wants improved is not delegable to the model.
Nobody can prove it worked. The system ships, dashboards glow, and a year later someone in finance asks what changed. If the answer requires faith, the project is in trouble. The only way to know a model beats the old way of deciding is to compare them honestly, usually with an A/B test against the baseline, and that measurement machinery has to be planned and budgeted like any other part of the system. The entire point of the investment is improvement; unmeasured improvement is indistinguishable from expensive noise.
Build and buy get conflated. The recent wave of pre-trained models, including the large language models behind tools like ChatGPT, is genuinely remarkable for language, images, and other unstructured data, and you should not rebuild what you can rent. But most business problems are tables: customers, transactions, sensor readings. For those, pre-trained giants offer little, and you will be training your own, smaller models on your own data. The two paths need different people, different infrastructure, and different budgets, and discovering mid-project that you picked the wrong one is a costly way to learn the difference. What almost nobody needs, in either path, is to invent algorithms in-house: the methods that will serve you are published, battle-tested, and free, so hire people who can apply research, and leave producing it to the institutions funded for that.
The humans get designed out. A model that runs a core process becomes business-critical the moment it works, and a business-critical system that nobody can inspect, question, or override is a standing risk. Keep people in the loop by design: reviewing samples of decisions, handling the edge cases the model was never trained for, empowered to pull the cord. This is also, quietly, what makes executives comfortable enough to let the system take on more, because trust in automation is built by the ability to check it, not by accuracy statistics alone.
If the answer is yes
Then start smaller than feels proportionate to the ambition. One use case, chosen because the decision recurs at scale, the data exists, and the outcome is measurable. A first hire or two who are pragmatic generalists, able to move data, build a model, and measure honestly, rather than a research team. A shipped heuristic as the baseline, then a model that must beat it in a fair test before it takes over. A monitoring and retraining plan written before launch, because the small box rots. Growth in the team only after the first system is visibly paying rent.
None of this is as exciting as the announcement your competitor made. But their announcement, in a few years, has to survive contact with everything above: the gray boxes, the pyramid, the baseline nobody shipped. Yours will be a working system with a measured return, and a leadership team that can name, precisely, the decision its models make. That is what an AI strategy actually is.
Sources & further reading
- Hidden Technical Debt in Machine Learning Systems (Sculley et al., NeurIPS 2015) — the “small box” paper; the canonical argument for why the model is the cheap part.
- Rules of Machine Learning (Martin Zinkevich, Google) — production ML wisdom in rule form, starting with “don’t be afraid to launch a product without machine learning”.
- The AI Hierarchy of Needs (Monica Rogati, 2017) — the readiness pyramid, from someone who built data teams at LinkedIn and Jawbone.
- People + AI Guidebook (Google PAIR) — practical worksheets for deciding whether ML is the right approach for a product problem at all.
- Designing Machine Learning Systems (Chip Huyen, O’Reilly 2022) — the book to hand the team once the answer is yes.
-
From Martin Zinkevich’s “Rules of Machine Learning”, Google’s distilled guidance from years of production ML systems. The rule continues: “Machine learning is cool, but it requires data. If you don’t absolutely need machine learning for your first version, don’t use it until you have data.” ↩
-
D. Sculley et al., “Hidden Technical Debt in Machine Learning Systems”, NeurIPS 2015. The paper’s framing is that ML systems accrue maintenance cost, “technical debt”, faster and more quietly than ordinary software. ↩
-
Monica Rogati, “The AI Hierarchy of Needs” (2017). Rogati’s framing: “Think of AI as the top of a pyramid of needs. Yes, self-actualization is great, but you first need food, water and shelter.” ↩