We were brought in to finish someone else's machine learning project. A logistics company had commissioned a model to predict late deliveries. The data science work was, genuinely, good. Careful feature engineering, sensible cross-validation, no leakage that we could find, 94% accuracy on a held-out set. The notebook was clean and well-commented.
It had been finished for eleven months and had never produced a prediction for a real user.
This is common enough to be a genre. It is worth being precise about why, because the reasons are not the ones people assume.
The features could not be computed at prediction time
The strongest predictor in the model was the delivery's total handling duration. Excellent signal. Available in the training data because the training data was historical, and every one of those deliveries had finished.
At prediction time, the moment a customer places an order and wants to know if it will be late, that number does not exist yet. It cannot exist. The model was, in effect, being asked to predict the past.
Not leakage in the classic sense; the validation split was correct. It was a subtler failure: no one had written down what information is actually available at the moment the prediction is needed. That document should exist before any modelling begins. One column, per feature: can this be computed at inference time, from data we will actually have, within the latency budget? Anything that fails is not a feature, no matter how predictive it is.
Accuracy was the wrong metric, and it was hiding the problem
Roughly 9% of deliveries were late. A model that predicted "on time" for every single delivery would score 91% accuracy. The celebrated 94% was, in context, close to worthless, and precision on the minority class told a far bleaker story.
Worse, no one had asked what an error costs. Those two errors are not equivalent. Flagging an on-time delivery as late means a needless apology and possibly a discount. Missing a genuinely late delivery means an angry customer who finds out from the absence of a parcel. The business would happily have accepted three false alarms to catch one more real failure. Nobody had asked, so the model had been tuned to a threshold that optimised a number nobody cared about.
Ask what the two errors cost before you choose a metric. Sometimes you discover the answer is "one of them costs nothing", and the entire problem gets easier.
There was no path from prediction to action
Suppose the model worked perfectly. A delivery is flagged as likely late. Then what?
There was no answer. No notification, no dashboard, no dispatcher workflow, no automatic rerouting. The prediction had nowhere to go. A prediction that does not change a decision is a very expensive way to generate a number.
We now start these engagements from the other end. Not "what can we predict" but "what decision is being made badly today, by whom, and what would they do differently if they knew?" If nobody can describe the changed behaviour concretely, there is no feature there, and the honest thing is to say so before anyone spends money on a model.
Nobody owned it after the handover
The model was delivered as a notebook and a pickled artefact. No serving code, no monitoring, no retraining plan, no defined owner. Even had it been perfect on day one, the delivery patterns it learned would have drifted within a season and there was no mechanism to notice.
A model is not a deliverable. The deliverable is: an inference path, a fallback for when it is unavailable, monitoring on both the inputs and the output distribution, an alarm when they shift, a retraining path, and a named person who owns the thing. Anything less is handing someone a component and calling it a system.
What we did instead
We threw the model away. Not because it was bad, but because it answered a question that could not be asked at the right time.
We replaced it, initially, with a rules engine: four conditions, written by the dispatch supervisor in a meeting, encoding what she already knew about which routes went wrong and when. It took two days. It caught most of what mattered and it plugged directly into the dispatcher's existing screen, because we designed the screen change first.
That gave us three things. It shipped value in a week. It created a baseline any future model would have to beat, which is a question nobody had thought to ask. And crucially, it started logging the dispatcher's overrides: genuine labelled data about the decision we actually cared about, generated by the system itself.
We revisited the model six months later with that data. It shipped, and it beat the rules. But the rules had been earning their keep the entire time, and the model had a real baseline, a real integration point, and a real owner.
Start with the decision. The model is the last thing you build, not the first.