Five questions that tell you whether a model is production ready
The gap between a model that works and a model that can be deployed is where most data science projects quietly die.
The evidence on this is unambiguous. Gartner's 2024 AI Mandates for the Enterprise survey found that only 42 per cent of non-generative AI prototypes and 41 per cent of generative AI prototypes reached production.1 MIT's Project NANDA reported in 2025 that 95 per cent of organisations deploying generative AI had seen no measurable return.2 Gartner has separately predicted that more than forty per cent of agentic AI projects will be cancelled by the end of 2027, citing unclear value and inadequate risk controls.3
What is notable is that these failures are rarely modelling failures. They are failures to answer questions that should have been asked before anyone committed to a date. These are the five I ask, in order.
1. Does it hold up on data it has never seen?
Cross-validated performance on the training set is a development metric. What matters is performance on a genuinely held-out sample the model never touched during training or tuning, reflecting the class balance of the real world.
Two things go wrong regularly. Evaluating repeatedly against the same holdout and adjusting based on what you see, which leaks information and produces optimistic numbers. And evaluating on a sample whose class balance differs from production, which makes every downstream metric misleading.
On imbalanced problems, be careful which metric you trust. With a low positive rate, ROC-AUC can look strong while the model floods a review queue with false positives, because the negative class is large enough to absorb a great many errors without moving the false positive rate. Precision-recall reflects the operational experience.
2. Do the probabilities mean what they say?
If a model outputs 0.4, do roughly forty per cent of those cases turn out positive? If not, the model is miscalibrated, and any threshold set on those scores means something other than what you intended.
This matters more than it is usually given credit for, because most operational systems do not consume the classification. They consume the score, to rank cases or assign them to tiers. Ranking survives poor calibration. Thresholds do not.
It is straightforward to test and straightforward to correct. It is rarely done.
3. Can you explain an individual decision?
Global feature importance tells you what matters on average. It does not tell a customer why they were declined, and it does not answer a regulator asking about a specific case.
For anything customer-facing in a regulated environment this is a design requirement rather than a reporting nicety, and it influences model selection. A scorecard that is marginally less accurate but trivially explainable is sometimes the better commercial answer. That is a conversation worth having openly with the risk function rather than defaulting to whichever model scored highest.
4. What happens when it is wrong?
Every model is wrong sometimes. The question is whether the surrounding system degrades gracefully.
False positives and false negatives almost never cost the same, and the asymmetry is frequently far larger than teams assume. In card fraud, industry research has put the cost of falsely declined transactions at roughly thirteen times the cost of the fraud actually completed.4 A threshold set on statistical convention rather than economic cost will systematically make the wrong trade.
Design for uncertainty as well. Where the model is not confident, the system should be able to say so and route to a human, rather than forcing a binary answer the evidence does not support.
5. How will you know when it stops working?
This is the question that separates a project from a system, and the one most often deferred.
Models decay. Input distributions shift, customer behaviour changes, and in adversarial domains the counterparty actively adapts. Without monitoring, that decay is invisible until it appears in a loss report.
A workable framework has three layers:
- Performance tracking against outcomes once labels arrive, on a window long enough to be stable rather than noisy
- Input drift monitoring, using a measure such as Population Stability Index, which gives early warning before outcome data is available
- Defined retraining triggers, so the response to degradation is a documented process rather than an argument
One point worth being firm about: a monitoring alert is a prompt to investigate, not an instruction to retrain. Retraining in response to a single poor week, without understanding what changed, is how teams end up chasing noise and destabilising a system that was working.
The pattern underneath
Four of these five questions have nothing to do with model accuracy. That is the point, and it is consistent with what the failure data shows. Gartner attributes the majority of failed AI projects to data and readiness issues rather than modelling.5
Accuracy determines whether a model is worth deploying. Calibration, explainability, failure behaviour and monitoring determine whether it can be deployed at all, and whether it will still be working in a year.