Shipping your first production ML model

The gap between a working notebook and a model your team can rely on in production is bigger than most first-time ML projects budget for — and it's mostly engineering, not data science.

What changes between a notebook and production

A model that performs well in a Jupyter notebook, evaluated on a static historical dataset, has answered exactly one question: can this approach learn the pattern at all. Production introduces a different set of requirements entirely: the model needs to make predictions reliably on new, live data (not just historical data it was validated against), it needs monitoring to detect when its performance degrades over time (a real and common phenomenon called model drift), and it needs a defined fallback behavior for when it fails or produces a low-confidence prediction — none of which show up as concerns in notebook-stage development.

The infrastructure a production model actually needs

A serving layer — typically an API endpoint (often built with a framework like FastAPI, deployed on infrastructure that can scale to your actual prediction volume) that the rest of your application calls to get predictions, rather than the model living only in a data scientist's local environment. Monitoring — tracking not just system uptime but prediction quality over time: is the distribution of inputs the model is seeing in production still similar to what it was trained on, and is accuracy (measured against actual outcomes, once they're known) holding steady or degrading. A retraining pipeline — because real-world patterns shift over time, a model trained once and left static will gradually degrade; production ML needs a defined process for retraining on fresh data, whether that's scheduled or triggered by detected drift.

Confidence thresholds and fallback behavior

We build every production model with an explicit confidence threshold below which its prediction isn't used automatically — instead routing to a human review or a simpler rule-based fallback. This matters because models are probabilistic; a model with 85% overall accuracy is still wrong 15% of the time, and treating every prediction as equally trustworthy, with no way to flag its own uncertainty, produces worse real-world outcomes than a system that knows when to defer.

A concrete example

A client's data science team had built a working churn prediction model in a notebook with promising validation accuracy, but had no path to actually using it operationally — predictions existed only as a static file generated manually, occasionally, by one person running a script. We built a production serving layer generating daily predictions automatically, a monitoring dashboard tracking prediction distribution and (once outcomes were known) actual accuracy over time, and a confidence-threshold system routing the highest-risk, lowest-confidence predictions to a human review queue for the customer success team rather than fully automating outreach decisions for those ambiguous cases.

Six months post-launch, monitoring caught a real instance of model drift — accuracy on a specific customer segment had degraded following a pricing change that shifted the underlying churn patterns — and the automated alert triggered a retraining cycle before the degraded accuracy caused any real operational impact, which is exactly the kind of issue that would have gone unnoticed in the original notebook-only setup.

How Ndakum approaches it

Production ML infrastructure — serving, monitoring, and retraining pipelines — is a core part of our Data Engineering & AI work, not an afterthought once a model shows promise in a notebook.