Model Observability
The ability to monitor, measure, and understand the internal state and outputs of AI models in production — including performance, data quality, and explanation consistency.
Model Observability is the AI-native extension of software observability — applying the three pillars of logging, metrics, and tracing to machine learning systems in production.
Why models require different observability: Traditional software behaves deterministically given the same inputs. ML models do not — their "behaviour" (predictions) changes as input distributions shift, even if the code is identical. This requires monitoring not just for errors and latency, but for statistical properties of data and predictions over time.
The three pillars of model observability:
Data Quality Monitoring — Tracking whether input feature distributions match what the model was trained on. This includes missing value rates, range violations, and population-level drift (PSI).
Model Performance Monitoring — Tracking accuracy, precision, recall, AUC — but only when ground-truth labels are available. For time-lagged outcomes (credit defaults, insurance claims), performance monitoring is inherently delayed.
Explanation Monitoring — Tracking whether the model's explanation structure is stable over time. If SHAP feature importance rankings change significantly without a business rationale, this is a signal of logic drift before accuracy metrics move.
Observable vs. interpretable: A model can be observable (you can see its inputs and outputs) without being interpretable (you cannot explain why it makes specific decisions). Full governance requires both.