SHAP (SHapley Additive exPlanations)
A game-theory-based method for explaining individual predictions by computing the contribution of each feature to the output.
SHAP (SHapley Additive exPlanations) is currently the most theoretically rigorous method for explaining machine learning model predictions. Developed by Lundberg and Lee (2017), it is grounded in cooperative game theory — specifically the concept of Shapley values from economics.
How it works: For each prediction, SHAP computes the average marginal contribution of each feature across all possible orderings of features entering the model. This guarantees three key properties: local accuracy (the explanation sums to the prediction), consistency (if a feature contributes more to a model, its SHAP value does not decrease), and missingness (features absent from the model get zero attribution).
Global vs. local SHAP:
- Local SHAP — Explains a single prediction by showing each feature's contribution to that specific outcome. Used for adverse action notices and customer-facing explanations.
- Global SHAP — Aggregates SHAP values across many predictions to show overall feature importance. Used for model documentation and regulatory submissions.
SHAP in regulated environments: SHAP has become the de facto standard for credit model explanation in regulatory contexts because its mathematically guaranteed properties make explanations defensible. The CFPB and RBI have both indicated SHAP-based explanations as acceptable methods for adverse action notice documentation.
Computational cost: SHAP is computationally expensive for tree-based models using exact methods. Claris uses TreeSHAP (an efficient algorithm for gradient boosted trees) for production-speed explanations at scale.