Clinical Decision Support AI: Explainability Requirements Under NABH and CDSCO Guidelines

Why Is Explainability Mandatory for Clinical AI in India?

India's clinical AI regulatory landscape has evolved rapidly since 2022:

NABH pathway: NABH accreditation requires that AI systems used in clinical decision support meet the same documentation and audit standards as other clinical protocols. Specifically:

• The clinical basis for AI recommendations must be documentable
• Clinicians must be able to understand and challenge AI recommendations
• AI performance must be monitored and reported in quality improvement processes
• Any AI system with an adverse outcome must be reviewed under the hospital's incident reporting framework

NABH does not require that AI be explainable in algorithmic terms (SHAP, LIME) — it requires that AI be explainable in clinical terms. This distinction matters enormously for implementation.

CDSCO pathway: AI diagnostic and decision support software is regulated as a medical device under Class B (moderate risk) or Class C (high risk) depending on clinical indication. CDSCO requires:

• Software Device Type documentation
• Clinical evaluation protocol
• Post-market surveillance plan
• Explainability as part of the intended purpose documentation

The intersection: Hospitals seeking NABH accreditation while using CDSCO-registered AI must satisfy both frameworks simultaneously. The practical requirement is: explainability that works for regulatory review (algorithmic transparency) AND explainability that works for clinical use (decision support that clinicians can act on).

How Do SHAP Values Translate into Clinician-Readable Explanations?

The translation problem in clinical AI:

A sepsis risk model might produce SHAP outputs that look like:

Raw SHAP: "Lactate_mmolL: +0.41, HR_bpm: +0.28, MAP_mmHg: -0.22, WBC_count: +0.19"

This is technically precise but clinically unactionable. A clinician needs:

Clinical translation: "High sepsis risk (87%). Primary concern: elevated lactate (3.8 mmol/L — above 2.0 sepsis threshold) combined with tachycardia (HR 124). Blood pressure currently stable but trending toward concern. Recommended: SOFA score assessment, blood cultures, consider early antibiotic initiation per Sepsis-3 protocol."

The translation architecture:

Claris's clinical AI adapter operates in three stages:

Stage 1 — Feature mapping: Each AI feature is mapped to its clinical concept with normal ranges and clinical significance thresholds. "Lactate_mmolL" maps to "Blood lactate level" with reference ranges from Sepsis-3 guidelines.

Stage 2 — Significance framing: SHAP attributions are combined with the actual feature values to produce clinically meaningful statements. A SHAP value of +0.41 for lactate becomes meaningful when combined with the actual value of 3.8 mmol/L (above the 2.0 mmol/L threshold) — the explanation becomes "lactate elevated above sepsis threshold."

Stage 3 — Protocol linking: High-SHAP features trigger references to relevant clinical protocols. This turns the explanation into decision support — not just an explanation of what the AI found, but a link to what the clinician should do about it.

Documentation format: Clinical AI explanations must be structured for EHR integration — formatted to be inserted directly into clinical notes without manual transcription, with clear delineation between AI recommendation and clinician decision.

What Audit Trail Does a Compliant CDSS System Require?

Why the audit trail standard is stricter for clinical AI:

In financial AI, the audit trail records the AI decision (approve/reject) and the eventual outcome (default/no default). The cycle is weeks to years.

In clinical AI, the audit trail must record:

• The AI recommendation at the time of clinical decision
• The clinician's response (accept/modify/override)
• The rationale for override (required by NABH)
• The clinical action taken
• The outcome (resolved, deteriorated, adverse event)
• A linkage back to the original AI recommendation for post-hoc analysis

This creates a feedback loop that is both a compliance requirement and a quality improvement mechanism.

The override rate as a governance signal: If clinicians are overriding the AI recommendation >40% of the time, either the AI is poorly calibrated or clinicians have not been adequately trained. Both are governance failures. The audit trail makes override rates visible — and NABH quality committees will ask about them.

If clinicians are overriding <5% of the time, there is a risk of automation bias — clinicians may be deferring to the AI without adequate critical review. This is also a governance failure, and CDSCO's post-market surveillance requirements implicitly address it.

Retention requirements: Medical records in India must be retained for a minimum of 3 years under MCI guidelines (longer for NABH-accredited hospitals). AI audit trails, as part of the clinical record, must be retained for the same period.

Claris's clinical module: The clinical AI governance module generates audit trail records in HL7 FHIR format — compatible with major Indian hospital information systems — and produces quarterly outcome reports that map AI recommendations to patient outcomes, enabling ongoing performance monitoring that satisfies CDSCO's post-market surveillance requirement.