Wrong Automated Advice Tied to Lower Clinician Accuracy

Clinicians were substantially more likely to make correct decisions when automated advice was correct than when it was wrong, according to a systematic review and meta-analysis of clinician responses to erroneous artificial intelligence (AI) and decision-support output, published as an article in press in npj Digital Medicine.

The review included 46 reports from 41 independent cohort clusters, 40 of them core cohorts, spanning rules-based decision support, computerized electrocardiogram (ECG) interpretation, image-based AI, and large language models (LLMs). Of the core cohorts, 35 were controlled or experimental.

Across three cohorts with adjusted estimates, correct advice was associated with higher odds of a correct clinician decision than incorrect advice (pooled odds ratio, 10.19; 95% CI, 7.43-13.98). Study-specific absolute differences ranged from 36.5 to 59.0 percentage points and were not pooled. In four real-world studies of deployed computerized ECG systems, 26.6% of erroneous interpretations were not corrected or recognized (95% CI, 6.4%-65.6%), with extreme heterogeneity; the authors said this estimate should not be interpreted as a common underlying failure rate.

In the only study reporting a complete decision-transition matrix, among dermatologist decisions that conflicted with decision support, 36 of 140 initially wrong decisions changed to correct and 50 of 217 initially correct decisions changed to wrong. Mitigation evidence was inconsistent: localization, selective suppression of high-risk outputs, and case-specific uncertainty cues showed some benefit in individual controlled or simulated studies, whereas image-based explanations and a generic warning that "ChatGPT can make mistakes" were not dependable safeguards. Patient-level adverse events were reported in only one retrospective cohort, and the authors said those observational findings did not establish that automated errors caused the outcomes.

The authors rated 26 of 40 core cohorts as having high methodological concern and none as low concern. They cautioned that the real-world estimate derives entirely from computerized ECG systems and should not be extrapolated directly to generative AI, since most contemporary AI and LLM evidence is controlled or simulated. The authors concluded that human oversight should be evaluated by how well it detects and stops errors before they cause harm, rather than treated as a guarantee of safety.

Source: Zeng D, Lei J, Li Z. Human oversight of erroneous clinical AI and decision support a systematic review and meta analysis. npj Digit Med. Published online October 23, 2026. doi:10.1038/s41746-026-03408-5