Junior Clinicians Often Missed GPT-4o Hallucinations in Clinical Simulations

Junior clinicians identified 15.8% of hallucinations generated by the large language model (LLM) GPT-4o in simulated clinical decision-making scenarios, according to a multicenter cross-sectional study published in npj Digital Medicine.

The investigators noted that current safeguards for clinical LLM use rely on a "clinician-in-the-loop" model, which assumes clinicians can reliably identify and correct hallucinations. They said quantitative evidence testing that assumption, particularly among junior clinicians, has been limited.

The study included 137 junior clinicians in postgraduate years 1 to 3 at 4 hospitals in China's Jiangsu and Heilongjiang provinces. GPT-4o answered 37 multiple-choice questions drawn from 16 clinical cases in a Chinese national case simulation database. Two senior clinicians classified hallucinations by type and by clinical risk level, from minor to major. Hallucinations appeared in 50.7% of GPT-4o responses, most commonly incomplete chains of reasoning (43.2%) and spurious correlations (27.0%). Participants reviewed each response under time-limited conditions without external references and indicated whether they agreed, disagreed, or were uncertain; uncertain responses were counted as missed detections. Overall, 18 participants (13.1%) did not detect any hallucination.

Detection rates did not improve with increasing clinical risk, ranging from 14.8% to 16.7% across risk levels (P=0.41). Variance decomposition attributed about 29% of the variability in detection to differences between clinicians and about 4% to scenario-level variation.

The authors noted that the study evaluated only GPT-4o without retrieval-augmented generation or other grounding tools, used a standardized classroom-based assessment rather than interactive clinical encounters, and included only junior clinicians. They concluded that future clinical applications will require structured human-AI collaborative workflows and tiered clinical certification pathways to ensure patient safety, while noting that such approaches require prospective evaluation.

Source: Zhang Y, Zhou J, Zhang Z, et al. A multicenter assessment of human oversight of generative AI outputs in simulated clinical decision making. npj Digit Med. Published online September 24, 2026. doi:10.1038/s41746-026-03294-x