AI Chatbots Vary in Responses and Safeguards in Simulated Suicidal Crises

News
chat on smartphone app for mental health support
Three widely accessible AI chatbots varied markedly in how they responded to simulated suicidal crises, according to a simulation study published as a journal pre-proof in the Journal of Affective Disorders. All 3 were vulnerable to attempts to bypass their safeguards.
The investigators simulated interactions with ChatGPT (GPT-5.2), DeepSeek (DeepSeek-V3.2), and the AI companion app Replika (v11.0.0). They used 3 depressive scenarios in which suicidal risk escalated over 7 days, informed by the Columbia Suicide Severity Rating Scale. The protocol included imminent-risk scenarios and attempts to bypass safeguards (jailbreaking). Outcomes included exploration of suicidal ideation, symptom assessment, referral to human support, and vulnerability to jailbreaking.
Overall, referral to human support occurred in 85.7% of daily records for ChatGPT, 76.2% for DeepSeek, and 9.5% for Replika. ChatGPT explored suicidal ideation in 74.6% of daily records, compared with 31.7% for DeepSeek and 7.9% for Replika. During the high-risk period, ChatGPT explored suicidal ideation more often than DeepSeek (adjusted difference, 25.9 percentage points; 95% CI, 2.4-49.5) and Replika (66.7 percentage points; 95% CI, 44.2-89.1).
On day 7, referral occurred in 9 of 9 trajectories for ChatGPT, 4 of 9 for DeepSeek, and 1 of 9 for Replika. Jailbreaking succeeded in 6 of 9, 7 of 9, and 8 of 9 attempts, respectively, and drew potentially harmful information from all 3 systems.
The authors noted that the simulation design and the small sample of 27 trajectories limit generalizability. They concluded that the findings raise concerns about unsupervised chatbot use during suicidal crises and support further evaluation of safeguards across repeated interactions.
Source: Pichowicz W, Kotas M, Kalka N, et al. Responses of AI chatbots to escalating suicide risk: a simulation study of repeated interactions. J Affect Disord. Published online October 5, 2026. doi:10.1016/j.jad.2026.122586
loading
Advertisement
×

Loading