As reported by Digital Health, more than half of serious medical emergencies were incorrectly assessed by ChatGPT Health in a new independent safety study, which also found the system frequently failed to recognise signs of suicidal thinking
The evaluation, carried out by researchers at the Icahn School of Medicine at Mount Sinai, tested the AI tool using 60 detailed patient scenarios ranging from minor illnesses to life-threatening conditions. Each case was first reviewed by three independent physicians, who determined the appropriate level of care based on established clinical standards.
Researchers then prompted the AI to respond under a range of changing circumstances, including altering patient gender, adding laboratory data and incorporating comments from relatives. In total, nearly 1,000 AI-generated responses were analysed and compared directly against doctors’ decisions.
The findings, published in Nature Medicine on 23 February, showed that the system under-triaged 52% of cases classified as gold-standard emergencies. In several instances, patients with conditions such as diabetic ketoacidosis or impending respiratory failure were advised to seek assessment within 24 to 48 hours rather than attend an emergency department.
The study’s lead author, Dr Ashwin Ramaswamy, said the technology handled clear-cut crises more reliably. “ChatGPT Health performed well in textbook emergencies such as stroke or severe allergic reactions,” he said. “However, it struggled when risk was less obvious. Those grey-area cases are often where clinical judgement is most critical.”
ChatGPT Health was launched by OpenAI in January 2026, enabling users in the United States to link their medical records and receive tailored health guidance. According to company figures, around 40 million US adults now use ChatGPT for health-related advice each day.
The researchers also raised concerns about the system’s ability to detect suicidal ideation, noting that subtle indicators of self-harm risk were frequently missed in testing.
Be the first to comment