AI Matches or Outperforms Specialists in Retina and Glaucoma Management
A groundbreaking study conducted by the New York Eye and Ear Infirmary of Mount Sinai (NYEE) reveals that a sophisticated artificial intelligence (AI) system can rival or even surpass human ophthalmologists in diagnosing and treating patients with glaucoma and retina diseases. Published on February 22 in JAMA Ophthalmology, the research suggests that AI tools like the latest generation GPT-4 (Generative Pre-Training–Model 4) from OpenAI, trained on vast amounts of data, could significantly enhance decision-making support for ophthalmologists dealing with these prevalent eye conditions.
The study, led by Dr. Andy Huang, an ophthalmology resident at NYEE, compared the capabilities of ophthalmic specialists with those of the advanced AI system, GPT-4. With medicine increasingly embracing AI for its potential to revolutionize diagnosis and treatment, particularly through the accuracy and depth of its responses, ophthalmology emerges as a prime candidate for AI integration, potentially allowing specialists more time for evidence-based practice.
"Our study's findings on GPT-4's performance were truly remarkable," says Dr. Huang. "We witnessed the immense potential of this AI system from the outset and were amazed to find that GPT-4 not only assisted but sometimes equaled or even surpassed the expertise of seasoned ophthalmic specialists."
The Mount Sinai team recruited 12 attending specialists and three senior trainees from the Department of Ophthalmology at the Icahn School of Medicine at Mount Sinai for the human aspect of the study. A selection of 20 questions (10 each for glaucoma and retina) commonly asked by patients, along with 20 deidentified patient cases from Mount Sinai-affiliated eye clinics, formed the basis for comparison. Responses from both GPT-4/AI and human specialists underwent statistical analysis and were rated for accuracy and thoroughness using a Likert scale, commonly employed in clinical research.
Results demonstrated that AI matched or exceeded human specialists in both accuracy and completeness of medical advice and assessments. Particularly, AI showcased superior performance in responding to glaucoma queries and offering case-management advice, while achieving comparable accuracy but greater thoroughness in addressing retina questions compared to humans.
Dr. Louis R. Pasquale, Deputy Chair for Ophthalmology Research for the Department of Ophthalmology and senior author of the study, notes, "The proficiency of AI in handling glaucoma and retina cases, with its accuracy and completeness in diagnoses and treatment suggestions, is truly surprising. Similar to how the AI application Grammarly enhances writing skills, GPT-4 can provide invaluable guidance on enhancing clinical documentation."
Dr. Huang underscores the potential of AI in ophthalmology, highlighting its role as a reliable assistant to eye specialists, particularly in complex cases or high-volume settings. "Integration of AI into mainstream ophthalmic practice could lead to expedited access to expert advice for patients, coupled with informed decision-making to guide treatment," he adds, emphasizing the need for further testing to validate these promising results.
