ChatGPT Overestimates Success When Predicting FTMH Surgery Outcomes

In a retrospective analysis of 50 eyes, ChatGPT matched postoperative anatomical closure in 90% of cases and predicted visual outcomes correctly in 66%. Predictions of actual retina specialists demonstrated lower accuracies of 72% to 86% for anatomical outcomes and 42% to 44% for BCVA. However, researchers call ChatGPT’s accuracy “superficial,” noting its tendency to “assume success rather than provide a nuanced analysis.” This optimism bias may lead the LLM to overpromise visual recovery or closure. Photo: Raman Bhakhri, OD. Click image to enlarge. A new study suggests that ChatGPT-5 can estimate outcomes after full-thickness macular hole (FTMH) surgery with surprising accuracy—even higher than that shown by retina specialists. However, the authors warn that the chatbot consistently overestimated success, especially when holes did not close or vision failed to improve, suggesting that its seemingly strong predictive ability may reflect a bias toward favorable outcomes rather than true clinical judgment.The goal of the study, published in Retina, was to evaluate whether the latest ChatGPT large language model (LLM) could predict long-term anatomical and functional outcomes after FTMH repair. They compared the model’s predictive performance against the clinical judgment of retina specialists and real-world 12-month results. Medical records were collected and reviewed from 50 patients who underwent pars plana vitrectomy for FTHM at a single medical center in Israel. All patients were 15 years or older (mean age: 66.2) and had complete preoperative data, OCT imaging and 12 months of follow-up. Baseline vision was poor (average: 20/100), and the mean minimal hole diameter was 441µm. For each case, the team removed personal identifiers and fed ChatGPT-5 a standardized clinical summary that included age, sex, refractive status, lens status, ocular history, symptom duration, preoperative BCVA, hole diameter, surgical details and a single foveal OCT B-scan. The model was asked to predict whether vision would improve, remain stable or worsen by 12 months; estimate the final BCVA; and predict whether the macular hole would close. Two senior retina specialists reviewed the same anonymized material and made parallel predictions.At 12 months, anatomical closure occurred in 44 of 50 eyes (88%), and mean BCVA significantly improved from 20/100 to 20/63. Functionally, 35 eyes (70%) improved by at least two ETDRS lines, eight (16%) remained stable and seven (14%) worsened. ChatGPT-5 predicted closure in 49 of 50 cases and visual improvement in 43 of 50, translating to 90% accuracy for anatomical outcomes (compared to 72% to 86% for retina specialists) and 66% for BCVA overall (compared to 42% to 44% for retina specialists). The chatbot correctly identified all 44 closed cases but missed five of the six nonclosures (17% accuracy), revealing a strong tendency to assume success. For functional outcomes, ChatGPT-5 performed best when vision improved, showing 60% accuracy in such cases. However, its accuracy dropped significantly in cases where vision remained stable (12.5% accuracy) or worsened (0%). The LLM’s mean BCVA prediction error was 11.4 ETDRS letters, with a median error of eight letters. “At first glance, ChatGPT-5 seemed as effective as, or better than, retinal specialists in predicting postoperative FTMH surgery outcomes,” the study authors wrote in their paper. “However,” they continued, “this apparent advantage was mainly due to a consistent overestimation of positive results, indicating a bias toward predicting FTMH closure and visual improvement.” Despite these shortcomings, the authors relay that these findings do validate the diagnostic potential of ChatGPT-5 in FTMH cases. In addition to correctly diagnosing most hole closures, the chatbot was able to recognize radiologic features of FTMH, “including intraretinal cystoid spaces, elevated hole edges and choroidal hypertransmission,” although the authors noted its detection of vitreomacular adhesion was less consistent. They added, “Despite lacking domain-specific training on ophthalmic imaging, ChatGPT-5 effectively interpreted OCT morphology from a single foveal B-scan and structured clinical summary, suggesting an emerging capacity for multimodal reasoning when textual and visual data are integrated.”In conclusion, the authors conveyed, “Although this model can provide quantitative visual acuity predictions and interpret OCT images, their outputs require cautious interpretation to avoid misleading confidence.” They argued that, at this time, AI-generated prognoses should only be used as supportive information, and clinicians should continue to rely on evidence-based counseling and specialist input when discussing expected recovery after macular hole repair.Click here for the journal source. Wattad A, Saadi R, Bez M, Loewenstein A, Goldstein M. Predicting postoperative outcomes in full-thickness macular hole repair surgery - ChatGPT vs. clinical decision. Retina. 2026;46:1015-23. This article was developed by the editorial staff in conjunction with experts in the field. In the process, AI may have been among the editorial tools used to meet the goals of human editors, who approved all content.