ChatGPT, which generates texts using artificial intelligence, achieved a score close to that required to pass a difficult medical exam in the United States, according to a study published Thursday.
OpenAI launched a "conversational robot" called ChatGPT last November, which was widely successful and caused a sensation due to its ease of use and its ability to engage in a dialogue-like conversation between humans, to answer questions asked to it in a somewhat accurate manner, and even to write novels or poems. in seconds.
In the study, published in the journal PLOS Digital Health, researchers from Unstable Health tested the robot's performance on an exam usually administered to medical students in the United States, which includes questions in various fields such as scientific knowledge, clinical reasoning, bioethics, and so on.
This exam, called the Medical Licensing Examination in the United States, is divided into 3 parts, the first of which students take after about two years of study and the second after 4 years, while they must succeed in the third in order to be able to practice medicine.
ChatGPT was tested on 350 of the 376 questions posted on the US Medical Licensing Examination website that were included in the June 2022 exam. Image-based ones were excluded.
The questions were divided into 3 types, one of which is open-ended questions. What is the diagnosis for this patient in light of the information provided? And another one can choose to answer it from a set of suggestions without the need for explanations. What is the most appropriate next step to follow up from among the following? And a third category that can be answered from a group of suggestions with explanations, what is the most likely cause of the patient's nocturnal symptoms? Explain what you based your answer on.
Two examiners corrected and scored ChatGPT answers, and a third judged the differences between them.
The percentage of correct ChatGPT answers ranged between 52.4% and 75%, noting that the percentage generally required to pass the exam is 60%.
The study concluded that ChatGPT is close to the margin of success.
Lucía Ortiz de Zarate, a researcher at the Autonomous University of Madrid, considered that this study shows the potential of artificial intelligence in the medical field.
She believes that ChatGPT can be useful for clinicians when they are making the diagnosis and prescribing treatment.
Another study conducted last January showed that ChatGPT could pass the exams of a US law school even if it came last in the ranking of successful students.