Dear Editor ChatGPT, developed by OpenAI, is a language generation model with the ability to generate human-like text. It had a significant impact on various industries, including media and science. Its strengths and weakness are subject to an important debate including copyright, practical and ethical concerns (Rudolph et al. Citation2023). Several media articles state that ChatGPT passes exams with open questions or paper assignments without revealing exact results or research methodology. A recent paper proved it can pass the United States Medical Licence Examination (Kung et al. Citation2023). We wanted to find out whether ChatGPT is able to pass a medical multiple-choice exam as well. We evaluated ChatGPT’s performance on the multiple-choice exam for the course of Family Medicine in the 3rd Bachelor year of the medical curriculum at Antwerp University. The exam consisted of 47 questions (Dutch language) with four possible answers and with a pass mark of 62.5. Each question was copied from the exam file to a new ChatGPT window. Questions were preceded by a prompt. Two prompts were tested: the Natural Prompt (‘Give one single answer’) and the Rank Prompt (‘Rank the possible answers’; the first ranked answer was used). Details upon the methodology and the results are available upon request.
展开▼
