Pilot Examination of Graduates' Competencies
October 16 - 19, 2003
Central South University, Xiang-ya School of Medicine
China Medical University
Fudan University Shanghai Medical College
Peking Union Medical College
Peking University Health Science Center
Sichuan University (West China) School of Medicine
Xi'an Jiaotong University School of Medicine
Zhongshan (Sun Yat-sen) University Medical School
 |
In October 2003, on the invitation from the Chinese Ministries of Education and Health, the IIME evaluated eight leading medical schools in China. Eight schools in China simultaneously administered the MCQ and OSCE examinations. The "MCQ" exam took place on October 16 from 8:30 AM to 12:00 Noon and included 150 items from the National Board of Medical Examiners (NBME) database related to the "GMER" competencies. While all schools had used the MCQ format in the past, these questions were usually of the recall type. The MCQs used in the exam were more complex, case-based questions and required considerably more integrative ability to answer correctly.
The OSCE examination began on the 17th and was performed in 15 stations, including five long SP cases, five short SP cases, and five Non-SP stations. Students were evaluated by at least one faculty member using a predefined checklist and sliding scale for performance. As only 30 students were evaluated daily, the examination took two days at Fudan and Peking Universities and three days at PUMC. As only two schools had OSCE experience prior to this examination, so workshops were held over a 6-month period, and faculty was trained in the organization, recruitment, training, and evaluation procedures for OSCEs.
 |
The faculty observation part of the exam occurred over a three-month period extending from August until October 2003, with a minimum of three faculty ratings per student (one per month). While faculty observations of student performance are routine in China, these are usually the result of the oral examination-single observation type and do not involve longitudinal observation as required for the GMER exam. Faculties from each site were trained to teach other faculty at their home institutions on how to use these forms.
The exam data were collected by CMB/IIME Observers and brought to the IIME for analysis. Approximately 500,000 data points were collected on a total of 380 students at eight schools that entered into a master database for analysis and score reporting. The reliability and validity of the collected data were established through statistical analysis, independent comparative analysis and international expert review.
All the CMB/IIME Observers during the debriefing meeting reviewing the exam organization in all Universities stressed that all students were very serious and determined in their performance and that the examination procedures were followed carefully and no breaks in security during examination were noticed. It is worthwhile to stress that substantial faculty resources were directed towards this examination, and it was estimated that in all, about 1,000 faculty members at the eight participating Universities were trained and involved.
Institute for International Medical Education.
Unauthorized reproduction strictly prohibited.