Let's Play Doctor: New Exam Measures Students' Clinical Skills



Texas Medicine Logo

Medical Education Feature -- November 2004

By  Ken Ortolon
Senior Editor

Thousands of U.S. medical students are ready to begin their careers in medicine. But before they do, they must prove they have the clinical skills to be competent physicians. They must prove they know how to communicate with patients. If they fail, they don't get a license.

In June, the National Board of Medical Examiners (NBME) began administering a clinical skills examination as part of Step 2 of the United States Medical Licensing Examination (USMLE). The test was designed by NBME, the Federation of State Medical Boards (FSMB), and the Educational Commission for Foreign Medical Graduates. Medical students fought it, saying it was too costly, and the American Medical Association wanted it delayed or canceled until concerns about the test's statistical validity and other issues could be addressed.

But NBME and FSMB officials say students' initial reaction to the exam has been favorable. Some educators say it focuses needed attention on a facet of medical practice often neglected by too many medical schools.

"Before now, licensure exams in this country have only assessed medical knowledge," said Steven Lieberman, MD, associate dean for educational affairs at The University of Texas Medical Branch at Galveston. "If we want our doctors to be good listeners and effective communicators, we sure ought to make sure they can do that before they've got a license to practice medicine," he said.

Playing the Part

The clinical skills exam was launched in late June at testing centers in Philadelphia, Atlanta, Chicago, and Los Angeles. A fifth center opened in Houston in September.

The test requires students to examine 10 to 12 "standardized patients," actors trained to present the same way to each student. During each 15-minute encounter, the student must establish a rapport with the patient, conduct a history and physical, answer questions, and provide counseling where appropriate. The student then has 10 minutes to make notes of the findings, develop diagnostic impressions, and present treatment plans or plans for further evaluation.

About 4,000 students had taken the exam by late September. Peter Scoles, MD, NBME senior vice president, says the board expects to test 15,000 by the end of 2004. Another 3,000 are registered through June.

One Texas medical student took a practice version this summer. Although impressed with how smoothly it was conducted, she questions whether it is a valid measurement of clinical skills.

"To a certain extent it can identify whether you can talk to a patient, whether you can elicit the right information and have an idea what you're looking for," said Megan Kielt Kressin, a fourth-year student at The University of Texas Health Science Center at San Antonio. "But is it anything beyond what we get in medical school already? I don't think so."

Ms. Kressin says the exam can determine whether someone can speak English, perform a physical examination, and write coherent notes, but it is not as strong in determining one's true ability to interview and communicate with patients. She says that is done better by medical schools, where faculty members observe students' interaction skills daily over a two-year period.

"If what they're really getting at is whether people can make their patients feel at ease, I think anyone can fake that for one test day," she said. "Unfortunately, I think the real aim of the test seems to be to avoid lawsuits in the future by weeding out people who don't interact well with patients. That, unfortunately, is probably the weakest element of the test."

Austin surgeon Thomas Kirksey, MD, immediate FSMB past chair, disagrees that students can fake their way through the exam. "We're very cognizant of one's ability to scam the examination, in other words, adopt a demeanor. The testing and the observations are so sophisticated that we can pick that up."

Pass or Fail

The test will be scored on a pass-fail basis, which Dr. Kirksey says prevents scores from being used in matching medical graduates to residency programs. The passing grade has not been set, even though thousands have taken the test.

Dr. Scoles says NBME did that deliberately so that physician experts who set the standard will have real test performance data upon which they can judge the potential impact of their decision. "You can't really do that with data from pilot tests," he said. "If you're taking an examination that doesn't count, your performance will never be the same as when you're taking an examination and you know it counts."  

The passing standard was scheduled to be set in October. The first scores will be sent out in mid-November, and NBME expects an initial failure rate of about 6 percent among U.S. medical students. That will drop to less than 2 percent after students who failed on the first try get some remediation and retake the exam, Dr. Scoles says.

While the projected failure rate is low, Dr. Lieberman says he believes it is worth the time and expense.

"I see this as a very positive thing. I know there has been resistance politically and I do understand the financial burden it places on students," he said. The exam costs almost $1,000. "We owe it to society and to our students' future patients to assure that [the students] are getting these skills, which are essential to them being successful physicians."

UTMB has been using standardized patients to teach clinical skills since the 1970s, and it uses an integrated curriculum evaluation exercise to determine if students are competent enough to graduate. UTMB also will require passage of the clinical skills component of the USMLE before graduation, Dr. Lieberman says.

However, not all medical schools place that kind of emphasis on clinical skills. Dr. Scoles says the number of schools spending time on clinical education has doubled in the past four years, largely due to the scheduled implementation of the clinical skills exam.

"The vast majority of medical schools now are clearly paying attention to the clinical education of physicians, something they weren't doing four or five years ago," he said. "That's got to have a positive effect."

Ken Ortolon can be reached by telephone at (800) 880-1300, ext. 1392, or (512) 370-1392; by fax at (512) 370-1629; or by email at Ken Ortolon .

SIDEBAR

Clinical Skills Testing Center Opens in Houston

When people ask Joe Guillory what he does for a living, he simply tells them he's an actor.

"People don't understand what a standardized patient is," he said.

But acting perfectly describes what Mr. Guillory and about 50 other employees of the Clinical Skills Evaluation Center in Houston do. They act sick so that officials who run the United States Medical Licensing Examination (USMLE) can evaluate whether medical students have the clinical skills necessary to be competent physicians.

The Houston center opened Sept. 27 as the last of five national testing centers administering the new clinical skills examination, which all U.S. and foreign medical graduates seeking licensure in the United States must pass as part of Step 2 of the USMLE. The center is operated by the Educational Commission for Foreign Medical Graduates.

Center manager John Repasch says the center can accommodate up to 22 exam takers at a time. When the facility is fully operational, exams will be conducted six days per week. Mr. Repasch expects as many as 1,500 medical students and foreign medical graduates to be tested there by the end of the year.

The center has two dozen examination rooms that look pretty much like those found in actual physician offices. The rooms are equipped with examination tables, sinks, blood pressure cuffs, and other equipment needed to conduct a basic physical exam. The rooms are wired for audio and video recording of patient encounters, and test administrators can visually monitor sessions through windows in each exam room.

Outside of each room is a desk with a computer. Each test taker waits at one of those desks until instructed to enter the exam room. He or she then has 15 minutes to interview the standardized patient inside and conduct a physical exam. Then, he or she returns to the desk outside and, with 10 minutes available, records notes on the encounter before moving on to the next exam room and next standardized patient.

Standardized patient Joyce Frank says she was attracted to the job because of a lifelong interest in acting. "I look at this as being something totally different than anything I've done before," Ms. Frank says. Her previous job was in legal records management.

In fact, Mr. Repasch says few standardized patients employed at the center have any acting background. Mr. Guillory previously worked in sales. Another standardized patient, William M. Barnes, is a retired petroleum geologist.

All went through extensive training to be able to play roles of patients with specific conditions. Mr. Guillory plays a patient with hip pain and Ms. Frank simulates a patient with severe right-side abdominal pain. Mr. Barnes, meanwhile, plays an elderly patient with memory loss. Some of the standardized patients will be trained to simulate more than one condition.

While the center is just getting off the ground, all three say they are enjoying their new careers and feel they are contributing something important to society.

"I feel like we may be performing a service," Mr. Barnes said. Mr. Guillory added that the careers of the medical students being tested "depend on our professionalism during the encounters."

Ms. Frank says it's sometimes hard to stay in character when a nervous test taker presses the wrong side of a stethoscope to her chest or forgets to put the earpieces in his ears.

"You want to show empathy for them but you must stay in character," she said.

November 2004 Texas Medicine Contents
Texas Medicine Main Page