Medical Education

Harlan Krumholz: Welcome to Health & Veritas. I’m Harlan Krumholz.

Howard Forman: And I’m Howie Forman. We are doctors and professors at Yale, and we’re trying to get closer to the truth about health and healthcare.

Harlan Krumholz: Howie, one thing I thought that might be good to talk about today is medical education and particularly the way that we choose doctors and how we educate them.

So, you and I both been here a long time. You’ve been on the admissions committee. We’ve been interacting with students and we’ve gone through medical school ourselves and through the whole process. I worry that the current process isn’t optimizing the selection of the people that the future of healthcare needs. A relative, someone I’m close to, is applying to medical school this year. And that brought me up close and personal with the kind of experience that people are having through the course of applying to medical school, thinking about where to choose, what gauntlet they have to run.

Howard Forman: Yeah, I agree. I spend a lot of my time advising undergraduates and people who are recently undergraduates who are applying to medical school, both because they were my students or they got sent to me to talk to about medical school. And it’s frustrating to me to realize that there’s almost a formula around what it takes to get into medical school, as opposed to a more holistic view of, what do we want our future doctors to look like? And I even have a sense in my head of what it’s going to take for you to be competitive in the 10 medical schools in the country versus the much larger set of medical schools that exist.

And even reminding people that going to medical school in the United States is not the only way to become a physician right now. Some of our best physicians have gone through other pathways.

Harlan Krumholz: Well, so there’s this issue about the gamesmanship of getting into medical school, but let’s think about it on our side too. So, if you’re setting up this system, you’re kind of projecting, what’s the world going to look like 15 to 50 years ahead of today? Because by the time these folks are through their training, maybe we’re 10 years out. They have four years of medical school. Some of them will have three to four years of residency. Some of them do a fellowship. I was amazed when I looked back, when I was ready to get my first job, I had just spent 12 years training and basically was at a very different point. The world was at a very different point than when I applied to when I got out. And so you’ve got to be thinking about, what’s the role of physicians? How will they be interacting with technology?

There’s a lot of stuff here. Hasn’t MCAT run its course? I mean, let’s just start there. I mean, what’s the real value? I mean, this becomes kind of a central gateway into most medical schools. Granted, there are a few that have abandoned the idea that the MCAT is necessary, but why is this thing so important? Why are people taking physics today? And why is it that if you are just the kind of person that doesn’t think in a way that gets you high grades in physics and a good score on the MCAT in physics, then you’re precluded from competing for the very best place. How does this make any sense?

Howard Forman: Look, I think to some degree doing well on a standardized test to get into medical school is a predictor that you’ll do well on a standardized test to graduate medical school. And as we know, there are three steps now involved in the United States medical licensing examination. One of them, the first one, has now become pass-fail, but the other two are not. And so if you are looking at it from the point of view of a leader of a medical school, one of the ways that you can hopefully avoid students failing out is by knowing that they do well on tests to begin with, but that’s not the same thing as saying they’ll be great physicians, or that we’re even balancing the skills.

Harlan Krumholz: Well, and it’s a self-fulfilling prophecy too, which is like, hey, we’re going to do standardized tests to see who gets in, and then we’re going to do standardized tests to see if you’re good enough to take care of people. And I can tell you, I sat on the American Board of Internal Medicine for maybe, I think it was like eight years. And I kept raising the question, how do we know that these tests, which are certifying and re-certifying internists and subspecialists in internal medicine, well, how do we know that it makes any difference in improving the quality of care? Or that the people who are able to pass these tests are any better able to provide high-quality care than those who don’t. And the answer was, we’re not sure. We’re not really sure that this is the best way to do it.

So we end up creating these systems that at least those tests are testing concepts that we believe are integral to being a doctor. But in this day and age, I think that the worst doctors are the ones that are depending entirely on their memory and the best doctors are the ones that have an extensive breadth of knowledge, but are able to access information in real time. So to me, most of these tests should be take-homes or access to other sources of information, because I don’t want to encourage people just to rely on their memory because I know how that can fail people. And so anyway, it starts to evolve into thinking about the whole way we evaluate people both before and after medical school.

Howard Forman: Look, within both graduate schools, as well as undergraduate programs right now, we’re in the midst of a great experiment where a lot of students are now applying without standardized test scores and we’re getting to witness what those outcomes look like. And at the moment I’m sort of hardened by this because I’ve just admitted one of my largest classes in the School of Public Health, to my healthcare management program. And right now I’m really comfortable with the fact that we’ve been working without standardized test scores in this admission cycle. And I’m hoping I’ll feel that way, obviously, in 18 months, but I’m sort of expecting that. And that may break our cycle of dependency on standardized test scores. So I mean that’s one big component.

The other, I think, is that maybe standardized tests could be useful, but maybe we’re testing for the wrong things, because we clearly are looking at people who come into medicine with, as you point out, almost an engineering background, or engineering inclination, when we emphasize things like organic chemistry and physics so much.

Harlan Krumholz: Well, let’s just say, we said physics isn’t essential. If we were going to substitute another course that would be essential, or at least a family of courses, what would you like to see every single person applying to medical school having had taken or at least in what domain of information would you like to see it?

Howard Forman: Yeah, I’m struck by how many students come to me and say, why aren’t we really taught about health policy in medical school, understanding the health systems that we work in. We say we do it. We give a little bit of time to it, but not on the level of detail that we do with the basic sciences or the clinical sciences for people to understand the systems that they work in. And that course could include things that cover social and behavioral sciences, that could also include topics like healthcare financing.

Harlan Krumholz: I wouldn’t want to specify a particular course, but I do think the family of courses, particularly around psychology and sociology, anthropology. I mean, understanding people and understanding how people interact and behave, and even in economics, particularly as you get to behavioral economics and understanding motivations and so forth. I mean, a lot of what we do as doctors is elicit preferences, understand goals, help people modify behaviors and achieve better health. And we need to, I think, instill a sense of respect for the kind of literature and work that’s been done in those fields and incorporate them into medicine in meaningful ways so that people can incorporate them into their professional lives. And right now they’re virtually absent. And so if we were going to do that, I think we could also leave open the fact that people should explain to us why they took the courses they took. If they are thinking about going to medical school, what would they think is an ideal set of courses for preparation as a compliment to their life experiences?

Howard Forman: And it’s obviously anecdote, but some of my absolutely best students in medical school—physicians now, all of them board certified—were people that majored in things like history, philosophy, English. They were less often—and no offense to this—taking these straight premed majors of biology, or biochemistry, or neuroscience.

Harlan Krumholz: Yeah. I mean, the message should be, be the best person you can be. Prepare yourself as a full-fledged individual and be prepared to learn what you need to learn to be a good doctor.

Well, one thing that I also think is an issue here is the fact that the schools are worried about their ranking. So then they’ve got these quantifiable metrics that end up being critical to their own stature. And then they’re working towards those. So that the MCATs are one—what’s our average MCAT score, especially if we’re not Harvard or Yale and we’re trying to establish ourselves as a top-tier medical school? The path to that is to say, we’ve got a highly competitive class. Similarly with grade-point averages. So one of the important things says…if people leave high school, they start college. Some people take twists and turns and yet, the schools want to brag about what the average grade-point average is of people coming in.

And that means that someone who’s a late bloomer, someone who finds themselves later in the course of their life and then starts to dedicate themselves to academia and to learning later is going to be markedly disadvantaged. And yet that person, because of life experience and so forth, is going to be better. Now, you might say, well, that could be picked up in the admission process, but because there’s such a large number of applicants, often there’s filters that make it so that it’s much less likely that someone’s going to be identified. And what we really want are compassionate, caring physicians who again, are able to tap the literature, and tap information, and help patients make the best choices and be skilled. And I just don’t see how this current process is optimized for that in the way that it’s currently configured and there’s lots of self-reinforcing features to it.

Howard Forman: I will say for Yale we have—and I hate to be making this about Yale, but it’s where I have the most experience, particularly through the admissions committee—we deemphasize things like MCAT scores and GPAs relative to other institutions. And it’s why you can go out there and figure are out which medical schools make the MCAT the be-all and the end-all, because they will have the highest mean MCAT scores in the end. We, I think, are a lot more holistic, but it doesn’t change the fact that when push comes to shove and you have comparable candidates, you going with the candidate that has the higher MCAT score, frequently.

Harlan Krumholz: Let me ask you this—and by the way, I have deep respect for the professionals in the admissions office and the work that they do—what percent of the Yale classes are from the Ivy league?

Howard Forman: I don’t know the actual percent, but absolutely the greatest predictor of getting into a great medical school is having already gotten into a great undergrad program.

Harlan Krumholz: So I’m just raising the point that these markers of, OK, you got into Yale. That means, by the way, that goes back, like this was someone who was, in high school, good enough to get into a competitive college. Again, it sort of precludes this evolution that people can go through. Not to mention, I believe that it sort of pushes down the value of some of the other institutions. So by disclosure, I went to Yale undergrad; I went to Harvard Medical School. I can tell you that the people in Harvard Medical School who came from schools that you hadn’t heard of much were like the most amazing people on earth. And I think it’s because in order for them to navigate the admission process, they had to walk on water whereas there were a whole gaggle of us who came from Yale. And all credit to my classmates and so forth, but I do think that we tended, probably, to get a closer look than someone who came from a small college in Montana. But I can tell you when you saw those folks, they are very, very impressive. And as you know, talent exists everywhere. And so it’s a question really of, how do we bring out the best in folks?

Just to pivot on this, if we want to get constructive about it, I mean if you and I were in charge of the world, like what would be the best ways to attract the best candidates? And who do we want to pull into medicine these days?

Howard Forman: Look, I think, one of the things that you face when you’re on an admissions committee is consideration of diversity. And most people, when they hear the word diversity, are thinking about racial diversity, gender diversity, but really socioeconomic status, where you came from, the level of adversity that you faced in climbing up to the position that you’re in right now, how well you’ve managed in complex social systems, in some cases, is probably a better predictor of how well you’re going to do in medical school and taking care of patients, frankly, in the end, that is how well you’re going to do on a physics exam.

And I can tell you, there were many times when I was doing interviews for the medical school, which I did for probably 10 years or more, where I would have somebody who on paper just seemed like they walked on water. And when you interviewed them, you just were convinced that they could never be a great physician or certainly weren’t going to fit in here. And that’s something that I think is why it’s good that we retain interviews in our process right now, but how we do the interviews then becomes another issue. And the Yale Medical School recently adopted…

Harlan Krumholz: But the science of it, the science of interviews, time and time again, says that it’s an extraordinarily flawed way to identify best candidates.

Howard Forman: And so about four years ago, Yale pivoted to doing more structured interviews, which is a partial solution to what you’re describing, but you’re absolutely right. Interviews are human interactions between flawed humans and they can be really problematic. And again, there are times where it was obvious to me that somebody should not be here, but there were also many times where two interviewers would interview the same person and come to vastly different conclusions.

Harlan Krumholz: By the way, when I was bringing this up about Yale, I want, since I am at Yale…we have extraordinary undergrads and I want to give a shout out to them and they are exceptional. And so I don’t mean that there should be a bias against either. It’s just a question how open our eyes are. But should we be going through a process where we’re giving people situations and seeing how they handle them? So it could be, for example, either a written form or that in the course of the interviews, someone meets an actor or something, and they go through a certain simulation where we sort of see, how do people manage challenging conversations? Granted there’s some artificiality to it but it’s trying to understand who people are.

And then there’s also this issue of really weighing their outlook and journey and what it is that they’re trying to get done. I don’t know. I mean, to me, this is a point of experimentation where we really should be thinking like, what’s the product we’re trying to produce? And if the product is someone who can pass standardized tests, then we’re probably optimizing for that. If it is about being able to relate to people and then in the future, it’s going to be also about how you leverage technology and how you’re able to bridge the new kinds of information that are going to be available with the human aspects of medicine that bring out the very key components of what are our profession is about, then that might be something else.

Howard Forman: So, I think part of what I was very impressed with when I transitioned off the admissions committee, Yale made this deliberate decision about moving to structured interviews. And it was a multi-month process of doing exactly what you described—how do we create simulations that are then comparable between candidates that different interviewers can discuss what they observed in a structured way—in an identical way, in some cases. And that’s a step in the right direction. I was very pleased with them. And I agree with you. We’re testing these things out real time. You can’t stop or randomized candidates to two different ways for five years and then look at outcomes 20 years later. It’s just not the way we do things, but we are moving in the right direction.

I think our goal is to attract people who will be compassionate caregivers, clinicians, leaders, people who are going to lead in different spheres of medicine. So that we’ve admitted people into our medical school who either explicitly or tacitly said that they were not going to practice medicine at all, but they had a clear focus. They wanted to write, they wanted to be communicators. And we’ve done well with those audiences.

Harlan Krumholz: Maybe there are more tracks. I mean, if somebody wants to be a lab scientist, that’s a different track.

Howard Forman: Yeah. Look, it’s 111 years since the Flexner Report, which was the largest wholescale relooking at medical education in America. And maybe it’s time that we have that, not just at a local level or an individual school level, but where we really look at it on a national level and rethink, how do we get the best physicians that’ll be the right physicians for this moment in history and the future.

Harlan Krumholz: Yeah. And the historic flaws of that report are really being illuminated now and the kind of harm that it did as well. I mean, it did some good, it did some harm and, but in any case, whatever came out 100 years ago, isn’t really relevant to how we’re training the doctors today.

Howard Forman: That’s right.

Harlan Krumholz: So, one of the things we were going to do, Howie, was talk about just hot takes as we kind of come to the end of this section on the week and what’s on your mind. So, what’s your hot take on this week?

Howard Forman: Yeah. So, look, last week when we first started working on the podcast, we talked about boosters and the issues around it. And this week, last Friday, the CDC director Rochelle Walensky sort of came up with her own recommendations that were slightly different than the recommendations of her advisory panel. And people were highly, highly critical of that. And I actually was not one of them. I looked at it and I thought the actual recommendations that were in print as opposed to what was conveyed in the newspapers was this group should, this group should, this group may, this group may get boosters. And I think that’s how medicine is actually practiced. That medicine is not as cookie cutter as people want to make it sound. And we should not expect our public health agencies to be so prescriptive about every single population that we just tell either you should, or you should not. I think there is room for may.

Harlan Krumholz: Yeah. I totally agree with you. I think it was a wise choice.

One of the things that I’ve been talking about, one of my hot takes this week, is I’ve been engaged in some tweets around the idea that there needs to be more accountability from investigators regarding the grant monies that they receive. So NIH hands out a lot of money, PCORI, other groups hand out a lot of money. We did studies a few years ago that showed that only about half of the clinical trials—these are experiments on human beings that are funded by the NIH—ever end up getting reported, ever end up getting reported out. That is either published or put on this registry site, clinicaltrials.gov. And the NIH didn’t believe us at first, but repeated the study and found the same thing.

Time and time again, there’s money that’s invested through the research enterprise, but not necessarily providing the returns. I think people ought to be accountable publicly for, where’d the money go, and what were the deliverables that were produced, and what’s the importance of it? And those kinds of reports should be publicly available and transparent. And I think that would propel us to be able to make better ascertainments of whether or not the research investments are really in society’s best interest and continue to iterate and improve on that. So I’m really interested in the transparency side of research. I’m researcher. I think I should be held at that same level of accountability.

The last thing here is that we said, we’d always end the podcast with either something that keeping us up at night or something that made us happy. I was just wondering what might fit into that category for you this week?

Howard Forman: I think, back to teaching, as I’ve mentioned before, and I’ve always loved teaching, but last year during the pandemic was not a fun year to teach, even though we made it work. And this year, teaching in person again has been great and fun for me, and I’m inspired by the students a lot. And ironically, the thing that I think is inspiring me the most is that I’m doing some offline asynchronous exchanges with the students after every class. And I feel good about the fact that we’ve learned something during the pandemic Zoom era that can be translated into our in-person teaching experience. And so I’m excited about the future. I think our youth will lead us and I may keep talking about that, but that’s what’s keeping me going.

Harlan Krumholz: Yeah. You’re one of my ideals for teaching. I mean, I don’t know—you’re inexhaustible and you seem always available to all your students. It’s a wonderful model for the rest of us.

I think the thing for me this week that made me happy in this time of confrontation, where no one can seem to get along, and polarization, that Yale and the unions reached a tentative contract agreement. As someone who actually as an undergraduate lived through a strike and then as a faculty member there was a strike and there was always strife between the administration and unions and somehow unable to bridge their different perspectives, often for long extended periods of time, which brought damage in relationships and financial threats to our colleagues who were in the union as they held out for better contracts…gosh, what great news that actually we can model behavior where that the groups can come together and it’s seeming that we’re not going to enter another phase where that’s going to happen again. And anyway, that just made me happy to see that could be bridged.

Howard Forman: I love that.

Harlan Krumholz: Yeah. That’s great. So Howie, another great week. I’m really appreciate the opportunity to be here. Folks who are listening, we hope that you’ll give us feedback or let us know what we can do better and keep the conversation going. If you want to see me on Twitter, I’m @HMKYale.

Howard Forman: And I’m @thehowie.

Harlan Krumholz: And Health & Veritas is produced by the Yale School of Management. Talk to you soon, Howie.

Howard Forman: Thanks very much, Harlan. Talk to you soon.