There is something more unsettling than the possibility that AI might outdiagnose a doctor: the possibility that adding the doctor back could make the answer worse.
That question sits at the center of the controversy surrounding Zeke Emanuel, Vinod Khosla, and their coauthors’ recent JAMA article, Will Autonomous AI Exceed AI-Aided Physicians as the Best Medical Care?. It challenges a comforting assumption in medicine: that whatever AI becomes capable of, a physician’s involvement will always make it better.
Adam Rodman is unusually well positioned to interrogate that assumption. A physician, researcher, and creator of Bedside Rounds, he combines a medical historian’s curiosity with the skepticism of someone actually testing these systems with patients. He has helped produce research that challenges our confidence in human diagnostic superiority. He is equally insistent about how much remains unproven.
Our conversation travels from ancient Greece to the modern emergency department, with a detour through self-driving cars. Throughout, we return to a question that feels increasingly urgent: What would it take to turn a machine that gives impressive answers into a system we can trust with someone’s care?
Three ideas stayed with me:
1. AI is challenging the story doctors tell ourselves about expertise.
We love the image of the brilliant diagnostician: part scientist, part Sherlock Holmes, assembling clues through extraordinary powers of deduction. Adam complicates that picture. He revisits Arthur Elstein’s classic research showing that novices could work through elaborate diagnostic hypotheses while experienced physicians considered fewer possibilities and arrived at better answers.
His interpretation: much of clinical expertise comes from accumulated knowledge and knowing when to activate it. That makes AI’s breadth of knowledge—and its ability to work through information that can overwhelm human attention—particularly consequential.
But Adam draws a crucial boundary around his own findings. His research includes diagnostic comparisons using real emergency department data, extending beyond textbook puzzles. Those results still do not establish that handing care over to an AI improves patient outcomes. Instead, it tells us that diagnostic accuracy is a milestone and reminds us that the destination is better care.
2. “A human in the loop” is a hypothesis we need to test.
Keeping a doctor involved sounds reassuring. The reassurance depends on what that doctor can actually detect and correct.
Adam describes an unsettling analysis from his work with Google’s conversational AI, AMIE: its explanations grew more confident as encounters progressed, including when its conclusions were wrong. Persuasiveness offered little help in distinguishing success from failure.
The human brings vulnerabilities, too. We can dismiss a correct recommendation, anchor on an initial diagnosis, or coax an agreeable model into validating our mistake. Effective collaboration requires understanding how both parties fail—and designing their interaction accordingly.
That is why the self-driving-car analogy becomes interesting. Adam sees a path toward greater autonomy through carefully evaluated tasks, explicit limits, and evidence about when human intervention helps. He also expects that path to be considerably longer than many enthusiasts suggest. Oversight has to demonstrate its value, just as autonomy does.
3. AI’s greatest contribution may be care we currently cannot provide.
One of Adam’s most compelling examples is also one of the least theatrical: an AI that checks in after a medication change, follows symptoms between appointments, and alerts the clinical team when something needs attention.
That is care many doctors would love to offer. Our current systems rarely give us the time or resources to deliver it consistently.
There is a related limitation in today’s answer engines: we usually consult them when we know we need help. Many consequential mistakes happen when we never realize there is a question to ask. Adam is interested in systems that could catch those omissions and extend our reach.
For me, this is where the conversation becomes most hopeful: earlier access, closer follow-up, and more chances to notice that a patient is heading in the wrong direction.
Also in this episode:
- Which skills should doctors preserve? The deskilling debate gets more useful when we ask which abilities patients will still need us to have—and how we will maintain them.
- Does writing the note help us think? Adam and I debate what medical documentation teaches us, and what might disappear when AI does the writing.
- Medicine has argued about this before. From nineteenth-century Paris to early computerized diagnosis, new ways of knowing have repeatedly challenged physicians’ authority.
- Pineapple on pizza, defended through an unexpectedly elaborate account of cultural exchange.
I came away thinking that medicine will need two kinds of humility: the humility to accept that machines may surpass us at work we consider central to our identity, and the humility to demand evidence before deciding what should follow.
If you are fascinated by human cognition, the history of medicine, and the pragmatic reality of building an “AI doctor,” you do not want to miss this conversation.
If you’re a US-based physician, continue the conversation with Dr. Rodman on Roon!
New Good Medicine episodes are released monthly.
If you’re a US-based physician, join the community on www.roon.com
Find us on Instagram and X: @roondoctors
If you have a question, comment, or suggestion for a future guest, please email us: jane@roon.care
