Michael Abramoff trained as a neuroscientist in Japan, working on neural network simulations of the brain.
“ultimately became a neuroscientist in Japan working on neural network simulations of the brain.”

26 Apr 2023 · 1h 5m · Michael Abramoff, Andrew Beam, Arjun Manrai · 24 quoted facts
Dr. Michael Abramoff argues that medical AI succeeds only when it is built around measurable patient benefit and ethics, and when the whole health system is brought along. His company's IDx-DR, which diagnoses diabetic eye disease without a doctor, became the first FDA-authorized autonomous AI. He says the hard work was regulation, payment and trust, not the algorithm.
The tension is between a clear scientific result and a resistant system. Publishing good research did not change care, so Abramoff went to the FDA, built an ethics framework, designed a payment model and raised investor money. He was mocked as the Retinator, yet he argues that starting from patient benefit and health equity made the path easier and the field more accepting.
Abramoff's algorithms matched retina specialists and were published, but nobody came knocking. He had to go to the FDA, raise angel and venture money, and spend years on approval and payment.
Abramoff's framework, published in the American Journal of Bioethics, lets engineers measure principles such as patient autonomy, equity and benefit. Because they can be measured, they can be optimized and checked with regulators.
Instead of pricing near the cost savings, Abramoff took what payers already spend on the 20 to 30 percent of patients now screened. He spread that money across all patients who need exams, which gave the 55 dollar equity-enhancing payment.
Abramoff splits explainability into validability, explainability and transparency, and notes that doctors cannot explain some diagnoses either. Unit-level testing of biomarker detectors and strong validation can build trust even when a model is not fully explainable.
Abramoff says an Obermeyer paper showed AI can increase bias, and that regulators now look hard at it. He also says randomized trials of his autonomous AI nearly eliminated disparities in who got eye exams.
IDx-DR is a device from Digital Diagnostics that detects more than mild diabetic retinopathy, an eye disease caused by diabetes, from eye images. In 2018 the FDA authorized it as the first autonomous AI that makes a diagnosis without a physician.
Abramoff says autonomous AI makes the medical decision by itself, while assistive AI guides a doctor who still carries the responsibility. He argues only autonomous AI really improves productivity and scale.
Abramoff says Medicare's CMS can only reimburse what a physician would charge, so he worked out a price of 55 dollars. It was based on spending the system already does on the 20 to 30 percent of patients now getting exams, spread across everyone who needs one.
In 2010 the chair of ophthalmology at Johns Hopkins wrote a front-page editorial in Ophthalmology Times calling his research the revenge of the machines. It was partly a joke and partly a warning about lost jobs and lower quality of care.
Abramoff says not always, because doctors often cannot explain a diagnosis such as melanoma either. He prefers strong validation and testing of the parts, and building in known biomarkers where possible, and Andy Beam largely agreed.
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Michael Abramoff trained as a neuroscientist in Japan, working on neural network simulations of the brain.
“ultimately became a neuroscientist in Japan working on neural network simulations of the brain.”
Michael Abramoff came to Iowa in 2003 and won interest from the National Eye Institute at the NIH for his machine learning research.
“came to Iowa in 2003, where I was able to get NIH, National Eye Institute, excited about the type of research I was doing”
Michael Abramoff says electronic health records slowed him down and lowered his productivity.
“electronic health records have actually slowed me down. They have lowered productivity.”
Michael Abramoff cites Bureau of Labor Statistics data as showing declining productivity for physicians in outpatient clinics.
“When you start looking then at the data, for example, the Bureau of Labor Statistics at the Department of Labor, you can actually see that productivity has been declining for physicians, outpatient clinics.”
In 2018 the FDA authorized Digital Diagnostics' IDx-DR, described as the first FDA authorization of an autonomous AI that works without physician input.
“Back in 2018, the FDA issued authorization for your company's device, which I understand was the first FDA authorization for an autonomous AI without physician input.”
When Michael Abramoff began this work in 1996, digital cameras were almost absent from medicine and were just starting in radiology.
“We're talking 1996 now. And so this was still the time that there were no digital cameras almost anywhere in medicine. It was just starting in radiology.”
Michael Abramoff says highly experienced retina specialists disagree with each other in about 20 to 30 percent of cases.
“when I look at myself and other of my colleagues who are highly experienced retina specialists, we differ in about 20, 30 % of cases.”
Michael Abramoff went to an FDA meeting in 2010 and told the agency he wanted a computer to make a diagnosis.
“I ultimately ended up going to FDA in 2010 at a meeting, said, I want a computer to make a diagnosis.”
Michael Abramoff says he founded his company because he needed angel investors to fund the long path to FDA authorization, not because he set out to start a company.
“i needed angel investors and that ultimately led to finding the company so it was not like i set out to find a company”
In 2010 an Ophthalmology Times front-page editorial by the chair of ophthalmology at Johns Hopkins, titled the Retinator, called out Michael Abramoff's AI research by name.
“there was an editorial in the most widely read ophthalmology journal in the world called Ophthalmology Times on the front page by the chair of ophthalmology at Johns Hopkins”
Michael Abramoff and bioethicists created an ethical framework for AI that was published in the American Journal of Bioethics and used in interactions with the FDA.
“I had to learn it to create an ethical framework for AI that was published in the American Journal of Bioethics.”
Michael Abramoff says randomized clinical trials showed his autonomous AI raised the share of patients getting needed exams and almost eliminated health disparities at clinics.
“we showed in randomized clinical trials that this autonomous AI improves, that it not only improves health equity, it literally allowed clinics to go from very low percentage of people getting the exams, they need to almost eliminating health disparities.”
Michael Abramoff says the Federal Register contains about 30 pages in which CMS discussed its concerns and guardrails about paying for autonomous AI.
“There's literally 30 pages in the Federal Register from CMS discussing their worries and concerns, what they called guardrails”
Michael Abramoff says John Deere showed its first autonomous tractor at CES, after healthcare had already used autonomous AI on patients.
“John Deere had the first autonomous tractor at CES last year.”
Michael Abramoff says he appeared before the Senate Finance Committee soon after FDA authorization to discuss how to pay for autonomous AI.
“I went in front of the Senate Finance Committee and explained early on, just after FDA”
Michael Abramoff says the relevant part of CMS can only reimburse what a physician charges because of how the Social Security Act was written.
“the part of CMS that we're talking about here is only allowed to reimburse what a physician charge. That's the way the Social Security Act was written.”
Michael Abramoff says diabetic eye exams are worth paying for even at several hundred dollars because they avoid blindness, vision loss and costly surgeries.
“it's still worth the cost savings on the back end because of avoided blindness, visual loss, very expensive surgeries.”
Michael Abramoff says society already pays for about 20 to 30 percent of patients to get diabetic eye exams, and he based his 55 dollar price on spreading that spending across all patients.
“there's a willingness to pay currently for about 20 % to 30 % of patients who are getting these exams.”
Michael Abramoff says research on bias in healthcare algorithms has led the Office for Civil Rights at HHS to look into AI bias.
“which now has led to the Office of Civil Rights in HSS looking into this”
Michael Abramoff says his company showed there was no racial bias in its AI, which the FDA examined.
“in our case, we proved that there was no racial bias in AI.”
Michael Abramoff says his paper with the FDA divides explainability into validability, explainability and transparency.
“three very different types of explainability we call them validability explainability and transparency in the paper”
Michael Abramoff says only about 3,000 melanomas of the eye occur in the world each year, which limits training data for AI on rare diseases.
“there's only 3,000 melanomas in the world in the eye per year”
Michael Abramoff says ophthalmology leads in gene therapy and automated surgery, with corneal surgery almost fully automatic.
“especially in ophthalmology, which is always leading like in gene therapy, but also in automated surgery, corneal surgery, that's almost fully automatic”
Michael Abramoff says that when he started medical school he had an Apple II computer that he built himself.
“When I started medical school, I was probably the only one in five city blocks with his own Apple II, which I built myself.”
Abramoff traces his path through computer engineering, neuroscience in Japan, software, residency and a machine learning PhD. It explains why he went all in on autonomous AI.
“I started training as both a medical student as well as a computer engineer”
As a resident in 1996, Abramoff saw patients who did not need care while others arrived too late. That led him to ask whether a machine could find the patients who needed care.
“Is there not a machine that can mimic my brain”
Research papers did not change practice, so Abramoff sought the FDA's judgment and founded a company funded by angel investors. The path was long and unlike the pharma model.
“rather than me saying it's safe, I want the highest authority in the world”
The mocking editorial taught him that a new technology needs support from patients, doctors, payers and regulators, not just good science.
“we need stakeholder support we need everyone in the healthcare system to be comfortable”
Abramoff explains how ethical principles can be quantified and balanced, which became central to talks with the FDA.
“And if you can measure it, we can optimize towards it.”
The discussion covers how to price an AI diagnosis, the CMS rules, and the 55 dollar equity-enhancing payment. This is the hardest part of bringing an AI product into practice.
“payment and so that ended up being 55 so it had to be 55”
A lighter segment reveals Abramoff's interests, including space exploration and music, and short views on documentation and AI art.
“I'm really excited about the potential for space exploration and especially asteroid mining.”
Abramoff and Andy Beam compare ways to earn trust in medical AI and find they largely agree on decomposing tasks into validated biomarkers.
“some specialists do not know how they come to a diagnosis either”
Abramoff says machine learning can both worsen and eliminate disparities, depending on how it is used and validated.
“we have the proof in hand that it can improve them”
Looking ahead, Abramoff hopes the pathway can be reused for many diseases quickly enough to reach underserved people.
“Can we apply it to many other diseases?”
Another thing which also happened in 2010, which was that I was doing all this research and my colleagues started to notice, my colleague ophthalmologists, very esteemed colleagues, and suddenly there was an editorial in the most widely read ophthalmology journal in the world called Ophthalmology Times on the front page by the chair of ophthalmology at Johns Hopkins, the retinator, revenge of the machines, about my research, calling me out by name, essentially saying that I'm the terminator of retina.
I'm the retina specialist, right? That hurt. And it's going to cost jobs and it's going to lower the quality of care. It was partially tongue-in-cheek and partially very serious.
That was Dr. Michael Abramoff for calling some of the challenges that he's faced in pioneering AI for ophthalmology. where at one point he was even called the Retinator, the Terminator of the Eye. So welcome to another episode of AI Grand Rounds. I'm Raj Manrai, and I'm here with my co-host Andy Beam. So Andy, I was really struck by Michael's amazing amount of persistence and passion. As we heard in that clip, he's had challenges, but he just keeps moving forward.
He's been a real pioneer in medical AI for a few decades. He's a professor at the University of Iowa and also the founder and executive chairman of Digital Diagnostics, an autonomous medical AI company. We got to explore both the genesis and the implications of his groundbreaking work, which led to the first FDA-authorized autonomous device. So no doctor required. This is the IDx-DR device, and it detects more than mild diabetic retinopathy from digital images of the eye.
I also have to say that at moments, the entropy, or if you prefer, the temperature setting was a little higher than I expected. And to our listeners, if you listen long enough, you'll hear about asteroid mining, Lana Del Rey, and ethical AI, which is a sentence that I never thought I would say. Overall, this was an illuminating and really fun conversation.
I have to agree, Raj. I did not see asteroid mining coming either. And I think you hit the nail on the head when you said that you were impressed by his persistence. In addition to his deep technical innovation in this space, I think what struck me the most was his willingness to go deep on the non-technical side of things to make sure that he was able to improve patient outcomes. And specifically, his work on defining new medical reimbursement models for AI, I found to be very fascinating, and his willingness to take seriously the bioethical considerations for medical AI.
So again, I totally agree. It was a fascinating conversation. I certainly learned a lot from it. And with that, we're very excited to bring you Dr. Michael Abramoff on NEJM AI Grand Rounds. The NEJM AI Grand Rounds podcast is sponsored by Microsoft and Viz.AI. We thank them for their support. Well, welcome to AI Grand Rounds, Dr. Michael Abramoff. We're excited to have you on today.
Thanks so much for having me. Very excited.
So, Michael, this is a question we like to always start with. Could you please tell us about the training procedure for your own neural network? How did you get interested in AI? What data and experiences led you to where you are today?
Yeah, absolutely. And, you know, my neural network is protected by a set of white hair. So you listeners cannot see me, but I'm the only one here with white hair. That shows my age and also therefore how long I've been involved in this. I started training as both a medical student as well as a computer engineer a long time ago in the 80s. ultimately became a neuroscientist in Japan working on neural network simulations of the brain.
So what we were trying to do at the time was mimic single neurons in computer code and then try to see how that related to cerebellar neurons where we had brain slices and try to mimic them and ultimately try to build larger neural networks, maybe three neurons, right? Compare that to the millions of neurons we use now. And there was this larger group there we were lurking on various ways of, well, how does a larger neural network process information, you know, about information theory of the brain, so to say.
So I was very excited about that. Ultimately decided to go into the software industry for work for a long time in software, went back to medicine, finished my residency in ophthalmology. I've always been interested in the intersection, like you say, of the audience here of AI, or as we used to call it machine learning then or neural networks and medicine and how you can apply it right not only as a theory of the brain but also can it actually be useful for what we do in clinic and so i kept back and forth and many people told me wow this combination of computer engineering and machine learning and medicine is so useful so exciting and there was literally nothing no
one was interested there was no funding nothing so i kept moving between doing my residency and ultimately ending up doing a PhD in machine learning. Finished that, did a fellowship in vitro retinal surgery, really focused more on the surgical aspects for a few years, got an opportunity. This was in the Netherlands, as you hear from my accent in Amsterdam, came to Iowa in 2003, where I was able to get NIH, National Eye Institute, excited about the type of research I was doing, where I said, well, can we use machine learning to mimic what the brain of a clinician like me does?
and so that went was a long path ultimately with me starting to be interested in either why why why do I think it's interesting and I started to see promise in healthcare like the high cost low accessibility lack of health equity access and in my view the solution in other fields like agriculture I'm here in Iowa and there's you know John Deere tractors all around me right there's cornfields and silos. And so I said, well, clearly automation, fertilizer, all these things, mechanization have helped tremendously in making food maybe 50 % or 80 % of people's expense 100 years ago or longer to now.
The problem right now for some people may be lack of food, but most people it's probably an abundance of food and how to deal with that. So that's really changed to automation. And in healthcare, I think we have tried it with things like electronic health records, within my own experience and I've done some research and collaborated with others, electronic health records have actually slowed me down. They have lowered productivity. When you start looking then at the data, for example, the Bureau of Labor Statistics at the Department of Labor, you can actually see that productivity has been declining for physicians, outpatient clinics.
And so how can we tackle that? And that's when I started talking about autonomous AI, meaning the medical decision, whatever it is, is made by the computer solely without assistance by a physician. Until then, most AI was more assistive, meaning you have a physician, another provider, a nurse in the clinic with the patient, and they are guided. And I use it daily in my clinic, but this is not the same because ultimately the responsibility for the medical decision is still typically with the physician, even from a medical liability perspective.
Also, you don't really move the needle on scalability and automation. You still need that physician there with that patient. You don't improve productivity. So I went all in for autonomous AI. And I think now I've talked long enough. But yeah, that's the background of my neural network.
So that was great. There's a couple of nuggets there that I want to put on the shelf and come back to, especially around the problem with abundance, the problems that the electronic health record present. But I'd like to just focus a little bit more on your training, because I think a question that we get a lot is from the clinician side, how do I get involved in AI? And then from the computer science side, how do I get involved in medicine?
And I just wonder if, given that you have an MD-PhD, your PhD is in a technical field, and obviously you're a practicing ophthalmologist, is the MD-PhD the gateway to this intersection? Or given the training that you've been through, are there sort of side routes into that?
There's a few answers. I'm afraid to say it, but learn to code is one thing which really makes a difference still, even if it's just Python. If you can build your own neural networks, there's just a lot of ground still to be covered.
I would just like to point out the sort of subtle, deep computer science jab there of saying, if you learn to code, even if it's just Python, for the clinical listeners, that's a nice little jab there. I like it.
Okay, so I didn't want to insult anyone. It's more, I mean, yeah, sure, some want to build really sophisticated recursive neural networks in C++, and that is fine, But I think we need more people who can just set up an existing inception or whatever it is and start training it. Because I think what is exciting about especially machine learning is that once you have the engine, the tool, it's more about the training data and how you use it.
And that's where the interesting discussion is going to be, I hope. but if you if you want to understand i i actually wrote or i was asked to edit the educational material on ai for the ophthalmology residents around the world which is called the basic clinical science course of the american academy of ophthalmology which is widely used even when i was in the netherlands we used that for our training and so they asked me to help you know what what does a practicing specialist or physician need to know about ai and it's mostly about both the potential, the risks, and what it's going to mean for your practice.
So that's important. If you then want to build it yourself, understanding is good, and trying it out yourself is good. But to really become involved, you need to be able to at least code, understand statistics, understand pre-rested clinical trials, understand back and forth about what type of reference standard to use. There's a lot of sophistication there that maybe is not for every physician. you really need to spend a lot of time there. On the engineering side, I had a lot of graduate PhD students in computer engineering.
I've now sort of focused more on digital diagnostics, but there, for me, it was really important that they were able to interact with clinicians, that they spoke the same language, that they understood what clinicians are dealing with and how getting your hands dirty and the noisiness of much of the data that clinicians deal with all the time, as well as, of course, the privacy risks, just the emotional aspects of medicine is really important, I think, for aspiring AI engineering students to understand.
And then ultimately, when it comes together, there's the more general risks and potential benefits of AI that we all need to know about.
Yeah, I think that makes a lot of sense to me and resonates with my own experience is that when I'm talking to clinicians, I try and tell them that a lot of the base models we use are essentially commodities now. And so if you're trying to think about how to use them clinically, you need to understand their operating characteristics, their inputs and their outputs. And that's probably sufficient. If you want to do research, then as you said, that's like another thing and you probably need to code and understand statistics.
I think that there's a wide range of opportunities for clinicians who don't even know Python but are still interested in this interface.
Yeah, and ideally you have a collaboration ongoing with maybe a grad student in engineering or maybe a group there that is of course, In some institutions, that's well done. In others, not so much. But yeah, even for when you want to commercialize it, ultimately right now, where digital health companies are, AI algorithms are typically seen as a commodity.
Great, thanks.
I don't know why it's free because this biomark approach that I happen to have a patent on is... Sorry, there's a conflict of interest here, but I do think there are certain advantages, and I think some regulators also see these advantages.
So I think that's a perfect segue, right, Raj?
Yeah, that was great. So Michael, we want to transition to your research and also to your efforts to move beyond academic publication to deploy medical AI. So you've been a trailblazer for several decades, both in academia and an industry where you're founder and chairman of digital diagnostics. Back in 2018, the FDA issued authorization for your company's device, which I understand was the first FDA authorization for an autonomous AI without physician input. And this was for the IDx-DR device for autonomously diagnosing patients with more than mild diabetic retinopathy.
So this is 2018, but I know this journey started several decades before that. Could you take us back all the way to the beginning of the technology journey? How did you choose this particular problem? And what were some of the key early challenges that you had to overcome?
I already sketched a little bit, you know, where my origins were. I really saw myself as a neuroscientist with an interest in applying what we learned. And we still don't know much about how the brain works, but at least we know a little bit and start to be useful. it was really when I was in clinic as a resident that I thought I have all these patients coming to me and I look at them and they spent half a day in my clinic and there's nothing wrong with them and meanwhile there's all these patients coming too late because they don't have access or there was another problem where they didn't feel like going and in many cases it's too late
so clearly we're not finding patients who need our care and we have too many patients that don't need our care Can we or can I not do something about it? Is there not a machine that can mimic my brain and it can find these patients wherever they are? We're talking 1996 now. And so this was still the time that there were no digital cameras almost anywhere in medicine. It was just starting in radiology. And so it was not like there was a bunch of training that you could just build an algorithm on.
but I said well let's try to see whether this can work and when I prove it works surely people will say wow this is brilliant and you know we need to do this okay so that was the naive thought I had and which I will be going back and forth between the commercialization aspect and the research aspect I cannot help that because I think applying research to actually benefit patients is in many cases what physician scientists are about and hopefully engineering scientists as well And so part of that was selling up literally a telemedicine network to have the images so we could scan them, so we could train the algorithms and see whether we could have a performance that was
acceptable. And at that time, there was no definition of, well, this is the threshold you need to meet, which has been a challenge for a long time. And we will get back to that. But it started to work, came to the US, got actual funding from NIH that these algorithms can actually perform equally good as me as then a retina specialist. So clearly there was promise there. So it said, well, what should the performance actually be?
Because I knew that when I look at myself and other of my colleagues who are highly experienced retina specialists, we differ in about 20, 30 % of cases. So the fact that I agree or disagree with an AI doesn't necessarily mean that the AI is wrong or right. I mean, there's a noise there. Can we do better? So that was, We'll set it aside, but that was an interesting question. But also, the research started to be fruitful.
There were publications, algorithms, successful publication track record, but that didn't move the needle. It was not people who were knocking on the door saying, well, yeah, we need to do this, CMS. I was very naive. And maybe some listeners are also thinking, how can this be done? And it's a little bit more complicated.
It's not just publish and solve all problems with a single PDF.
And also I've been saying in other parts of my research about neurodegeneration as a cause by diabetes, there's publish and perish. So you can literally publish and still perish. But yes, absolutely. Definitely publications are important. They absolutely help with the IP, with convincing investors. But at that stage, I started to realize, well, rather than me saying it's safe, I want the highest authority in the world, which I consider the FDA, to say this is safe and effective.
How do I do that? So I ultimately ended up going to FDA in 2010 at a meeting, said, I want a computer to make a diagnosis. And here's why. And I said, ho, ho, ho. And we worked together very closely, continued to work together very closely, but they were skeptical, to say the least. And so I realized that there's a long path here. I need money. and funding is not really NIH funding is not really for that they don't want to fund FDA interactions and $1 ,000 per hour consultants that is just not what we as taxpayers should be paying for when we pay researchers with an R01 and so people told me well what works in pharma the pharma
model is that you patent your idea maybe new protein maybe a folding mechanism, new CRISPR thing and then Pfizer or Regeneron or wherever will come in and take a path all the way to phase one, phase two, phase three. It's a well-trot path. It's well-known. And they said, well, why don't you do the same with your AI algorithms? So I did. And, you know, I was hopefully waiting for IBM or Google to show up.
And nothing moved, right? And so, okay, then other people were doing philanthropy. Very successfully, you know, you can raise a lot of money. But that's, for example, here in Iowa, very successful Institute for Vision Research and other institutes. but that's for people who got blind at a young age because of inherited eye disease people love giving money for making blind kids see that is you know very motivating if you say well i'm going to generate 10,000 pages of paperwork for iso certification and fda not so exciting for a retired farmer who made a few $1 million from their farm and you know can either continue to making blind kids see again or a bunch of paperwork so i realized
i needed angel investors and that ultimately led to finding the company so it was not like i set out to find a company it was hey i want to solve this problem i realized i need to i'm looking at a long runway i now call it um and how do i solve that and that turned out to be with a business plan and i was successful raising first angel investment money and later vc and now very proud to say for the first time growth equity last year uh stepped in so for the first time ever ai is not seen as a debt right which is what venture capital really does but now growth equity is this is going to happen
this is not a bet anymore and and so i'm really excited about that newest aspect of our fundraising but yeah long journey this is the sort of more commercial aspect and why that worked because part of it was wanting to meet the highest standard of safety and part of it was i've shared with with at least raj another thing which also happened in 2010 which was that i was doing all this research and my colleagues started to notice my colleague ophthalmologists very esteemed colleagues and suddenly there was an editorial in the most widely read ophthalmology journal in the world called ophthalmology times on the front page by the chair of ophthalmology at johns hopkins the retinator revenge of the machines
about my research calling me out by name essentially saying that i'm the terminator of retina i'm the retina specialist right that's that hurt and um and it's going to cost jobs and it's going to lower the quality of care it was partially tongue-in-cheek and partially very serious and actually immediately got a phone call from my chair hey what's going on here i mean and i was trying to make tenure at the time so you can understand it was somewhat upsetting but actually I have it now framed in my other office it was a really important moment because I realized that even though you think you do the science right and the goal is to help patients it may
not always go over smoothly and you need actually healthcare is complex as we all know to our maybe detriment, maybe not but it's very complex and we need stakeholder support we need everyone in the healthcare system to be comfortable with the introduction of a new technology that is very scary. So A, let's figure out what is scary about it. And I'm talking about racial bias, which we started talking about before then. Data usage, liability, right? Who's medically liable for an autonomous AI making an erroneous diagnosis?
Does it actually help patients? How are we going to pay for all of this? So these are the issues that I was summing up and starting to realize we need to address these. And on the other side, who are the stakeholders? These are patients, first of all, patient organizations clearly my colleagues professional medical societies the american medical association it is payers cms medicare private payers like united and aetna ethicists very important and then of course ai creators these are all stakeholders and then ultimately even investors and and then the ppp is what i call physicians patients and payers right that is the the nucleus of that.
We need all of them to support this, to be able, and failure-based care organizations. Sorry, I forgot one. I don't have a list in front of me, but normally I have a slide where I list these, like NCQA, USPSTF, PCORI. And they all look at each other and they all have a little bit of a literate take on what should happen in healthcare. And we can talk about it, but I don't think it's necessary. Now, more importantly, that to get these all on the same page about something as provocative as an autonomous AI, a computer making a diagnosis, which had never been done, was what led me to develop with real biothicists.
I wouldn't call myself a biothicist, but I had to learn it to create an ethical framework for AI that was published in the American Journal of Bioethics. And that was very important in the interactions with FDA. How do we deal with bias? How do we deal with the clinical trial? Is there something you can measure about these ethics? And we now call it metrics for ethics, where you say, well, there's an ethical principle like autonomy of the patient.
How can you measure that when you have a healthcare system, but you can measure it for a physician, but you can also measure it for an AI? No one ever did that for physicians, not for healthcare systems. We talk about it. Many papers on ethics in medicine are about words, but not really about measuring. And if you're an engineer, you need to have something that you can measure so you can meet it or not, right? You develop something, you want to know, does it meet this ethical criteria?
Does it meet this ethical criteria? Same for justice or equity, same for patient benefit or non-maleficients. We can go into the ethical discussion, but I don't think that's what your audience wants to hear. But more importantly, A, ethics are something you can understand as an engineer or an AI creator. More importantly, you can measure it. And if you can measure it, we can optimize towards it. And then what is interesting, you can never meet all of these ethical criteria 100%.
You can optimize the patient benefit by not allowing them to smoke, telling them exactly what to eat, but you lose a lot of autonomy. And patients do not always want that. So you need to find a balance between the different ethical principles. And that is, I think, the contribution we did in that framework.
I'll just say I am completely struck by how many different parties, groups of individuals you've had to navigate in just taking this from a paper, this sort of early technology into something that's now deployed in the clinic. And so we asked you about your training in terms of MD-PhD being the gateway, but I'm wondering how you learned all of what you needed to learn to sort of navigate all of these parties from the FDA to payers to other folks and just thinking about the physicians and machine learning scientists in the audience who are interested in those skills and and commercializing, did you have a mentor or did you have sort of the right mix of people or did you
just figure it out on the fly? What advice would you give folks who are in a similar position and want to commercialize their technology?
Ultimately, the objective function was patient benefits, right, and health equity. So that was the North Star, the guiding principle. That really helped. Now, I had to figure it out on my own and then decided, but part of it is as a physician, you are aware of payers. you're aware of HEDIS and MIPS and things like that and what they want to see versus what you need to record in the chart, medical liability. So that's a little bit more easy to understand for a physician than for an engineer.
But you can learn that. Don't forget, all of this has been done now and we've shown it can be done. So it's not like everything needs to be revented again and again. There's a lot. The stakeholders are known. There's not like every year there's a new group of stakeholders. It's sort of a fixed system. I literally drew maps for my board showing almost like D-Day. You know those maps with all those arrows to the different Normandy beaches?
Well, so I drew maps of, well, this group influences that group because it's all interrelated, right? The National Committee of Quality Assurance looks at the standards of care developed by patient organizations and professional societies. So if you go to one group, it may help another group get more comfortable. So part of it is just navigating that. Part of it is learning who are the stakeholders, being very open and not thinking that because you have this cool technology, what I call a glamour AI, people should just swallow it.
It's glamour AI technology that is really cool. And I love technology. It doesn't benefit the patient, or at least that has not been shown. And so it is more than that. Ultimately, your North Star, I think, should be the patient benefit. How do you benefit healthy populations? And then it's just necessary to do this. but no there was no mentor i wish yeah i wish but no we had to reinvent it and i think it couldn't have been because autonomous ai is so confrontational right we never think about how many rules and regulations are written but it all assumes a human because that's been the way that way for thousands of years and suddenly with a computer you need to
change these almost implicit regulations and now people start to think well do i actually want it but they never thought about a doctor doing it that just it was all implicit and now it had to be made explicit and that's why
you know some of the swirl was created got it so you also mentioned that you faced you know some resistance here and there along this journey and i really like that you've now framed the copy of the ophthalmology times that had the retinator as the the cover page which honestly is a i know it It sounds like it was a difficult time, but it is a pretty impressive nickname to also have at the same time. But, you know, I think what this signals to me and what you've clearly overcome is that there's sort of resistance from folks at key moments along this journey.
And so I'm also curious, you know, you've been able to overcome that, but what advice you would have for young machine learning scientists or clinicians who are also facing resistance as they try to develop and deploy new technology?
If it doesn't destroy you, it makes you stronger. So that's totally true.
When in doubt, go to Nietzsche.
I think more importantly, do you want to benefit patients or not? And then everything else is sure is a hurdle on the way, but can be overcome. The more evidence you collect that, hey, this works, maybe, hopefully you can show patient benefit. And I want to say that about this. I like to show this timeline where we had an algorithm, literally a multilayered network at the beginning and you have patient outcomes at the end. It was a long journey to bring it to patients a lot of evidence, a lot of stakeholder support, the ethics, the reimbursement.
Don't forget, we haven't even talked about that because if it's not reimbursed, if it's not part of the HEDIS MIPS, people won't use it. If it's not usable, and there's an entire team of digital diagnostics that is focused on what we call customer experience, success meaning you take a clinic, you introduce AI, you don't drop it and walk away. You literally continue to support them to make sure it works for their patients. So all of that is necessary.
And now finally, and I'm really excited about the papers coming out very soon, we showed in randomized clinical trials that this autonomous AI improves, that it not only improves health equity, it literally allowed clinics to go from very low percentage of people getting the exams, they need to almost eliminating health disparities. So it is really doing what we envision to do. So part of why I'm mentioning is this, it was worth a journey. I mean, it was rough, but it was absolutely worth it.
We did it. And with an entire team of more than 100 people at Digital Diagnostics, and thanks to stakeholder support from everyone, because what you say is true, there was initial resistance, and the Retinator is just one example, but you can imagine me going into CMS and saying, hey, you know, let's set reimbursement at $55, because here's why it should be $55. And there's a whole reimbursement framework based on an ethical framework that went in wide that specific number that you can read up on, right?
And so very grateful to all these others who were willing to do what is right for patients. There's literally 30 pages in the Federal Register from CMS discussing their worries and concerns, what they called guardrails, and whether they should do this, which they ultimately ended up doing. You can see it live in their proposed rules, how they were dealing with this, with this new thing in healthcare. So very grateful to everyone involved. And finally, then you need to stop me again.
Don't forget that healthcare was the first to use autonomous AI. So we always complain that healthcare is so complicated and how can you change it? There's no self-driving car. John Deere had the first autonomous tractor at CES last year. but it has been in wide use in US and now elsewhere. It was an announcement for the Middle East yesterday but widely used on patients in healthcare and nowhere else until last year with autonomous tractors. So we were actually the first.
Isn't that exciting? That doesn't get you excited that we can actually change this and make it better?
Yeah, I think that's amazing. And I think it's a perfect segue for something that I think we want to focus on for our next topic. So just to pin down a couple things, one of your first release products was autonomous AI for the diagnosis of diabetic retinopathy. And just to be clear, that means that you have a device that can be operated by a technician that can render a diagnosis of that disease without the oversight of a physician, correct?
Correct. And so that's the autonomous in the autonomous AI. And so I think the thing that is the real differentiator and the real thing that you figured out first that no one else has is how do you get someone to pay for that? So I would like to focus on the reimbursement model because just to be clear, sort of what I do is kind of cheap. Like technology development is difficult sometimes, but it's kind of cheap and easy if you have the right data.
Going that last mile, getting an actual device that can go into a clinic with a patient, and then actually getting a payer to pay for that is millions of times more difficult than the original tech dev is. So I'd like to understand a little bit, one, how you got it from whole cloth, got a reimbursement model for that, and then how that reimbursement model operates today. So if someone goes into their clinician's office and a digital diagnostics device comes in, sort of what does the back end of that transaction look like?
Absolutely. So reimbursement is extremely complex. Set it out. It's fraught with legal risks. If you do it wrong to what seems like in other fields normal, it's a felony. So I will be careful in how I respond. I hope you understand that. We published our reimbursement framework for AI half a year ago. And it's probably best if I go a little bit through that. I think there's two challenges. As an AI creator, what do you charge for that?
And I literally went through that in interactions with Congress. So I went in front of the Senate Finance Committee and explained early on, just after FDA, the no-vatorization, because actually it's not approval. I'm not allowed to say that it should be the no-vatorization. FDA doesn't allow me to say approval. And I went in front of them and said, well, how are we going to pay for this? we can pay it based on the marginal cost meaning you have a million diagnosis with the ai so the the rnd and all the science that you did is is amortized and now let's do one more diagnosis well that's maybe a little bit of electricity and maybe a lot of wearing
down of some atoms in the you know gpu circuits right that's very little and definitely it's not a sustainable business model that no investor will go for that and ai creators will probably say i'm not going to do it because it will be a money drain. There's no money to be made. And to go back to your previous
example, is that kind of the pharma model also where you're recouping R &D and then you have some sort of marginal cost that you're getting on every patient that would do a drug?
Yeah, it's a little bit more complex because then the part of CMS that we're talking about here is only allowed to reimburse what a physician charge. That's the way the Social Security Act was written. So it might seem that you want to pay for something, but there is actually very strict laws and regulations and jail time if you don't do it in the right way. So that's why it needs to work. But that's the other aspect.
But the first problem is literally, what do you charge an AI creator? Another way what people then recommend is what is called cost-effective analysis meaning you look at there's this service we know it is benefit to patients and for example the diabetic eye exam is a good example but there are many others where we know that if you do diabetic eye exams and i as a retina specialist get about 170 to 300 for those per patient even if you do 500 to 600 even if you pay that it's still worth the cost savings on the back end because of avoided blindness, visual loss, very expensive surgeries.
So one way for an AI creator is to say, I will just go to just below the cost-effective threshold, but then you would make it more expensive than it currently is. My whole goal, my North Star was improving access, lowering costs. So we don't pay 100 % of our income to healthcare in 20 years. And so not acceptable for me as an AI creator. And so there are various strategies that you can do for what you charge as an AI creator.
That is irrespective of whether you get it reimbursed. What should it be based on? So ultimately, what I said, and that may not be possible for all, but definitely for autonomous AI in some fields, is there's a willingness to pay currently for about 20 % to 30 % of patients who are getting these exams. So as a society where we're paying for other healthcare, there's a willingness to pay. we're already paying a certain amount of money for 20 to 30% of patients let's take the same amount of money we wouldn't be paying more and just get it to 100 of patients who deserve and need it and that's what we based it on i called it the equity enhancing
payment and so that ended up being 55 so it had to be 55 for clearly right i mean there's a formula here and so that was the basis for what you charge and then ultimately many many meetings with payers why it should be this and then you know we we could talk about the mechanics but i think that's less interesting that the fact that you need to start with what is an appropriate payment if you start with 600 for this case where currently cms is paying but it's 50 very unlikely that someone will use it because the reimbursement will probably not go up.
So you as an AI creator can charge $600, but then the physician only getting way less than that, no one uses it. It's a tricky balance to be found, but this is for what is called the physician fee schedule. Now I'm getting into the really wonky details. There's other payment schemes, which are very different. Other AI companies do things with Antep, which is a temporary reimbursement. It's a challenge right now. So as you can see from these internal discussions in CMS that they exposed and in Congress as well, are we going to pay for all of this because we want to make sure it doesn't increase healthcare costs?
Hopefully, and my goal is definitely to decrease healthcare costs, make it more affordable, more accessible, especially for people who currently are not getting the care they need.
Stepping outside of like digital diagnostics, if there is a wave of similar devices that are created that are essentially sort of screening tools, I believe you in your sincerity that your goal is to drive down healthcare costs. But are you at all worried that this sort of writ large will increase costs for the healthcare system as a whole? Meaning you can look to other areas of medicine where screening hasn't improved patient outcomes and we're sort of spending money for nothing.
Are you at all sort of worried of that sort of outside of the digital diagnostics ecosystem?
Absolutely. So I care greatly about patient benefit, population benefit, and all the things. These are the North Stars, right? And then you work your way back from what is the evidence for what is best for this patient and what role can AI play? That's how we do it and how we see it. So it's not Glamour AI, I have this cool technology, let's find a use for it. No, it's either way or not. What does the patient need?
What does the population need? And so if you start with that, there's a lot of things to be done. and there's a lot of evidence already. So let's start with that. And then what you see is all stakeholders aligning around that. And it becomes much harder for Glamour AI where people get excited about technology and it's $1 ,000 per patient. And it sounds great, but it doesn't, like you say, actually move the needle. Less willingness to pay, clearly, right now, especially.
And so, sure, you can try that and maybe patients self-pay for it. And that's their choice. but I think especially based on the value-based care movement which your listeners probably are aware of there's more and more resistance to just doing things because of coolness of glamour and so I think the guardrails are in place already there's a regulatory process there's an ethical framework and there's a way to get payment so why not that's probably the easiest path you can fight for a long time to get a thousand dollar for your AI that is probably maybe have a negative benefit for a patient It's a long journey.
Why do that and not do the easier journey that we just sketched?
Yep. And so I think one more reimbursement question before we conclude this policy walk corner and move on to the lightning round. So if I could summarize why digital diagnostics has been successful in this area of ophthalmology, there's a clearly defined task that a physician already does. There's a billable interface to that task. So reading a retinal scan, there's already sort of an existing way to bill for that service. And then you can sort of productize this into a medical device that essentially does that service.
So there's a lot of pieces in place. Obviously, you broke the path to get a reimbursement model for that. But I'm curious what you think about other areas that people are excited about medical AI where at least one of those conditions doesn't hold. So, for example, a lot of people are very excited about risk scores. So you can have an AI that pulls in the patient's entire EHR data and predicts things for like mortality, decompensation, sepsis.
And to me, those business models have always been very tricky. But sort of given your deep expertise in this field, I wonder if you think that other areas of AI that are kind of more amorphous, if there's a sort of a sustainable reimbursement model behind some of those risk scores or predictive models where there isn't this sort of natural billing procedure built into it.
Yeah, so indeed, I focus right now on the per patient because it's about the evidence. It's also about the regulatory approach and the safety aspects. And so as we have seen with Dr. Obermeier's paper in science a while ago, you're probably aware of. He's a friend of the podcast. Oh, yeah. Okay. Well, even better. So I don't have to tell you anything new. but clearly the problem there that it's so easy to fall into the trap to use proxies for what you you think you're measuring and are optimizing as an objective function rather than what you should be optimizing for and there is no regulatory oversight whatsoever and so i think yes there is a potential benefit but showed it
in improved patient outcomes it may be harder to get definitely someone like CMS to pay for that, but I'm not CMS. Payers may be more interested, but I'm very worried about the backlash that we already saw from his research, which now has led to the Office of Civil Rights in HSS looking into this and essentially saying any racial bias or any other bias in an AI is the responsibility of the provider using it rather than the liability being taken care of because the FDA looked at racial bias And in our case, we proved that there was no racial bias in AI.
So I think I would be careful until the safety aspects are better understood. And especially when I work with the Federal Trade Commission, they're looking into this. OCR is looking into this. I don't want to damage the field or I have to feel damaged by Congress saying, let's shut down AI because we worry too much. and there's too much bias and harmful effects going on. And as you know, there were actually harmful effects of this AI.
Yeah.
So first, do no harm applies to medical AI just as well as it does to human intelligence.
This is back to the ethical framework. Patient benefit or do no harm, non-maleficient. Justice or equity, do we do it equally well for everyone? And are we leaving the patient autonomous? Meaning, what about their data, right? Whose ownership of the data? Are we taking care of privacy? then can they make their own decisions? Absolutely. It starts with ethics.
Great. So I think we will transition to the lightning round now. The lightning round is a series of short questions that we're curious of your thoughts on. Also, just an opportunity for the listeners to get to know you a little better. So some non-medical, non-technical questions. The goal is to keep the answers as brief as possible, but no briefer. So a couple sentences should suffice, and we'll sort of knock these out in quick succession.
So are you ready?
I'm ready.
So the first question is, if you weren't a medical AI researcher, what would you be doing professionally?
My three kids are all, well, one is becoming an engineer. They're all engineers. They're all service academies. I'm really excited about the potential for space exploration and especially asteroid mining. So there's a lot there. But in the medical field and AI, there's so much to do. There's so many exciting opportunities right now. I'm just amazed. I wish I was 30 years younger.
And I admit that I did not have asteroid mining on the bingo card. So that was an excellent lightning round response.
Yeah, I love it. I love it. Michael, will doctors still be responsible for documentation in five years? Or will generative models like ChatGPT have taken over that task?
it was interesting the way it went with electronic health records where i was a big fan originally when this you know was at the short-lived level remember the 60s and my sin where it was the excitement was about the doctor typing stuff not about the ai behind it and so i was excited about the money or the time-saving aspects are more efficient and like i mentioned we lose productivity not to gain it and then what turned out is that if i don't type in my own finding, there's no reimbursement on the back end.
If I copy a note, reimbursement falls apart. And so, yes, you may become more efficient, and there's probably going to be a way in where that is, depending on how you see the payer, they want to pay for what the doctor actually does, rather than, and hopefully, this is the value-based care product, what benefits the patient, not for the time that the doctor spends that the doctor is now saving because they use an AI and so it's going to be interesting but I'm not expecting five years that this will be widespread
got it, alright so next lightning round question is what is your favorite band or musical artist
I wasn't expecting that right that's the point of the lightning round we saw the Chicago Symphony with Beethoven which was great last weekend. So that was wonderful. And I've been listening to Lana Del Rey, which I love singing. Oh, nice. Yeah, it's all over the place.
That's excellent musical breadth. Lana Del Rey and asteroid mining are fantastic answers to the loading room. All right, next question. Do you think things created by AI can be considered art?
Ah, that's a cool one. Do artists consider it art? I would say that, you know, the experts, and we're seeing some discussion there, right? if you look back at healthcare, what patients want is not whether doctors agree or disagree or AIs agree or disagree. They want the best patient outcome. Whatever gets them to the best outcome in an ethical way is best. As a user of art, I care about the quality of the art. And, you know, that can be newness.
It can be what looks visually appealing or emotionally appealing. I don't care about the process we got there. So as a user, yes. As an artist, I would say no. Fair.
Okay. And so again, lightning round question. So your responses.
Yeah, but these are complicated. Yeah.
So this one, I know that you're going to have a long answer to. Try and I think compress as much as you can. So, will AI in medicine over the next decade primarily be driven by computer scientists or clinicians? And this is a forced binary response.
I'm not answering, but I need my time. I either have a long answer. I need time to think.
It takes time to run the compression algorithm, I understand.
Yes. Take some time. Wow. I think computer scientists are the rate limiting step of these both. There's many rate limiting steps. Many Many. These are just two of them. But then I would veer towards the engineering computer science aspects.
Got it. Great. Last lightning round question. If you could have dinner with one person dead or alive, who would it be?
I had an interview recently. I said Elon Musk because I'm interested in autonomous cars and I think having an ethical framework to start with will really help get this across where now it's sort of who's liable do we want this? I think it can be easy if you just start with ethics and that's another AI field as well I don't even know the name of the CEO of John Deere but I would love to have dinner with him.
He's probably somewhere around here in Iowa, but I've never met him. I would love to meet him.
I'm guessing it's not John Deere, right? It's not John Deere. All right. So I think that that was a top five lightning round performance. I think that the entropy of the responses was excellent, and we got a little bit more of Michael Abramoff than I think we had going into that. So I think that was an excellent performance. So we're just going to wrap up with some big picture and some conclusions here where we kind of zoom out from the work that you've done so far and try and take stock of the AI medical field as a whole.
One question that I'm curious about, I was a snotty computer science undergrad. I have this story about when I met my wife, who's a clinician, that I told her I was going to replace her with AI and computers. That's why it's got over well, I'm sure. Yeah. As far as pickup lines go, I would say that's bottom five, nonetheless. So I have some sense that there was deep skepticism towards AI in the medical field, certainly not to the extent that you've sort of lived the experience.
So the Retinator, I think you sort of hinted at was not exactly something that you were flattered by. So what I'm curious about is over the course of your very long and impressive career, do you think the medicine as a whole has become more receptive to AI or how has that sentiment changed?
Absolutely, it changed entirely. Part of that is just a generational change where people are just more comfortable with computers. When I started medical school, I was probably the only one in five city blocks with his own Apple II, which I built myself. And so that has entirely changed. And like I said, I think healthcare should be proud that they were the first to use autonomous AI widely in a very ethical, sustainable manner. And now I think it's a wonderful environment where we can innovate, but we need to do it the right way, not innovation for innovation's sake.
And by the way, about the Retinator, and sorry about these long answers, there was an interview with me in the American Medical Association on their website. This guy is doing AI the right way. So one can turn it around also. And what doesn't destroy you makes you strong. And in this case, the retinitis absolutely made me stronger. And the field of the whole, I think, stronger.
Yeah, I have seen that too, that just having seen my wife go through training, that medical students now and residents kind of just take for granted that something like this is going to be integral to their practice. And, you know, I don't want to speak out of turn here, but I think that there's less of a mystique around the clinical reasoning process. Certainly safety, patient benefit, all of that's always top of mind. But I think that they're just used to working with computers to make all kinds of decisions in their life.
And so it almost would seem unnatural if that weren't also part of their patient interaction in clinical practice. So I think that you're right that there's like a generational thing going on here too.
It's still interesting. You're so right. But that's, you know, some, you still see so often that people think the doctors are always right. and that when there's a disagreement between a doctor and AI, therefore the AI is wrong. The whole continuous learning debate that we didn't get into is part of that. And then you have the others, and they suddenly think that the computer is always right. And let's try to find a middle ground here where we know that they're both not perfect, which they are.
Right. Thanks.
So I think you've touched on this question quite a bit, but maybe you can give us some just concluding pearls. you've given everything happening in the field given your work how do you think clinicians should think about the impact of ai on medicine so i discussed it with the residents and the
fellows and even medical students all the time of course i think there's going to be a lot more of it now that the the path is sort of at least it's been demonstrated it can happen so there's a lot more going on right now there's a lot of ais and development have reached commercial success not only in direct patient vision but but elsewhere elsewhere as well so there will be a lot more of that and they have to come to grips with that but don't forget in most fields there's so much underserved patients 50 60 70 percent of patients do not get the appropriate care that they need and deserve and will have better outcomes.
So there's so much work to be done. So we should see it not as, well, this limits my opportunities. It expands my opportunities. We can now reach more population and I actually can be of more benefit to these patients and to this population. So part of it is that. Will there be less diagnostic and more interventional? Yes, because I see autonomous surgery, the real autonomous surgery, where there's a full surgical procedure as something a little bit harder to validate and prove that it's safe because you literally make a decision every microsecond, a medical decision.
And so do you need to validate it as a series of decisions or FDA and the field hasn't decided on that, I think. I think that's more decades away before we are feeling comfortable with that. There's definitely, especially in ophthalmology, which is always leading like in gene therapy, but also in automated surgery, corneal surgery, that's almost fully automatic, but that's a very, very specific, very narrow field. So anywhere there's interventions, it's good to have those skills more than the diagnostic skills, which will slowly, computers are typically better.
Great. So I think the next question is an area of disagreement that we might have, but the disagreement may be on semantics versus on the foundational thing. So I've been on the record before as being somewhat of an explainability skeptic. We wrote a paper a while ago trying to lay out why we think that explainability for AI is not a good trust mechanism. Because I'm sure that you hear this a lot too. One of the things that I always get asked is, how can I trust this algorithm if it can't explain its reasoning to me?
So I'm curious if you believe that explainability is important for winning trust of the healthcare workforce for a particular AI model or algorithm.
So I'm going to give a complex...
This one deserves a complex, so go for it.
Polychromatic answer. So A, physicians, some specialists do not know how they come to a diagnosis either. It's literally a black box. Skin melanoma, famous example. There is no biomarker for skin, or let alone a set of biomarkers for skin melanoma. It's literally gestalt. Does that mean we cannot figure those out? active research right now are there needs of the population where we may need and I will go into the semantics of explainability but where we don't know what it's doing and it's literally learning from examples and we validate it enough that we trust it because we wouldn't know what to look for if we said it explainable it cannot tell us well there's a hemorrhage here or a pigmentation there
or the border is irregular or it's a classification so but then i think there's different aspects and i tried to define it in a paper i wrote with fda on this because it's about metrics for ethics right and one of them is how do you measure explainability or at least quantify it i think there's explainability after the fact where it's about well these are the steps i took as a decision maker to come to my conclusion and here's the reference in the literature maybe right that's there's one way of considering explainability if If I have a colleague and I ask them, what do you think?
Typically, that's the type of answer. There's another more algorithmic level, which is at the unit level, can you prove the different parts? That's how we validate typical computer code, right? At the unit level testing and then larger aggregates at the system level. That's another type of explainability, more at the algorithmic level. And then ultimately, almost at the pathophysiological level, do we know what it is doing in terms of the pathophysiology of the disease we're trying to detect and now i'm talking about my markers these are three very different types of explainability we call them validability explainability and transparency in the paper and we differentiated them so i think it's a little bit more complex but i think you refer mostly
well actually i don't know what you're referring to but i think you're referring mostly to this biomarker versus non biomarker where I think if you have biomarkers, these are priors. Why not use them? One of the biggest limitations in AI in healthcare is going to be the sparsity of data. Unlike self-driving cars where you can just have a car drive around and you have millions of images to train from, there's all these ethical aspects with getting images from patients.
There's radiation, it's dangerous, it can harm them, let alone normals, which is actually more of a challenge, as you probably know because normals why would you expose them to radiation just to have a normal and you need many normals to make a good ai and so we will always in my view will be data limited especially this is a problem for more rare diseases where there's only 3,000 melanomas in the world in the eye per year so if you want 50,000 examples to train your ai you know there's a there's an issue here so if we know that there are well established priors that we can build in them, build detectors for that are racially invariant, for example,
probably a better way to go. But if you don't have those, we will have to do, you know, where it's non-explainable. But that's from the pathophysiological algorithmic aspect. That helps you, by the way, at the unit level testing that I mentioned earlier, because now you have detectors, you have a combination of detectors, you can test each of them. and if in our case the algorithm fails on some patient or some image because oh this detector didn't fire this probably you know we need to see what's flaw there it makes it easier to tune it to new populations or new data sets not that we're doing that with different applications than the diabetic eye exam so i think there are
uses for this but it shouldn't prevent us that oh it's something is not explained we should never do it because ultimately it's about patient benefit, about equity, not about, and if you can prove it safe.
So as I suspected, we actually don't disagree. I think that I'm totally on board with you as verification and validation as a really strong trust mechanism that if you give a doctor evidence that these parts have all been battle tested, we know how it works. We do that in other areas. It's usually called an RCT for drugs, where maybe we don't understand the mechanism of action and things like that. And then I really liked your other part of that, which is I'm going to take what computer scientists would call an end-to-end task, which is where you put in the raw images and the label for the disease and have the computer sort out all of the different components
that may or may not be present in an eye that gives rise to a disease. And I'm going to decompose that into a set of biomarkers that I know are more reliably captured, don't have all of these other noise artifacts may not be influenced by gender or racial biases and things like that. So again, like I think we actually totally agree with that. Like a smart decomposition of the task that you want the algorithm to learn is something that I certainly don't find controversial and I think is kind of the secret sauce in a lot of areas where deep learning has had success, where we actually sneak in some domain knowledge to give the algorithm kind of a head start
on what we actually want it to learn versus having it learn everything tabula rasa from scratch.
And so indeed, it's gray. It's not black and white. I would argue, why waste so many training samples on detecting the edge, right? I mean, why does it learn? You waste 50 ,000 samples to learn that an edge is an edge? Well, I could have told you that, right? I mean, and there's ways to do that. So that's why I mentioned prior. So I couldn't agree more. Yeah. Awesome.
So we have a related, I think related question. Will machine learning, in your view, exacerbate healthcare disparities? I think I know what you're going to say, but please, please go ahead.
So we have the proof in hand that it can improve them and literally eliminate them. So that's all I want to share because, you know, read the paper when it comes out. I wish they would just, you know, accept it, but it will come soon. and so yeah it absolutely has the potential to increase them as Obermeyer's paper showed right and so it is again the right use at the right time and for the right group of patients validated in the right way and it can randomized clinical trial proof it's only one for the diabetic eye exam but it can be done
great so I'm really happy that I get to ask this question because I have no idea what you're going to say. But what is your most controversial or contrarian opinion on medical AI?
That has shifted a lot over the years, I think. So it used to be, you know, this can never work. You're just a doctor. You don't know what you're talking about. So this can work, but there's, you know, never going to be. people are still saying FDA will never quote approve this unquote literally and so so these have been controversial takes never reimbursements never will improve health disparities will never improve outcomes so been through all of that so it seems that it's not controversial anymore I'm still running into a lot of interesting debates about autonomous versus assistive AI and why we need that but where it seems so obvious to me and I can explain it in terms of productivity
and what the economy is all about so that's interesting
what I love is that you've had controversial opinions your entire career but you keep proving them to be non-controversial
I'm afraid but one thing is what is interesting what you run into is what we call incidental findings things and so and it's interesting when you go into a healthcare system and you propose to use ai because it's better for the patients better for the populations they serve it's billable etc so everything has been taken care of and there's still resistance because clinicians may object and say but this doesn't replace what i do which is a full eye exam and it doesn't it's for a very specific disease, but what is interesting, and that is maybe controversial, I say, let's look at what is beneficial for patient outcome, that it should be doing this.
Why are we paying for this? And that's a very interesting debate, and I have a somewhat controversial take that I think we should start and work our way backwards from patient and population benefit, meaning both health equity, you know, equally applied, but also patient benefit. First, do no harm, and then second, can do actually good.
Great. Right. Got it. Thanks. Michael, this has been a lot of fun and very illuminating. I have one last question for you. What are you most excited about for the next five years in medical AI?
The scalability of what we have done, meaning like you said so rightly, it's only one specific disease. Can we apply it to many other diseases? We have AIs in our fields. Many other groups, research groups, companies have AIs in other fields. There's a pathway, but can we scale it quickly enough to benefit a lot of people on the short term? Because we need to. Healthcare is too expensive. Costing too much is not reaching the people who need it in many cases.
It can be better. I mean, equality is awesome here in the U.S. I mean, the reason it came to the U.S. is because healthcare is so much better than anywhere else in the world. but it's unequally distributed. And so can we use our AI to equally distribute it in this short period of time with, you know, five, 10, 20, 40 AIs? That is, you know, let many flowers bloom. That is really what needs to happen here and what I hope will happen sooner rather than later, if only for my kids' sake and your kids' sake.
Right, right.
Well, I think that's all we have today. I would just like to say thank you for being on and sharing what I think is just an amazing amount of tenacity and grit and foresight as one of the pioneers in this field. And I think the entire community will have benefited from the pathbreaking that you've done and kind of showing us how to go from a real unmet clinical need to an actual product that can help patients.
So, Michael, I just want to say thanks for being on AI Grand Rounds today.
It was great having me, and thanks for having me.
Thank you, Michael.
Have a good day. Thank you.