OpenNashPodcasts

CXOTalk

Data, AI, and Patient Outcomes: Inside the Mayo Clinic Platform

1 Apr 2024 · 52m · John Halamka, Michael Krigsman · 25 quoted facts

▶ Play episodeOriginal source ↗

In brief

Dr. John Halamka argues that healthcare AI will only help patients if it is built and tested on carefully curated, diverse, privacy-protected data. Mayo Clinic Platform pursues this through a federated network, where partner hospitals keep their own data and algorithms travel to the data. Shared standards, transparent model labels and public testing registries are meant to build clinician and patient trust.

The central tension is between AI's promise and its credibility. Many algorithms exist, but few are used daily, because clinicians cannot see what data trained them or where they fail. Halamka contrasts verifiable predictive AI with hard-to-validate generative AI. His answer is human responsibility, plus transparency, ethics and equitable access, so AI augments doctors rather than replacing them.

Key insights

  1. Federated data keeps privacy while enabling global AI

    Each partner keeps its de-identified data in its own sealed cloud container, and nobody holds keys to anyone else's data. Algorithms are sent to the data for validation, and this also shrinks the attack surface.

  2. Most healthcare AI is built but not deployed

    Of the 200-plus algorithms Mayo developed, only about 30 are used in clinical practice every day. Halamka cites a Stat article saying healthcare AI has a credibility problem because developers do not disclose training data or performance.

  3. Model cards should work like mattress labels

    The Coalition for Health AI wants every algorithm to ship with a data card and model card. Clinicians could then see whether it was built on patients like theirs, as with the cardiac mortality tool that only works below a body mass index of 31.

  4. Generative AI is far harder to validate than predictive AI

    Predictive models can be checked against a ground truth such as an echo test. Generative answers change with each prompt, and a New England Journal chest-pain case got a dangerous answer when asked one way and the right one when asked another.

  5. AI augments doctors, who stay legally responsible

    Halamka cites a draft CMS rule saying a doctor who delegates decisions to AI is responsible for bad results. At Mayo, AI auto-contouring cut head and neck radiation planning from 16 hours to one while an oncologist still reviews the plan.

Questions

What is the Mayo Clinic Platform trying to do?

Halamka says it curates health data in a privacy-protecting way, works with startups and large companies building AI, and helps put the resulting tools into healthcare systems with little friction. The goal is to shrink the time for new ideas to reach patients from years to far less.

Does Mayo sell patient data?

Halamka says Mayo never sells or exfiltrates (moves out) the data, and keeps it under Mayo's control. Patients can opt out at any time, and about 3,000 of 10 million have.

What does federated data mean, and why does it matter?

Federated means the data stays at each hospital or country rather than being pooled in one place. Questions or algorithms are sent to it, which respects national data-sovereignty laws and lowers security risk.

Why don't more doctors use AI tools yet?

Halamka says many tools do not disclose what data trained them or which patients they work for, so clinicians cannot trust them. He says only about 30 of Mayo's 200-plus algorithms are used daily.

Who is responsible if AI gives a wrong answer?

Halamka says the doctor is. He cites a draft CMS rule stating that AI should augment human decision-making and that a doctor who delegates the decision to AI is responsible for bad results.

Facts

Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.

John Halamka was hired on January 1, 2020 to lead Mayo Clinic's front door for partnerships with innovators.

“I was hired January 1 of 2020 to be that kind of front door for Mayo Clinic as it seeks partnerships and collaborations with innovators around the world.”

0:01:02 · John Halamka · Mayo Clinic Platform · 2020

In 2019 Mayo Clinic CEO Gianrico Farrugia asked what it would take to bring innovation to patients in 18 hours rather than 18 years.

“So in 2019, Gianrico Ferruja, the CEO of Mayo Clinic, asked the question, what if you wanted it to be 18 hours, not 18 years?”

0:00:31 · John Halamka · Innovation speed · 2019

Mayo Clinic Platform puts de-identified data into the OMOP standard format under the control of the organizations that generated it.

“put them in an industry standard format, OMOP, under the control of those who generated the data.”

0:04:48 · John Halamka · Data standards

Brad Malin certifies each de-identified data set in the Mayo Clinic Platform federation.

“Brad Malin, one of the world leaders in DID, REID, does the certification of each of the resulting data sets.”

0:07:37 · John Halamka · Data certification

Mayo's data set was graded as 99.6 percent de-identified when it was certified.

“In the case of Mayo, it was graded as 99.6 % de-identified and highly curated and normalized.”

0:07:37 · John Halamka · De-identification

Mayo's data sits in Google Cloud, Mercy uses Azure and Albert Einstein uses AWS, so the federation is cloud neutral.

“In my case, our data happens to sit in Google Cloud, but in the case of Mercy, which works with us, they're Azure. In the case of Albert Einstein, they're AWS.”

0:09:31 · John Halamka · Cloud

Working with the International Standards Organization, Mayo Clinic Platform extended OMOP to cover digital pathology and genomics.

“We've extended OMOP to include certain other kinds of data elements like digital pathology and genomics that are quite in core OMOP.”

0:11:04 · John Halamka · Data standards

In one partner data set Mayo's tools found that more than half of the social determinants of health data were missing, so AI based on them was discouraged.

“what we found is more than half of the social determinants of health were missing.”

0:11:39 · John Halamka · Data quality

Mayo Clinic Platform's accelerator program has brought in about 40 startup companies.

“On the one hand, we have an accelerator program where we brought in about 40 companies so far.”

0:16:54 · John Halamka · Accelerator · 2024

Mayo Clinic and HIMSS announced a program called Solution Studio to test and deploy company AI models.

“Mayo and HIMSS announced something called Solution Studio”

0:18:10 · John Halamka · Solution Studio

Of more than 200 algorithms Mayo has developed, about 30 are used in clinical practice daily.

“of those 200 and some algorithms we've developed, say about 30 are used in clinical practice every single day.”

0:19:35 · John Halamka · AI deployment · 2024

Stat reporter Casey Ross wrote an article saying healthcare AI has a credibility problem.

“Casey Ross, who is a reporter at Stat, a Boston Globe affiliate, wrote a brilliant article last year which said, healthcare AI has a credibility problem.”

0:20:27 · John Halamka · AI transparency · 2023

Mayo's cardiac mortality prediction tool works well for patients with a body mass index under 31 and poorly above 35.

“that tool works really well if your body mass index is less than 31 and not so well if your body mass index is above 35.”

0:22:24 · John Halamka · Model performance

The Coalition for Health AI began in September 2021 with six organizations and had 90 by October.

“in September of 2021, we brought together six organizations Then by October, there were 90 organizations.”

0:24:19 · John Halamka · Coalition for Health AI · 2021

The Coalition for Health AI plans to assign testing to a network of laboratories and host a national registry of results rather than test products itself.

“So CHI won't do the testing itself, but will assign the testing to a nationwide network of laboratories that will report on these products.”

0:26:47 · John Halamka · AI testing

John Halamka has worked with the Bill and Melinda Gates Foundation for 15 years in sub-Saharan Africa and Northern India.

“I've worked with the Bill and Melinda Gates Foundation for the last 15 years in sub-Saharan Africa and Northern India.”

0:29:48 · John Halamka · Global health

Mayo CEO Gianrico Farrugia set Halamka a goal of touching 4 billion people by the end of his presidency.

“And by the end of your presidency, I want you to have touched 4 billion people.”

0:33:09 · John Halamka · Global reach

Mayo is working with Kenya to deploy algorithms there, funded by philanthropy.

“we recently worked with Kenya and we'll be deploying a variety of algorithms in Kenya. and philanthropy will fund the deployment of the algorithms in Kenya.”

0:33:45 · John Halamka · Kenya · 2024

John Halamka did a postdoc in AI at MIT in the 1990s, writing rule-based systems.

“My postdoc was in AI at MIT in the 1990s.”

0:34:29 · Michael Krigsman · Halamka background · 1990s

At Mayo, AI auto-contouring reduced head and neck tumor radiation treatment planning from 16 hours to one hour.

“it used to take 16 hours for a human to develop a sophisticated head and neck tumor treatment algorithm. Today, it takes one hour.”

0:41:41 · John Halamka · Radiation oncology · 2024

About 3,000 of Mayo Clinic's 10 million patients have opted out of data use.

“Of our 10 million patients, about 3 ,000 of 10 million have opted out.”

0:42:33 · John Halamka · Patient consent · 2024

Halamka says a draft CMS non-discrimination rule states AI should augment human decision-making and the doctor stays responsible.

“He said AI should be an augmentation to human decision-making.”

0:39:36 · John Halamka · AI regulation · 2024

Halamka's wife was diagnosed with stage 3A breast cancer in December 2011, which led to founding Unity Farm Sanctuary.

“In December of 2011, my wife was diagnosed with stage 3A breast cancer.”

0:46:38 · John Halamka · Unity Farm origin · 2011

In 2012 the Halamkas sold their Wellesley home and bought 15 acres in Sherborn, Massachusetts, to start Unity Farm Sanctuary.

“So in 2012, we sold our small home in Wellesley, Massachusetts, and purchased 15 acres in a rural town called Sherbourne, Massachusetts.”

0:47:17 · John Halamka · Unity Farm origin · 2012

Unity Farm Sanctuary now covers 100 acres with 22 barns, 360 large animals and about 1,000 volunteers.

“We're now 100 acres and 22 barns with 360 large animals, 1 ,000 volunteers, programs for the elderly, programs for the disabled, and a preschool”

0:47:55 · John Halamka · Unity Farm Sanctuary · 2024

Chapters

  1. 0:00:31Mayo Clinic Platform's mission

    Halamka explains why Mayo created the platform: to shorten the time it takes for innovation to reach patients. It rests on curating data, engaging innovators and deploying solutions.

    “what if you wanted it to be 18 hours, not 18 years?”
  2. 0:05:30Federated data, de-identification and quality

    Explains why data stays with its owners and how it is de-identified, certified and standardized in OMOP. This shows how global AI can be built without centralizing sensitive records.

    “the data never leaves the organization, the data is never sold.”
  3. 0:13:35Business drivers and the accelerator

    Halamka ties AI to hospital pain points such as negative margins and burnout. He also describes the startup accelerator and Solution Studio.

    “my margins are negative. My staff is hard to recruit and retain”
  4. 0:21:03Credibility gap and the Coalition for Health AI

    Many algorithms lack transparency, which slows adoption. CHAI aims to create testing standards, labs and a public registry.

    “Every algorithm needs to come with a data card and a model card”
  5. 0:31:27Ethics, equity and global reach

    Uses cancer versus Alzheimer's prediction and examples from India and Africa to show that AI must fit patient preferences and local infrastructure.

    “We all agree we're going to meet the patient at the level of their technological sophistication”
  6. 0:39:36Predictive versus generative AI and responsibility

    Contrasts verifiable prediction with unreliable generative output and addresses who is accountable. It also shows radiation planning as a burden-reducing use.

    “a doctor should never delegate decision-making to an AI.”
  7. 0:44:50Privacy, trust and Unity Farm Sanctuary

    Covers opt-outs, values alignment with partners and the Digital Hippocratic Oath. Halamka then describes the animal sanctuary he runs with his wife.

    “It's called the Digital Hippocratic Oath.”

Transcript

0:00:00
Michael Krigsman

Today, on Episode 829 of CXOTalk, we're discussing the intersection of health, data, platforms, and AI with Dr. John Halamka. He is truly, and I'm not exaggerating, a towering figure in this field. John, you're president now of the Mayo Clinic platform. Please tell us about

0:00:31
John Halamka

the focus of your work. I have been in academic IT and healthcare for 40 years. And what do you learn over those 40 years? The pace of innovation, the dissemination of new ideas is slow, right? I think we've learned from Clayton Christensen and others, you know, It could take 20 years. So in 2019, Gianrico Ferruja, the CEO of Mayo Clinic, asked the question, what if you wanted it to be 18 hours, not 18 years? What would you need to do?

0:01:02

What would be the technology, the policy, the process? I was hired January 1 of 2020 to be that kind of front door for Mayo Clinic as it seeks partnerships and collaborations with innovators around the world. And so in some ways, it's three components, which is how do you curate data and do so in a privacy-protecting and ethical fashion? And how do you do it not just for one institution like Mayo, but globally? And we can talk about that.

0:01:31

How do you engage with solution creators, small and large? And how do you take the innovations and place them into healthcare systems or others in our ecosystem, could be pharma, payers, et cetera, and do so with little friction? So in effect, that's my role. Simply curate the world's data, encourage innovators to build more faster

0:01:54
Michael Krigsman

and place it in the hands of those who need it. So you're aggregating this data, lots of data, you've built a platform. And as we talk, we'll get into the specifics about the platform and the kinds of data. But for what purpose? Why are you curating this data? All of us have figured

0:02:13
John Halamka

out over the last two years, that predictive and generative AI will likely impact our future workflow. And the question, of course, is if you are going to create models that are going to do good for patients of the future, you better have a great curated data from patients of the past so those models perform as we expect them to. You should also have highly diverse data sets globally so you can validate and fine-tune these AI models.

0:02:45

Because just an AI model developed in Minnesota may not necessarily work so well in rural Georgia. And so, as you say, it's not data for data's sake. It's creating a global federation of a distributed data network for the creation of AI that we hope is fair, appropriate, valid, effective, and safe. It's testing,

0:03:10
Michael Krigsman

and it's ultimate use and monitoring. You are raising a whole bunch of issues. can you start to drill in what kinds of data are you collecting? You mentioned a federated model. Tell us about that. You mentioned or implied safety and security, privacy. Tell us about that. There's a whole bunch of very crucial issues that get wrapped up together here.

0:03:39
John Halamka

Again, having done this for 40 years, and I'll ask you a rhetorical question. If somebody said, you know, we're going to build the world's largest healthcare database. We're going to put it in the basement of the White House. It's going to be fabulous. Chances are a centralized, aggregated anything will never happen. And there's issues of technology and policy and trust, but also issues of privacy and the logistics of it all. So what have I learned over the last 40 years, really important to decide what data you have that's good enough for purpose.

0:04:16

And that is who collected it? Why did they collect it? Is it standardized? Is it believable? And then once you have an understanding of data and metadata, the purposes it will be used for, curating it, normalizing it, maintaining it within the organization that created it and not sending it out of that organization. That's something that's really important for persisting privacy and trust. And that means that if you say global, well, many countries have data sovereignty laws.

0:04:48

They're not going to send their data across the border. So you say, oh, could I work with Canada? Well, sure, as long as the data stays in Canada. Brazil, as long as it's curated by those who created it and there's consent, Israel has to stay with data sovereignty under the control of Israeli authorities in Israeli data centers. So my role has been define standards and processes and tools to de-identify structured data, unstructured text, telemetry, images, genomics, digital pathology, and put them in an industry standard format, OMOP, under the control of those who generated the data.

0:05:30

but with the nature of cloud computing, FHIR APIs, peer-to-peer governance, enable organizations or even countries to work with each other so that models can be developed and models can be tested. But really important, the data never leaves the organization, the data is never sold. And if a patient doesn't want to participate, their data is redacted.

0:06:00
Michael Krigsman

How do you ensure that the data that is being federated and ultimately aggregated is reliable, accurate, and adheres to the privacy standards that you have set up?

0:06:19
John Halamka

First, you ask who in the world has the data science and the experience to collect data, curate and normalize data? Well, I mean, this may be an imperfect metric, but if you look at the U.S. News & World Report top 10 healthcare organizations in the U.S. or top 10 organizations internationally, that seems like a reasonable place to start. So we did. And so our federation currently is the top five nationally and internationally organizations because they have worked with data for a long time.

0:06:59

They are willing to run tools that we provide to help us understand the quality of the data. How many 150-year-old patients do you have? Have they had three kidneys removed? You know, these kinds of things you look for just to assess, is the data sparse? Maybe you're missing a lot of social determinants of health. So you assess the data hygiene. The tools not only technically do de-identification through natural language processing, machine learning techniques, but once DID is done, we seek certification of the data that resulted.

0:07:37

Brad Malin, one of the world leaders in DID, REID, does the certification of each of the resulting data sets. In the case of Mayo, it was graded as 99.6 % de-identified and highly curated and normalized. Then it's placed in a sealed cloud container for which, in this case, Mayo has the keys to its data. But Sheba has the keys to its data. Albert Einstein, its data. At UHN Canada, its data. Neither of us, none of us have keys to each other's data.

0:08:12

But for example, if I want to validate an algorithm, I can send that algorithm in a containerized fashion to my colleagues in Brazil and say, run this against your data. Let me know how it works. And that way, we're never exfiltrating the data, but we're also protecting intellectual property.

0:08:33
Michael Krigsman

How does this actually work? Because the data never leaves, or correct me if I'm wrong, never leaves their location And yet the ultimate value of what you're doing depends upon this aggregated data that we can all operate on in order to determine that will ultimately yield better clinical results.

0:09:00
John Halamka

Let me just explain to our two use cases. One would be federated query. right so that is the data never leaves but i can ask questions of each of the data sets and aggregate the result that comes back you know how many patients do you have that have diabetes blood pressure on a torvostatin and have muscle pain and you'd get oh institution one has this many institution two has that many institution three has this many yeah so that kind of federated query.

0:09:31

And then the other way, as I say, is that very often the issue is we develop an algorithm in one geography and we need to tune it for another geography. So in that case, you're literally working in a sub-tenancy of the cloud in a given locality. In my case, our data happens to sit in Google Cloud, but in the case of Mercy, which works with us, they're Azure. In the case of Albert Einstein, they're AWS.

0:10:00

It's cloud neutral, doesn't matter, but they're creating subtenancies for people to come and work in the context of their cloud.

0:10:08
Michael Krigsman

So the issue then is the ability to query that data across the collection.

0:10:16
John Halamka

Correct. The second you start mixing data across geographies, you run into incredible regulatory challenges, but also real privacy and security concerns. And so, you know, you'll always use this term attack surface. And if you've got the data distributed across the world in different clouds operated by different organizations, your attack surface is smaller. How do you ensure then

0:10:46
Michael Krigsman

the quality of the data going across all of these different institutions. Because if I am querying the system, the platform, I want to be sure, obviously, that the results that are returned

0:11:04
John Halamka

are going to be valid. We happen to use international standards like OMOP. And we've extended, by working with the International Standards Organization, We've extended OMOP to include certain other kinds of data elements like digital pathology and genomics that are quite in core OMOP. So what happens is each institution is charged with taking its data, which could be in an EHR that is unique to their location, mapping it to a common set of vocabularies and schema structures in OMOP.

0:11:39

So now, although you have all these various federated partners, the underlying data store they're using is identical across all these geographies. And we have tools that then run across that data store, as I mentioned, and assess quality by looking for data sparsity. Example for you, I mean, what entity we worked with had an extraordinary data set. But what we found is more than half of the social determinants of health were missing. And so our biostats people said, you know, you really shouldn't run AI that is based on social determinants of health because it could be the people who are the lowest income, the lowest educated are the ones not providing the social determinants of health.

0:12:28

So that's sort of these tools running on those local enclaves help you to understand variation

0:12:33
Michael Krigsman

where you can develop algorithms and not. Check out CXOTalk.com. Subscribe to our newsletter and subscribe to our YouTube channel. We have a very interesting question from Twitter, from Arsalan Khan, who is a regular listener. He always asks excellent questions. And he says this, that what you're describing, using AI in medicine requires a rethink of many things, from training doctors, disrupting processes, to how insurance works. Where do you begin to manage this? How does one start to manage this?

0:13:17

And how does the culture of the medicine, medical and insurance industries affect what you're doing? So now he's asking about the cultural dimensions behind all of this? When you talk to CEOs of hospital systems,

0:13:35
John Halamka

they say, God, I woke up this morning and what I need is an algorithm. Never, never, right? What they say is my margins are negative. My staff is hard to recruit and retain and everyone's burned out. So you say, oh, well, if those are your three critical business problems, what if there was technology that could help you with those business problems. Oh, that would be amazing. Okay. Well, if your margins are negative, are there other, say, activities you'd like to begin with?

0:14:10

Remote patient monitoring and interpretation across cardiology or neurology or other specialties of patients in their homes. Delivering care in their homes for serious and complex high-acuity illness. Chemotherapy in the home. Oh, yes, these are all the wonderful new things we'd love to do because there's going to be reimbursement. And if AI can empower those, wonderful. That helps my margin pressure. Well, doctors and nurses are hard to retain. Specialists, hard to recruit. Well, what if, in effect, a specialist knowledge could be encapsulated in an algorithm and you could actually have every one of your primary caregivers enabled with a cardiologist in their pocket, so to speak.

0:14:56

Oh, that would be amazing. Well, what if I could get every doctor or every nurse home for dinner every night? Oh, that would be incredible. Reduce the burden and help with longevity. So as you start to look at some of the 240 plus algorithms Mayo has developed, each of them is justified in some fashion by one of those three business imperatives. Now, you did ask about reimbursement. Here is, again, it's a dream I have. This is not quite the reality, but let's start with the dream.

0:15:31

Mayo has 10 million patients, birth to death. I have every event that has occurred in their lifetime. I know their signs, symptoms, phenotype, genotype, and expozo. What if insurance companies and provider organizations agreed to work together as two sides of the same coin. We could say, we're going to take hundreds of millions of patient journeys, figure out what worked, and then the standard of care isn't just a set of rules that a, say, payer clerk reads and says, approved or disapproved, pre-offed or not pre-offed.

0:16:11

but it's actually based on continuously improving AI models of what's going to benefit the patient the most. And when I've talked to payers, they say, we're not interested in denying care. We just want to pay for the effective care. Okay. Well, that's where payers and providers can work together. And these are some of the things we're beginning to pursue. When it comes to algorithms,

0:16:37
Michael Krigsman

what is the role of the Mayo Clinic platform? Because so far, you've mostly described the platform as a way of gathering data for queries to be used very broadly. What about algorithms?

0:16:54
John Halamka

I'm, of course, not here to advertise or sell you anything. But Mayo looked at where are the gaps, especially say startup companies and they want to create a novel product or service. What you find is startup companies may have great engineering and great energy, but don't necessarily have access to clinical expertise, workflow, understanding, de-identified data. So on the one hand, we have an accelerator program where we brought in about 40 companies so far. Those 40 companies are given the clinical guidance, given access to de-identified data, given an opportunity to test things in workflow.

0:17:36

The end result is a product co-developed with Mayo Clinic Platform, and that goes out to the marketplace. Some of these startup companies have gone on to be sold or have significant funding events, and that was a result of the collaboration that we had together. On the other hand, And some large companies say, well, we may have clinical experts, we may even have data, but we don't have the capacity to deploy these things in a clinical setting and understand their impact.

0:18:10

And so Mayo and HIMSS announced something called Solution Studio, where it's for any company we work with, we can take a model. If you've already developed one, fine. We'll then run it against all of our data, understand its biases, and give mechanisms to deploy it in workflow and be able to gather feedback. How is it used? Did it make a difference? Were the suggestions reasonable? So think of it in the wheel of AI, from ideation to building a model, to testing a model, to deploying a model, to monitoring a model.

0:18:45

Our role is to enable all of our stakeholders to have all those capabilities.

0:18:49
Michael Krigsman

Where is medicine today with respect to the deployment of these kind of technologies actually in practice to help clinicians solve patient problems?

0:19:04
John Halamka

I think we need a William Gibson answer to that, which is the future is already here. It's just unevenly distributed. And I think in your question is a perfect observation, which is as you walk the floor at him and say, oh my, there are hundreds of these algorithms. How many are deployed in practice every day? How many are making a difference? How many, like a drug, have had a post-market surveillance study or a randomized controlled trial?

0:19:35

And so at Mayo, for example, of those 200 and some algorithms we've developed, say about 30 are used in clinical practice every single day. A number of them have gone through FDA, a software's medical device. A number have gone through randomized clinical trials. But I think your audience should know today, 2024, there's a gap between the generation of AI models and the use of AI models in workflow. And it's getting better. And I could argue that 2024 HIMSS, I started to see last year's hype being turned into a bit more deployed reality.

0:20:18
Michael Krigsman

Can you elaborate on that? Adam, I'm assuming what you mean is lots of stuff is being developed, but it's just not yet being put into practice.

0:20:27
John Halamka

Casey Ross, who is a reporter at Stat, a Boston Globe affiliate, wrote a brilliant article last year which said, healthcare AI has a credibility problem. And here's what he meant. He said, well, here is an algorithm. Well, what data was used to develop it? Oh, we're not going to tell you. Does it work on tall people? short people, rural people, urban people, we're not going to tell you. So why would a clinician want to adopt something where there is no transparency as to how the product was developed and how it works?

0:21:03

Is the product reliable? Is it consistent? Can I make sure that it does no harm to my patient? So the reason the Coalition for Health AI, which we'll talk about, was established was to create a set of nationwide standards for testing and evaluation so that just as a mattress has a little label on it that says, oh, it's new, it's got polyester, it's got cotton, don't remove this label. Every algorithm needs to come with a data card and a model card so a clinician could say, oh, I see.

0:21:37

It was developed on patients like mine. It's reasonably credible, it's consistent, it's reliable. Oh, I will use it. And therefore, oh, it will have good benefit to my patients and to me. And that's kind of a gap that without all those things as foundationally, you're not going to see quite the adoption of the new technologies because doctors won't trust.

0:21:59
Michael Krigsman

So you're saying that when it comes to data, lack of transparency and provenance of that data is a major issue here. And when it comes to the algorithms, lack of explainability, which is another way of saying transparency, is a major issue. Am I understanding this correctly?

0:22:24
John Halamka

How do these things perform in the field? And let me give you a real example. So Mayo Clinic developed an incredible tool to predict cardiac mortality. that tool works really well if your body mass index is less than 31 and not so well if your body mass index is above 35. And so what we do, because we analyze the performance of all these things, is we say, doctor, never use this on a patient with a body mass index above 31, and you can be assured it's going to be useful, will do no harm, will be credible as long as you follow that guideline.

0:23:05

And that's the kind of guidance that people need. Where do I use it? Why do I use it? What's it likely going to do? And that's really what I meant by transparency, not only in the data used, but the performance of the model that resulted.

0:23:18
Michael Krigsman

You mentioned the Coalition for Health AI with the abbreviation CHI. I'll mention every day I brew up chai tea and I grind my spices every day. I love it. So tell us about the Coalition for Health AI Chai and where that fits into addressing this set of issues.

0:23:41
John Halamka

Many of us who've been in the industry for decades know that government alone creating regulation without input may create either laws or regulations that have unintended consequences. Wouldn't it be better to have public-private partnerships so that everyone's at the table and plan these things thoughtfully? Well, to do that takes a coalition. Starting two years ago, a number of us said, we believe there needs to be nationwide guidelines, guardrails, testing and evaluation. So why don't we start bringing people together?

0:24:19

And so I was mentioning to you that in September of 2021, we brought together six organizations Then by October, there were 90 organizations. Well, as of today, there are 1 ,600 organizations, including all the big tech organizations, medical centers large and small, the FDA, ONC, NIST, and the White House, all working together on these questions. The board of directors, which we tried to make very representative, has Mikitropathy from the ONC as a director. Troy Tazbaz, the person overseeing AI regulation for the country, as a director.

0:25:02

Eric Horvitz, who chairs the President's Council of Science and Technology as a director. And then provider organizations, entrepreneurial organizations, and patient advocacy organizations. So you try to be highly representative on the board level, support that by advisory groups in each domain. So these affinity groups come together and suggest where we go. And we do this for the benefit of society and the hegemony of none. And so assume what's going to come out in 2024. Government and industry working together will have mechanisms for testing data and algorithms that will then be posted publicly in a nationwide registry.

0:25:48

So you'll say, oh, I would like to go buy this thing for cardiology. Oh, here's where it will work. Here's how it will perform. and here is an official laboratory that tested it so you can believe in the results.

0:26:01
Michael Krigsman

So you have all of these interest groups. At the end of the day, what is the product

0:26:10
John Halamka

that Chai will produce? I want an algorithm to be fair, appropriate, valid, effective, and safe. What does that mean? How do you measure fairness? How do you measure efficacy? Well, that's a question society has to answer. So what you'll see if you go to the coalition for healthai.org is the testing and evaluation framework. All this is downloadable at no charge. Defines each of the measures and metrics, processes for quality management. How do you analyze your data?

0:26:47

How do you analyze and test an algorithmic result? So that's on the one hand, you define all that, call it the what. Well, then Chai likely, I mean, it's early, will have a hand in helping pick or assure the testing laboratories for this country to make sure that they're credible, that they perform to what we'll call sort of ISO standards and to audit them to make sure they're doing a good job. So CHI won't do the testing itself, but will assign the testing to a nationwide network of laboratories that will report on these products.

0:27:23

And then CHI will host the nationwide registry, which makes all the

0:27:29
Michael Krigsman

testing results available to all. So essentially then it becomes a centralized trust mechanism.

0:27:37
John Halamka

Correct. I mean, that's a really good point, which is if we've decided as a society how to measure goodness, and then we have a database for the society that shows what is, for each product and service, its measure of goodness, that starts to build credibility.

0:27:54
Michael Krigsman

You've been talking, John, about fairness and ethics. where does that whole set of issues come into play with this imagine michael i had a wonderful

0:28:09
John Halamka

algorithm i'm gonna ask this two questions because you're gonna see why they're different what algorithm would predict the likelihood you would develop cancer five years from now And by the way, that cancer is curable. And question is, would you want to know, knowing that if we predicted the cancer, well, the algorithm maybe had an AUC of 0.9 or something, wouldn't be perfect, but it would be pretty good. And you could make a life decision. Do you take a cure?

0:28:44

Do you do something proactive surgically? You say, oh, well, that sounds okay. okay, as long as the patient decides and it's curable, oh, that sounds like a kind of good thing for society because you could avoid disease. How about this? I have another algorithm that will predict your likelihood of Alzheimer's 10 years from now and will let you know if you're going to develop an incurable disease. Oh, how do you feel about that? Do you want to know?

0:29:10

Because if I tell you it's going to create immense anxiety for which you can take no action. And so this is why I mentioned a couple of cases like this, because these are real cases. We have to decide not only how does the algorithm perform, but what human factors are involved in using it? What's the patient preference? And every human, I would argue, will have a different preference as to what they want to know and when they want to know.

0:29:36
Michael Krigsman

What about access, equitable access to care and then bias in the algorithms, which are two other dimensions of this topic?

0:29:48
John Halamka

I've worked with the Bill and Melinda Gates Foundation for the last 15 years in sub-Saharan Africa and Northern India. And each of these, every society has sort of different challenges with how one delivers care or brings technology to a patient. What Northern India decided to do, it was really a good idea. Bandwidth in Northern India is ubiquitous and cheap. Every family has an Android phone. And so we said, oh, what if a community health worker in each village would be given a set of devices like a blood pressure cuff, a pulse oximeter, a small one lead EKG device, and could work with every family in that village and exercise AI algorithms on their behalf to help the families derive

0:30:40

the benefit of some of these things? Would that work? And the Gates Foundation actually funded a lot of the community health workers to do this. And the model was great, right? Because bandwidth is ubiquitous. And that enabler, that community health worker, brought the best of digital health to the village. Sub-Saharan Africa, bandwidth is expensive and hard to find. The model has to be different. So give me an example of what we did there. Could I, on my phone, instantiate a computer vision algorithm so that when a community health worker went to the home and ran a rapid diagnostic test, could be a malaria test or something else, in effect, they're able to run the algorithm in the home by

0:31:27

using computer vision on the test in the home. Again, Gates Foundation funded that. So I think it's really important. We all agree we're going to meet the patient at the level of their technological sophistication, income level, and the infrastructure of their country. In the United States, here's one more quick story for you. An engineer came to visit me and wanted to come to the Medicaid clinics in Boston to see how they're delivering digital healthcare. So I brought this engineer to a low-income clinic.

0:32:05

the engineer went up to an 80 year old homeless gentleman and said what's your favorite wearable and he said socks and i tell you that story because as all of us cxos start to think about this don't assume everybody is going to have an iphone 27 and an apple watch and infinite bandwidth. Meet the patient at the level of their sophistication and recognize you all have to deliver digital solutions to those who have no digital technology.

0:32:39

I realize that this question

0:32:41
Michael Krigsman

now starts to drift from the center of the discussion on platforms. What you're describing sounds like a great set of ideals to meet the patient where they are. But the fact is we know that as you were just describing, resources and access to healthcare are not evenly distributed, and the economic incentives and business incentives almost militate against accomplishing

0:33:09
John Halamka

the goal you've just been describing. I have a set of KPIs at Mayo. Gianarico Ferruccia says, John, I want you to, as you build this platform, to of course help Mayo be the best practice it can be. But I want you to capture specialty knowledge. And I want you to assist in global transformation. And by the end of your presidency, I want you to have touched 4 billion people. Now, how are you going to touch 4 billion people?

0:33:45

Well, that will take partnership and collaboration at massive scale, at country scale, and the delivery of algorithms. And it will require that sometimes philanthropy is used to engage countries, especially low and middle income countries, to get either data capture or the use of algorithms. So as Mayo has gone across the world, for example, we recently worked with Kenya and we'll be deploying a variety of algorithms in Kenya. and philanthropy will fund the deployment of the algorithms in Kenya.

0:34:22

So as you said, it's a complicated question, but I have been charged with touching 4 billion people,

0:34:29
Michael Krigsman

so my incentives to do so are aligned. Chris Peterson says healthcare was one area that adopted rule-based expert systems and earlier forms of AI in the past. Does that give the industry a leg up on others that did not have this history. My postdoc was in AI at MIT in the 1990s.

0:34:56
John Halamka

And what did I do? I wrote rule-based systems. And so let me give you an example of a rule-based system. So Michael, is a giraffe a mammal? Well, we better define what's a mammal. Oh, has live young, feeds its young, certain characteristics. Does a giraffe have live young? It does. Does a giraffe feed it? Yes, a giraffe is a mammal. And I give the audience this example of literally it took me a month to create a Lisp-based system based on a set of rules that would answer questions like that.

0:35:30

So good, we have experience creating what I call knowledge graphs. And those knowledge graphs are good to be pretty useful in medicine as we use more generative AI. And I know this is probably not the answer that you expected, but here's the problem with generative AI. How do you assess its quality, its accuracy, its validity? Very hard. We actually might end up going back to the rules-based systems to look at the validity of the output of generative AI, and that's going to help us assess its utility.

0:36:02

So sure, we've got a lot of examples in healthcare of having done this for years with understanding the complexity. But still, I think we do have to be careful of saying healthcare is in a perfect place to adopt these emerging technologies because there's risk, there's a lack of rapid adoption of change in our industry, and there's regulatory constraints.

0:36:27
Michael Krigsman

You mentioned generative AI. Can you talk about the differences between predictive AI and generative AI in this platform product that you're building?

0:36:45
John Halamka

One of the algorithms we've developed is able to take your lead one EKG from your Apple Watch or a live core device, whatever it is you have in your home, and assess your ejection fraction. That is, does your heart have a good pump or not? Well, that's a predictive mathematical algorithm and we can assess, is it good or bad by simply doing a gold standard test like an echo and say, oh, the algorithm worked or didn't work.

0:37:16

So predictive is mathematical and you can measure its performance against a ground truth. Generative AI, as we know, creates novel content. In the Quest large language model context, it's predicting a next word in a sentence, and then the next word, then the next word, then the next word. So here's a problem. How do you assess that against ground truth, right? If every prompt yields a different result and at 1.41 in the afternoon, you could get a brilliant result and at 1.42, get a nonsensical fictional result, it's really hard to assess its accuracy.

0:37:54

So that's why I tend to separate the mathematical prediction, which is verifiable against the generation of new content, which is just so difficult to assess for accuracy and quality.

0:38:08
Michael Krigsman

I guess if you're diagnosing patient illness, you really do want that consistency.

0:38:15
John Halamka

Well, it's worse than that. As you've probably worked with prompt engineering, depending on how you ask the question, the response can be brilliant or devastating. And here's just this quick example. We took a New England Journal case, patients of 59-year-old, crushing substernal chest pain, hypotension, and left leg radiation. And we say, hey, Journal of AI, what's the diagnosis? Says, oh, having a heart attack. Anticoagulate immediately. We'll ask the question differently. And what diagnosis should I absolutely not miss?

0:38:51

Oh, don't miss a dissecting aortic aneurysm. And of course, the answer in this case was, dissecting aortic aneurysm, anticoagulation would have killed the patient instantly. So, wow. Depending on the nature of the question, you're going to completely give doctors good or bad advice. Got to be really careful.

0:39:12
Michael Krigsman

Arsalan Khan comes back and he says, what happens when the specialist doesn't agree with what the AI recommends? Who is responsible when AI or the specialists are wrong about the prognosis or the diagnosis? And should patients be aware that AI is being used in their care? Let me give you an

0:39:36
John Halamka

answer from CMS. I'm sure, Michael, you have read the CMS drafts non-discrimination rule, and page 175 answers that question directly. And what it says, this is a draft rule, right? So it's not completely finalized. He said AI should be an augmentation to human decision-making. It should be like the smart medical student is providing you a review of the research and suggesting that there's some things to look at or test. But a doctor should never delegate decision-making to an AI.

0:40:07

In fact, if a doctor delegates decision-making to AI, the doctor is responsible for any bad result. So the way we use AI at Mayo Clinic is we say, oh, you've got your experience, you've got the literature, you've got the community standard of care. Oh, but there's AI providing just, again, a set of data points that you put into your entire decision-making process. And absolutely, we would say, patient, we're going to go, just as we order lab tests, We're going to go order this AI test, see what it says.

0:40:42

And then together, we'll decide how to interpret that, ignore it or not.

0:40:47
Michael Krigsman

So ultimately, it is the physician takes responsibility. And AI is one, we could say, reference source to feed into the physician's judgment.

0:41:01
John Halamka

Exactly. And there are other ways we're using AI. So in the field of radiation oncology. radiation oncology for those of you who are familiar with the practice is physics and math it's figuring out how to target a tumor without affecting the nerves arteries and veins around the tumor turns out ai is really good at physics and math and so we give ct or mri images of the tumor to an algorithm and it produces an initial what's called auto contouring algorithm that is then handed to the expert radiation oncologist for review, fine-tuning, and adjustment.

0:41:41

On average, in our use of this, it used to take 16 hours for a human to develop a sophisticated head and neck tumor treatment algorithm. Today, it takes one hour. Does that mean we're going to need fewer radiation oncologists? No. It means we're going to be able to treat more patients in more geographies than ever before. And so, truly, it's just seen as a reduce the burden

0:42:07
Michael Krigsman

kind of algorithm, not replacing a human. And this is from Elizabeth Shaw, who says, patients grant permission to the Mayo Clinic to use information for research and altruistic purposes. How will these patients react when they find out that their data is being monetized?

0:42:33
John Halamka

We never sell the data. In our notice of patient privacy rights, what we say is we would like to use data collected during a care process, anonymized and aggregated, to produce new knowledge and to care for patients in the future. And at any time you can opt out. Of our 10 million patients, about 3 ,000 of 10 million have opted out. But we're very careful to never sell data, to never exfiltrate data, to keep it always under Mayo's control.

0:43:10

and what we're producing as capture of specialty knowledge in algorithms. Those algorithms we'll use internal to Mayo Clinic, and they may be used external to Mayo Clinic, but always in the interest

0:43:24
Michael Krigsman

of patient help. So, Mayo has this set of policies, and obviously other institutions, organizations are going to have their own over which you have no control, even though they may be part of the federated system that you've built? Mayo is a very values-driven organization.

0:43:47
John Halamka

And that means that it's been around for 150 years and the values are real. I mean, every person at Mayo asks, what would the patient want? And I will be very candid with you that as we evaluate partners throughout the world, we ask about values alignment. And we're quite selective and making sure that organizations philosophically are going to work in the same spirit of capturing knowledge, not monetizing data. And so, so far, so good with the patients that we've engaged so far, each of which, of course, in their local institutions have kinds of consent for the secondary use of their data for deriving new cures.

0:44:35
Michael Krigsman

Should we adjust the Hippocratic Oath to implore medical professionals to be aware of the pros and cons of using AI? And should other industries have similar oaths?

0:44:50
John Halamka

What I'm going to share with you is something that I have each of my employees sign when they begin work. It's called the Digital Hippocratic Oath. It is an entirely redesigned Hippocratic Oath around the notion that we are going to work together to protect the sanctity and the privacy of the data, to validate algorithms before their use, and to strive to do no digital harm. And you can post this. It's a quite comprehensive oath that each of my employees signs,

0:45:22
Michael Krigsman

and you could post it on LinkedIn. This issue of trust becomes central to what you're doing and really the use of data and AI for these purposes.

0:45:34
John Halamka

Well, right. And a question you need to ask all yourself, what would you want? what would your family want? And I think your answer to that would be, wow, I want to be treated with empathy and compassion. I want respect for my own care preferences. And you know, I can understand the value if a doctor like me, I went to medical school in 1980. Things have changed since then. What if I have the benefit of hundreds of millions of patient experiences and the world's literature giving me some augmentation, that's going to make me a better doctor, but only if I respect the needs and the care preferences

0:46:20
Michael Krigsman

of the patient. Earlier, you mentioned Obamacare, and you said we could talk about Llamacare, And you run the Unity Farm Sanctuary. Tell us about that. That seems disconnected to a lot of your other work, or maybe it's not.

0:46:38
John Halamka

In the patient always comes first paradigm of Mayo, Unity Farm Sanctuary is this wonderful 360 large animal community we've put together, same principles of multi-specialty care and the creature always coming first. Here's how it started. In December of 2011, my wife was diagnosed with stage 3A breast cancer. It was metastatic. She wasn't quite sure how things would go. And while she had radiation, chemotherapy, and surgery, she said, I want to leave something behind. She's doing fine today.

0:47:17

I'll cure it. So in 2012, we sold our small home in Wellesley, Massachusetts, and purchased 15 acres in a rural town called Sherbourne, Massachusetts. And everyone said, you're going to move to a socially distanced, biologically isolated community? Why would anyone want that? Now, of course, in 2020, everybody thought we were completely prescient. But in 2012, we started with small animal rescue. And this would be poultry, some alpaca. We were able to acquire the land next door and the land next door.

0:47:55

We're now 100 acres and 22 barns with 360 large animals, 1 ,000 volunteers, programs for the elderly, programs for the disabled, and a preschool so that students are able to live their early childhood in a farm and forest setting. And this was done as a community benefit 501c3. It is truly a community resource for the New England area. Every abused, abandoned, and neglected large animal comes to us. Let me give you, here it is at the afternoon.

0:48:31

Already this morning, we've been asked to bring in a pig, a goat, a calf, and a dog. All four of them were going to be euthanized today. And all of them now are brought to our facility where they will live their natural lives. Or if we can't accommodate every one of them, we will find a placement somewhere in the United States in a similar sanctuary.

0:48:53
Michael Krigsman

And for folks who want to get in touch with you at the Unity Farm Sanctuary, how can they do that?

0:48:59
John Halamka

Oh, very simple. Just go to unityfarmsanctuary.org. You'll find all our contact information. And we're on every bit of social media you can possibly imagine.

0:49:09
Michael Krigsman

How do you accomplish all the incredible things that you do? It's unbelievable.

0:49:15
John Halamka

I think there's a couple of answers to that. And so why don't we start with, I'm an emergency physician. And so much of what I've been trained to do over these last, I'm 62. So right over these last decades is triage. what must be done now, right? And so my wife will tell you, if there's an animal in need, the animal always comes first, right? Triage. Figure out then what you can delegate. And so sometimes in my roles, I will create a vision, create a plan, turn it over to a team, monitor that team, support that team, but won't do the work directly.

0:49:53

Whereas some work I have to do directly, and this is going to sound completely ridiculous to you, Michael. Have you tried to get your tractor serviced anytime during the COVID or immediate post-COVID period? Who knew that I would be going through school for seven degrees and I would be doing hydraulics on tractors or replacing plumbing and electrical carpentry, et cetera. but yet those are some things I have to do out of necessity because there's no one else to do them so triage delegates learn continuously and how about this tolerate discomfort you're going to be tired you're going to be cold and wet it's okay and you know so far I don't know what each day will bring but I

0:50:45

try to live it to its fullest do you still play the Japanese flute I have had less time to play the shakuhachi than in previous years, because these days with 360 children, I'm doing a fair amount of animal care, fence bending, and just keeping so many moving parts in the world of AI, as we've been talking about, running so that society will better.

0:51:10
Michael Krigsman

On that note, I think given all of your work, society certainly does benefit. And I want to say thank you so much to Dr. John Halamka for being our guest today. John, thank you so much

0:51:25
John Halamka

for being here. Well, absolutely. And I know I was in one of your earlier episodes in 2014. Let's do this more often because the world is moving so fast. Happy to share my experience.

0:51:36
Michael Krigsman

Well, thank you. Look forward to the next time. And a huge thank you to everybody who listened and especially you folks who asked such excellent questions as you always do. We have incredible shows coming up. You should check out CXOTalk.com, subscribe to our newsletter, and subscribe to our YouTube channel. And we will see you again next time. Have a great day, everybody. Take care. Thank you.