Facts
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Marzyeh Ghassemi's Oxford master's work examined whether acute hypotensive episodes in intensive care could be predicted, possibly as a precursor to sepsis, and benchmarked neural networks among the methods.
“could you predict acute hypotensive episodes in the intensive care unit, maybe as a precursor to sepsis?”
0:03:07 · Marzyeh Ghassemi · Early research
Ghassemi says neural networks performed poorly in her master's project because it predated GPUs and datasets with millions of examples.
“they did very poorly, as we all know now, because that was the pre-GPU, pre millions of examples”
0:03:45 · Marzyeh Ghassemi · Neural networks
Before her master's degree, Marzyeh Ghassemi worked for Intel in Portland on predicting from health data collected by mobile platforms.
“I actually went to work for Intel in Portland for a couple of years before I started my master's.”
0:06:10 · Marzyeh Ghassemi · Career
Marzyeh Ghassemi directs a research group at MIT's CSAIL, the Computer Science and Artificial Intelligence Laboratory.
“This is the Computer Science and Artificial Intelligence Laboratory.”
0:07:12 · Arjun Manrai · MIT CSAIL
For the Hurtful Words paper, Ghassemi's student used SciBERT, a publicly available contextual language model trained on scientific abstracts.
“This is the Seibert model, and it's one that has been trained on scientific abstracts”
0:10:13 · Marzyeh Ghassemi · SciBERT
In Ghassemi's group's test, SciBERT completed a note about a belligerent patient with 'sent to prison' when the patient was described as Black, whereas a white patient was 'sent to the hospital'.
“African, African-American, or Black patient was belligerent and violent, then the model would autocomplete the note with the patient was sent to prison”
0:10:46 · Marzyeh Ghassemi · Language model bias
Ghassemi said the LaTeX source of the GPT-4 paper posted on arXiv contained a section on unprompted toxic language generation that was commented out of the published version.
“And then in the actual paper, that whole section is commented out.”
0:13:53 · Marzyeh Ghassemi · GPT-4 paper · 2023
Differential privacy protects data by adding noise to and clipping gradient updates from data points that influence the model too much.
“anytime there's a point during the machine learning process that pulls the model's weights around too much, it's influencing the gradient too much in this batch of data you selected, you noise and clip the gradient”
0:20:26 · Marzyeh Ghassemi · Differential privacy
Ghassemi says Canadian healthcare data do not collect self-reported race, so outcomes such as childbirth deaths by race cannot be verified.
“they do not collect self-reported race in the Canadian healthcare data, which makes it impossible to verify how many, for example, Black or white women die in childbirth”
0:27:12 · Marzyeh Ghassemi · Race data
In an ICML paper, Ghassemi's group showed that adding sex and age to a sleep apnea prediction model can raise overall performance while lowering it for subgroups such as older female patients.
“if you include sex and age, then you can train a logistic regression model and your overall performance will go up.”
0:28:10 · Marzyeh Ghassemi · Group attributes · 2023
Raj Manrai said medicine is moving away from race-based adjustments in physiological equations such as the estimated glomerular filtration rate for kidney function.
“there's a movement now across medicine with common physiological equations to move away from indexing on race because of reification of race as biology”
0:26:45 · Arjun Manrai · Race in equations
Ghassemi says her group tested radiologists and found they could not detect patients' self-reported race from chest X-rays, even though machine learning models could.
“Radiologists cannot detect self-reported race from chest x-rays. We tested them and they can't.”
0:36:51 · Marzyeh Ghassemi · Chest X-rays
A collaborating radiologist found old papers on X-rays of bird feathers revealing melanin levels, which the team considered as a possible explanation for race detection in X-rays.
“One about x-rays of bird feathers and how you could tell the color of the bird feather, the level of melanination in the x-ray.”
0:33:35 · Marzyeh Ghassemi · Melanin hypothesis
Ghassemi reports that models whose internal representations made patient race easier to decode had larger differences in true positives between Black and white patients.
“we found that the more obvious it was from the model who was black and white, the worse the disparity was in the model's predictions.”
0:39:52 · Marzyeh Ghassemi · Model disparity
Andy Beam said the paper on the false hope of explainable AI in healthcare arose after he and Lauren Oakden-Rayner were asked at a meeting whether explainability should be required for medical AI.
“Lauren and I had been at a meeting and had been asked about, should explainability be a requirement for medical AI?”
0:41:17 · Andrew Beam · Explainability paper
A paper from Ghassemi's group, The Road to Explainability Is Paved with Bias, found popular explanation methods made model results less fair across several settings.
“we found that when you use these state-of-the-art, very popular local and global explanation methods, they make your results less fair.”
0:43:27 · Marzyeh Ghassemi · Explainability and fairness
Ghassemi argues medicine already relies on black boxes, citing acetaminophen and lithium as drugs whose mechanisms are not known.
“like we do not know how acetaminophen works or lithium”
0:45:31 · Marzyeh Ghassemi · Medical black boxes
Ghassemi cites aviation and space research on pilots and automation as a model for studying how doctors should work with AI.
“all the research that was done in the aviation and space industry to try to figure out how pilots would best be served by having lots and lots of automated systems”
0:49:01 · Marzyeh Ghassemi · Human-machine teaming
Ghassemi says her main lesson from her year at Verily was that collaborations with large health systems are hard to extend beyond limited studies.
“collaborations with larger health systems can be very hard to extend beyond limited settings of study”
0:53:07 · Marzyeh Ghassemi · Industry collaboration
Ghassemi says datasets for machine learning speech research are privately held by companies such as Amazon, Microsoft and Google, whereas vision datasets are usually public.
“All of those data sets are private. They're owned by Amazon or Microsoft or Google.”
0:58:41 · Marzyeh Ghassemi · Data ownership
Ghassemi says health institutions that cite patient privacy for withholding data can legally sell HIPAA de-identified data to companies.
“they will de-identify that data, so it is legal, HIPAA de-identified, to sell it to a company.”
1:00:22 · Marzyeh Ghassemi · Health data
Ghassemi says the undergraduate machine learning class at MIT enrolls 400 to 500 students each semester.
“The undergraduate machine learning class at MIT gets 400, 500 students per semester.”
1:09:03 · Marzyeh Ghassemi · ML education