OpenNashPodcasts

Person

Saleema Amershi

Saleema Amershi is a researcher at Microsoft Research who works where human-computer interaction meets artificial intelligence, and she appeared on the Microsoft Research Podcast to discuss responsible AI. She described building tools and techniques for people who create and use AI systems, and her path from intelligent tutoring systems to machine learning.

Amershi said she deliberately calls responsible-AI problems "failures" because calling them issues tends to lower their priority. She also argued that humans and AI have different regions of error, so they can complement each other. She recalled that the Guidelines for Human-AI Interaction began with about 200 pieces of guidance, and she said AI policy should be driven jointly by industry and government agencies.

1 episodes · 20 quoted claims

Saleema Amershi works at the intersection of human-computer interaction and artificial intelligence, creating tools and techniques for people who build and use AI systems.

“I work at the intersection of human-computer interaction and artificial intelligence. And so I create technologies and techniques to help people both build and use AI systems.”

Microsoft Research Podcast · 0:01:57

Saleema Amershi earned her PhD at the University of Washington, near Microsoft's Redmond campus, which made it easy to collaborate with Microsoft Research.

“I did my PhD at the University of Washington, which is really just across the lake from Redmond here. And so that made it really easy to collaborate with MSR.”

Microsoft Research Podcast · 0:31:10

Saleema Amershi completed three internships at Microsoft Research before joining the lab.

“And so I ended up doing three internships here at MSR. I just kept coming back.”

Microsoft Research Podcast · 0:31:10

Saleema Amershi attended the University of British Columbia for her undergraduate and master's degrees and began as a math major.

“I went to UBC for undergrad and my master's. And I actually started off as a math major.”

Microsoft Research Podcast · 0:32:12

Saleema Amershi worked on intelligent tutoring systems at the Laboratory for Computational Intelligence at the University of British Columbia.

“And then I started working at the Laboratory for Computational Intelligence at UBC, working on intelligent tutoring systems.”

Microsoft Research Podcast · 0:32:41

Saleema Amershi says early intelligent tutoring systems used giant belief networks hand-tuned by experts for a single system, which were powerful but not scalable, and this pushed her toward machine learning.

“you would create like these giant belief networks that were hand tuned by experts for just one system. And that was like, yes, it was powerful, but it was not scalable.”

Microsoft Research Podcast · 0:33:34

Saleema Amershi says she was the first human-computer interaction hire in the machine learning group at Microsoft Research.

“I was the first like HCI hire in the machine learning group.”

Microsoft Research Podcast · 0:19:54

Saleema Amershi explains that an algorithm is tuned to optimize a metric such as precision or recall, which affects how many false positives and false negatives it produces.

“by design an algorithm will be tuned to optimize for some metric so maybe you try to optimize for precision or recall which really like affects the amount of false positives and false negative errors you can have.”

Microsoft Research Podcast · 0:07:32

Saleema Amershi argues that in a facial-recognition access-control system, false positives are costlier than false negatives because someone else could access your account.

“in the facial recognition scenario, false positives are much worse than false negatives, right? If somebody accesses your account, that can be much more costly than if you have trouble accessing it yourself.”

Microsoft Research Podcast · 0:08:38

Saleema Amershi gives the example of a doctor who overestimates a clinical AI system's abilities and over-relies on it, which can cause harm.

“if a doctor is relying on a system to make clinical recommendations about their patients, if they think that the system is smarter than it actually is, they may over rely on it and that can result in harms.”

Microsoft Research Podcast · 0:10:27

Saleema Amershi says that when an AI hiring system is biased against recommending women for engineering jobs, many people respond that this is just reality.

“maybe your system is biased against recommending women for engineering positions. If you ask people about this fairness and bias issue, a lot of people will come and say, you know, well, this is just reality.”

Microsoft Research Podcast · 0:12:58

Saleema Amershi argues that humans and AI systems have different regions of error and uncertainty, so ideally they can complement each other.

“I think we should think about these humans and machines as having different error regions, right? So yes, each of these systems will have uncertainty, but they'll be different.”

Microsoft Research Podcast · 0:14:26

Saleema Amershi says that if a system must explain itself to users, that need dictates the choice of an interpretable model.

“If you need a system that can provide an explanation to a user so that they can make an appropriate decision, you're going to need an interpretable model.”

Microsoft Research Podcast · 0:16:13

Saleema Amershi deliberately uses the word failures for responsible-AI problems because calling them issues tends to lower their priority.

“I use the word failures really intentionally because we used to call responsible AI issues, you know, issues or problems, but just that term will put it lower in the priority stack, right?”

Microsoft Research Podcast · 0:20:47

Saleema Amershi says roughly 20 years of research advances on developing AI systems effectively motivated the creation of the human-AI interaction guidelines.

“there's actually been a lot of research advances over the last like 20 years around how to develop these types of AI systems effectively.”

Microsoft Research Podcast · 0:22:40

Saleema Amershi recalls that the team behind the Guidelines for Human-AI Interaction began with about 200 pieces of guidance collected from many sources.

“We collected guidance and best practice recommendations from a wide variety of sources. I think we had like 200 to begin with.”

Microsoft Research Podcast · 0:23:39

The team grouped and revised the collected guidance and tested it with real practitioners to see whether people could detect it in interfaces.

“And then we iteratively grouped them, revised them, tested them with real practitioners to understand if they were really something that people could detect and notice in interfaces.”

Microsoft Research Podcast · 0:24:08

The human-AI guidelines and related studies came out of Microsoft's Human-AI Interaction and Collaboration working group, called the HAKE working group.

“the guidelines work, all came out of this human AI interaction and collaboration working group, which we call the Hake working group.”

Microsoft Research Podcast · 0:26:01

Saleema Amershi says Wizard of Oz prototyping, a common early-testing technique, is hard to use for AI systems because their varied behavior is hard to mimic.

“prototyping techniques like Wizard of Oz techniques, for example, that's often used to do early prototyping and testing. But that's really hard with AI systems because it's hard to mimic the different ways an AI system will behave.”

Microsoft Research Podcast · 0:27:00

Saleema Amershi says AI policy is needed and should be driven jointly by industry and government agencies.

“I think we need policies around this, but that needs to be driven both by industries and government agencies working together, right?”

Microsoft Research Podcast · 0:29:40