OpenNashPodcasts

Person

Daniel Kahneman

This page collects what Daniel Kahneman said on the Masters in Business podcast, where he discussed his book Noise, which he wrote with Olivier Sibony and Cass Sunstein, and how organizations can reduce inconsistent judgments. Each fact comes with his exact quote and a link to the moment.

Kahneman told the podcast that an insurer's noise audit found underwriters differed by an average of 55%, against the 10% executives expected. He also described decision hygiene, a set of general procedures for reducing errors, and talked about his work with Amos Tversky and his return to studying well-being.

1 episodes · 24 quoted claims

Kahneman defines judgment noise as variability among judgments of the same object that should be identical.

“we speak about judgment noise, and this is a situation in which judgments should be identical.”

Masters in Business · 0:03:45

Kahneman says the idea for the book came from a noise audit he ran while consulting at an insurance company seven or eight years before the interview.

“So I was consulting in an insurance company seven, eight years ago. and I had the idea of running what today we would call a noise audit”

Masters in Business · 0:06:25

Executives surveyed by Kahneman's team typically said a 10% difference between two people's judgments of a quantity would be reasonable.

“and 10 % seems to be what we expect a reasonable difference to be, which is tolerable when two people make judgments of a quantity.”

Masters in Business · 0:07:12

In the insurer's noise audit, the average difference between pairs of underwriters was 55%, more than five times the 10% executives expected.

“Now, the correct answer among the underwriters in that company was 55%, more than five times as much as expected.”

Masters in Business · 0:07:57

Executives estimated that a 15% pricing error by underwriters could cost a very large insurer hundreds of millions or billions of dollars.

“but people estimated on that basis that this could be in the hundreds of millions or billions of dollars this is a very large company”

Masters in Business · 0:10:07

Kahneman says averaging two underwriters' ratings of the same risk is usually more precise than either individual judgment.

“if you have two separate underwriters estimating the same risk and you average their ratings, then the average will be usually more precise than the individual judgments.”

Masters in Business · 0:12:00

Kahneman describes an admissions committee that originally collected independent judgments and averaged them, before switching to speaking in sequence.

“The admissions committee used to operate in what we consider the correct manner. And that is, everybody would individually make their judgments, and then they would reveal all judgments together and average them.”

Masters in Business · 0:15:32

Kahneman defines naive realism as the assumption that one sees the world correctly, which stops people from noticing noise.

“we think we're right. We think we have the right view of situation. We think we understand things correctly. In short, we see the world as the world is. That's naive realism.”

Masters in Business · 0:19:11

Kahneman says bail judges receive one-sided feedback because they learn about crimes by released defendants but not about those held in custody.

“So a bail judge may get feedback on somebody who was released and committed a crime, but a bail judge will never know if somebody who was incarcerated would have committed a crime.”

Masters in Business · 0:21:18

Kahneman says well-run airlines and air traffic systems track near misses closely so they can learn without accidents.

“in well-run airlines and air traffic systems, keep track very closely of near misses because those are their opportunities to learn without tragedies.”

Masters in Business · 0:22:35

Kahneman says COVID lockdowns led him and co-authors Olivier Sibony and Cass Sunstein to write Noise through daily video calls, which proved more productive than in-person visits.

“Zooming one or two hours a day turned out to be a much better way of writing the book, and this is what happened.”

Masters in Business · 0:24:22

Kahneman says structured interviews, which gather information systematically about different characteristics, hire better than unstructured interviews.

“A much better way of doing it is what is called a structured interview, a structured process, where you accumulate information systematically about different characteristics of the person.”

Masters in Business · 0:28:13

Kahneman recommends that several people form independent impressions of a candidate, average them and then discuss the average.

“And better yet is having several people do the hiring, each of them forming an independent impression, and then they average, and then they discuss the average.”

Masters in Business · 0:28:13

Kahneman and his co-authors introduce decision hygiene, a set of general procedures to reduce errors without identifying specific biases.

“we introduce a concept that we call decision hygiene. And, you know, the word is not particularly appealing, but it's intended to bring to mind the image of washing your hands.”

Masters in Business · 0:30:17

Kahneman says a key principle of decision hygiene is to delay intuition until information has been gathered systematically.

“That is a key principle of decision hygiene is not to avoid intuition altogether, but to delay it.”

Masters in Business · 0:31:59

Kahneman cites evidence that judges sentence more severely on hot days and after their football team has lost.

“judges who pass sentences on criminals are more severe on hot days and they They're more severe if their football team lost the game last Sunday.”

Masters in Business · 0:34:36

The book identifies three sources of noise: differences in average level, pattern noise from personal taste, and occasion noise within one person.

“Those are the three are differences in average level for a judge's severity, differences in taste, what we call pattern noise, and within subject, within person variability.”

Masters in Business · 0:37:17

Kahneman says the biggest source of noise, which surprised the authors, is pattern noise, where individuals have distinct consistent tastes in cases.

“The biggest source of noise is that people really don't see the world in the same way.”

Masters in Business · 0:35:43

Kahneman says averaging multiple judgments of a case is the surefire way to eliminate noise, though it does nothing to reduce bias.

“The only way, by the way, to get rid of noise, and that's really quite important, is average judgments.”

Masters in Business · 0:38:08

Kahneman says precise rules tend to eliminate noise, while vague standards reduce it only partly.

“And rules, if followed, they're like computations. A computation is a rule, and rules tend to eliminate noise.”

Masters in Business · 0:40:20

Kahneman recommends that organizations where several people make the same judgment without good feedback run a noise audit by giving them the same problem.

“if you're in an organization where you have multiple people making the same judgment and no very good feedback, conduct what we call a noise audit.”

Masters in Business · 0:44:04

Kahneman says he has returned to studying well-being, a topic he worked on about 20 years earlier.

“Almost by accident, I'm back studying well-being, and I'm involved in several research projects.”

Masters in Business · 0:44:45

Kahneman says he and Amos Tversky worked closely together for about 12 years, spending many hours a day together.

“amos and i worked very closely together for about 12 years and we spent many hours a day together”

Masters in Business · 0:48:11

Kahneman says a new edition of Nudge by Richard Thaler and Cass Sunstein, titled the final edition, was due in August, and that the original appeared around 2008.

“I'm reading the new edition of Nudge, which is coming out, I think, in August. And it's called Nudge, the final edition by Dick Thaler and Cass Sunstein”

Masters in Business · 0:49:15