# Daniel Kahneman on Behavioral Economics (Podcast)

- **Show:** Masters in Business
- **Published:** 14 May 2021
- **Source:** https://omny.fm/shows/masters-in-business/daniel-kahneman-on-behavioral-economics-podcast
- **Transcript:** https://podcasts.opennash.com/shows/masters-in-business/daniel-kahneman-on-behavioral-economics-podcast-c1852e/
- **Provider:** OpenNash Podcasts (machine-generated)

## In brief

Daniel Kahneman argues that organizations suffer from two kinds of judgment error: bias, which pushes everyone the same way, and noise, which is unwanted disagreement among people who should agree. Noise is larger than leaders expect, wherever human judgment is involved, from insurance and courts to hiring. It stays hidden, and it can be reduced with structured processes.

The tension is between how reasonable disagreement feels and how costly it is. Experts expected underwriters to differ by about 10% but found 55%, and confidence grows even without feedback. Groups often hide noise by letting people influence each other. Kahneman's remedy is to measure noise with a noise audit, gather independent judgments, average them, and delay intuition.

## Key insights

1. **Insurance underwriters disagreed five times more than expected** Executives expected two underwriters to differ by about 10% on the same case, but the audit found 55%. Kahneman says executives guessed this could cost hundreds of millions or billions of dollars at a large insurer.
2. **Errors cancel only when judging the same thing** Averaging two ratings of one risk improves accuracy, but overpricing one policy and underpricing another leaves two mistakes. Kahneman says a good average can hide poor individual performance.
3. **Letting people influence each other hides noise** An admissions committee had officers speak in sequence because open disagreement was uncomfortable. Judgments stopped being independent, so the noise was concealed rather than removed, and a CEO speaking first makes the others' agreement uninformative.
4. **Pattern noise is the biggest source of noise** Kahneman names three sources: differences in average severity, differences in personal taste (pattern noise), and mood or occasion. Pattern noise is the largest, because each judge reacts to cases in a consistent but personal way.
5. **Decision hygiene delays intuition rather than banning it** Break a decision into parts, assess each factually, build a profile and only then consult intuition, much like a structured interview. Kahneman says it works against errors without knowing which specific biases are at play.

## Questions this episode answers

### What does 'noise' mean in decision-making?

Kahneman defines judgment noise as disagreement between people, or the same person at different times, when their judgments should be identical. Examples include sentencing, pricing insurance or diagnosing illness.

### How is noise different from bias?

Bias is the average error, with everyone leaning the same way. Noise is scatter, with some people guessing too high and others too low, and Kahneman says you can have zero bias and still perform poorly.

### What is a noise audit and why would a company run one?

A noise audit gives several employees the same realistic cases and compares their answers. Kahneman says he first ran one at an insurer, found 55% disagreement among underwriters, and predicts most organizations will find more noise than they expect.

### How can a business reduce noise in hiring?

Kahneman recommends structured interviews that collect information systematically about different traits. Better still, several people form independent impressions, average them and only then discuss, which he says Google does.

### Why do experts stay confident when their judgments are inconsistent?

Kahneman says people suffer from naive realism, assuming they see the world correctly, and often get little feedback on whether they were right. Underwriters become faster and more confident because they agree with themselves.

## Facts

- **Barry Ritholtz** [0:02:25]: "He was awarded the 2002 Nobel Memorial Prize in Economic Sciences, which he shared with Vernon Smith for his empirical findings, the work he did with Amos Tversky."
- **Barry Ritholtz** [0:02:25]: "His previous book, Thinking Fast and Slow, was the bestseller of 2011 and won a variety of different awards"
- **Daniel Kahneman** [0:03:45]: "we speak about judgment noise, and this is a situation in which judgments should be identical."
- **Daniel Kahneman** [0:06:25]: "So I was consulting in an insurance company seven, eight years ago. and I had the idea of running what today we would call a noise audit"
- **Daniel Kahneman** [0:07:12]: "and 10 % seems to be what we expect a reasonable difference to be, which is tolerable when two people make judgments of a quantity."
- **Daniel Kahneman** [0:07:57]: "Now, the correct answer among the underwriters in that company was 55%, more than five times as much as expected."
- **Daniel Kahneman** [0:10:07]: "but people estimated on that basis that this could be in the hundreds of millions or billions of dollars this is a very large company"
- **Daniel Kahneman** [0:12:00]: "if you have two separate underwriters estimating the same risk and you average their ratings, then the average will be usually more precise than the individual judgments."
- **Daniel Kahneman** [0:15:32]: "The admissions committee used to operate in what we consider the correct manner. And that is, everybody would individually make their judgments, and then they would reveal all judgments together and average them."
- **Daniel Kahneman** [0:19:11]: "we think we're right. We think we have the right view of situation. We think we understand things correctly. In short, we see the world as the world is. That's naive realism."
- **Daniel Kahneman** [0:21:18]: "So a bail judge may get feedback on somebody who was released and committed a crime, but a bail judge will never know if somebody who was incarcerated would have committed a crime."
- **Daniel Kahneman** [0:22:35]: "in well-run airlines and air traffic systems, keep track very closely of near misses because those are their opportunities to learn without tragedies."
- **Daniel Kahneman** [0:24:22]: "Zooming one or two hours a day turned out to be a much better way of writing the book, and this is what happened."
- **Daniel Kahneman** [0:28:13]: "A much better way of doing it is what is called a structured interview, a structured process, where you accumulate information systematically about different characteristics of the person."
- **Daniel Kahneman** [0:28:13]: "And better yet is having several people do the hiring, each of them forming an independent impression, and then they average, and then they discuss the average."
- **Daniel Kahneman** [0:30:17]: "we introduce a concept that we call decision hygiene. And, you know, the word is not particularly appealing, but it's intended to bring to mind the image of washing your hands."
- **Daniel Kahneman** [0:31:59]: "That is a key principle of decision hygiene is not to avoid intuition altogether, but to delay it."
- **Daniel Kahneman** [0:34:36]: "judges who pass sentences on criminals are more severe on hot days and they They're more severe if their football team lost the game last Sunday."
- **Daniel Kahneman** [0:37:17]: "Those are the three are differences in average level for a judge's severity, differences in taste, what we call pattern noise, and within subject, within person variability."
- **Daniel Kahneman** [0:35:43]: "The biggest source of noise is that people really don't see the world in the same way."
- **Daniel Kahneman** [0:38:08]: "The only way, by the way, to get rid of noise, and that's really quite important, is average judgments."
- **Daniel Kahneman** [0:40:20]: "And rules, if followed, they're like computations. A computation is a rule, and rules tend to eliminate noise."
- **Daniel Kahneman** [0:44:04]: "if you're in an organization where you have multiple people making the same judgment and no very good feedback, conduct what we call a noise audit."
- **Daniel Kahneman** [0:44:45]: "Almost by accident, I'm back studying well-being, and I'm involved in several research projects."
- **Daniel Kahneman** [0:48:11]: "amos and i worked very closely together for about 12 years and we spent many hours a day together"
- **Daniel Kahneman** [0:49:15]: "I'm reading the new edition of Nudge, which is coming out, I think, in August. And it's called Nudge, the final edition by Dick Thaler and Cass Sunstein"

## Transcript

**[0:00:00] Speaker 3:** At KKR, we've spent 50 years understanding what matters to our clients and companies. From pioneering private equity to investing across credit, real assets, insurance, and sports, KKR has built the experience to navigate complexity, the judgment to recognize opportunity, and the conviction to create lasting value. Because when people and institutions are counting on us, understanding what matters isn't just important, it's everything. KKR, Understanding what matters. Visit kkr.com slash what matters. Running a business means keeping pace with the latest tech.

**[0:00:34] Barry Ritholtz:** You've put a lot into building your enterprise software stack and scrapping it to integrate AI is out of the question. That's why IBM technology works with all the apps, data, and models you need. It's another way IBM helps you build on the technology you rely on. Let's create smarter business.

**[0:00:53] Speaker 3:** IBM. Visit IBM.com to learn more.

**[0:01:01] Barry Ritholtz:** This is Masters in Business with Barry Ritholtz on Bloomberg Radio. This week on the podcast, what can I say? Another extra special guest, Danny Kahneman, Nobel Prize winner, author of Thinking Fast and Slow. His new book is Noise, A Flaw in Human Judgment. And Danny is just so knowledgeable. Please call me Danny. I feel like I have to call him Professor Kahneman. And he insists. He's 87 years old and incredibly sharp and insightful and just so much wisdom and knowledge.

**[0:01:41] Barry Ritholtz:** If you liked Thinking Fast and Slow, which is about judgment error in humans, in individuals, well, noise is about how flaws in judgment within broader institutions come about. And it's a totally different area, and it's absolutely fascinating. I'm a big fan of behavioral finance in general, plus all of Danny's work. Historically, if you are remotely interested in this, then strap yourself in. This is another doozy. With no further ado, my conversation with Danny Kahneman.

**[0:02:19] Speaker 3:** This is Masters in Business with Barry Ritholtz on Bloomberg Radio.

**[0:02:25] Barry Ritholtz:** My extra special guest this week is Danny Kahneman. He was awarded the 2002 Nobel Memorial Prize in Economic Sciences, which he shared with Vernon Smith for his empirical findings, the work he did with Amos Tversky. And what's so fascinating about that Nobel Prize is that Danny is a psychologist. The work they did challenged the prevailing thoughts in economic theory by establishing a basis for common human errors. His previous book, Thinking Fast and Slow, was the bestseller of 2011 and won a variety of different awards, including the National Academy's Communication Award for Best Creative Work.

**[0:03:11] Barry Ritholtz:** His latest book is just out, Noise, A Flaw in Human Judgment, which Danny Kahneman wrote with Olivier Sibony and Cass Sunstein. Danny Kahneman, welcome back to Bloomberg.

**[0:03:26] Daniel Kahneman:** I'm delighted to be here.

**[0:03:28] Barry Ritholtz:** You always say, call me Danny, and I always feel awkward, and I feel like I should call you professor, but let me just get that out of it.

**[0:03:35] Daniel Kahneman:** I insist. Call me Danny.

**[0:03:38] Barry Ritholtz:** All right, Danny. So let's start very basically. What is noise? How does it happen? And where does it come from?

**[0:03:45] Daniel Kahneman:** Okay, well, the term noise is an accepted term in statistics. We talk about statistical noise, which is variability, and that's where it comes from. We talk about noise of measurement, which is unreliability in measurement, where measurements that should be identical turn out to vary. So that's the background in the use of the term. As we use it specifically, we speak about judgment noise, and this is a situation in which judgments should be identical. people or the same individual judging the same object at different times or different people, judging the same object.

**[0:04:32] Daniel Kahneman:** If they don't agree and are expected to agree, we speak about judgment noise. And in general, people are expected to agree when they're trying to be accurate. So when you have a group of people trying to make their best guess about a quantity, it could be the sentence that somebody should get for a crime. It could be the value of a company. It could be the premium that somebody should be charged, or it could be a diagnosis, a medical diagnosis.

**[0:05:04] Daniel Kahneman:** In all these cases, you might have several people looking at the same information, making judgments. If they don't agree, there is noise. And noise is the topic of the book we wrote.

**[0:05:17] Barry Ritholtz:** So it's fascinating how we start to see noisy decision-making come up over and over again in the same fields. And you just mentioned a few, medicine, criminal justice, finance. Are there certain fields that are more susceptible to problems in expert judgment than others? Or is it just that the results of those sort of noisy decisions are so much more significant than other fields?

**[0:05:47] Daniel Kahneman:** Well, we use the word judgment when there is room for reasonable disagreement. That is, you know, we don't use the word judgment for computation. And when computation is appropriate, we wouldn't be talking of noise. We would be talking of people making mistakes. And we talk about noise when it's a matter of judgment. And so the existence of noise by itself is not a surprise. What is a surprise is the amount of noise. There's just a lot more than would be expected.

**[0:06:25] Daniel Kahneman:** And here, I think the best way to explain this is to tell you the story of how I started to work on noise and where the whole thing began. So I was consulting in an insurance company seven, eight years ago. and I had the idea of running what today we would call a noise audit that is underwriters to take one example we had several underwriters saw some realistic cases the same cases they were constructed by executives of experts in underwriting so they were completely realistic and you might have 50 underwriters looking at the same premium.

**[0:07:12] Daniel Kahneman:** Now, nobody would expect the numbers to be exactly the same, but I asked executives, if you take a pair of underwriters at random, by how much would you expect them to differ in percentages? That is, you take the average of the pair, you take the difference, you divide the difference by the average, what percentage looks reasonable to you? And their answer typically was 10%. And we have, by the way, we have surveyed hundreds of executives since then, and 10 % seems to be what we expect a reasonable difference to be, which is tolerable when two people make judgments of a quantity.

**[0:07:57] Daniel Kahneman:** Now, the correct answer among the underwriters in that company was 55%, more than five times as much as expected. That's the phenomenon. So we expect disagreement where judgment is involved. We just don't expect that much disagreement. And this basically was the observation that started us on that path of writing a book, Because it turns out that you find astonishing amount of disagreement when you look for it. And you find it wherever judgment is involved. So engineers who make estimates on the basis of objective data, they don't have a problem of judgment.

**[0:08:40] Daniel Kahneman:** But to the extent they do have a problem of judgment, you will expect a lot of noise. So that's the basic finding. And, you know, wherever precision is important, wherever it is important to get to the right number, noise is a source of error. So if you have some people overestimating and others underestimating, they're making errors.

**[0:09:02] Barry Ritholtz:** So let me roll back to that insurance company, which you discuss in the book. And there were two particular areas where there were these broad disagreements. The first was when people were trying to estimate the risk involved with some insurance. And so how you price that very much determines if you're too expensive, meaning you think it's a high risk, you're not going to win the business. And if it's too cheap relative to the risk, well, you'll win the business, but it won't be profitable.

**[0:09:40] Barry Ritholtz:** The cost will be higher. And then on the other end, in the appraisal of, hey, what are the damages here, figuring out how much something should be covered by insurance, what the dollar amount is, and the same situation. You can't be too stingy or you lose customers, but you can't be too generous or you give the house away. How significant a financial issue was this for the insurance company?

**[0:10:07] Daniel Kahneman:** Well, you know, it's not easy to estimate that exactly, but I can tell you the question that I asked some executives. I said, suppose there is a correct number, say, for the underwriters, and you have somebody who overestimates the underwriting cost by 15%. How much would you expect that to cost the company? and the same question for underestimating by 15% now in fact 15% on either side is much less than noise than we had discovered but people estimated on that basis that this could be in the hundreds of millions or billions of dollars this is a very large company so So underwriters have a lot of important decisions to make.

**[0:11:01] Daniel Kahneman:** Claims adjusters make important decisions which are really consequential for the company. And errors of the magnitude that we observe are costly. The main reason that they may be less costly is that if error is present in all insurance companies, if all insurance companies are noisy, then some of the damage to each individual company will be reduced. But that's the best that we can say.

**[0:11:29] Barry Ritholtz:** Well, one would imagine the insurance company that could reduce noise would find itself at a competitive advantage. There was something you had written that really stood out to me. There's an assumption when you have noisy systems in everything from criminal justice to medicine to insurance that these errors tend to cancel out. But you found out that noisy systems have errors. Not only do they not cancel out, they tend to add up. Explain.

**[0:12:00] Daniel Kahneman:** Well, if you have two separate underwriters estimating the same risk and you average their ratings, then the average will be usually more precise than the individual judgments. because errors in measuring the same object do cancel out. But errors when you're responding to different objects do not cancel out. So if you overprice one policy and you underprice another policy, that doesn't cancel out. You've made two mistakes. And, you know, it's the same thing with two judges. if one defendant is punished too much and another defendant is punished too little.

**[0:12:49] Daniel Kahneman:** On average, you know, punishment was right. But who cares about the average? Two mistakes were made. So there is some confusion because people think about canceling out, but that happens when people evaluate or judge or measure the same thing there, and errors do cancel out.

**[0:13:09] Barry Ritholtz:** I recall a book a couple of years ago called The End of Average that looked at that exact issue and said, you know, we tend to look at these averages as if anyone is experiencing an average. But what you're really saying is, hey, if it averages out to be the right answer, it means you have a lot of long answers. That's right.

**[0:13:32] Daniel Kahneman:** Averaging out to the right answer is not a guarantee. And that is a nice example of the phenomenon we're discussing in the book, the neglect of noise. People really tend to focus on bias, which is the average error. But you can have a zero bias and a very poor performance if you have a lot of overestimates and a lot of underestimates.

**[0:13:57] Barry Ritholtz:** estimates. Quite interesting. So one of the things in the book that I was so taken by had to do with the admissions committee for a university, and they used to have all the admission officers do a blind review and get together and try and hash out who they thought would be a good fit for the school and who wouldn't. But it led to a problem, and they started having the first person who reviewed the application put their review number on the corner, like they would actually put their rating on the page and then hand it off to the second person.

**[0:14:38] Barry Ritholtz:** And you describe this as the illusion of agreements in organizations. Tell us about that.

**[0:14:46] Daniel Kahneman:** Well, you know, this is an experience that any teacher has had, for example. when you are looking at a test booklet, a student has written several essays. If you score a test booklet, you score the first question, then the second, then the third, then in general, you'll find that your grades do not vary very much. On the other hand, if you read the same test across all students and write the score at the back of the booklet so that you don't know when you read the second question where the first question was, you will often be shocked by the discrepancy between the first and the second.

**[0:15:32] Daniel Kahneman:** There is a mechanism by which people, if you gave a good grade the first time, you're going to be inclined to give the benefit of the doubt to the student if there is any ambiguous answer. Exactly the same thing happens in deliberations and in the example that we gave. The admissions committee used to operate in what we consider the correct manner. And that is, everybody would individually make their judgments, and then they would reveal all judgments together and average them.

**[0:16:07] Daniel Kahneman:** But they changed the system so that now people spoke in sequence. And the question was asked, why do you do this? This is not optimal. And they say, well, we used to do it the other way. We used to have people prepare their judgments individually, but there was so much disagreement that we stopped. And that's an example where people manage to avoid finding out how much noise there really is. Because when people are allowed to influence each other or influence themselves, in the case of a teacher reading multiple booklets, when judgments are not independent, they are less effective statistically.

**[0:16:50] Daniel Kahneman:** You just have less information. Think of the example in which the first person to talk is a CEO, and then everybody agrees. Then the agreement of other people is not informative. In fact, you have one person making the judgment. That's the extreme of abolishing, of eliminating the appearance of noise without eliminating the reality.

**[0:17:15] Speaker 3:** At KKR, we've spent 50 years understanding what matters to our clients and companies, From pioneering private equity to investing across credit, real assets, insurance, and sports, KKR has built the experience to navigate complexity, the judgment to recognize opportunity, and the conviction to create lasting value. Because when people and institutions are counting on us, understanding what matters isn't just important, it's everything. KKR, understanding what matters. Visit KKR.com slash what matters.

**[0:17:46] Barry Ritholtz:** So it sounds like groups and corporations, institutions, schools, they seem to amplify noise. Is that just the nature of bigger numbers of people working together, that they're going to create additional noise?

**[0:18:03] Daniel Kahneman:** No, not necessarily. What happens in a group, if they make their dreadments individually, is not that noise is amplified. The true noise is revealed. So suppose you had underwriters. Suppose you had multiple underwriters judging routinely every risk. Then the optimal procedure would be to have them making independent judgments and only then revealing the two judgments and averaging them. That's clearly the optimal procedure. Sure. are. And the optimal procedure reveals noise and then reduces it by averaging.

**[0:18:46] Daniel Kahneman:** But when a single individual makes a judgment, that judgment will be noisy. And when individuals are allowed to influence each other, then it's more like a single judgment than it is like having multiple judgments are the same object.

**[0:19:03] Barry Ritholtz:** So you use the phrase naive realism. What does that mean relative to noise in groups?

**[0:19:11] Daniel Kahneman:** Well, what naive realism means is a statement which most of us are most of the time, that we think we're right. We think we have the right view of situation. We think we understand things correctly. In short, we see the world as the world is. That's naive realism. And if I see the world as it is, and there are friends and colleagues looking at the same world, and I like and respect them, then naturally I assume that they see the world as I do because I see it right.

**[0:19:45] Daniel Kahneman:** And if I respect them, they see it right as well. So that's naive realism. And naive realism prevents us from becoming aware of the amount of noise that there is. We just assume noise away. We saw that very nicely among underwriters. When you interview an underwriter, what happens to them? How does an underwriter become expert in the absence of any feedback? Because they don't get any feedback from reality about their underwriting. and what happens is that they become increasingly confident and largely because they agree with themselves so when you agree with yourself a lot and you think you're right and you make judgments with increasing speed and confidence so that makes you think that you're even brighter that's naive realism

**[0:20:38] Daniel Kahneman:** allowing massive noise to occur with everybody convinced that they're doing the right thing, but in fact, they may not be doing the right thing because if they were looking at the

**[0:20:49] Barry Ritholtz:** same problem, that would be the fruit. Quite fascinating. So we become familiar with a particular area. That familiarity leads us to think that we're developing an expertise. We tend to make more snap judgments. And without any sort of feedback loop, how can we possibly We know that we're right, and yet that absence of feedback seems to strengthen people's self-confidence. Do I have that right? That's right.

**[0:21:18] Daniel Kahneman:** And think of a number of situations in which exactly this holds. There's a judge who doesn't have feedback as to whether a judgment was correct or not. A bail judge, sometimes there is feedback, but it's asymmetric. So a bail judge may get feedback on somebody who was released and committed a crime, but a bail judge will never know if somebody who was incarcerated would have committed a crime. So feedback is a massive problem, and many professionals act with minimum feedback, and yet they become confident and they feel they're experts.

**[0:21:55] Daniel Kahneman:** But in those cases, there is a high risk of noise.

**[0:21:59] Barry Ritholtz:** And a lot of that feedback seems to be only at the extreme. A bridge collapses, a plane crashes, somebody dies, someone out on bail commits a crime. What about all of the, for lack of a better word, near misses where there is a bad judgment, something happens, it's not quite as terrible as an airplane crash, and it's resolved before there's damage, but it's pretty clear the basic judgment was wrong. How does that affect a person's future judgment?

**[0:22:35] Daniel Kahneman:** Well, in situations where there are near misses, there is an opportunity to learn. And in well-run airlines and air traffic systems, keep track very closely of near misses because those are their opportunities to learn without tragedies. But in many situations, you get no feedback at all. I mean, the idea of having sensors in bridges that gives you a sensitive measurement of how much stress there is, that is fairly recent. There used to be very poor feedback on whether a bridge would collapse or not.

**[0:23:15] Daniel Kahneman:** And in many situations that professionals make judgment on, there is no feedback at all.

**[0:23:21] Barry Ritholtz:** Quite interesting. So let's talk about this book, which was a collaboration. What was it like working with those two gentlemen versus Thinking Fast and Slow, which I kind of get the sense was you sitting down and putting a lot of your previous work into a context for public consumption.

**[0:23:47] Daniel Kahneman:** Well, writing fast and slow was mostly a very lonely experience. And writing with collaborators was really a pleasure. So it was a relief to be able to count on people, to find mistakes, to correct them. And a lot of the text was actually written by Olivier and by Cass. I had a lot to do with outlining and with critiquing and with rejecting drafts, but I was spared much of the things that I'm most afraid of in writing.

**[0:24:22] Daniel Kahneman:** So it was a very good collaboration. And by the way, we benefited a lot from COVID because that forced us into quite an efficient way of collaborating. We used to visit. Olivier would come to New York from Paris, and I would visit Paris for a few days every month. And we had a very good time, but it wasn't productive. Zooming one or two hours a day turned out to be a much better way of writing the book, and this is what happened.

**[0:24:54] Barry Ritholtz:** It sounds like it was just a good excuse to get together in New York and Paris and have a little bit of fun.

**[0:24:59] Daniel Kahneman:** Well, I mean, you know, we didn't think of it as a good excuse, but it turned out that we'd waste a lot of time and a fair amount of money.

**[0:25:07] Barry Ritholtz:** So you mentioned you reviewed a lot of manuscript from Olivier and Cass and rejected stuff. You and Amos very famously would agonize over every sentence in all of your publications. You seem to have spent a lot of time writing meticulously and very thoughtfully. How has that evolved over time? Is this a little easier to sort of be the orchestrator and the editor as opposed to, you know, just agonizingly putting down every single word?

**[0:25:44] Daniel Kahneman:** No, it isn't. I mean, this is part of sort of my intellectual personality or character that I think most clearly when I find flaws in existing texts, and I'm not good at anticipating the flaws. So I see a flaw and I correct it, and then there is new text, and then I discover a new flaw. And I tend to work that way, which is infuriating to my collaborators, and waste a lot of time and effort, but that's the way I am.

**[0:26:14] Daniel Kahneman:** On the other hand, I do tend to be very critical, and most of the flaws that I find do exist. So it tends to lead to a good project in a very inefficient way.

**[0:26:25] Barry Ritholtz:** So despite that perfectionism, we all evolve over time. As you were preparing Noise, did you find any of your previous writings or research that you either disagree with or see from a different perspective or light when you were putting this book together?

**[0:26:45] Daniel Kahneman:** No, not really. I mean, in the book, we actually relied on ideas from Thinking Fast and Slow. But the book is really fundamentally different. Thinking Fast and Slow is a book about individuals. And it was a book about the average or a typical individual and how the average or typical mind works. noise is about individual differences. It's about the way that different people think differently. And so this is a really different cut about thinking. It's a different way of looking at thinking.

**[0:27:22] Daniel Kahneman:** So we did use some of the material, but noise is not a revision of thinking fast and slow. It is about a truly different topic that we didn't even touch in thinking fast and slow.

**[0:27:36] Barry Ritholtz:** It clearly, it goes in a very different direction and it looks at some systems and some organizations that I don't believe you touched on in Thinking Fast and Slow. So it's kind of interesting because we've already talked about medicine and criminal justice and finance. There was one section I was fascinated by where you discussed hiring and promotions and how, I don't want to use the word random, but how much noise is in that system and how unreliable many organizations' hiring processes are.

**[0:28:11] Barry Ritholtz:** Tell us a little bit about that.

**[0:28:13] Daniel Kahneman:** Well, it turns out that people like hiring by interviewing people and forming a general image of the individual that they are thinking of hiring. And it turns out this is not a good way of doing it. A much better way of doing it is what is called a structured interview, a structured process, where you accumulate information systematically about different characteristics of the person. That is less pleasant. It's less enjoyable, but much better. And better yet is having several people do the hiring, each of them forming an independent impression, and then they average, and then they discuss the average.

**[0:29:05] Daniel Kahneman:** And this is a procedure, for example, at Google, and it's about state of view. but many places are way short of a state-of-the-art. I should add that state-of-the-art hiring doesn't mean that you're guaranteed a perfect fit. There's so much luck in the world. There's so much uncertainty that the person that you hire may be very good but may run into difficulties with a boss who doesn't like her or something like that.

**[0:29:39] Daniel Kahneman:** And by chance alone, you can get a lot of variety. Chance, by the way, is not noise. Chance is something that happens in the real world. Noise is differences among judgments. So hiring is by and large really very poorly done. And it's very poorly done because it doesn't control noise.

**[0:30:00] Barry Ritholtz:** Quite fascinating. So the book goes over how noise affects judgment and how it introduces a variety of errors into our institutional decision-making process. What can we do to improve that process?

**[0:30:17] Daniel Kahneman:** Well, in the book, we introduce a concept that we call decision hygiene. And, you know, the word is not particularly appealing, but it's intended to bring to mind the image of washing your hands. And there is a contrast between debiasing and decision hygiene. Debiasing is like medication or like vaccination. It's specific to a particular disease. When you wash your hands, you don't know which germs you're killing. And if you're successful, you'll never know. So decision hygiene is oriented to improving decision-making and avoiding errors, specifically avoiding noise, but incidentally also avoiding bias, without knowing precisely what biases you're trying to control.

**[0:31:12] Daniel Kahneman:** And we have a variety of procedures that we think of as decision hygiene.

**[0:31:17] Barry Ritholtz:** Give us a few examples. What are some of the procedures?

**[0:31:20] Daniel Kahneman:** Well, I'll give you an example that has to do with decision-making. So suppose you are making a decision, and the first step that everyone will tell you is you have to consider your options and have the best possible set of options. But now you come to evaluate options. How do you do that? And here, actually, our advice, we have a slogan. We say options are like candidates. You should think of options in the same way that organizations are advised to operate when they hire candidates.

**[0:31:59] Daniel Kahneman:** And we were talking about that earlier. Structure the thinking. Break up each option. Look at the various aspects of it. Assess these aspects in a fact-based way, which is the equivalent of interviewing somebody about different aspects of her character or her experience. and then create a profile of all the information you have about that option and only then invoke intuition. That is a key principle of decision hygiene is not to avoid intuition altogether, but to delay it.

**[0:32:41] Daniel Kahneman:** Because intuition is way more effective if it is preceded by a period in which you accumulate information systematically. So that's an example. I have many others.

**[0:32:53] Speaker 4:** With ChatGPT, you can do more than get answers. You can get things done. When you're running a business and looking to expand, you need a clear view of the landscape before making any decisions. ChatGPT can help you prepare. Connect your Google Drive, point to relevant files and reports, and ask Chat to research competitors, industry forecasts, and market signals. It can turn the findings into a strategic deck in your team's template that you can review before presenting to your leadership team.

**[0:33:22] Speaker 4:** Growing a business is ambitious, and ChatGPT can help you do it with confidence. Get started at chatgpt.com.

**[0:33:30] Barry Ritholtz:** This is what... And there were quite a few in the book. There were some things that really surprised me about that decision-making process, how people's moods affect their decisions, even the weather affects decision-making. We are essentially different people at different times.

**[0:33:46] Daniel Kahneman:** Oh, yes. That is, there are different sources of noise that we talk about. So there are three of them, actually. But one of them is what we call within-person noise. And that is that the individual is indeed making different judgments depending on circumstances that are irrelevant. So it's true there is evidence that mood really affects the way that people think. people tend to be more creative when they're in a good mood but they tend to be also more gullible and they are more critical when they're in a bad mood so mood affects the way we think and it also affects we're more prone to see good things when we're in a good mood mood is important there is evidence

**[0:34:36] Daniel Kahneman:** that judges who pass sentences on criminals are more severe on hot days and they They're more severe if their football team lost the game last Sunday. So there are a lot of irrelevant events or circumstances that influence our judgment. This is one of the three major sources of judgment.

**[0:34:59] Barry Ritholtz:** Let's get to the other two. What are the other two sources?

**[0:35:02] Daniel Kahneman:** Well, one other, which is easy to think about, it's very easy to think about it in terms of judges. Some judges are more severe than others. So their sentences on average are more severe than the sentences of other judges. That's one aspect. And the same is true by the web underwriters. Some underwriters write large premiums on average, and other underwriters write small premiums on average. So there are differences of that kind. But it turns out that the biggest source of noise, And that came as a surprise to us.

**[0:35:43] Daniel Kahneman:** The biggest source of noise is that people really don't see the world in the same way. So that different judges have different tastes in crimes and different tastes in criminals. So somebody may be particularly severe about repeat offenders. Somebody else might be extremely lenient, say, about white-collar crime, but really upset by violence. And it turns out that there is, we call that a pattern noise. That is, each judge, each individual has a pattern of judgments, which is different from the pattern of judgments of other people.

**[0:36:29] Daniel Kahneman:** And that is the major source of noise. And people are consistent in that way. So, for example, suppose you're a judge and somebody reminds you of your daughter, whether that makes you more lenient or more severe, probably more lenient. Now, on another day, that same person would also remind you of your daughter. So this is not noisy within the individual. This is a characteristic of the individual, but no other judge shares it. And it turns out that this highly case-specific differences in attitudes that are difficult to pin down, they are noise.

**[0:37:07] Daniel Kahneman:** Judges have personalities, and judgments differ as much as personalities do.

**[0:37:13] Barry Ritholtz:** And then what is the third source of noise that you identified in the book?

**[0:37:17] Daniel Kahneman:** Those are the three are differences in average level for a judge's severity, differences in taste, what we call pattern noise, and within subject, within person variability. We call that occasion noise because on different occasions you make different judgments. And it's the sum of these three sources of noise that creates, that's the noise that we observe. So in the system, all three operate on any particular judgment.

**[0:37:50] Barry Ritholtz:** So I'm going to ask the question I was thinking about a little differently based on what you just said. What fields seem to manage reducing noise better than others? And are there any fields that are especially susceptible to noise?

**[0:38:08] Daniel Kahneman:** That's a very good question to which I do not have a very good answer. because in our work, we found noise wherever we looked for it. Indeed, our summary conclusion is wherever there is judgment, there is noise and more of it than you think. You know, this has been our conclusion. So we haven't found places that control noise very efficiently. The only way, by the way, to get rid of noise, and that's really quite important, is average judgments.

**[0:38:44] Daniel Kahneman:** Take multiple judgments of a case and average them. And this mechanically eliminates noise. If you have enough judgments, the average, it may be biased because averaging does nothing to reduce bias, but it eliminates noise. So that's a surefire way of eliminating noise is averaging multiple cases.

**[0:39:07] Barry Ritholtz:** Very interesting. Let me throw a curveball at you. If you were designing a system to introduce noise to short-circuit human judgment, what would you create to make judgment less effective, noisier?

**[0:39:25] Daniel Kahneman:** I don't think I would do things very differently from the way that they are done in many institutions now. I would let people make individual judgments without feedback. That's all that's needed. Make their individual decisions without feedback, which is a situation that's very common. And that will create a lot of noise eventually. And noise is reduced by feedback. Sometimes it's the feedback of other people. So case conferences can be arranged to some extent control noise. But, you know, you don't have to try very hard to create a lot of noise.

**[0:40:06] Daniel Kahneman:** I think the existing organizations do very little to control noise.

**[0:40:11] Barry Ritholtz:** So let's talk a little bit about ways to control noise. And you describe a difference between rules and standards. Tell us about that.

**[0:40:20] Daniel Kahneman:** Well, standards is a way of when you say, for example, that obscenity is something that you recognize. So there is a standard to avoid obscenity, but you do not define it. That's a standard. A rule is more precise than that, and it tells you specifically what you have to do. And rules, if followed, they're like computations. A computation is a rule, and rules tend to eliminate noise. Standards sometimes reduce noise, but standards do not eliminate noise because they're vague.

**[0:40:58] Barry Ritholtz:** So the seven words you can't say on television is a rule, but pornography, I know it when I see it, is a standard? Is that the difference?

**[0:41:07] Daniel Kahneman:** Precisely. Precisely.

**[0:41:08] Barry Ritholtz:** Quite interesting. So given all of the work you've done over the years, all of your research, you seem to have continually identified flaws in cognition, flaws in human judgment. Has this affected the way you view other people? Do you turn around and say, wow, this species is a terrible decision-making apparatus, or is it something less comprehensive than that?

**[0:41:38] Daniel Kahneman:** No, actually for all my career, I've been interested in intuition and intuitive thinking. And I've been interested, and that's a lecture I used to give for many years, intuitions, marvels, and flaws. Because intuition is marvelous, but it's also flawed. And it's true that I have found it more interesting to study the flaws of intuition than its marvels. And there is a lot to do to correct the flaws of intuition. But to say that this has turned me into a pessimist or that I dislike people because their minds are flawed, I think the minds are pretty marvelous.

**[0:42:20] Daniel Kahneman:** But they are certainly far from perfect.

**[0:42:23] Barry Ritholtz:** Right. So you're focusing on the small bits that we get wrong. But overall, we manage to navigate through life pretty effectively.

**[0:42:31] Daniel Kahneman:** Well, we certainly manage to navigate through life. And, you know, it would be absurd to focus on the flaws of human beings when you see what they're capable of. On the other hand, if you want to do things better, then you'd better focus on the flaws rather than on what is going well.

**[0:42:52] Barry Ritholtz:** You know, one of the things you said when we spoke last about thinking fast and slow, I asked you about your own investing process. And you said despite knowing everything that you know about human decision making, you still catch yourself making the same sort of mistakes that everybody makes. Is that still the case? Do you still feel that way?

**[0:43:20] Daniel Kahneman:** Oh, yes. I mean, you know, I've been at it for more than 50 years, and I'm really not better than I was. In general, my thinking has been, and it was true when I wrote Thinking Fast and Slow, which was focused on individuals, that the hope of improving thinking is in organizations because organizations think slowly, and they have procedures. and it's by imposing procedures, by adopting procedures, that you can improve things. And in the case of noise, we have a procedure that we recommend to get started, and that's measure noise.

**[0:44:04] Daniel Kahneman:** If you're in an organization where you have multiple people making the same judgment and no very good feedback, conduct what we call a noise audit. Give them the same problem and look at their solution. We predict that you'll find more noise than you think you will. That's our prediction. And that's a recommendation to organizations. It's not something that you can recommend to an individual.

**[0:44:33] Barry Ritholtz:** Quite interesting. I have to ask you before we get to our favorite questions, what's the next project? What's the next book look like? What is tickling your curiosity these days?

**[0:44:45] Daniel Kahneman:** Well, actually, I'm back to a topic that I was working on 20 years ago. Almost by accident, I'm back studying well-being, and I'm involved in several research projects. None of them is as big or ambitious as noise was or thinking fast and slow, but all of them are quite interesting, so I'm not bored.

**[0:45:12] Barry Ritholtz:** I can't picture you bored because you always seem to have a lot of different things going on. Let me ask my favorite questions that I ask all of our guests. And let's start with what are you doing to stay entertained during this pandemic lockdown? In addition to working on the book, what are you streaming? What are you watching on Netflix, if anything?

**[0:45:34] Daniel Kahneman:** Oh, I've been watching several series, several very good series. Let's see, the last ones. There is a political series on Netflix, Le Baron Noir, which is a French political thriller. That is very good. There is a Danish political series, Borgen, that is very good. I am now watching Babylon Berlin, about Berlin in the 1920s, which is excellent. And so I do my watching mainly when I exercise, and I exercise a fair amount. But I've seen a lot of series since, well, for the last few years.

**[0:46:16] Barry Ritholtz:** So Balm Noir was the French one. What was the Danish one?

**[0:46:20] Daniel Kahneman:** Bogen. Bogen is bridge. Actually, the Danish one, Bogen is a thriller. It's a Scandinavian thriller. There is a Danish one about a woman prime minister, and it's not Borgen, and I now block on its name, but it will be easy to find. And I really recommend it. It is superb.

**[0:46:40] Barry Ritholtz:** All right, I will check that out. So let's talk about your early mentors. Who helped to shape your career? And I guess we have to include collaborators in that as well.

**[0:46:51] Daniel Kahneman:** Well, I mean, there's been one major influence on my career, and it was Amostrosky. The collaboration with him completely changed my life, and it changed the way I do things, but it gave me a—yeah, it changed my life. And it was the best period of my life, too, professionally.

**[0:47:16] Speaker 4:** With Chachi Bt, you can do more than get answers. You can get things done. When you're running a business and looking to expand, you need a clear view of the landscape before making any decisions. ChatGPT can help you prepare. Connect your Google Drive, point to relevant files and reports, and ask chat to research competitors, industry forecasts, and market signals. It can turn the findings into a strategic deck in your team's template that you can review before presenting to your leadership team.

**[0:47:45] Speaker 4:** Growing a business is ambitious, and ChatGPT can help you do it with confidence. Get started at ChatGPT.com.

**[0:47:53] Barry Ritholtz:** The thing I recall from Michael Lewis's undoing project is that people said you guys would lock yourself into an office or a classroom, and all they would hear all day long is peals of laughter coming from within. Is that true? Is that an exaggeration? Or did you guys...

**[0:48:11] Daniel Kahneman:** No, that's really not an exaggeration. amos and i worked very closely together for about 12 years and we spent many hours a day together and he was very funny he had an excellent sense of humor and he loved laughing and in his presence also became funny so we were amusing each other and the field that we studied uh was was one that was conducive to laughter because we were looking for mistakes in our own thinking. And to trap ourselves or to see that you are tempted to make a stupid error, that is quite funny.

**[0:48:57] Daniel Kahneman:** And that's the game that we engaged in, in studying judgment and in studying decision-making, was looking for errors in our own thinking. And that was very amusing. I can imagine.

**[0:49:09] Barry Ritholtz:** So let's talk about books. What are some of your all-time favorites, and what are you reading right now?

**[0:49:15] Daniel Kahneman:** Well, I would say my all-time favorites of recent years were Sapiens. I think it's many people's favorite book by Yuval Noah Harari. I read it twice, which is something that I rarely do. And right now, well, I'm reading the new edition of Nudge, which is coming out, I think, in August. And it's called Nudge, the final edition by Dick Thaler and Cass Sunstein, the same Cass Sunstein. And it's quite different from the original Nudge, which appeared, I think, in 2008.

**[0:49:53] Daniel Kahneman:** But it had the same characteristic that Nudge had. It's wise and it's funny.

**[0:49:59] Barry Ritholtz:** Right. Dick said it's about two-thirds new, and I think that's August 4th that comes out.

**[0:50:05] Daniel Kahneman:** Yeah, that's right. So I happen to be reading that right now.

**[0:50:09] Barry Ritholtz:** August 3rd, I'm looking at a message from him. He's a very amusing person to begin with, and if you're telling me the book is funny, then I am really looking forward to reading that.

**[0:50:23] Daniel Kahneman:** Oh, the book is, you know, he just, he can't help himself. He's funny all the time. He's my best friend, my best living friend.

**[0:50:31] Barry Ritholtz:** Let me ask you this question. If a recent college graduate asked you for some advice, if he was interested in a career in either psychology or behavioral finance, what sort of advice might you give him?

**[0:50:47] Daniel Kahneman:** Well, you know, I tend to refrain from advice because I don't believe I have a crystal ball into the future. I can tell you what I would have been doing if I was starting today. The fields that are very exciting from my perspective are neuroscience, including neuroeconomics, which is the neuroscience of decision making, and artificial intelligence. I mean, in those two areas right now, there are extraordinary developments, very exciting. And so when you see that, and they're attracting massive talent, both areas.

**[0:51:28] Daniel Kahneman:** So you know that for the next decade or so, they'll be cooking. A lot is going to happen. And what happens after that, I have no idea.

**[0:51:38] Barry Ritholtz:** And our final question, what do you know about the world of psychology, behavioral finance, economics today that you wish you knew 50 or so years ago when you were first getting started?

**[0:51:52] Daniel Kahneman:** Oh, well, so much has been learned. I can't say that I wish I had known earlier, but it has been so much fun to find out over the years, both in my work and in the work of others. So I can't think of things that would have made me act differently. But all I can say to you is, oh, yes, things have really changed since I've been in that field.

**[0:52:22] Barry Ritholtz:** Huh, quite fascinating. Thank you, Danny, for being so generous with your time. We have been speaking with Danny Kahneman, whose new book, Noise, A Flaw in Human Judgment, was co-authored with Olivier Sibony and Cass Sunstein. If you enjoy this conversation, check out any of our previous 400 such interviews. You can find those at iTunes, Spotify, Google, Bloomberg.com, wherever you get your podcast each week. We love your comments, feedback, and suggestions. Write to us at mibpodcasts at Bloomberg.net.

**[0:52:57] Barry Ritholtz:** You can sign up for my daily reads at Ritholtz.com. Check out my weekly column on Bloomberg.com slash opinion. Follow me on Twitter at Ritholtz. I would be remiss if I did not thank the crack staff that helps put together this conversation each week. Tim Harrow is my audio engineer. Matiko Valbron is my project manager. Michael Boyle is my producer. Michael Batnick is my head of research. I'm Barry Ritholtz. You've been listening to Masters in Business on Bloomberg Radio.

**[0:53:35] Speaker 3:** Buy a digital pack, rip it open, and reveal a real physical card. Football, basketball, even Pokemon. Packs start at just $5. Ship it straight to your door or sell it back instantly at full market value. Max value up to $125 ,000. Download Underdog and get in on the chase. Deposit $5, get five free packs. Real rips, real cards, real cash. Only on Underdog. Unleash your dog. 18 plus terms apply. Top grails are extremely rare. This is Jana Kramer.

**[0:54:03] Speaker 3:** Chat GPT for teens is designed to help teens learn, think critically and grow their skills. Chat GPT for teens supports active learning by breaking concepts down rather than giving teens the answer. Study mode helps teens work through problems and interactive quizzes test their knowledge and provide immediate feedback. Safety protections are on by default and parental controls offer families additional options like setting study hours and quiet hours. Visit chatgpt.com slash parents to learn more.
