Person
1 episodes · 24 quoted claims
Cassie Kozyrkov said neuroscientists do not know the detailed mechanism by which humans make decisions, even though they can study the brain regions involved.
“to summarize this for you, John, neuroscientists have no idea how we make decisions. So that's that's the funny conclusion, right?”
Kozyrkov said decision analysts commonly define a decision as an irrevocable allocation of resources.
“they say that it is an irrevocable allocation of resources”
Kozyrkov said psychology's field of judgment and decision making separates judgment, which is deciding how to decide, from the decision making that follows.
“Judgment is separate from decision making. Judgment is where you do all the effort of deciding how to decide.”
Kozyrkov argued that the math, data and code in a decision are a thin objective layer surrounded by subjective choices about what matters.
“that little, uh, math piece and that data piece and that code piece is a thin layer of objectivity in a big fat subjectivity sandwich.”
Kozyrkov said that before AI chatbots, data science teams might take a week to six months to answer a leader's question.
“by the time that that team has come back with an answer, it's been well, a week if you're lucky, but you know, could be six weeks, could be six months.”
Kozyrkov said a top leader's main job is delegating decisions, including deciding which people or automated systems to trust.
“In fact, most of the job is figuring out how to delegate decision making to everybody else, choosing whom to trust or what to trust.”
Kozyrkov said that by giving an AI system details about her sore feet, she discovered she tied her shoelaces too tight.
“And then I've even improved silly things like I discovered that I, uh, tie my shoelaces too tight. I had no idea.”
Kozyrkov described how an AI system could be run about a hundred times behind the scenes to measure how often each answer recurs and show a confidence score.
“in the back end, you could run this, let's say, a hundred times, and you discover that ninety nine out of a hundred times the answer comes back with a four in it”
Kozyrkov used Australia as an example, noting that its capital is Canberra, not Sydney or Melbourne, to show an AI could confidently repeat a common wrong answer.
“For those for whom that's a surprise, it's Canberra is the right answer.”
Kozyrkov said psychology research on the fluency heuristic finds that fluent, hesitation-free speakers are perceived as more trustworthy, and machines can exploit this.
“there is this concept of the fluency heuristic. So a system or a person that appears more fluent, less hesitation, less uncertainty, is perceived to be more trustworthy.”
Kozyrkov claimed that AI agents such as Operator, Claude computer use and Project Mariner are deliberately run below their potential because running them well is expensive.
“Agents like Operator, the Claude Computer Use, Project Mariner, all these things, they are underperforming relative to where they could be performing on purpose because it is expensive to run them well.”
Kozyrkov said she majored in economics at the University of Chicago after her father suggested economics when she was about thirteen.
“that's how I found myself in economics classes and ended up majoring in economics at the University of Chicago.”
Kozyrkov said both of her parents were physicists and that she rebelled against studying physics.
“it's not always true that what your parents put in there makes its way out, of course, cause both of my parents were physicists.”
Kozyrkov said she went to Duke University for graduate study in neuroeconomics, a field she described as how the brain implements decision making.
“I actually went to Duke for neuro neuroeconomics, that was the the field neuroeconomics, and, uh, that is, you know, you know, you get macroeconomics and microeconomics.”
Kozyrkov said her second graduate program was in mathematical statistics, where she did what was then called machine learning.
“my second one was for mathematical statistics. My undergraduate work was econ and statistics, so then I I went to math stat”
Kozyrkov defined statistics as the discipline of changing your mind under uncertainty.
“Statistics, as I realized as I began to soak in the philosophy, is the discipline of changing your mind under uncertainty.”
Kozyrkov said her University of Chicago undergraduate advisor doubted she could find a job with her mix of studies.
“my um undergraduate, um college advisor said, I have no idea what job you think you're gonna get with all this stuff.”
Kozyrkov said that around age eight or nine she collected gemstones to add rows to a spreadsheet, which sparked her interest in data.
“So I had this gemstone collection. Its entire purpose is to give me another row for my spreadsheet.”
Kozyrkov said her PhD advisor at North Carolina State accepted a job at Berkeley where he could not bring his students.
“my advisor, this was at North Carolina State, my advisor had just taken an offer at Berkeley, where he could not bring any of his students along with him.”
Kozyrkov said her first application to Google was for a summer internship, after a Googler gave a talk at her department.
“the application was just for a summer internship, which seemed like a harmless thing to do”
Kozyrkov said her Google internship title was decision support intern.
“the formal internship name was decision support intern, and I thought to myself, we'll figure out the support and we'll figure out the intern, but decision”
Kozyrkov described the Google decision support team she interned with as a statistics-focused SWAT team for data-driven decisions, close to Google's primary revenue.
“The team that I was in was is sort of a a SWAT team for data driven decision making. They were very, very close to the revenue, to Google's primary revenue”
Kozyrkov said that after a weekend in New York she asked a Google recruiter for a role in the New York office instead of Mountain View.
“I called the recruiter and I said, whoa, whoa, whoa, whoa, can I get a role in New York City instead?”
Kozyrkov said she built an early statistical consulting service at Google by persuading statisticians to offer their 20% time to other teams' projects.
“I built early statistical consulting because that wasn't there like you, you couldn't just go ask a statistician who'd sit down with you”