# How Microsoft is democratizing safe AI

- **Show:** Pioneers of AI
- **Published:** 17 Sep 2025
- **Source:** https://prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/tracking.swap.fm/track/7G0M4p2bOLP0mP50O3CD/pscrb.fm/rss/p/rss.art19.com/episodes/7ad32e1b-1422-4296-97a2-bbb90ff4e1bf.mp3?rss_browser=BAhJIhlwb2RjYXN0Lm9wZW5uYXNoLmNvbQY6BkVU--4dcea1dd70768d62b2466884c2f8513b5f848fbf
- **Transcript:** https://podcasts.opennash.com/shows/pioneers-of-ai/how-microsoft-is-democratizing-safe-ai-733918/
- **Provider:** OpenNash Podcasts (machine-generated)

## In brief

Sarah Bird, who leads responsible AI product work at Microsoft, argues that making AI trustworthy is not a brake on innovation. It is part of building a finished product. Safety, fairness, privacy, security and human accountability have to be designed in from the start, and the tools should be shared widely so every developer can use them, not just Microsoft's experts.

The main tension is between speed and safety. Bird says the trade-off only appears when safety is bolted on at the end. Microsoft spent years building scalable testing and protection systems, and gives its AI red team early access to new models. Agents sharpen the problem, because they cut the human out of checking every output, so oversight has to be redesigned.

## Key insights

1. **Safety bolted on late creates the speed trade-off** Bird says the supposed conflict between shipping fast and shipping safely appears only when responsible AI is added at the end. Microsoft avoids this by giving its red team first access to new AI technology and building reusable testing platforms over years.
2. **Responsible AI covers far more than bias** Bird lists fairness, human oversight, security and privacy as parts of the field. Fairness itself splits into quality of service, resource allocation and representation harms, which is why she calls the term nebulous.
3. **AI red teams hunt many risks, not just hacks** Traditional security red teams looked for vulnerabilities; AI red teams also probe stereotyping, data leaks and weapons information. Microsoft's tool, called Pirate in the transcript, multiplies a human's attack ideas into many variants at scale.
4. **Microsoft retired gender and emotion prediction features** After its responsible AI program matured, Microsoft reviewed shipped products and removed facial-recognition features that predicted gender and emotion. Bird says a smile does not show someone is happy, and that outward appearance cannot reliably reveal inner identity.
5. **Agents force a redesign of human oversight** Chat tools let users check each reply, but agents run longer tasks and that checking defeats their purpose. Bird says automated checks, task-adherence guardrails and approval steps for risky actions must replace constant manual review.

## Questions this episode answers

### What does 'responsible AI' actually mean at Microsoft?

Sarah Bird defines it as making AI systems people can rightly trust, with controls covering fairness, human oversight, security and privacy. She says it is broad because it means doing everything needed to make a system behave as expected every time.

### What is a prompt injection attack, in plain terms?

It is an input that confuses an AI system into doing something its developer did not intend, such as answering a non-coding question when told to answer only coding questions. Bird says it can also hide in websites or data an agent reads, and she compares its name to SQL injection.

### Does responsible AI slow down shipping products?

Bird says no, if it is designed in from the start rather than added at the end. She argues a product that makes mistakes or produces harmful content is not really finished, so the trade-off is false.

### What is an AI red team and why do companies need one?

A red team is a group that tries to break a system so its defenses can be strengthened. Bird says Microsoft's AI red team looks beyond security holes to stereotyping, data leaks and potential weapons information.

### How can a small startup do responsible AI without a big team?

Bird advises having some people who know best practices, reusing existing tools from Microsoft and open-source communities, and spending original effort on risks specific to the startup's domain. She says you cannot set it and forget it.

## Facts

- **Sarah Bird** [0:20:56]: "I think our first responsible AI standard, uh, sort of came into effect about 2018, but if you're, you know, following the timelines of AI, deep learning really started taking off before that in, like, 2013."
- **Sarah Bird** [0:20:56]: "the Gender Shades study that showed facial recognition systems did not work as well for, you know, dark-skinned women as everybody else."
- **Sarah Bird** [0:22:41]: "we moved these kind of things like emotion detection into a sort of unsuitable use of AI, um, at Microsoft, and then we retired kind of the features we had doing that."
- **Sarah Bird** [0:06:17]: "Microsoft's responsible AI standard kind of breaks fairness into three types, and the types of fairness are about how the system behaves."
- **Sarah Bird** [0:06:17]: "your speech-to-text system could transcribe, you know, better for someone with one type of accent than another."
- **Sarah Bird** [0:11:53]: "It's named after a SQL injection attack if you're familiar kind of with that."
- **Sarah Bird** [0:16:20]: "we prioritize the very first people getting access are actually our AI red team because we want the responsible AI work to start, you know, before or at the same time as all of the other exploration"
- **Sarah Bird** [0:16:48]: "Microsoft ships thousands of different AI features every year, and so we don't have experts working hand in hand with every single product because not all of them need that level of expertise."
- **Sarah Bird** [0:19:28]: "we have hundreds of people working in responsible AI. Most of those were not jobs even a few years ago"
- **Sarah Bird** [0:20:06]: "we have a tool called Pirate that helps augment that idea with many different variants or other attack techniques so that you can test many more things at scale"
- **Sarah Bird** [0:18:00]: "And so a traditional security red team was just looking for security vulnerabilities."
- **Sarah Bird** [0:18:26]: "We're starting to prepare for new types of risk around ability to produce, you know, chemical or biological weapon information."
- **Sarah Bird** [0:28:45]: "We had already had, you know, at scale, generative AI, responsible AI systems running in production, for example, for GitHub Copilot, and we had to adapt them to new risk."
- **Sarah Bird** [0:31:05]: "what we try to do with Foundry, which is our AI platform, is offer as many different models as we can."
- **Sarah Bird** [0:34:53]: "that's part of, um, a Microsoft Research project that we've released called Magentic UI."
- **Sarah Bird** [0:32:53]: "we've really in the last couple months kind of crossed a critical threshold where this technology is starting to be good enough for these, you know, agentic use cases."
- **Sarah Bird** [0:33:22]: "an agent can get confused and go off task. And so we have to build new guardrails around things like task adherence."
- **Rana el Kaliouby** [0:30:04]: "So Elon Musk recently announced MacroHard, a company trying to compete directly with with Microsoft."
- **Rana el Kaliouby** [0:27:11]: "seatbelts were only introduced into cars after, you know, vehicles became available on the market, and we started to see"
- **Rana el Kaliouby** [0:23:25]: "my PhD. thesis, which I did at Cambridge, like over twenty years ago, was exactly that."
- **Rana el Kaliouby** [0:09:51]: "at my company, Affektiva, we were in the automotive space. We did a lot of work in the semi-autonomous and fully autonomous vehicle space."

## Transcript

**[0:00:01] Ad read:** Stripe is built for whatever comes next because AI isn't just changing business, it's evolving fraud. Stripe Radar detects fraud with the fast scale of Stripe's 1.9 trillion dollar annual transaction volume. In fact, when a card is used, there's 92% chance the Stripe network has seen it before. That means emerging fraud patterns are detected sooner, blocking bad actors, welcoming good customers. Learn more about fighting fraud with Stripe at stripe dot com slash radar.

**[0:00:31] Ad read:** On Masters of Scale, iconic leaders reveal how they've beaten the odds.

**[0:00:36] Speaker 7:** Asking really strong questions is a superpower.

**[0:00:39] Ad read:** You want to show up with something radically different and how they've grown companies to incredible heights.

**[0:00:45] Speaker 6:** The greatest rewards always come from the greatest risks.

**[0:00:48] Bob Safian:** Let's hit the gas.

**[0:00:49] Ad read:** Airbnb, Zillow, Microsoft, Liquid Death, and more. Hear from the founders who've changed the game. It's anything but business as usual. Find Masters of Scale on Apple Podcasts, Spotify, YouTube, or wherever else you get podcasts.

**[0:01:13] Bob Safian:** The business world is moving faster than ever. And when change hits, we need to learn in real time. On Rapid Response, you'll hear candid conversations with CEOs and leaders making tough calls about AI, human talent, responsibility, and the bottom line. How they navigate uncertainty, pressure, and high-stakes moments. I'm Bob Safian, former editor-in-chief of Fast Company, and I'll be your host. As each episode breaks down what you need to know right now, you can find Rapid Response wherever you get your podcasts.

**[0:02:02] Sarah Bird:** The thing that's amazing about working in this space is that it feels like every six months we have some sort of amazing breakthrough. We first had kind of the huge jump with general purpose models starting with GPT four. But then, you know, in the last year, we've started seeing huge improvements in reasoning models as well as agents. And so in some ways, I think I'm just along for the ride to see, like, what's coming next.

**[0:02:30] Rana el Kaliouby:** That's Sarah Bird, and as chief product officer of responsible AI at Microsoft, being along for the ride doesn't mean sitting back. It means being responsive to ensure the safety and security of Microsoft's AI products.

**[0:02:48] Sarah Bird:** The attackers are innovating as much as everyone else. We have to respond to the new threats we're seeing as well. And so I kind of just wake up every day and say, let's see what happens today. And, you know, that's kind of some of the fun of it.

**[0:03:02] Rana el Kaliouby:** We talk a lot about responsible AI on this podcast, and this week, we dig into what exactly that means. We're taking a deep dive into how Microsoft makes their AI product safe, how innovation doesn't mean sacrificing ethics, and why red team jobs are on the rise. I'm Rana El Kaliouby, and this is Pioneers of AI, a podcast taking you behind the scenes of the AI revolution. Hi Sarah, thank you for joining us on Pioneers of AI.

**[0:03:42] Rana el Kaliouby:** I'm so excited for our conversation.

**[0:03:44] Sarah Bird:** Thank you so much for having me. It's great to be here.

**[0:03:47] Rana el Kaliouby:** So before we dive into your role at Microsoft and responsible AI, I I love to ask our guests and leaders in the AI space how they're actually using AI in their everyday lives. So I also know you're a big runner, and I wear Whoop religiously, and I've started to run again. I used to a few years ago and then I took a break. But yeah, do you use any wearables or technologies to track your running and what have you found?

**[0:04:13] Sarah Bird:** Yeah, I I got really into running in the pandemic because I live in New York City and staying in a tiny New York apartment for a year and never leaving, you know, was maybe a bit much. And so running was kind of a great way to be able to see the city and and do something healthy. So I wear an Apple watch for that. It's really helpful to have data, right, to be able to see that you're going faster than last time or you're going further.

**[0:04:38] Sarah Bird:** But I also, you know, don't want to have kind of too much of it. I think something that's nice about running is just being able to, like, almost like meditate, right? And kind of just not think about things. And so I also try to be a little disconnected when I'm doing it.

**[0:04:52] Rana el Kaliouby:** So you're not, like, continuously looking at the metrics, you try not to.

**[0:04:55] Sarah Bird:** May maybe if it's getting really painful, I'm, you know, start just staring at the watch. Have I made it another mile? How many more miles to go? But, you know, I try not to be doing that.

**[0:05:04] Rana el Kaliouby:** Yeah, that's awesome. Okay, so let's talk about responsible AI. You know, a lot of the big tech companies have teams that are focused and strategizing around responsible AI, but at this point, it's really become a buzzword. So how do you define responsible AI?

**[0:05:20] Sarah Bird:** You know, I think responsible AI is about making AI systems that are trustworthy. So it's not just that we do trust them because people trust technology that they shouldn't, but putting in all of the controls in place so that people can actually trust the outcome they're getting from the system. And so to do that in practice, it encompasses a wide range of things. We have to look at fairness, and we have to look at how we implement human oversight, and we have to look at security, and we have to look at privacy.

**[0:05:50] Sarah Bird:** And so I think that's one of the reasons it becomes kind of such a nebulous term, because it is really about everything you need to do to make a system work the way you expect every time. In, for example, high-stakes applications.

**[0:06:05] Rana el Kaliouby:** Yeah, I want to kind of double-click on that a little bit, and maybe you can walk us through, like, how you define each of these terms, because I think that's very important to our listeners. So what do you mean by fairness?

**[0:06:17] Sarah Bird:** So Microsoft's responsible AI standard kind of breaks fairness into three types, and the types of fairness are about how the system behaves. So the system could, for example, not work as well for two different types of people. Um, so your speech-to-text system could transcribe, you know, better for someone with one type of accent than another. And so that would be what we call a quality of service fairness issue. You could also have a system that makes different decisions on how it allocates resources.

**[0:06:52] Sarah Bird:** So it could choose to offer loans to people like me, but not offer loans to people like you, and that could be viewed as unfair, and that's the the decision the system is making. And then the last type, which we see a ton in generative AI, is how people are represented in the AI system. So, for example, if it puts out stereotyping comments about women, or when it's asked to sort of produce a list and it totally ignores, you know, a whole group of people, then those would be representation harms.

**[0:07:22] Sarah Bird:** And so those are the three types we look at in how the system behaves. And to be clear, that says nothing about the bigger picture of are there disparate impacts if an AI system is applied. So you could have, for example, using AI reinforce traditional power structures, and that could result in it impacting different groups differently. And some people could say that that's unfair. But when we look at implementing responsible AI, we're really focused on fairness in the system behavior itself.

**[0:07:51] Sarah Bird:** And so then that's the three types that, um, I just mentioned.

**[0:07:55] Rana el Kaliouby:** Very cool. What about human oversight?

**[0:07:58] Sarah Bird:** Yeah, so our kind of last principle or foundational principle is accountability, and that's really important because, you know, humans ultimately need to be accountable for the technology that they're developing, the technology that they're using. And it's really important that we don't forget that because a lot of times we think about AI, especially as we start getting into these exciting technologies like agents that are automated. We think of AI as kind of having a life of its own and, oh, you know, it's going and making the decisions.

**[0:08:29] Sarah Bird:** But ultimately, we have to design it in a way that humans are accountable for the technology's behavior. And so when we think about human oversight, there's a lot of techniques we use for that. So in the era, for example, the chat systems, you know, the chat system puts out a response and you, the human user, can actually review that and decide, you know, does it have a mistake? Should I use this? Should I trust it? And so that's a case where we have a human in the loop, you know, kind of in every single interaction.

**[0:09:00] Sarah Bird:** But that's not the only type of human oversight we want to have. And so, for example, when we think about agents, which are going to do much longer running tasks, they're going to be more automated. We want to make sure that we have human oversight in the development and the testing of it so that we ensure that you've tested it, it works as intended, so that you can trust it to go do that longer task. And we can also insert human oversight in those kind of things as the system's running.

**[0:09:27] Sarah Bird:** So maybe if your agent is going to do a particularly high-risk task, it goes back to the human user for approval, or it goes to the system administrator for approval. And so there are many different techniques that we have for human oversight that we implement throughout the lifecycle. But it's a core sort of foundational principle that we need to apply to kind of every system that we develop and every system that we're using.

**[0:09:51] Rana el Kaliouby:** Yeah, you know, this is sparking two thoughts for me. One is at my company, Affektiva, we were in the automotive space. We did a lot of work in the semi-autonomous and fully autonomous vehicle space. And accountability is a big thing there, right? And we noticed that people were starting to just, like, delegate responsibility to the well, the car did this, the car decided to do this. And and and it's so important to your point to keep the accountability on the human.

**[0:10:18] Sarah Bird:** Every person has a role to play and they have a different role to play. And so, for example, the model developer should be looking at, am I, you know, imbuing this with dangerous new capabilities that are going to change the external landscape? They should be building safeguards into the model itself. At the platform level, we build in additional controls like real-time guardrails or monitoring for abuse. Then the examples we were just talking about are the user looking at the output of the system and deciding if it's appropriate to use, or if there's a mistake, or if they need to make an adjustment.

**[0:10:54] Sarah Bird:** And we also have things like, you know, we look at academics and regulators to help set the standard for the world on how these things to be used. And so we really need many, many different people to contribute to actually making this work. It's not just, you know, one single person that's, you know, accountable for an AI system. It's it actually takes all of these roles playing the part that they're supposed to play to get to trustworthy AI.

**[0:11:20] Rana el Kaliouby:** Yeah, so you're chief product officer of responsible AI at Microsoft. What does this role entail?

**[0:11:27] Sarah Bird:** Yeah, it's a great question. I really look at how we put this into practice. So part of my team works on identifying the risk that we see emerging in AI. So, for example, you know, a couple years ago was the first time we started seeing prompt injection attacks, or some people call these jailbreaks.

**[0:11:50] Rana el Kaliouby:** What's a prompt injection attack? I've not heard that before.

**[0:11:53] Sarah Bird:** Hm. Yeah, this is the more, um, technical term we use for jailbreaks. It's named after a SQL injection attack if you're familiar kind of with that. But the idea is really simple, which is to say that the data coming in the prompt confuses the AI system and results in it behaving in a way that is not aligned with its programming. So maybe the developer wrote a system prompt that said, you know, you should only answer coding questions, but, uh, if the user puts in, you know, a input that confuses it and it responds to a non-coding question, we would call that like a prompt injection attack.

**[0:12:31] Sarah Bird:** And so it's a class of techniques that people are using to basically get the AI system to do something different than intended. And it can be both the user interface, but it can also come through tools that an agent calls or data that something consumes. You can hide these types of attacks in a website, and so it's a really important new risk with AI that we kind of have to defend against. And so exactly to this point, when this first came out, we had to figure out, okay, what is this risk?

**[0:13:01] Sarah Bird:** What does it really mean? How do we define it? But then we also had to figure out, okay, now what do we do about it? So, you know, one of the first things we do is develop testing systems to figure out how we really test for this system. Where is it occurring? Where is it not occurring? What does it look like when it happens? Um, but then we want to build mitigations, and so we want to build like a defense in depth approach to say, well, what do we do about this risk?

**[0:13:22] Sarah Bird:** We don't want to just test it. We want to, you know, reduce it as much as possible, mitigate it. And so that part of my team works a ton with Microsoft Research, with external academics, with our red teamers to really kind of find these early patterns. But once we've developed the patterns, we don't want to just publish a paper and call it a day. We want to make it easy for everyone to do this. So a lot of my team then focuses on taking those ideas and turning them into production tools and technologies.

**[0:13:54] Sarah Bird:** So, for example, we build out at-scale testing systems that allow us to test our AI for these different risks before we ship things. But we also want to make it easy for our customers to do that. And so we integrate these in our AI platforms that our customers can also use. And so that same testing system is something developers, um, who are building AI applications, can leverage themselves too.

**[0:14:18] Rana el Kaliouby:** We're going to take a short break. When we come back, why Microsoft brings in their responsible AI team at the start of new projects and how AI is creating new kinds of jobs. Stay tuned. The companies winning right now aren't just building great AI. They're building the infrastructure to actually run a business on top of it, which is where Stripe comes in. Stripe is built for whatever comes next, with planetary scale financial infrastructure to help you power any business model.

**[0:15:03] Rana el Kaliouby:** As a matter of fact, 86% of the Forbes AI fifty run on Stripe, along with industry leaders like Salesforce and Pepsi. Last year, Stripe users processed $1.9 1 trillion combined. See how Stripe can power what's next for your business at stripe dot com. I've seen so many cases where the responsible AI team is brought at the very end of the pro, you know, the the the building process, right? So you're building an AI product, you've ideated, you started building, uh, and then it now it's like validation or testing.

**[0:15:39] Rana el Kaliouby:** And now it's the time to think about ethics and responsible AI, which which you can, you can hear the bias in my voice. I don't believe that that's the right way to do it, but how do you ensure that, you know, the thinking around responsible AI is brought in from the from the get-go? And and does that happen?

**[0:15:57] Sarah Bird:** Yeah, so, um, certainly, uh, the example of bringing at the end most of the time is the anti-pattern, and it's something that we've, you know, worked really hard in our practice at Microsoft to to not do. And so one of the things that Microsoft does is when we get a new AI technology, you know, there's many people that want to start using it because we're gonna start seeing how can I integrate it in all these different applications?

**[0:16:20] Sarah Bird:** What can I do? Should I start a new business line? But actually, we prioritize the very first people getting access are actually our AI red team because we want the responsible AI work to start, you know, before or at the same time as all of the other exploration so that we have the maximum amount of time to, you know, understand the risk, build the testing tools, build these mitigations, and co-develop all of the responsible AI alongside the product development.

**[0:16:48] Sarah Bird:** But, you know, Microsoft ships thousands of different AI features every year, and so we don't have experts working hand in hand with every single product because not all of them need that level of expertise. And so your point around how do we train people is really important here. Because if someone is just shipping kind of a standard AI feature with a known pattern, we want them to be able to do all of the best practices, all of the required testing, put in all of the mitigations on their own.

**[0:17:19] Sarah Bird:** And so then we might only see them at the end because we'll run a release review process to make sure that they did the right things and they didn't miss anything.

**[0:17:27] Rana el Kaliouby:** Yeah, very cool. Like you're democratizing access basically to how to build AI responsibly. Like you don't have to be there, right?

**[0:17:34] Sarah Bird:** That is my, you know, entire life's goal, right? I want everyone to be able to do this. And if the if I have to be there to, uh, get them to do it, it's really not gonna work, right? So we try to run ahead and figure out the patterns, but then we want to f- figure out a way to make it easy for everybody to do it.

**[0:17:50] Rana el Kaliouby:** I love that. You've mentioned the red team a few times now. So for some of our listeners who are not familiar with what that means or looks like, can you kind of visualize this for us?

**[0:18:00] Sarah Bird:** Yeah, so, um, this is really, I think, fun and kind of amazing work. So red teams are a concept in security for a long time, and the idea of the red team is the team that goes in and tries to break stuff and just see, you know, what they can get a system to do. Um, and then we use that information to shore up our defenses. And so a traditional security red team was just looking for security vulnerabilities.

**[0:18:26] Sarah Bird:** For an AI red team, we need to look much broader because there's a much wider range of how a system can misbehave. You know, going back to those principles we talked about, you can have issues with it producing stereotyping results. It could be leaking sensitive data. We're starting to prepare for new types of risk around ability to produce, you know, chemical or biological weapon information. And so we've built, uh, an AI red team that uses the same techniques of trying to break the system, but looks at a much wider range of risk.

**[0:18:59] Sarah Bird:** And so this includes a core team that we've built that's dedicated to doing this. So these are the hacker mindsets, but with just many, many more different types of risk in mind. And then, of course, build the mitigations so that we can, ideally, you know, remove the risk from the end system.

**[0:19:16] Rana el Kaliouby:** You know, we? There's a lot of, obviously, conversation around the impact of AI on jobs, but I believe that these are K- like the red team example is an example of a type of job that's really new because of AI.

**[0:19:28] Sarah Bird:** No, it it's you're exactly right. Um, you know, we have hundreds of people working in responsible AI. Most of those were not jobs even a few years ago, and so it's absolutely right that it changes what is needed. I can tell you I am just horrible at red teaming. I can never get the system to do anything interesting, you know, it's always just producing like rainbows and kittens when I talk to it. And so, like, these are skilled experts who are really creative in figuring out how how to break systems.

**[0:19:59] Rana el Kaliouby:** What kind of skill set do you need to or background or education do you need to have to be a member of the red team?

**[0:20:06] Sarah Bird:** I, you know, I don't think you have to have anything. If you are really good at breaking AI systems, we probably want to hear from you, right? The interesting thing is we actually also use AI to help assist with that. And so if a red teamer has an idea, we have a tool called Pirate that helps augment that idea with many different variants or other attack techniques so that you can test many more things at scale and helps you just kind of get more diverse inputs.

**[0:20:35] Sarah Bird:** Maybe it's not quite as clever as our expert AI red teamers, but it's pretty good because we can use it every day. And hey, I can use it and actually break the system. And so, you know, maybe if I don't want to wake the red teamers up in the middle of the night, I can at least use the agent to get started.

**[0:20:49] Rana el Kaliouby:** Very cool. Are there any products that you've had to sunset because they did not meet Microsoft's responsible AI standards?

**[0:20:56] Sarah Bird:** So we've made changes to products. I think our first responsible AI standard, uh, sort of came into effect about 2018, but if you're, you know, following the timelines of AI, deep learning really started taking off before that in, like, 2013. And so there was actually really important, you know, kind of landmark studies in external academia that were showing that there were problems in these systems that people weren't thinking about. So I'm thinking, for example, like the Gender Shades study that showed facial recognition systems did not work as well for, you know, dark-skinned women as everybody else.

**[0:21:35] Sarah Bird:** And so that that work and these kind of known problems are what helped catalyze, you know, things like Microsoft's responsible AI program as well as others. As our program got more mature, we did go back and look at, you know, products we had at the market and said, you know, would we do this differently today? And so one of the things we did is make adjustments to our facial recognition system. And so our facial recognition system originally would also not just identify, you know, is this person Sarah or does this face match this face?

**[0:22:11] Sarah Bird:** But would also try to predict someone's, uh, gender and also try to predict someone's emotion, right? And, uh, you know, I think what we've learned is gender prediction is something we think just doesn't make sense. Because looking at someone's outward appearance and then trying to predict their internal identity, well, that might work well for many people, it doesn't work for everyone. And so it's just not something we think that makes sense for AI to be doing.

**[0:22:41] Sarah Bird:** And I think, you know, the same thing applies even more so to emotion detection, where just because I'm smiling, that doesn't mean that my inner state is happy. And so, well, it might be reasonable to use AI to detect that someone's smiling, it's not the same to detect that they're happy. And so, you know, we moved these kind of things like emotion detection into a sort of unsuitable use of AI, um, at Microsoft, and then we retired kind of the features we had doing that.

**[0:23:11] Sarah Bird:** And so we actually absolutely expect that we're gonna continue needing to make adjustments and adapt as we learn and the world learns, and, you know, we kind of all learn together. And so, um, so yeah, we've done it, and, uh, and we'll, you know, continue to do it as we need to.

**[0:23:25] Rana el Kaliouby:** Yeah, I've spent my entire career basically building facial expression recognition technology, not fa- not identity detection, but emotion detection. And my PhD. thesis, which I did at Cambridge, like over twenty years ago, was exactly that. Like, you cannot assume that because I'm smiling, that I'm happy. And in fact, there's like hundreds of different types of smiles, but when you reduce it to, like, smile equals happy, eyebrow raise equals surprise, it's simplistic. It's not how humans work.

**[0:23:52] Sarah Bird:** Right, exactly. And and I think then it's misleading to people who are, you know, cause people have a tendency to want to trust like a machine or something that they don't understand as well. It must be some sort of oracle. It must be right. And you're like, no, it actually, there's no way it can be right in that case. And so it's really important, we think, to either make sure that we provide a right amount of transparency and education so people know what this tool actually does so they use it appropriately.

**[0:24:20] Sarah Bird:** Or, in the case of that, we just looked and we found there are certainly some valid use cases. For example, accessibility is a place where, like the blind and low vision community can benefit even from imperfect AI to help them have capabilities that they wouldn't otherwise have. And so it's not that there's no suitable use in the world, it's just that we've generally found that the sort of unsuitable uses really outweighs the suitable ones.

**[0:24:50] Rana el Kaliouby:** Microsoft has the luxury of a big budget and an army of people working on responsible AI. But the reality is even smaller businesses need to invest in implementing safe AI. After a short break, Sarah gives us some guidance on how to exactly do that. I spend a lot of time investing in early stage startups in in the AI space, and I would love to hear your thoughts. Like a lot of these startups are, you know, they're early in their journey.

**[0:25:30] Rana el Kaliouby:** They they can't afford to employ, like, an incredible responsible AI team. So what are your advice to these founders with regards to how they can still kind of implement responsible AI without, you know, necessarily building out a a a big team?

**[0:25:46] Sarah Bird:** Yeah, so my advice for startups is you do need to invest in having some people who are going to understand the best practices and apply them. So as much as possible, leverage existing tools. We have great ones at Microsoft. There's open source communities producing tools, and so, you know, use what's already there. But you also still you can't just sort of, you know, set it and forget it. You you have to use your brain and makes you know, think about what is specific to your domain, right?

**[0:26:16] Sarah Bird:** And so, like, a lot of the capabilities that Microsoft develops are the general purpose capabilities, but you still need to customize them for your specific domain, or you need to augment them to your specific domain. Like, I'm not building, you know, tools that are gonna look for very specific kind of financial risk or something. And so you want to build on top of that foundation and then you want to put all of your energy and what is unique about your application.

**[0:26:44] Sarah Bird:** And that's where you should kind of invest your your innovation in, you know, AI, like responsible AI or security. So, yeah, so you can't, you can't avoid it, but definitely make sure your team is using what's already out there. Also, because what's already out there is changing very quickly. And so you might build a great version yourself, but the, you know, the external community's gonna move faster than you cause it's a whole community. And so making sure you're building on top of something so you can stay up to date with the latest is really important.

**[0:27:11] Rana el Kaliouby:** Yeah, great segue to my next question. So there's this inherent tension between innovation and kind of speed of bringing stuff out, shipping product, and of course, doing so safely. And I love Reid Hoffman's example. He talks about how, when, you know, seatbelts were only introduced into cars after, you know, vehicles became available on the market, and we started to see, okay, there are, you know, these are the dangerous situations for which we need seatbelts. So how do you balance kind of bringing products to consumers quickly, but also doing that safely?

**[0:27:46] Sarah Bird:** Yeah, this is definitely, I think probably like the number one question I'm asked, and I think I would say that it's really, like, important question and topic, but also I I wanna say that I feel like the framing's kind of wrong. Because to your point about the anti-pattern earlier, if we do everything and then we add responsible AI or security at the end, of course, it looks like there's a trade-off between shipping quickly and not. Because you've put it at the end, you've, like, uh, you know, increased the time by definition.

**[0:28:17] Sarah Bird:** But when you're designing alongside and doing that investment, it doesn't have to change your speed to shipping. And so it's not trivial to achieve that. You know, a lot of what we've done to successfully invest in responsible AI at Microsoft is make sure that we're building our responsible AI systems to be scalable platforms that we can adapt really quickly so that we can move agilely and at pace and at scale so that we can ship AI very quickly.

**[0:28:45] Sarah Bird:** And it took us years to build that up. That was not something, you know, that we we just woke up on, like the day ChatGPT was released and started working on this. We had already had, you know, at scale, generative AI, responsible AI systems running in production, for example, for GitHub Copilot, and we had to adapt them to new risk. But we did not just, you know, start from scratch. But the other thing I'll say, you know, most customers I talk to, uh, uh, you know, enterprise customers, but also consumer startups.

**[0:29:14] Sarah Bird:** Like, if your AI system, uh, you know, regularly sort of makes mistakes if your agent is going off task or if your system is producing harmful content that's not aligned with your brand. I think most of the people say the product doesn't seem finished, doesn't seem like it's actually working. And so you're not shipping a high quality product. And so, sure, you can ship quickly without responsible AI, but it's not gonna be what people want, right?

**[0:29:39] Sarah Bird:** And so that's where I think it's a false trade-off because you have some idea that that it's just extra and it's not about the quality of the product. But it's so fundamental to how AI systems behave that you really just aren't even done until you've done that. And customers will call you on that and they will demand more, which I think is a fantastic thing about the ecosystem these days, that the world has very high expectations that we do this well.

**[0:30:04] Rana el Kaliouby:** Yeah, I love this answer and I love this framing because building AI responsibly isn't just the like, the right thing to do, it's actually good for business to your point, and I I think that's so important. All right, there's a lot of competition right now in the AI space, and one particular example. So Elon Musk recently announced MacroHard, a company trying to compete directly with with Microsoft. What do you think of this announcement? And in general, how do you think about competition?

**[0:30:33] Sarah Bird:** Yeah, um, you know, I don't know if I have particular feelings about that announcement, um, since I spend my time mostly on the, you know, responsible AI and security side of the house. But I would say that I think, you know, competition is really important in the ecosystem. It pushes all of us to be better. And, you know, I think the other thing that's really important is kind of choice. So part of Microsoft's approach to how we're building our AI platforms is about giving developers and customers choice.

**[0:31:05] Sarah Bird:** For example, a particular model might be great for Microsoft's use cases, but it might not make sense for, uh, you know, an application that's being made for, like, rural farmers in India, um, because it uses a different language, or if there's a different technical domain or something. And so, um, what we try to do with Foundry, which is our AI platform, is offer as many different models as we can. And so, you know, these models have different trade-offs in terms of quality, in terms of cost, in terms of safety.

**[0:31:38] Sarah Bird:** We put all of that information in front of the customers so that they can pick, as well as tools that then allow them to adapt it to their application. And so I think that if we're really gonna change the world with AI and and power, you know, every application and every profession, we need a lot of people working on this, and we need a lot of different types of ideas. And so, like, generally, I think the competition is great and is making, you know, is making the field better and is gonna result in more impact.

**[0:32:04] Sarah Bird:** And so I hope that we continue to see an ecosystem like that where people are competing, but also collaborating to help everyone, you know, get better at this.

**[0:32:12] Rana el Kaliouby:** So I wanna talk about agents, because there's this inherent tension between giving these agents autonomy, but also still having control over what this agent does. How do you think about responsible AI in the agentic world? And, uh, yeah, what are some of the biggest concerns that you're trying to tackle?

**[0:32:31] Sarah Bird:** Yeah, I think that agents, first of all, are just incredibly exciting because I think there are a lot of the real promise of the technology. Uh, you know, as fun as it is to have a chat system attached to everything you do now, uh, I don't want to just chat with something. I want to assign it a task and I want it to just go get it done, right? Like, that's what's really gonna multiply my impact.

**[0:32:53] Sarah Bird:** And so, uh, you know, we've really in the last couple months kind of crossed a critical threshold where this technology is starting to be good enough for these, you know, agentic use cases. But it does bring new challenges for us. So, for example, I mentioned prompt injection attacks earlier. In a chat system, you may just see those coming in through the user interface, or maybe some of the data coming from the search engine. Now those could be coming through all of those different tool calls.

**[0:33:22] Sarah Bird:** So we have a bigger surface area that we need to secure. Or you can have new types of risks, like an agent can get confused and go off task. And so we have to build new guardrails around things like task adherence. And so with the sort of expanded risk portfolio, we have to take everything that we've built and extend it more. I think the other challenge that we also talked a little bit about earlier is that, you know, for a lot of the AI systems that are in production today, people are still using humans as a key mitigation kind of in the inner loop.

**[0:33:56] Sarah Bird:** Oh, the human will check the thing. But what's so exciting about agents, or even something like vibe coding, is that they're sort of, by definition, not about having the user do the oversight. And so that's where we have to invest a lot more in automated testing tools or automated checks that help ensure that the agent stays on task or ensure that the agent completes the task every single time. And so we're redesigning what those human oversight mechanisms look like and what those interfaces look like so that we still have accountability, we still have humans in control.

**[0:34:36] Sarah Bird:** But the way they're in control is not just checking every single output of the system, cause that would defeat the point with automation.

**[0:34:42] Rana el Kaliouby:** I imagine one important kind of design factor here is how do you build trust with with the user, right? So that the user can actually l- let go of this control and let the agent do its thing.

**[0:34:53] Sarah Bird:** Yeah, I think that's that's ex- uh, exactly right. And it's not just, um, building trust, like, okay, you can print out the execution plan, but also if something goes wrong, what's the impact of that? And how do you debug what went wrong so you understand and you can adjust it for next time? And so I think there's, you know, research to figure out how to show that type of information effectively to end users. And actually, that's part of, um, a Microsoft Research project that we've released called Magentic UI.

**[0:35:21] Sarah Bird:** It's exactly designed to allow researchers to experiment with these different user interfaces and figure out what patterns are really gonna empower users to understand and feel in control versus what's kind of overwhelming. And they're just gonna ignore it the way we all just say, you know, okay, to all license boxes or all of the privacy, you know, cookies.

**[0:35:41] Rana el Kaliouby:** Okay, last question, and it's a question I ask of all my guests. In this world of AI, what do you think it means to be human?

**[0:35:49] Sarah Bird:** It's a question people are certainly asking. And, um, you know, I I guess I feel like, in some ways, we just already know the answer, right? It it still feels different to be human and connect with other humans than it does to an AI system. And so I think there's going to be fascinating research and art and all of this really exploring this topic. Um, and uh, you know, I love people digging into it, but I don't see it as creating some, you know, existential risks for us.

**[0:36:23] Sarah Bird:** I think this is a tool and we're gonna use it like many other tools that humanity has already used. And I think that that's gonna, you know, make make our lives richer. But also we have to go into it eyes open, knowing that there are a lot of, you know, things we need to think about and a lot of risk that, like with each new technology, we have to address.

**[0:36:44] Rana el Kaliouby:** I love that. Thank you, Sarah, for joining us. This was great.

**[0:36:48] Sarah Bird:** No, I really enjoyed the conversation. Thanks for having me.

**[0:36:53] Rana el Kaliouby:** In the world of AI, some terms don't have clear definitions. For example, people don't agree on what AGI actually means, but there's one word that should be crystal clear responsibility. In my experience, a lot of companies building AI products think about the ethics or safety aspects at the very last step, just before shipping the project. And what I admire about what Sarah and her team are doing at Microsoft is that they take responsible AI so seriously from the get-go.

**[0:37:25] Rana el Kaliouby:** It's not an afterthought or an add-on for brownie points. It's integrated into every step of product development. And what I find especially compelling is how they're democratizing access to responsible AI. They're training their partners and creating tools for other developers to implement AI safety into their workflows. To me, that shows a real commitment to having AI that works for everyone. Before you go, if you like what you heard on this episode, don't forget to rate and review us.

**[0:37:56] Rana el Kaliouby:** It helps other people find the show. Plus, we love reading your feedback. Thank you. Pioneers of AI is a Wait What original. Our executive producer is Eve Trow. Our producer is Rachel Ishikawa, and our associate producer is Jordan Smart. Our senior talent executive is Stephanie Stern. Mixing and mastering by Brian Pew. Original music by Brian Holliday. And our head of podcasts is Lital Molad. You can join the conversation on LinkedIn, Instagram, TikTok, YouTube, and X. Just search for at Pioneers of AI.

**[0:38:43] Rana el Kaliouby:** Thanks so much for listening.