Sharing AI Mistakes: Partnership on AI’s Rebecca Finlay

Stay tuned after today's episode to hear Sam and I break down the key points made by our guest.

We can all certainly learn from companies successes with AI, but what about from their failures? On today's episode, we speak with one leader who encourages organizations to share the bad along with the good in the hopes that we can all learn together.

I'm Rebecca Finlay from Partnership on AI, and you're listening to me, myself and AI.

Welcome to Me, Myself and AI, a podcast on artificial intelligence and business. Each episode, we introduce you to someone innovating with AI. I'm Sam Ransbotham, professor of analytics at Boston College. I'm also the AI and business strategy guest editor at MIT Sloan Management Review.

And I'm Shervin Khodabandeh, senior partner with BCG and one of the leaders of our AI business. Together, MITSMR and BCG have been researching and publishing on AI since 2017, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities and really transform the way organizations operate. Hi everyone, thanks for joining us today. Sam and I are very happy to be speaking with Rebecca Finlay, CEO of Partnership on AI.

The organization is a nonprofit that brings together a community of more than a hundred partners to create tools, recommendations and other resources to ensure we're all building and deploying AI solutions ethically. Rebecca, this is super exciting and important work, and we'd love to be speaking to you about it. Thanks for joining the show.

Thank you so much for having me. I've been looking forward to this conversation.

Wonderful. So let's get started. Tell us more about the organization's mission and purpose.

The Partnership on AI was formed in 2016 with the belief that we needed to bring diverse perspectives together in order to address the ethical and responsible challenges that come with the development of artificial intelligence. And also to realize the opportunities to truly ensure that the innovation of AI benefits people and communities. And so with that belief in mind, a group of companies and civil society advocates and researchers came together to chart out a mission to build a global community.

That has now come together for many years, focused on ensuring that we're developing AI that works for people, that works for workers, that drives innovation that is sustainable and responsible, privacy protecting and really enhancing equity and justice and shared prosperity.

Maybe give us an example or some examples of the companies and the type of research.

The very first investment that was made into the Partnership on AI was by the six large technology companies. So that's Amazon and Apple, Microsoft, Facebook, now Meta, Google DeepMind and IBM. And it was really at that moment when this new version of AI, or what was new then, you know, deep learning, pre generative AI, that wave of AI was really starting to be deployed in internet search mechanisms, in mapping mechanisms and recommendation engines. And the realization that there were some important ethical questions that needed to be answered.

And so that brought together a whole group of other private sector companies, but also organizations like the ACLU and research institutes at Berkeley and Stanford and Harvard and internationally as well. So organizations like the Alan Turing Institute in the UK and beyond. And so that group came together and now we have a number of different working groups that are really focusing both on the impact of that predictive AI, but even more importantly, the potential impact of generative AI, foundation and frontier models.

What are some examples of some progress that you feel like the partnership has made? What are some specifics here?

Particularly in this area, it's clear that we need to think about it through what we would call a sociotechnical lens. Yes, there are technical standards like watermarking or standards like C2PA that are thinking about how you clearly track the authenticity of a piece of media through the cycle. But you also need to think about what are the social systems and structures in place. So one of the efforts that we developed and now have eighteen organizations signed up and evolving with us is the framework for responsible development of synthetic media.

And that is really looking across the value chain. What are the responsibilities of creators, developers, deployers, that's platforms and otherwise, when it comes to thinking about how to disclose appropriately to make sure that whoever comes in contact with the media that is developed is aware that it is AI generated in some way. And also to make sure that the media that is being developed is not being maliciously used or being used in any way to harm.

And we have a whole series of lists and information about what those harms are and why we need to be protecting people from them. So that's a really important effort. And of course, the question is, how is it being used? And so one piece of this work that we've been doing is to ensure that the companies and organizations and media agencies that have signed up to support this work are really being transparent about how they are using this to respond to real world circumstances.

And we so we make that available. And the goal is, yes, both to be accountable in terms of the framework itself, but also to try to create case studies that other organizations can use and learn from as well.

That makes a lot of sense. I guess, uh, what I was particularly interested in is the aspect of deployment there. So, you know, in my mind, it's really hard to imagine the fang companies that you mentioned that were part of your origin story. You know, they have enough economic incentives not to let's say let's pick deepfakes as an example. They have a lot of incentive not to create this sort of media. And so I guess I'm less worried about them being involved in the creation.

But you also mentioned deployment, which I think is is really where they have a huge role. And it seems like this is a case where maybe the people that the partnership is focusing on are probably what we would call the people who are likely to behave well in the first place. From that generative standpoint, how do you reach the people who are not, let's say, not inclined or not incentivized to behave well?

Great question. None of the work that we're doing at the Partnership on AI in any way should stop appropriate regulation. I've always been a supporter of governments attending to and being aware of and acting upon harms and ensuring that citizens are protected. And so regulation is a key part of thinking about an innovative accountability ecosystem. But at the same time, we do think it's really valuable to be helping those organizations that do want to be good and responsible actors to know what good looks like.

It's both helpful in terms of companies, in terms of the work they're doing. It's also helpful, I think, to ensure that civil society and academics have a seat at the table in saying what good looks like. And then we also are finding that it's very helpful for policymakers. So helping them to better understand the details of the emerging technology and where regulation is appropriate and useful is part of the work we do as well.

What are the partnerships views on impact on workforce?

Yeah, this, I think, is one of the most fundamental areas that we all need to be thinking about. And it's funny that it's the one area in the whole AI development conversation that has this sort of air of inevitability to it. You know, it's sort of like the robots are coming, they're taking our jobs. We need to look at UBI. We need to think about all these other mechanisms as well. And so we've really rejected that inevitability and said, no, there are choices that firms and employers and labor organizations and policymakers can make to ensure that AI is developed to augment workers and to ensure that workers voices are at the table when thinking about developing this technology.

And this is a very complex question. Yeah, it's a question of education and reskilling, but it's also a question of how are we measuring and evaluating these systems to ensure that we're focusing on development that works for people. We issued a set of guidelines, we call them the shared prosperity guidelines, which really were trying to get very specific. They were based on a series of interviews and some research that we've done with workers themselves to better understand when and how AI deployed into their workplaces was beneficial and when it wasn't.

And so

Hm, hm. Beneficial to who?

beneficial to the workers. There are all sorts of times when workers realize the value in having AI systems working with them. That is so that they can be more creative, so that they can make more decisions, so they can be more innovative in the way in which they approach their work. When it's not working for them, of course, is when they feel as if they're being driven to rote tasks or repetitive tasks, or being surveilled in some way through the system.

So taking those insights and saying, okay, so how do we make choices, clear choices when we're deploying these systems to ensure that we're doing it in a way that works for workers? And so I think, you know, if you're an employer and thinking about how do you even begin to wrestle with these questions of how to use generative AI in the workforce? You need to be thinking about it in a very experimental way, right? Rather than thinking, all right, it's gonna save me all these costs.

Let's deploy it, you know, 100% in this direction. Thinking about it as a way to sort of pilot new technologies, hear from your workers. One of the things we know is that a lot of AI systems failed when they're deployed because we haven't thought through what it actually means to put it into a workforce setting, what it actually means to put it into a risk management setting.

Yeah, I love that, particularly the lens of it clearly can give a fair amount of efficiency, but if that's the only lens, then what happens is you're missing the bigger opportunity. But I also think in that bigger opportunity of how is AI and human sort of grow the size of the pie together. I think AI is not gonna replace an entire worker, but it will replace tasks. So now 30% of my tasks have been replaced.

I have to do something with that 30%.

I've got no shortage of things to do with my 30%.

You can't, you can't fire 30% of me. But if you have not just a cost lens and you have much more of a growth and productivity and profitability and innovation lens, then the sum of all those thirty percents can go into creating all those new opportunities. I was wondering if there is any research or thoughts that the partnership has on what are some of these opportunities that are gonna create jobs versus replace jobs, right? Like, cause, there's a lens here, which is we've gotta be ethical, we gotta make sure that harm is not done, and we also have to have a longer term perspective of not just replacing workforce with, like, pure blind automation.

So that's just sort of more like protecting. But I wonder if the partnership has any sort of points of view on how do you expand the art of the possible through creation of new roles and new jobs?

Well, 100%. I mean, I think the way you've just described it, right? It's not a trade-off. It's not like responsibility and safety are a counterweight to innovation, and you have to constantly be choosing between innovation and benefits. We know that in order to be innovative, in order to be opening up new markets, in order to be thinking about new beneficial outcomes, you need to be thinking about how you are doing this safely and responsibly as well.

The whole opportunity for generative AI to become much better, because today it still has real challenges, whether it's hallucinations or other ways in which it is deployed, just hasn't really gotten to where it needs to be. But if we're starting to think about once it's there, how it can be deployed to really deal with some of the biggest global challenges of our time. That's why I'm at the Partnership on AI is because I believe that AI does have that transformative potential to really support important breakthroughs, whether it's in healthcare or really the big questions in front of us around our environment and about sustainability.

We're already seeing this in the predictive AI world, right, where we're starting to see it just becoming integrated into the scientific process across all sorts of disciplines. I do think getting back to this question of, you know, the trade-off between responsibility and innovation, that one of the things that I hear from companies right now is they feel alone as they're trying to disentangle the risks of deploying these technologies and the benefits to their productivity and the innovation and how they serve their customers as well.

And so one of the reasons why I think the work that we do at PAI is important is I want to say there is a community of organizations that are wrestling with exactly these same questions that are trying in real time to figure out what does it mean to deploy this responsibly in your workforce? What does it mean to think about safety of these systems and how they're operating, whether that's auditing, oversight or disclosure or otherwise?

How do you experiment and what is best practice? And so I think more and more if we can let companies and organizations know that there's a communities who are actively working on these questions where you can get some insights and really in real time develop what will become best practice, that that's a good thing for them to know.

Yeah, it sounds like education and collaboration and sort of sharing all these things is key because absent that, it's really easy for many people, including many executives, to think of AI as it's a tool is going to give me 12% productivity. I'm gonna need 12% less people and let's figure out how to do it, which is a valid way of thinking if you're not exposed to everything else that's happening. And this is really interesting.

You mentioned that so many of the participants in in this partnership are actually big tech firms that are shaping the very thing we're talking about.

Yeah, I mean, I think it also goes to show that it's all very new. So many of the questions that we're dealing with, whether you're a large technology company or a small startup who wants to develop systems based on the release of these foundation models or otherwise, so many of these questions are being sorted out in real time. Now, that doesn't mean that we shouldn't be looking to other sectors, and I know you both have experience in sectors where we should be able to learn better about what does it mean to be safe and responsible as we deploy these AI systems as well.

But I do think that even for policymakers, both keeping up to speed on the latest developments and the pace of the technology, but also understanding the nuances of the technology. It's a tricky time, and so finding places where we can learn together is really crucial.

Learn together makes a lot of sense because I think one of your key missions is about collective good and the idea of you just said that we are learning as we go. And this is a technology that we can't look at the back of the book and see the right answers. But we tend to learn better from things that don't go well. News media, they pick up examples of extreme wonders and extreme terrors. There there's very little of this nuance that you just mentioned.

How do we elevate? How do you get people to share this nuance and the good experiences and the bad experiences? How do you get them to learn for this collective good?

Yeah, that is just such a fundamental question. And of course, from your experience and from other sectors, we know that building a culture of safety means building a culture of transparency and disclosure, right?

Which we don't see much of.

Right? We were really happy a number of years ago to work with some researchers to develop an incident reporting mechanism, which is now thriving. And this you'll see if you're looking at any of the emerging frameworks around foundation model or frontier model, whether it's the G7 work or work coming out of the OECD or elsewhere. This question of incident reporting is becoming very, very clear. And how do we create ways in which so that we better understand once these systems are deployed, what is actually happening and how can we all learn from them?

So I think that's one piece of it, which is going to be crucial.

When I think about, like, the last twenty five years that I've been in this field, it's all typically a one sided story from the perspective of the technology developer or proponent, which then forces a very negative sort of dialectic on the other side. From the group or organization or people who are trying to rein it back in, which gives this, you know, very polarized way of dealing with this. But we are at a precipice or inflection point where the speed with which we learn and adapt is really critical, which means we need to share things that aren't working.

Just a couple of days ago, a YouTube video came on my field that says how we messed up this customer's order. And the whole point was about how that company screwed up and what mistakes they made and then how they correct it. And like, that's all they do, like, they basically post all these videos on how they've messed up something and how they respond to it. And they get a lot of views because they just literally celebrate their mistakes and learning from it, which is not something you see often.

Maybe I should start a YouTube channel with all the screw ups I do in class.

We get a lot of views, for sure.

You know, I've been thinking a lot about this question of openness lately, because you'll both know that there was a debate last year between open models and open source models and closed models and which were safer. And so in the executive order, for example, over the last year, there was a real interest in finding out more about open open source and what are the marginal risks associated with those type of models being deployed. But for me, it's it.

This question of openness relates directly to this point I think you're making about sharing, and how do we build a culture of open sharing into the AI ecosystem? And so I think, first and foremost, it has to start in the classroom, it has to start in the research community itself. As you know, in the AI research community, we don't have the same, uh, culture of ensuring that everything is published openly. We have a lot of research that's now happening behind closed doors.

We don't have the same sort of journal and editorial and oversight perspective that we do in some of the other fields of science. We have a lot of things that are that um are published via conferences or published directly to archive. So what do we need to do? What does publicly funded research need to do at that very, very early stage to in order to incent a culture of true scientific open openness and scrutiny? And then, yes, the next piece of it has to be.

What is the level of transparency and disclosure when these systems are being deployed out into the world? We put together a set of guidances all the way from pre deployment R and D to post deployment monitoring. What does it mean at twenty two pa- twenty two places along the development and deployment ecosystem to be consciously disclosing and attending to risks and ensuring that guardrails are in place? We think about the disclosures that are required in the financial services sector, for example, or elsewhere.

What does that mean to have that type of disclosure regime in place for some of these very, very large models? And then I think the last piece of it is how are we making sure that we are bringing the public and citizens into this conversation about the way in which these tools are developed and deployed? So what does openness mean when we think about a citizen engagement in this process as well, being really part of the tech development process to ensure that their voices are heard about how they want the technology to work for them and not on them.

So I do think as we sort of build our skills and capacity around the deployment of this technology, we need to be thinking about openness, sharing, um, and disclosure, um, in order to ensure that it really does work for us.

You mentioned public funding, but I think that's part of the thing too. And, you know, that's one thing that I'm interested in the the partnership in that. So much of basic science used to come out of universities and it used to come out of NIH or NSF funding. But that's not who's really on your board. That's not the fang companies that started the partnership. That's what seems particularly important that maybe the partnership has a role here that might not have existed with prior types of technologies.

Because the costs are staggering and universities are not really able to participate in the research budgets of, you know, OpenAI has a uh, close to $10 billion of funding from Microsoft, but the NSF had about 9.9 billion dollar budget in 2023. So that one project, it would wipe out the NSF budget entirely. So it's not clear exactly how that plays out. And because we talk about openness, I think the community's been very open about sharing algorithms.

You know, those algorithms are very widely dispersed, but openness really depends on data too.

I also feel like is really openness on deployment and

Like, where it's being used, is that what you mean,

h- how it's being used, where is it messing things up? You know, it's beyond the algorithm, right? It's the data that gets fed into it, it's all the decisions that get made, it's the human role. Yeah, it's just in general, like Sam, your comment around sharing and Rebecca, your point around transparency. It really resonates with me because I just feel like, generally speaking, when it comes to technology, it's sort of an alpha game of like, you know, this is the best one, it's better than the other one.

Its accuracy is more, its error is less. Like everything is always so great and is better than everything else. But I do feel like there is something here, and maybe is also more on the evolution of us as a society as to just just accepting and acknowledging that these systems make mistakes. And if you're not hearing about it, it's because somebody's hiding it rather than it's not. And so I think the more some of these big players begin to openly talk about these things so that they're not the only one, right?

That's a great example.

I mean, like, if many of these larger players in this world really exhibit the same kind of openness that we're talking about here. Look, look, we made a mistake. This is how we're fixing it. This is what happened. I actually think that would demystify a lot. I think that will increase a lot more the public trust and will allow a real dialogue. Cause I actually feel like this conversation is very much polarized at all levels. It's very polarized for those who understand AI and those who don't.

Within those who do understand AI, it's very polarized between the proponent and the the folks against it.

Fear mongering.

Yeah, I feel like we need to break that polarization and bring these two sides together.

So maybe just to shift that slightly, Rebecca. We've been talking a lot, I think, about a Western-centric world, but so much of this is going to affect the globe. What does the partnership think about, and how does it try to work with those outside of San Jose and the rest of the world?

Absolutely. And of course, we know today that many workers outside of the global North are very much consumers of the technology that is being developed in the in the global north as well. So we need to make sure that they are part of the global governance conversation. I was really heartened to see just this September around the UN General Assemblies, that there was a really big initiative to think about bringing many, many, many of the voices that are not around the table at the G7 or many of the other discussions, whether it's the, you know, safety institutes that are being developed in the different countries.

To bring those countries and those voices into the conversation about what is needed from a global governance perspective. We had the high level advisory body to the Secretary General at the UN release their report, really starting to think about what does it mean? There's a lot of work that we've been doing to try to hear from organizations across the globe. In how is AI currently working within your populations? How would you like it to be working?

And that's both from companies who are developing and startups who are developing technologies, through to many academics who are developing all sorts of new technologies. Thinking about the way, for example, to develop data sets with languages that are not prevalent in those that are being developed out of the West. So all there's really, I mean, so much interesting work that's happening and thinking about how to get those voices into this conversation is core to what we do at PAI as well.

So now we have a segment it's called five questions, where I'll ask you a series of rapid fire questions. Just tell us the first thing that comes to your mind. What do you see as the biggest opportunity for AI right now?

The biggest opportunity for AI is for companies to start experimenting with it to see how it can truly drive innovation and potential in the work that they're doing. Start experimenting in a low risk, high value way as soon as possible. The more you use the technology, the more you understand what it can do for you and what it can't.

What is the biggest misconception about AI?

That it is a technical system that sits outside of humans and culture. I always say AI is not good and it's not bad, but it's also not neutral. It is a product of the choices we make.

Hm, hm, very good. What was the first career you wanted? What did you wanna be?

I think when I was youngest, I wanted to be a journalist. I wanted to tell the stories of people and societies, and I'm very happy that as part of my job, I get to do that today.

When is there too much AI?

I think there's too much AI when we see people deferring to AI systems or overtrusting AI systems. You know, one of the things that we know is that we tend to overtrust something that happens through computers or machines.

What is the one thing you wish AI could do right now that it cannot currently?

Oh my goodness, where do I get started?

You have to pick one.

You know, my favorite use of AI right now is my bird app. I don't know if you've used the Merlin bird app, but it's a great app for bird watchers. Because you can take a very fuzzy picture of a bird and it will give you a nice match in its system, or you can take a recording of a bird song and it will tell you what the bird is. So I guess what I would love AI to do is to help me see the birds, help me find the birds in the trees.

It's just like Pokemon Go for birds, I guess.

I would really love that. I've been enjoying so much learning more and more about birds, but I need to find them.

Rebecca, it's been wonderful speaking with you. Thank you.

It has been my pleasure entirely. Thanks so much for having me.

Sure, Rebecca, I thought, has a really interesting perspective. This idea of we're all in the same boat, we're all developing this technology. None of us have any experience with it, but it affects all of us. That's a great perspective, and it makes me optimistic and worried at the same time.

Well, I agree with you. I also think that you had a very good point on, uh, importance of transparency and sharing and sharing. Like the question you asked about, you know, how do we get people to share more about the things that don't work? We hear all the positives, like I I actually think between the two points of look, we're all in this together. It will help us all if we are transparent and collaborative about it.

And in many ways, I mean, AI has gotten to what it is mainly because of the open source nature of it, right? But when we say open source, we don't just mean let's share the algorithms. It also means, like in this new paradigm that we're entering, it's also means share lessons from deployment and things that don't work. And for me, the YouTube thing I was referring to was really eye opening. Because, like, usually when you see a tagline that's like how we screwed up this customer's order, you're like, why is this have like so many views?

And you're looking for something chaotic. And they literally are showing all the mistakes they made and who made those mistakes, and and it doesn't matter the mistakes you make, it matters how you correct them and what you're doing correcting them. I actually think we've entered really a phase where we need to, as a society and as a community of technologists and innovators, share a lot more about things that don't work and why they don't work. If we're truly gonna be open source, we should be open source about that too.

Yeah, that's a great expansion. Uh, if I hand back an exam to people, they skim right over all the things that they did correctly and they focus on the ones they did wrong. Cause that's where we have the opportunity for learning. And actually, to make it even a sort of more machine learning example, that's the fundamental idea behind boosted trees and the boosted algorithms. Is that you pay a lot more attention to the places where your model makes an error than where the model gets it right.

That's right.

Sure, and I think we've got another example here where we're getting to a little more depth behind what it means to go beyond the platitude. You know, we've got these ideas that no one disagrees with of doing the right thing, don't do the wrong thing. But like Jeremy Kahn said back when we were talking with him, adaptability, sure, be adaptable, but how do you be adaptable? Well, today we were talking about open. How do you be open?

I don't think we have the right answer for this, but it's an interesting thing to think about.

We need to celebrate transparency around mistakes and how we correct them.

Um, one thing I liked that she also said was about this idea of sociotechnical. I think that a lot of people view this as a technical problem, which has a technical solution. For example, we have copyright infringement, and so watermarking is the technical answer for that.

That's right, that's right. Yep, yep,

And, you know, I'm not against watermarking. I think there's a lot of potential for that, but it's not just going to be solved by a technical solution. These are things that are operating within within societies, within cultures, and that's a theme that keeps coming through. We did the report a couple of years ago about the intersection between culture and the technology of artificial intelligence. It seems like there's a growing recognition that it's more than just a new set of algorithms.

that's right, that's right. I mean, it, it is. When I think about any technology that's widely available to people, whether it's cars or whatever it is, it is a question of responsibility of an individual too, right? But I think that the cultural aspect of this, which can start from within corporations, right? I mean, we did talk about how AI makes individuals and then teams, and hence organizations more proud and happy, et cetera, right? I do really think that the point around the intersection of society and culture with technology is going to be really, really key here, rather than just a bunch of regulations and a bunch of technology artifacts that that sort of correct mistakes or prevent mistakes, right?

It's how we choose to use something, right? When I get behind a vehicle, it doesn't matter whether there is a speed limit. I mean, it does matter, but like, it doesn't matter whether I'm being followed or there is a cop or whatever. There is something about being responsible, and it's in it's ingrained in us as as

That transcends regulation itself.

as human beings. That, yeah, that's like, look, you, you can't be reckless because you're gonna endanger your own life, other people's life. It is something about that, uh, we've now begun to accept as a society that just just goes with being responsible with a tool. And Rebecca said it very nicely. It's not good or bad, it just is, but and it's how you use it is gonna make a difference.

If I step back and I think back of some some folks we've had, we've had Mozilla, for example, we've had the partnership on AI, we've had Amnesty International. And these are not some of the traditional companies using artificial intelligence. And, you know, I think some of these organizations may fail, some of their initiatives might not end up working well. But I think the fact that they're trying and they're pushing towards these things can help that collective good, even if they don't quite reach the goals that they set for themselves, even if they come close.

I think it's an important force and I'm I'm glad that we've had some of these guests on here to share that. Thanks for listening. Our next episode is a bonus episode where I speak with Oxford's Carl Frey and LinkedIn's Karen Kimbro at a recent conference on jobs in the age of artificial intelligence. Also, if you have a question for us about AI strategy, implementation, use, Shervin's favorite flavor of ice cream, or anything else, we have an email address, SMR-podcast at MIT dot edu.

We'll include that email in the show notes. Send us your name, where you're from, and what question you have, and we'll dedicate an episode to airing some of those questions and the best we can come up with answers for them. Thanks.

Thanks for listening to me, myself and AI. We believe, like you, that the conversation about AI implementation doesn't start and stop with this podcast. That's why we've created a group on LinkedIn specifically for listeners like you. It's called AI for Leaders, and if you join us, you can chat with show creators and hosts, ask your own questions, share your insights, and gain access to valuable resources about AI implementation from MITSMR and BCG. You can access it by visiting MITSMR dot com forward slash AI for leaders.

We'll put that link in the show notes, and we hope to see you there.