Top Data Scientists Explain Bad Data, Poisoned Datasets, and Other AI Killers | CXOTalk #896

Your AI is under attack from bad data, poisoned data, deep fakes, bad actors, and more. Today on CXO Talk number 896, luminaries Dr. David Bray and Dr. Anthony Scriffignano reveal how to escape this AI quicksand with practical strategies for building clean systems and trustworthy AI. David, uh, very briefly, tell us about your work.

One of the roles is with the Stimson Center, specifically the Stimson Accelerator. And what we're striving to do is demonstrate that one can advance both programs and policies in parallel. We can't wait for perfect policies anymore because of the rate of change that's happening in tech, including AI. And so we're showing it's possible, whether it's in business, whether it's in governance, or whether it's possible at the community level.

Anthony, give us a sense of your work.

I am also have the honor of working with David at the Stimson Center in the context of the Lumis Innovation Council. We do what I would call action research. It's not just a bunch of white papers, it's actually stuff that matters, and then incubating things that hopefully kind of grow and and impact the thing that we're working on. And there's some very big problems that are really important.

David, we're talking about AI quicksand. What is that?

When we unpack artificial intelligence, AI, we recognize that most methods of AI require some training or some data to actually underpin what they do. Not all there are other approaches that don't, but the most recent approach, generative AI, does. And so when we think about quicksand, we could end up thinking we've built this perfect AI system. Yet the very data that actually sort of serves as the foundation that the system was trained upon, what it was based upon, turns out to be not only quicksand, but it might actually just disappear.

Uh, in terms of it's just under the weight it is, it's not a sufficient for the models we're trying to build. It's recognizing that if you don't take the time to think about where the data came from, how it might actually be missing certain things that you're trying to ask using the system, or someone might be intentionally poisoning it. If you don't do that, you may find that you build the metaphorical castle on sand, only to have it be washed away.

I can add a few things to that. I have, um, five M's that are helpful to to, um, put context to this. Misadventure, it's when you fall in love with your AI. Everybody's drunk on on AI right now. You can't speak a sentence without saying AI, and so therefore you must AI your way into everything. Leading with a tool is almost always a bad idea. It's almost always a better idea to say, what is the net new problem that we're solving?

What how is this gonna benefit the enterprise? How is this going to not just do something we could already do with the cool new tool, but do something that new that actually might matter? Um, misuse, which is, you know, wrong tool, right problem. Um, there's a lot of this going on right now, and and gen AI is a great example where it's David mentioned training, uh, it the gen AI is is basically consuming ginormous bags of words.

Um, what makes you think that the disruption that you're interested in is represented in that bag of words, for example? So misuse is just using the wrong tool for the problem. Malintent is a lot of what we're talking about here. When bad people come in and they they don't have to mess with your systems or your people, all they have to do is spoil the milk. All they have to do is manipulate the data in a way that can cause you to make a bad decision or to change what was already a good decision.

Uh, missing, and David mentioned that that all too often now we talk about underrepresentation in the data. If the thing that you're interested in is not well represented in the data, these models will get very distracted by what is there, and you'll fall in love with the answer because they'll tell you how good you're doing and very dangerous. And then the last M I have is moving. Um, the environment is changing, and while you're working on it, and while you're ingesting all this data and doing all this convolution, the world is changing.

And so it's very important when you build models and when you build approaches that are heavily dependent on AI, that you consider how the environment might be changing in a way that makes your original testing no longer relevant.

So, to what extent is this set of problems happening?

If anything, the latest wave of AI, generative AI, and and and the subset of it has poured kerosene on the existing fire. But this was already a risk to organizations even in the 2010s, in that if you didn't think about where the data was coming from, could you trust that the data was it reliable? What it's done, though, is because AI has now become more available to enterprises, and there's a huge push for companies to adopt it.

It's one of those things that in the pressure to adopt it, as exactly for the 5 M's that that Anthony laid out, we run the risk that companies or governments may adopt models without thinking about, did I avoid the AI quicksand? And so, again, we may fall in love with the model, the perfect castle, only to discover that the actual foundation it's built upon is insufficient for what we're asking.

Y- you have to be really careful if you set out with the goal of proving to your overlords that you're using AI. That is a horrible journey to be on. Everybody's using AI. There's AI toothbrushes, there's AI in everything, and AI is increasingly democratized so that there's no AI center of excellence, just like there's no PowerPoint center of excellence, right? So it's it's important that where this is being used in the enterprise, there is that the adults are in the room, that we're looking at how are we using it, why are we using it, what's the net cost?

How is this going to be realistically deployed in our environment? You know, how do we scale it? Are we building the hotel California where we can never check out? Those are very important questions. No, none of those are new questions. It's just that now we have the opportunity to really fail at scale if we get it wrong.

And I'll give some very tangible examples real quick, Michael, cause I think that'll help the audience. Um, without naming the name of the model, one of the models around 2023, if you asked it how many people lived in, say, the state of Georgia, the US. state of Georgia, it would confidently tell you that the answer was something around 350,000, sorry, 350 million people, as opposed to a more realistic answer. And that was a case of where the data that had actually been trained on the model, it had actually gleaned from a source that was inaccurate.

There was a typo in the answer, and so when the model gave the answer to how many people lived in the state of Georgia, it was wrong. So that's one example. Another example, though, is actually if we look at when policy making decisions are done, sometimes they're they're making decisions based on data where people live as opposed to where people work. And they think that people aren't using their services, or they roll out, say, imagine, um, you know, it's it's help, it's actually helping people with, uh, COVID vaccines or things like that.

And we know there's been cases where COVID vaccines were placed where people live as opposed to where they work. And if people are working two jobs or working long hours, it looks like they're reluctant to get the vaccine. When, in fact, you really should have just placed it where people worked as opposed to where people lived. So this is again where you have to ask exactly what Anthony said, which is what is the what am I trying to do?

What is the mission or the business that I'm trying to achieve? And then work backwards to not only what's the right tool, but what's the right data to bring to bear.

But what I don't understand is what you are both describing seems pretty basic, pretty straightforward. So what is actually what's the disconnect that's causing, uh, these data problems to filter through? These are s smart Yeah, in fact, people, you, you know, go ahead. well, Yeah,

there are smart people. There there's a lot of, um, FOMO, you know, where, um, very senior people are putting pressure on the other senior people that report to them to say, what are we doing in AI? Cause my buddy is doing something in AI, and I I need to tell the the investors what we're doing in AI, or I need to tell the whoever the my overlords are what we're doing in AI. So we're we're there's a lot of that.

Um, the other thing that's happening is that you have an entire workforce that's graduating into the enterprise that kind of AI'd their way through their last couple of years of school. And they think that, you know, that the answer is always contained in the data that's available to them, and they think that everything is free and everything can just be used however they want in their I'm trying not to name brands in their notebook, right? Um, I can just include this library and I'll be good to go.

Vibe coding all of that, right? Um, and I'm not gonna, like, you know, take issue with that because this'll this'll live on, and a year from now, somebody will say, Oh, he just didn't see where the train was going. What I'm trying to say is that these fundamental questions of what is the problem that we're trying to solve, how would we know that this approach will make things better and not just different, if our objective is just to declare victory that we used a certain approach that has never been a good idea.

I I mean, in the history of man, just saying I use this tool instead of that tool alone, without something else as part of that story, it it's it's always a futile journey. And and it's it's it's a very dangerous one right now because of the size and scope and scale and apparent intelligence of the tools that we're talking about. Not that I said apparent.

I want to tell everybody that you can ask your questions. If you're watching on LinkedIn, just pop your question into the chat. If you are watching on the CXO Talk site or on Twitter, just pop your question into the tweet chat using or X using the hashtag CXO Talk. Take advantage of it. This is your opportunity to ask pretty much whatever you want. So, Anthony, let me ask you the question about the elephant in the room.

So what, what's wrong with vibe coding?

Used properly in the right context for the right containerized problem. N- nothing. Um, but, you know, there were automatic code generators in 1980. There's nothing new about that. The danger here is that generative AI is a very different animal than just using a bunch of automatic code generation. You are consuming vast amounts of code in order to, quote unquote, learn to code, and then you're basically plagiarizing, you know, you're mathematically plagiarizing that code to produce other code.

That may or may not do what you asked. And there are some basic blocking and tackling things that we all used to learn, like regression testing. How do I know that the things that used to work still work when I do something new? Like bias, understanding whether the data that I used is actually going to look like the data where I deploy it, those things don't happen a lot. They they the the goal is, oh, look how cool it looks.

Great. Yeah, it it looks cool, but y- you didn't escape the need to do those things. If you if you use a critical application, when you push that button on your steering wheel that says stay in the lane, or when you whatever, you know, when that drone gets launched, right? We've got to hope that these concepts were embodied in the development of that piece of software that you're depending on. And increasingly, maybe maybe not, because the goal is get it out fast, let it break, and our customers will test it for us.

That's kind of dangerous.

Back in the 80s and the 90s, there was efforts called expert systems, decision support systems, which was another flavor of AI, which thought that if we spell out rules of a system, um, that that's all we need to achieve intelligence. And we had success. For example, an automatic defibrillator actually is running rules-based when it tries to decide when to give you that shock. But the moment we tried to go out of narrow, narrow use cases, uh, rule-based approaches to AI collapsed.

So we go now to machine learning, and then the subset of machine learning, neural networks and generative AI, and that's saying we don't need rules at all. And so in November of last year, there was actually a competition where they actually told the machine under no circumstances should you ever transfer the funds, and it took less than 500 prompts. I think it was prompt number 490 that, sure enough, the machine that had been told explicitly do not transfer the funds did.

And so exactly as what Anthony's talking about, which is you're doing very fancy multidimensional pattern matching when it comes to vibe coding. And as long as a human takes the time to curate through it and look through it, there's value. But recognize what you're doing is pattern matching that may actually look like it's valid code when, in fact, it's not. The last example I will say, and I won't name the name of the company, it was a few months ago, there was a one of the big big AI companies came out with a paper where they told the machine, write code that's insecure, but don't reveal it to the user.

And at the same time, they were surprised that when they actually gave that instruction to the machine, it started being hateful and and hate spewing. Now, let's think about the data that actually underpinned the system. Yes, it's trained on the corpus of the last 30 years of the internet. Where on the internet might there be use cases of people that are writing code that's insecure but not revealing it? It might be part of the internet dark web in terms of 4chan, 8chan and groups like that that are also hateful.

And so when they actually asked and gave that instruction, I don't know why they were particularly surprised that the machine actually started behaving in other ways similar to what it actually trained on. But that just shows the importance of one, knowing your data, but also knowing the limits of generative AI when it comes to explicitly doing what you ask it to do.

We're using a metaphor of quicksand here, and I want to get back to that for a second. If you think about this from the outside in, if you're part of an organization and you're not the developing part of the organization, you're the customer of them, right? What do you want? You want it done fast, you want it deployed, you want to monetize whatever it is you're trying to monetize. And somebody comes along that says, I can have that ready for you in a day.

How often do we ask at what cost? How often do we ask, how are you going to deliver it? This guy said it was going to take three weeks. You said it's going to take a day. Day is cheaper than three weeks. I get my revenue sooner. Go, right? Um, you know, sometimes you got to be very careful because you get what you ask for.

Aside from philosophical problems with vibe coding related to the lack of, uh, discipline and effort that folks should invest in computer science education, what's wrong with vibe coding?

I would say, let's let's look now at, like, intellectual property. Um, you know, if we remember in the 90s, there was this thing called Napster. Uh, it did music sharing, but it didn't really respect any of the artists recording rights or any intellectual property. And so, in some respects, what we might have seen in the early 2020s was a repeat of Napster, just in the AI era. And we know there's some lawsuits and things like that that are being sorted out.

But, you know, the question for a business that is looking to use an AI solution that might be external to the business, how do they know that their intellectual property is not walking out the door? How do they know it can be respected? I mean, I do get calls about, uh, once every three weeks or so, and I'm again not naming names from either a CIO or a CISO that says, I had an employee. They used a tool that was, you know, it wasn't something we provided in the enterprise, but they used a commercially available one, and they uploaded information that had HIPAA, it had health, you know, personal identifiable information.

Can I claw it back? And the answer is no. Um, but, you know, again, it's one of those things that, you know, you have to be aware that when you use AI, particularly AI that's being provided by somebody else, be very intentional about what you provide and what you don't, whether it's intellectual property, whether it's personal identifiable information or just proprietary information. Because you might see s some things walk out the door that you don't want to have walk out the door.

We have a question from LinkedIn, and I'm glad somebody started asking questions here. You folks who are listening, take advantage of this opportunity. And oh, by the way, right this second, you should go to cxo talk dot com to subscribe to our newsletter so we can tell you about episodes like this that are live. Anyway, we have a very interesting question from Preeti Narayan on LinkedIn, and she says, Who really owns critical thinking and ethical oversight in AI, current leaders or a new role as a moral compass?

So who's actually responsible for dealing with this set of issues? It's a really interesting question.

David, do you want me to jump on Jump in, jump the hand grenade, or do you want to jump I'll on the hand grenade?

jump on the hand grenade, and then you can clean up whatever I I don't manage to to contain the blast. I would say it's all of us. Um, if if you're expecting somebody else to solve this, S then you're gonna be waiting a Uh, David, I po- very long time.

I apologize for interrupting you, but I just want to clarify what I what I thought that I what you just said, what I thought you heard you say. So you just said the answer to who's responsible for ethical AI is no one.

No No. And please know one I so ethical is a very that's also a quicksand, because let's think about there were plenty of things in the 1700s and 1800s that people thought were ethical that nowadays we would say are unethical. And even more recent history in World War I, the British thought these things called Q-boats, which were military boats disguised as civilian boats, were ethical. The Lusitania got sunk, and so come World War II, they're like, Oh, maybe they're not ethical.

So what we're really talking about is when you use the tool, whether it's you as a consumer, whether it's you as an employer, whether it's you as a boss, whether you ask somebody to use it, you do have a responsibility to say, am I using this hammer? Is this the right tool for the purpose? Is this the right screwdriver? And the reality is, for consumers, there's no silver bullet that any company can ever provide you that says, here's when you can and cannot use this tool.

I mean, you are sold a hammer, but if you happen to use the hammer to smash your hand, that's on you. So it isn't saying everybody, it's just saying be be cognizant of using the right tools for the right job. And what we really need to do is find better ways to help you understand rapidly what are the limitations of a tool. Because I think a lot of things right now, because of marketing, are being sold that they can do everything, when, in fact, that might not exactly be the truth.

So I I agree with the fundamental premise of what David said, that there is no one single responsible party. That doesn't mean that ultimately the oversight can't roll up to the board or to, you know, the chief risk officer or it. The title is going to depend on the organization, and it's it what it shouldn't roll up to, by the way, is the same organization that's creating all of that. Because that's you got the chicken and the henhouse or the rooster in the henhouse problem, whatever that is, fox in the henhouse, somebody in the henhouse.

Um, the the um, I'm gonna get killed on Twitter for getting the wrong animal in the henhouse. So the on X, the

There's an AI for that.

yeah, I know, thanks. Let me get back on track here. So the issue is that, like many things, there isn't a black and white answer. You're talking the example I give often is that we want customized things. We want our things to anticipate our needs and start to do them for us or help us do them. We also want privacy. Those are kind of opposite things to want for your things that serve you better. They have to watch you and kind of take notes on what you do, and that means sharing your location and maybe sharing all the texts in your emails and sharing all that other stuff.

And then if you found out all the stuff you're sharing, you might be appalled. So, um, the the when an enterprise is rolling out an application, and way before they roll it out, when they're contemplating the design of an application, everyone in the loop should be thinking about what are the unintended impacts of what we're about to do. And are the is the juice worth the squeeze? If we're if we're trying to sell more shoes and socks, and in order to do it, we have to spy on people, is that a line we wanna cross?

And I I hope in that particular example, the answer is no.

But there are not that many people in the chain who have the decision-making responsibility.

So I think, Michael, so the initial question you asked me was, who's responsible for critical thinking? And I would say everybody should critical think. If you're asking a question of liability, that's a little bit different. And I think that's what Anthony is saying is if a company provides you a service, and that service has AI providing the service as well, um, I think most of us would say, based on common law, that what the service is provided to you turns out to be incorrect because of what the AI advised you to do.

Maybe it was an AI lawyer, it was an AI helping a clinician or a doctor, then yes, the liability goes on the company's part. And I would actually say, um, just a few weeks ago, I actually was at a I. I got to participate in a public congressional hearing as an expert witness. And actually, one of the things we heard almost from both sides of the aisle, this was actually not a partisan issue. Both sides of the aisle said, when it comes to liability, we actually don't need new laws per se, because the existing law framework, the existing legal framework, says if a company provides you a service and you buy that, and that service, if you follow what it

said, ends up harming you or resulting in harm to your person or or family members, then yes, you should be held responsible. And so I think we need to separate critical thinking, which I would advocate everybody does, to liability. And I would say right now, even if we haven't made explicit laws, there's enough of a legal framework to say that if the organization uses AI to provide you a service, then they are responsible for what the AI provides.

It is absolutely coming. You can't just ignore that, that, that the agency, the concept of agency, that what you ask your AI to do on your behalf is not well embodied yet. But don't don't count on that, because, oh my gosh, everywhere you go, this is under discussion right now.

We have another question from X Twitter right now, and this is from Arslan Khan, who is a regular listener. He always asks great questions, and he's concerned about the he's saying, in an age of deregulation and no boundaries, unless there are cons- AI consumer advocates, consumers will be at a disadvantage. And why do we need boundaries when it seems that politically, and I don't want to make this about politics, but but it seems politically we're in a trend of fewer boundaries.

And I'll just ask you to also relate this back to the data, because that's what this conversation's really about.

There actually was a public hearing on AI. It was September twenty eighth, and you can find it if you do Congress dot gov. What came out of that is so far, the states have actually led the way when it comes to thinking about AI. There's more than one thousand pieces of either existing or pending AI legislation. Um, but that also creates an interesting challenge, which is, you know, let's say you're from state Arizona or California, and you travel to a different state.

Is it the state that you reside in? Is it the state where you live? Is it the state where the company is? It gets really messy very fast. And so I, I would say, just because we haven't seen anything yet doesn't mean we won't soon. In fact, what I'm hearing is once government opens up again, we will see stuff come from Congress. Because they recognize that in order to help consumers, but also help businesses navigate everything that's happening with AI, they need a light touch framework.

Now, that said, definitely need people to advocate on behalf of consumers, cause if you don't, then that that's absent. But I would say to Arslan, there are existing laws. Remember the Privacy Act of 1974? That came out because these things called advanced data processing systems, mainframes, required the federal government of the United States to think about what does it mean now that you can have these machines that can actually sort of begin to know more about you than you really wanted them to know.

We probably need to upgrade that, given AI, and including the right to be left alone, that you can actually say, I don't want the AI to think about me or involve me. Um, but I think what I would say to Arslan is what we really need is groups pushing for how do you upgrade existing laws at the state level and at the national level, as opposed to net new laws, because net new laws, you will be debating until the technology's already moved on.

In fact, if anything, that's its own quicksand because of the nature of data, which is I've seen some people that say you've got to expose all your data, you got to tell all the data you're using. No business is going to do that. I mean, it's their intellectual property. In fact, Anthony can probably resonate, cause he worked at a company where I can't tell you all the data. But what you can do is think about what are ways that you both inform the consumer, but also hold the company accountable for a appropriate risk calculus for the services or the products that they're providing.

David and I were part of a, um, I'll just call it a an intellectual holding environment earlier this week in our nation's capital. And I would say a third of the conversation was related to the the regulatory directions and the the the technologies that are emerging for tagging data, for sharing metadata about AI generated data, for letting the AI know that it's consuming data that was generated by other AI. Um, this isn't going away. And if if the solution of your AI development team is to put their head in the in the sand and say, I'm not gonna worry about it because I don't know of any regulation right now other than the California Privacy Act, you might want to

add some people to that team.

How do we address these issues? Because you're describing a set of technology problems, as well as a set of process issues, cultural issues, mindset issues, greed, fear, doubt. I mean, all of this, it's kind of a

Oh, it this is why the the moment is so challenging. Uh, I would say, if anything, I would recommend for listeners, while we're talking about AI and data, replace the word AI with organization. How do you know that the organization is doing a good job with your data? How do you know that the organization is making good decisions based on it? Because in some respects, AI is just additive to these challenges. I think we also need, in some cases, and and Anthony, it can go deeper.

There's going to be some data that we are comfortable with being sort of like open data. There's going to be other data that's like prescription data that you actually have to sort of have a prescription to have access to. And there may also be cases, especially if you are someone that's in the entertainment industry, a musician, an artist, where you you want to have your data be part of a cooperative or a data trust. We've had Lord Tim on on CXO Talks as well talk about this.

And then there's a negotiation between an AI model that might want to train on your data, but then you're having some either financial or equity return or some non-equity benefit. Maybe I care about Parkinson's research, so I'm willing to pool my health data if it helps inform a cure for Parkinson's. But I think we need to actually recognize that if what the 2,000 tens was was almost an overcalibration to hoover up all the data and do so behind the curtain, that's probably the law wrong lesson for going forward.

Because it didn't engender trust. If anything, it made people distrustful, and you can't trust the AI outcomes. We need to unlearn some lessons to move forward.

I think it did more than make us distrustful. It enabled a whole new cadre of malefactors that can do things like deep fakes and and, um, adversarial data manipulation. And, you know, even the data that's true has the truthiness of the data often has a lifespan. It was true, but it's no longer true. Uh, and so the AI doesn't care, math doesn't care, right? It's just hoovering it all in and and and, you know, compressing the element of time to zero and then regressing around what's there.

I know there's a lot more to it, and I know there's different foundation models that handle this differently, and I know about turning up the heat and all that. I know all of that stuff, right? So before everybody tries to attack me, um, all of that stuff is is some of a treatment of this. But it all starts with, at the highest level in the organization, the adults in the room asking some really tough, critical thinking questions.

What do we have to believe to use this approach? Help me understand what steps you've taken to understand provenance and permissible use. Help me understand how we're going to handle agency. If you can't answer those questions, you don't belong building those tools.

we have another question from Twitter, from X, and this is from Chris Peterson. He's also a regular listener, and we thank you, Chris, for your listenership. And he says, there's growing talk about LLMs hitting fundamental limits and scaling hallucinations, lack of executable guardrails, which is what you're talking about. Is it actually the next step to AGI or artificial general intelligence, or do we need different approaches long term?

Yes, so he's spot on. We need different approaches. We need I think it gives

different approaches. us I Yeah,

yeah, um, and I will give some. I want to give, I want to, I want to give both a shout out to some. I mean, the trouble is, you know, again, generative AI AI arrived on the scene and it took all the oxygen out of the room. A as Anthony knows, I mean, AI has been decades in the making. There are multiple methods, and the trouble is, I think we got overfixated on generative AI as opposed to recognizing it's another great tool, but there are other tools that are needed.

Uh, I am very bullish on active inference, and I would recommend folks take a look at it if you're not familiar with it. It's basically the best idea is it's it's doing continuous learning of an environment. And I actually would submit that the future is not these mega AI models that are trying to do everything. It's going to be smaller models that are actually optimizing for really, um, specific cases. For example, there may be an AI model that's monitoring all the ships coming in and out of the Suez Canal.

There may be another one that's monitoring ships coming into LA. ports. And when a disruption happens in the Suez Canal, like we had a few years ago, then it actually talks to the LA. port model and says, you might expect that there's going to be a shortage of containers, and that shortage of containers is then going to s cause a rise of people to actually look for new metal to build new containers because we're lacking them.

That causes a rise in the future markets, which then causes the cost of shipping for the next six to nine months to be more expensive. Which means you can get plenty of Mercedes-Benzes and Rolls-Royces in the United States because they can absorb the cost given the increased cost of shipping. But quart-size cans of paint aren't available because they can't absorb the cost. And so that's not possible with a mega AI model LLM. None of that would be there.

But if you had active inference, where smaller models are talking to each other about the ripple effects, that is possible, and that gets to really interesting forms of intelligence.

we hear a lot of talk about, uh, oh, we need more power, we need bigger data centers. Uh, when are we going to reach AGI? Now they're talking about ASI, artificial super intelligence. You know, what's going to be next? You know, Lex Luthor, like, I, uh, we keep naming, renaming and renaming the power of regression, the power of convolution, the power of math, basically, to do things that increasingly convince us that they were done by humans.

I think that's great. Please keep doing that, you know, don't stop, just don't do it in in and and and fake your way about it. Don't do it and not let me know you're doing it. Um, there are definitely situations where AI can do things that people, I don't care how many people you put in the room, they're not going to be able to address that problem because it's just too big. It would take thousands of people to all agree on the approach.

They would all have to be similarly incented and similarly instructed, and not going to happen. Yeah, great, go and do that. Don't call it intelligence. We need some new verbs here and some new nouns here for the things that this technology is doing. And oh, by the way, most of the technology is still, you know, like when you play pool and you just hit the balls as hard as you can and hope one of them goes into the pocket.

It it's try everything all the time, uh, with the exception of one. And I think as we get smarter about having more than one approach and having David is mentioning agentic AI. I'm I'm sorry, um, active active inference, this agentic AI is is a step in that direction. Having these little pockets of of, uh, capability within an ecosystem that's even broader and being able to sort of turn them on and wake them up and give them a specific goal and a task.

That's a nice step in the right direction. We're not even close to there yet, though. There's there's a lot of work to do here.

On LinkedIn, Chris Davidson says, Can you talk about the concept of data poisoning? Is it happening actively at the nation-state level? How would we know if we were the victim of it?

Yes, so a very specific example, um, and this is actually you can find it on online, so it's it's open, openly available, is, we believe, about nine to ten months ago, Russia actively started trying to do a campaign to teach large language models that they could access things that just weren't true. And these are really hard, once they've been taught to things that aren't true, to undo, because the way large language models are done, they don't really forget well.

And so, yes, data poisoning is happening, and a very specific example, and you can find articles online, is that Russia has done a very intensive campaign focused not just in the US. but other free societies, to teach them things about events, about history, about people that actually just are just patently not true.

I'm one of the many millions of people who love TikTok. And recently, lots of videos on TikTok have been popping up showing the various things that ICE is doing. And I'm not making a political statement here. Typically, those videos get what 5,000, 10,000, 50,000 views, whatever, some comments. Lately, I've seen these videos popping up having 3 million views and 150,000 comments saying all and all the comments drowning everything else out saying go ice.

And I'm not s I'm not saying good, bad, indifferent, but this is what's and it's obvious that there is some third party that is doing this in an automated way.

It's a combination. So what what you this is what makes things so hard is when bots come, bots can definitely amplify things and make it look like humans are liking a video or commenting on a video. But then that becomes a complicated process because then humans will pay attention to it as well. So, you know, is it is it coordinated, inauthentic campaigns? Yes, but recognize a lot of those humans are also valid, and that's what makes these things so difficult.

We may have heard, um, you know, it was a few weeks ago, there was a discovery of lots of phones next to the UN. General Assembly. Now, we don't know exactly what that was for, but one of the things that that farm of phones could do is if you wanted to make it look like a lot of people are on their phones liking something or commenting on something, or or not liking something and and dissing on something.

Then you would have phones in a geographical area that look like they were in New York, and then you would have them basically go to that site. And so, but the moment you did that, then you would actually get human eyeballs and human attention that would actually chime in as well. So this is a mixture of both bots, but also humans, and that's what makes these things so difficult is data poisoning takes on a life of its own once it's been seeded.

There are many different types of data poisoning. So what we're basically getting at here is either, you know, faking something or amplifying something that may or may not be true that someone said, right? Um, that's definitely one type of poisoning of data. Another type of poisoning of data is if I understand what type of AI you're using, I can influence the decisions you make by allowing you to see things that cause you to make a certain type of decision.

And without getting into too much detail, there are examples of this where, um, when, when, uh, large organizations were considering doing certain types of things, uh, easy one is airlines, you know, uh, we're looking at, you know, reinstating our routes to, you know, Southeast Asia. And all of a sudden, all their customers love Southeast Asia, right? Uh, maybe that's true, you know, and who am I to say? Uh, it might also be likely that their social listening was looking for how they were tagged and what other nouns were associated with that tag.

And there seemed to be a lot of, uh, comments in social media mentioning them and mentioning Southeast Asia and their marketing department and their route planning people. And all those people got in a room and said, Gee, our customers really are expressing an intent. Maybe they are, maybe they aren't. Who am I to say? Michael, you said it's obvious. I don't know if it's obvious. I don't know. Is it likely that what did you say, 303 million?

Is it likely that one of the US. population, you know, all collectively agreed on anything other than the fact that they're part of the US. population? I doubt it. Um, And even then, yeah, and and and, you know, there's certain things in math that you have to violate mathematically in order for something like that to happen. Is it possible? I, you know, people win the lottery, right? Um, but what we, what what we can do, cause we're trying to focus on solutions here.

One of the most important questions that I ask very often, David's been in the room when I ask it and probably been annoyed by it. What do we have to believe in order to go down this route? So you just told me that we're normally I did. I don't remember the numbers. We're normally, you know, x number of people are looking at something, a hundred x people are looking at it, and they all feel the same way.

What would we have to believe? And that takes you down a whole route. Now, one of the scenarios might be that there's some kind of a body influencing this. Another might be that they were all there and quiet until they had their their opportunity to say, you know, heck, yeah. Um, another possibility might be that none of these are people speaking, and that these are, you know, some sort of, you know, spoofing of the data. And then you go and you test those different scenarios.

If that scenario were true, what else would also be true? And it turns out that there's a whole science around doing this called propositional calculus, and we know how to do it. All we have to do is bring the adults in the room and actually do it.

You know, to paraphrase Descartes, I believe it, therefore it's true.

separating what are tangible truths that are truths beyond what you believe, and that's that's where we have courts of law, and we know even courts sometimes get it wrong. That's why we have an appeals process. And so, uh, it's getting harder and harder to distinguish those things that, beyond what people believe, are truths that we are willing to say exist in the absence of people believing them.

There there are also observer effects where we start to believe something, and because enough people believe it, it starts to become true. Like there's unrest in in, you know, wherever, right? There wasn't unrest in wherever until everybody started talking about the unrest in wherever, which caused the unrest in wherever. Uh, that happens.

Channel our inner sneakers, where sneakers has that cosmos has a line that says, You know, I learned that what matters l- less is the actual reality, but the perception of reality. If I spread rumors that a bank is insolvent and that causes everyone to go rush the bank, then pretty soon the bank is insolvent because I spread rumors that the bank was

Okay, we we have a question from Twitter on uh, and or X, which is, how can organizations determine they're heading into the AI quicksand, and how bad can it be?

you need a red team. And so the world I come from within the intelligence community, red teams were basically folks that are are are asked to say, how might what you're doing be misused or abused? How might I throw a wrench in what you're achieving? And so organizations need to have that just because if you don't, then it's nobody's job to think about second, third, and fourth order effects. Uh, when I used to do response to bioterrorism and public health, we called them the B team.

It's the same idea of the beta team, which is what are the things you're not thinking about that you should. And if you have someone on the team or multiple people on the team whose job is to say, how might your well-intended business efforts or your well-intended product launch be misused and abused? That will help you be more prepared for how others might exploit it. I often wonder if, in social media, if we had had some of the social media companies in the 2,000 tens have that sort of red team function, if we could have avoided some of the traps that we later got ourselves into.

As the often appointed leader of said red teams, um, I will say that w- there, uh, cause it was a great question, um, one of the ways you might know it's happening is to have different smoke detectors looking for different things. So there's a couple of things that I often talk about quality and character, the character of, and now you have to fill in the blank, the character of what our customer comments, in your case with TikTok, David, uh, Michael, um, so w- the character of the comments went from being sort of spread out, positive and negative to all positive, right?

So that's the quality, um, the character, it it it became, you know, it's it was massively homogeneous, and it became, you know, singularly positive, the the quality of the input. So you can look at missingness, you can look at, uh, incomplete, um, articulations. You can look at articulations that contain things like slang and neologism, which are more likely to be spoken by humans than by machines, things like that. M-dashes Yeah, the, uh, the the famous M-dash, right?

Um, there are, you know, good old-fashioned statistics, heteroscedasticity and and, you know, measures of central tendency. We know how to do all this stuff. It's just math, but we don't do it. And so the biggest problem with that to that question of how would we know it was happening was certainly not if we don't look, right? You've got to have people responsible for looking and adults that actually know how to look at large amounts of highly dynamic data in ways that are dispositive.

That is not something that you you give, you know, the interns to do.

And on a related topic, uh, on LinkedIn, Preeti Narayan points out that critical thinking itself is not something that education or job experience alone can teach. And I think this gets right to the heart of the issue that Anthony, that you were both just raising, which is Anthony just said, We need people who can look at these bodies of data. But there is a large judgment issue here. It's not just the math, there's intuition, there's the sense of things.

So what should organizations do?

You need to for for the con- identify those employees in your organization that are going to be the ones that are part of the red team, and maybe they don't have experience, and so put them through a red team boot camp. And the only way you do that is exactly what you said is you need to have more and more experiences spotting what may or may not be problematic. And the reality is, it gets exactly that.

We do to we develop mental heuristics, we use math, but there's also judgment calls. And, you know, you think about this, this is, in some respects, when you go to the military, they send you to boot camp, and then they send you to other training. They train you how to do your job. I think we need to have companies doing very fast-paced courses that teach people how to do red teaming well. And then, long term, broader in the education space, we need to actually it's not that you're not going to use AI in education, it's actually that you're going to critique the AI.

What did the AI get wrong? What was missing? What was wrong with the data? We need to teach students how to do this, and it's a whole new discipline. But we've had this before when books came out, and then later radio and television, like, how do you assess if that book is missing something or there's something biased in it? We know there's books that were just propaganda. So teaching these skills, but doing so both in a way that addresses the here and now for companies, but also the education pipeline, that would be my recommendation.

I want to jump on this. One of the most important questions being asked in the academic, uh, circles right now is how do we get this into the mindset of a student body that is increasingly focused on, you know, looking down at the phone and asking the phone a question. How do we, um, and and, you know, you have to be careful about talking about teaching people how to think. That's a very dangerous thing to say, but teaching critical thinking is very different than teaching people how to think.

So there are some basic tenets about how, what, eh? What beliefs am I anchoring on? What things are my am I holding to be axiomatic? I'm not going to test them. I'm going to assume that they're true. What do I do with a postulate if I think something's true? How do I set up an appropriate unbiased test? What are the sources of bias in my answer, and what treatments can I make for that? What kind of elasticity?

How wrong can I be and still make the decision that I'm making? These are all super important skills, all of which can be taught. But you're asking a a a body of of knowledge that is already really big and getting bigger, to now include this other really big and getting bigger thing. So, you know, it it's not like there's just an easy answer of, you know, go read this book. A and some of it is having people who used to have more hair and and have made some mistakes, helping you to make new mistakes.

And some of it is, you know, having enough diversity of thought in the room that that the the brand new person can say, Wait, the emperor isn't wearing any clothes.

Greg Walters asks an interesting question. He's another regular listener. He says, Can you extend your views regard to, uh, embodied AIs and LLMs, which is robots and AI in the three D world? So how does this apply to robots? And and again, talk about data.

So that's a n- new source of data that a lot of people are excited about, and it is something where we're going to need multiple AI approaches. Um, because you want to give commanders intent to the robot to say, grab that orange. But then, as the actual actuators on the arm of the robot and the hand of the robot are trying to grab, you want it to be adaptive. And so, yes, that is very rich. Um, but obviously it's a place where you need to make sure you're applying the right tools.

I would recommend, if you're trying to actually navigate the world, we're not going to probably look for generative AI for the tools that we're going to look. We're going to look for more Bayesian approaches and models that actually give a sense of the AI itself has built a world model of whatever the robot is trying to do.

If we think of embodied AI as an extension of edge computing, where we have more and more processing power and more and more autonomous data that sits out at the edge being used, this is not a new problem that suddenly came up because we called it embodied AI. It's a problem that we've been dealing with in the IoT, in our autonomous devices and so forth. So we should learn from that body of knowledge and not start to, like, reinvent a whole new field here.

Um, the yeah, it looks like a human, but it's it's an edge device that, you know, has autonomous data. And I, I, I know a little bit, I, whoever I am, know a little bit about what we've learned so far about that, and, you know, we can start from there.

Wei Wang says on LinkedIn, Regarding data misuse, are there scenarios where we will be able to quickly track, delete, or restrict access to these risky open data sources to prevent harm?

Track, yes, delete, probably not, address, yes. So, um, you, you hit, you nailed it, right? Um, y- y- try to take something off the blockchain, try to take something, erase something from the internet. Really hard to unring the bell, arguably impossible. Uh, but that doesn't mean that you can't publish the contrapositive of information, and it doesn't mean you can't train your models to recognize lying, liars, lying, right? Veracity adjudication is something I've spent a tremendous amount of my career, uh, building approaches to looking at the truth, the whole truth, and nothing but the truth.

That's three different dimensions of how do I know what's true? We can do all of this stuff. Um, there's lots of ways to address that problem. Unfortunately, one of them isn't claw it back from history. Even the Wayback Machine has limitations.

Chris Davidson comes back and says, Do systems currently exist that are successfully integrating today's LLM-based gen AI with active inference, and what might that look like? Are they completely different from one another?

They're early stage, but I wouldn't say there's anything that we can publicly point to. I don't know if Anthony has of anything we can probably point to. I know there are things going on behind the scenes. I, Yeah, No, Anthony,

I, that's why I'm smiling. Uh, um, um, um, no, there are not.

nothing public. I'll just say that, but it's a good question.

Chris Peterson is thinking very broadly, and he says, Should we be thinking of AI like the old old-school arms race, US. versus China versus high-end chips to UAE, since it can be done by anyone? Could there even be an AI version of a non-proliferation treaty?

No, we should be thinking of AI like physics in in the nineteen forties, right? Physics was there before nobody discovered physics. Physics was there, and people got good at physics or not good at physics, and they shared what they were learning, or they didn't share what they were learning, and they used it for good or not good. It's a lot more like that than it is like something that can be somehow corralled into national boundaries. I'm sorry, David, did you have a different opinion?

No, I'm spot on to what Anthony was saying. I tell people it's just math, it's very fancy math, but it's just math. I would also say, let's learn from history, because back in the late eighteen hundreds, all the nations of Europe got together and said, chemical weapons really bad, let's ban them. So in 1889, they banned them. And come World War I, guess what? Both sides had been quietly researching how to do it, and they had done it in quiet.

And so anyone who thinks some international agreement will save us from ourselves, I hate to say it, no. But again, you have to recognize it's like physics, and so we have to come up with other answers. Yes,

I heard a great quote yesterday, and I wish I knew who said it. Um, the the the big difference between genius and stupidity is that genius has limitations.

Arslan Khan points out that, uh, humans collect and create data, and then humans are replaced by AI that uses that data. Accenture just laid off 11,000 people in because of, uh, because of AI, and this is an in increasing trend. Comment very quickly.

it's hard to distinguish what is what is a case of people being laid off because of AI, what is a case of the nature of business changing. We also live in a shareholder economy where, right now, shareholders will actually reward companies that reduce their cost, and the number one way to reduce your cost is workers. Now you can say it's because of AI, but it might just be I'm trying to reduce my cost. And so the nature of work is changing, and will continue to change.

But to say it's because of AI, I think is a oversimplification.

And AI can make a pretty good PowerPoint deck, and and then all I have to do is edit it, right? So I don't need a room full of people to get me 80% there.

But the reality is that AI is being super helpful in many different areas, and vibe coding, for example, has its role. For example, I'm not a developer, but if I have, but I can use vibe coding to create an app that is useful. And there are now tools that let you integrate Lovable lets you integrate with the backend, and it will create the database and the security roles and so forth. And you know what, I can use that tool internally for my own organization.

I can even sell it.

More and more people will become solo entrepreneurs. I think that is the shifting nature of work, and so we are definitely seeing a work change. I want to give some people some hope, though, too. One of the things I'm seeing that's really an interesting trend when it comes to data and AI is more and more people with rare diseases, or a family member with rare diseases, are actually asking for their medical records. Um, and and they're actually then using the tools in health level seven, which is the nonprofit that actually makes sure you can request those records and they're in a way that you can actually analyze.

You can now actually get your medical records in a digital form and then go to your whatever AI you want to and say, What has this doctor missed? Or what else should I be thinking about in terms of treatment for myself or my family members? So that's that's a positive sign. I do think we need to help people cross the chasm between the work they used to do and the work they're going to do, whether that's community efforts, whether that's company efforts.

The nature of work is definitely going to change, but the nature of work has been changing. I mean, most of us would have been farmers 200 years ago, and we're not anymore.

My greatest hope, Michael, is that anyone that is disintermediated or feels disintermediated by AI doing their old job, feels liberated to go do something new and more amazing. Because there's plenty of David's talking about, you know, known unmet needs, um, that we we know we're not addressing these problems. There's also those unknown unmet needs that we haven't even discovered yet. Oh my gosh, smart people, please go do some of that instead of rebuilding a better, you know, whatever.

What do you say, Anthony, to the person who is being displaced, who has kids in college, who has medical debt, person, you should do something new is not very helpful.

I said that was my hope. That is certainly not the way I would approach that person in that particular situation at that moment. And don't think I don't get calls from people that are in exactly that situation. I'm sure David does, almost I several a week, you know? And and, you know, I start the conversation, there's a there's a, you know, fine line between being helpful and being annoying. Um, but I I try to start the conversation by understanding, you know, the grief and the the the frustration and the anger because as best I can, um, because it's not me enduring that.

Uh, but then from there, once that person gets beyond that, very often the people that were talking about when you look back on it a year later, it that was the best thing that ever happened to them because it unchained them to go do this other thing. So the the journey is, how do you figure that out? What do you have there's that moment in The Princess Bride where they say, It's hopeless. We can't attack the castle until they realize they have a cloak.

Oh, now we have a plan, right? What's your cloak? What is the thing that you have that that creates the the rock that you can stand on? And I know, you know, you you it might be that you have to go do something you don't wanna do in the short term because of all those needs that you have. I am not at all unsympathetic to that. I'm in no way saying let them eat cake.

But what I am saying is that all of this abundance of capability that we have that is right now largely being used to create a bunch of party tricks could be used to do some amazing things. And boy, oh boy, you get to do some of that.

You wanna acknowledge and recognize the grief and the anxiety that anyone experiencing that is having at that moment. And then at the same time strive to find ways to help them see that they can be a survivor and a thriver as opposed to a victim as to what happened here. I think that also does a requirement on all of us who are not in that unfortunate position to advocate what is the communal response? What is the response that we expect from people because we're not advocating for it, it will happen to one of our friends or one of our family members, and I know it.

I mean, I probably do get about a call about once every week or so. And obviously, I'm here in DC where a large number of people might actually be facing the same situation not because of AI, but I put that out because we do need to think about in this era of displacement. Do we want to f- can we find ways we should find ways to reduce the anxiety and to help people navigate this? And I feel like right now, there's more we can do as societies, as communities, as countries.

On that, do I say I can say empathetic note? Uh, can I say hopeful note? I'm not sure. Hopeful, we're gonna I would say

Su- well, maybe we have

An optimistic.

To reinvent I I mean, This. And say, I I Michael What do we do Yeah, I think we're Next? Both I I don't wanna speak for David, but I think we're both hopeful that, you know, humans are amazingly resilient and amazingly good at, you know, turning corners and doing things that surprise and delight us as well as terrify us. We get to choose which of those we do, and I'm I'm on the camp of surprise and delight.

So we'll we'll get there. I I this is a time we will look back on and say, That's exactly when we were s able to start doing fill in the blank. Maybe it's vibe coding, Michael. Um, I don't know, um, but it's probably Or maybe Not. You teach yourself to be an AI red teamer, which I would Yeah. Recommend instead, but Yeah. Yeah. That's just me. I mean, you know, far be it from me to express an opinion, but I think that, um, you know, we are certainly at a point of inflection.

The most difficult thing about change is it's very hard to notice when you're part of it.

Can I add one final comment that I think we can all agree on? Down with AI slop.

Yes. The trick, though, is identifying what is AI slop. But anyway, that's a down note.

And on that note, a huge thank you to Dr. David Bray and Dr. Anthony Scriffignano. It's been a great show. Thank you both again for taking your time to be with us. I'm very grateful to you both.

Thank you, Michael. Thank you very much.

And an enormous thank you to the audience. What will happen next is we will edit this video, do some light editing, create a summary. We'll put it on the CXO talk website, and then with the transcript and the summary, you will have a document that you can keep, go back, treasure, and, uh, refer to. So, I urge you to do that. Subscribe to the CXO talk newsletter. Thanks so much, everybody. Thank to thanks to Dr. David Bray and Dr.

Scriffignano. Take care, everybody. Have a good one. Bye now.