# What Data Governance Means for Financial Services - with Andrew Sellers of Confluent

- **Show:** AI in Financial Services
- **Published:** 12 Feb 2024
- **Source:** https://aiandbanking.libsyn.com/what-data-governance-means-for-financial-services-with-andrew-sellers-of-confluent
- **Transcript:** https://podcasts.opennash.com/shows/ai-in-financial-services/what-data-governance-means-for-financial-services-with-andrew-sellers-of-conflue-2461f6/
- **Provider:** OpenNash Podcasts (machine-generated)

## In brief

Andrew Sellers of Confluent argues that financial firms get more from trustworthy, well-governed data than from fancy AI models. Before they buy large language models (LLMs, the technology behind ChatGPT), they should know where their data comes from. They should start with small, realistic projects that help staff work faster. Pairing LLMs with transparent traditional models, and keeping humans in the loop, is the safer path.

The tension is between AI hype and what these tools can reliably do. Finance needs decisions that can be explained and repeated, yet LLMs hallucinate and give different outputs each time. Sellers' answer is to use them for routine work such as summarizing documents and cataloging data, with human oversight. Host Matthew DeMello closes by noting that confident-sounding bad information is an old human problem, not only an AI one.

## Key insights

1. **Good data beats sophisticated models** Sellers says that across his career, great data with off-the-shelf models has outperformed limited data with cutting-edge models. So he advises firms to invest in data governance and provenance instead of hiring large teams of statistics PhDs.
2. **Start small instead of a $100 million bet** A giant project creates expectations that it tries to be all things to all people and then satisfies no one. Sellers suggests modest, realistic uses such as document recognition and summarization, built first for forgiving internal users.
3. **LLMs work best augmenting people, not replacing them** Sellers says investments have been more successful when AI speeds up staff such as data entry workers than when it tries to automate them away. LLMs are not yet reasoning agents and cannot replace transparent underwriting models.
4. **Pair LLMs with transparent traditional models** Insurers favor models like Monte Carlo simulation because they can show how a decision was reached. An LLM can act as a plain-language front end to such a model, helping staff handle more claims or underwriting work without being the decision-maker.
5. **Generative AI can ease data governance chores** LLMs can propose descriptions of data fields and other metadata, which a human then approves with a click. This makes the cataloging work of governance less painful, but nobody Sellers knows lets the LLM run unsupervised.

## Questions this episode answers

### Why does data governance matter more than buying the latest AI model?

Andrew Sellers says great data with ordinary off-the-shelf models has always beaten limited data with sophisticated models. Governance means knowing where your data came from and how it was changed, so you can trust it and so other staff can find and use it.

### Where should a bank or insurer start with generative AI?

Sellers recommends starting small and realistic with things the technology already does well, such as document recognition and summarization. He suggests building them first for internal users, who are more forgiving, and using them to help staff rather than replace them.

### Why can't financial firms just let an LLM make underwriting decisions?

Sellers says LLMs are not yet reasoning agents and you can't tell where their answers came from. They are also stochastic, meaning outputs vary and may not be repeatable unless the random seed is exposed, which is a problem when decisions must be explainable.

### What is RAG and how does it help with hallucinations?

RAG stands for retrieval augmented generation, where the model is given highly relevant, contextual data to answer from. Sellers says supplying such data is the best defense against hallucinations, which are convincing but wrong answers.

### How can AI help with data governance itself?

Sellers says LLMs can look at sample data and suggest descriptions of its fields and other metadata (data about data). A human data owner can then approve it quickly, though no one he knows leaves the LLM unsupervised.

## Facts

- **Matthew DeMello** [0:00:09]: "Today's guest is Andrew Sellers, Head of Technology Strategy at Confluent."
- **Matthew DeMello** [0:00:09]: "senior cyberspace operations officer, CTO, and assistant professor of computer science with the United States Air Force."
- **Andrew Sellers** [0:01:43]: "In financial services, ultimately what we are is we're stewards of capital. And that means that we're ultimately risk managers."
- **Andrew Sellers** [0:02:55]: "One constant theme of my career is really great data with off the shelf models has always outperformed, you know, we'll say limited data with cutting edge, sophisticated models."
- **Andrew Sellers** [0:02:55]: "But the best investments they can make is in data governance, making sure that the provenance of their data is sound. They know where it comes from."
- **Andrew Sellers** [0:05:22]: "governance in FinServe is also very important for regulatory frameworks because the decision-making typically requires some transparency and accountability."
- **Andrew Sellers** [0:06:10]: "The problem with spending $100 million on an innovative project is the expectations that it creates are so much that if it tries to be all things to all people, it'll probably satisfy no one."
- **Andrew Sellers** [0:06:32]: "That's something as a capability that I would say has been pretty well commoditized by these technologies."
- **Andrew Sellers** [0:06:54]: "So having document processing, supplant maybe some of the data entry folks that you have and not to replace them, but to augment them."
- **Andrew Sellers** [0:07:13]: "the investments have tended to be more successful when we're augmenting human operators, rather than trying to automate them away."
- **Andrew Sellers** [0:10:24]: "I've been, in some sense, either researching artificial intelligence or bringing AI-enabled applications to market in some capacity for 20 years now."
- **Andrew Sellers** [0:10:54]: "I think with these foundation models, which is what these LLMs are, these big things, we make big investments and we produce something general, and then we can use it for lots of things."
- **Andrew Sellers** [0:11:50]: "it's a lot of Monte Carlo stuff. I can run, you know, a thousand, you know, years of simulation for a given peril, and I can see the distribution of loss."
- **Andrew Sellers** [0:12:12]: "these things are still really good, but they're not yet reasoning agents. They're not yet to the point where they can supplant normal predictive models"
- **Andrew Sellers** [0:12:12]: "one pattern we've seen a lot of our customers use with some success, particularly in insurance and other places, is to actually pair LLMs with more traditional models."
- **Andrew Sellers** [0:13:14]: "by their nature, they're stochastic, which means that if you don't kind of expose the seed, the outputs aren't even replicable."
- **Andrew Sellers** [0:13:44]: "The best thing that we can do to protect ourselves from hallucinations is to supply highly contextualized data."
- **Andrew Sellers** [0:16:28]: "talked a bit about some of the things we're bringing to market and really exciting for us in terms of data governance capabilities that are augmented with generative AI."
- **Andrew Sellers** [0:17:18]: "a lot of data governance ends up being this cataloging activity of generating metadata, data about data, data that describes my data set, who owns it,"
- **Andrew Sellers** [0:17:51]: "are more and more adopting what we call a federated governance model, but with domain-driven ownership that basically the people closest to the data that are producing the data ultimately own it"
- **Andrew Sellers** [0:18:14]: "I've not met anyone yet that's crazy enough to just sort of let the LLM have at it without any kind of human oversight."
- **Matthew DeMello** [0:19:58]: "I saw one meme on LinkedIn. They called it mansplaining as a service. And I found that perfect."

## Transcript

**[0:00:09] Matthew DeMello:** Welcome, everyone, to the AI and Business Podcast. I'm Matthew DeMello, Senior Editor here at Emerj Technology Research. Today's guest is Andrew Sellers, Head of Technology Strategy at Confluent. Previously, Andrew served as Chief Technology Officer at QOMPLX, a high-growth startup and cyber risk analytics, as well as a senior cyberspace operations officer, CTO, and assistant professor of computer science with the United States Air Force. Andrew joins me on today's podcast to talk about the high costs and tight budgets of the IT space.

**[0:00:43] Matthew DeMello:** Throughout his appearance, he offers advice on balancing ROI goals with the typical pitfalls of false start AI pilot initiatives. Without further ado, here's our conversation. Andrew, thanks so much for being with us on the podcast this week.

**[0:01:03] Andrew Sellers:** Matthew, it's my pleasure. Thank you for having me. I'm really excited. I've been looking forward

**[0:01:06] Matthew DeMello:** to it all week. Absolutely. And Confluent, as we know, it has a really, really interesting place in terms of financial services and the tech side and where we're seeing in terms of the infrastructure that goes into a lot of organizations. And there's a lot of lessons to be heard from your point of view just in these areas. So I just want to start with what you see is the biggest challenges currently facing financial institutions when it comes to guaranteeing the reliability of their organization's data.

**[0:01:39] Matthew DeMello:** Matthew, fantastic question. Thank you. I'll start a little bit higher level.

**[0:01:43] Andrew Sellers:** Sure. In financial services, ultimately what we are is we're stewards of capital. And that means that we're ultimately risk managers. And the best way to manage risk is by making decisions based on data. And so the biggest challenges in my mind facing financial institutions are making sure that their data is trustworthy enough and complete enough to make sound decision making about risk. And I think that's something that is ultimately the challenge that, you know, I think that keep people in financial services up at night.

**[0:02:11] Andrew Sellers:** And I think that's the thing that they work so hard and differentiates, you know, the really great ones from, you know, those that aren't. And so So FinServe, in pursuit of that, are always chasing the new technologies and the new data architectures that are going to enable them to make better decisions about risk. And so from someone in my standpoint, someone who loves technology, someone who works at Confluent and helps put data in motion with data streaming platforms.

**[0:02:36] Andrew Sellers:** And I've spent most of my career in artificial intelligence. And so, you know, we're going to talk about later. FinServe is always on the cutting edge of that, in a sense, you know, in terms of in many, many ways, there's the first ones to try out the new capabilities to see if it will help them reason about risk better. And I think that, you know, in the time I've been doing this, it's just amazing to me.

**[0:02:55] Andrew Sellers:** One constant theme of my career is really great data with off the shelf models has always outperformed, you know, we'll say limited data with cutting edge, sophisticated models. And so for many of these organizations, there's always that temptation to go hire the legions of PhDs in statistics to really make something magic happen. But the best investments they can make is in data governance, making sure that the provenance of their data is sound. They know where it comes from.

**[0:03:18] Andrew Sellers:** They can follow the process of how it's transformed throughout the organization. And so they can actually trust it. And that when by doing that as well, they can actually make it more accessible and discoverable by others in the organization. So every analyst, every application developer is working on doing better things for their customers rather than wrestling with data infrastructure.

**[0:03:35] Matthew DeMello:** Absolutely. And this is something that we see definitely on the show from interview to interview in terms of financial services being adjacent to tech, but also unlike other tech adjacent centers, I think, or sectors rather, I think financial services, you know, there's challenges in healthcare. it's more regulated, you know, there's greater ethical concerns in terms of people's bodies, you know, the data that goes into, you know, healthcare that comes out, how that's all regulated to the point of, you know, facilitating payment and doesn't really facilitate care and systems need to make up for that.

**[0:04:17] Matthew DeMello:** In financial services, the problem seems to be in terms of adoption and data governance of taking a tech stack that, you know, for the most part for the last 40 years has been transitioning to digital in many ways, but still very paper reliant. And I know, especially with, it's not quite exclusively generative AI capabilities, but really that first generation, you know, those first generation capabilities, optical character recognition, you know, machine learning, predictive analytics, They can make functions like document processing that I know are becoming widespread use cases throughout financial services just in terms of taking that paper-based tech stack and making it digital.

**[0:04:59] Matthew DeMello:** Where financial leaders are hearing about these use cases and they're getting jumbled up in new generative AI capabilities, how do they know where to start just in terms of looking at their organizations, looking at that tech stack, looking at the capabilities of their IT to help compensate? and saying, all right, where do we need this technology?

**[0:05:22] Andrew Sellers:** Matthew, fantastic question again. I would start again with the data. And so as we were saying before, and I think you made a great analogy to an adjacent sector in healthcare about the HIPAA regulations that we have, some version of that in most of the world now. And so governance in FinServe is also very important for regulatory frameworks because the decision-making typically requires some transparency and accountability. And if you have this inaccurate, delayed, or compromised data, it can result in significant financial and reputational hits.

**[0:05:47] Andrew Sellers:** And, you know, and it can also really lengthen innovation cycles. And if in order to combat that, you're constantly doing extra validation. And so the first thing for financial leaders, you know, when looking at the ROI and starting down this path is to get your data in order. I think before you even start contemplating, you know, chat GPT or LLMs or anything else, make sure your data is in a place where it's trustworthy and it's complete.

**[0:06:10] Andrew Sellers:** I think after that, you really want to start small and realistic. I think that in some of these things, you really want to innovate faster than everyone else. And so there's this temptation to make a really big investment, to throw $100 million at something and leapfrog everybody else. The problem with spending $100 million on an innovative project is the expectations that it creates are so much that if it tries to be all things to all people, it'll probably satisfy no one.

**[0:06:32] Andrew Sellers:** And so when we talk to customers, one thing that we try and emphasize is to start small and realistic, to take advantage of the things that these technologies are already good at. You gave a great example like OCR, document recognition, document summarization. That's something as a capability that I would say has been pretty well commoditized by these technologies. There's not a lot of science to bringing that into market. And I think it's a fairly trustworthy thing to put into one's workflow.

**[0:06:54] Andrew Sellers:** One thing that we always encourage people to do as well is when they develop those things, to do them first for internal customers, they're a lot more forgiving than people that are paying you. So having document processing, supplant maybe some of the data entry folks that you have and not to replace them, but to augment them. to basically take the things that they do well and allow them to do it faster or to do it with a lower skill level.

**[0:07:13] Andrew Sellers:** And that, you know, particularly as we're looking again, early days with things like the LLMs, that is, is tended to be something where the investments have tended to be more successful when we're augmenting human operators, rather than trying to automate them away. I think that there will come a time with the technology where we can automate more of this work with, you know, generative AI. But, you know, right now, it seems like the things that it has done very well is being partnered with humans and helping them work faster by eliminating some of the rote parts of their jobs.

**[0:07:41] Matthew DeMello:** Absolutely. And I think for the most part, especially now that we're at this point of 2024, I think that's really circulating through the media. And a lot of those anxieties from the labor side have kind of died down, especially now that we're kind of seeing the light at the other side of the tunnel, which is how great our jobs can be, which is what that augmentation actually looks like now that it's in view. And now that people are seeing kind of the better world for the trade-off, the headlines about replacement seem to be overblown.

**[0:08:15] Matthew DeMello:** But in terms of you brought up ROI and just being able, I loved what you said just about how failure in a lot of these projects comes from kind of the promise of it being all things to all people. And that's kind of a problem that's entrenched in, right? Especially with now the hype of the media cycle. Just because it seemed, you know, five years ago, it seemed MSNBC or, you know, whatever channels, they knew what artificial intelligence was, but it was just something that social media companies used.

**[0:08:47] Matthew DeMello:** Now it's the headline every single day. You know, we were just talking on job replacement a moment ago. That headline seems to have faded. But other anxieties, of course, haven't. you know, there's there's misinformation. But just in terms of all things to all people, on the one hand, I do think that's kind of coming off its pedestal in terms of the larger media narrative. But that does seem to be where, you know, my parents, you know, or everyday folks rather seem to start from of why can't it do everything I expect it to?

**[0:09:18] Matthew DeMello:** You know, people logging on to chat GPT and expecting it to be as conversational as a human, you know, if they've never used it before based on the hype. And I'm wondering, you know, how do you facilitate those ROI conversations? Because we know, you know, from talking to other folks on the show that especially in ROI and getting that executive buy-in, it comes with sort of this trade-off of like, listen, this might not look like ROI in hard dollar amounts over time.

**[0:09:44] Matthew DeMello:** This might be a business function by which, you know, in an abstract way, you're seeing the return on investment for the time you're putting in. You know, how are you handling or how do you recommend handling those conversations, especially in this media climate where a lot of subject matter experts might come into the conversation expecting all things to all people from the

**[0:10:04] Andrew Sellers:** project? Matthew, you make a really great point that there's just so much hype around these technologies right now that it can be hard to separate where they are. And so I think my first piece of advice to someone that's trying to have a reasoned investment conversation with an executive about it is to have a good understanding of what the technology is and what they offer and do well, and also what they don't. They have some pretty significant limitations.

**[0:10:24] Andrew Sellers:** So in financial services, I'll start a little bit maybe back with my background. I've been, in some sense, either researching artificial intelligence or bringing AI-enabled applications to market in some capacity for 20 years now. A lot of that had to do less ML, more symbolic reasoning, knowledge engineering things. And this whole time, it's always felt like AI is this really important, inevitable thing, but it's always been three to four years away. This moment feels different, it really feels like that we're at an important inflection point.

**[0:10:54] Andrew Sellers:** And I think that we are. I think with these foundation models, which is what these LLMs are, these big things, we make big investments and we produce something general, and then we can use it for lots of things. And that's counter to the more traditional ML predictive models, which tend to be very bespoke and trained on specific data to do a particular kind of prediction, for example. And what that enables us to do now that we have these foundation models is we can use these for more and more things without having to hire all the expertise to actually build a model.

**[0:11:22] Andrew Sellers:** But the data engineering challenges still remain. As you've said, these things hallucinate, they give bad information, and they give convincing bad information. And that's something that I think is also something that's diminished the hype a lot. And so it's important to understand that in financial services, for example, transparency ends up being really important. And so you look, at least in my experience, and just to describe my bias, most of it has been in insurance and has been in automating, underwriting and automating claims processing.

**[0:11:50] Andrew Sellers:** And when you're doing that, you tend to favor models that are highly transparent. So it's a lot of Monte Carlo stuff. I can run, you know, a thousand, you know, years of simulation for a given peril, and I can see the distribution of loss. And I can say that, okay, this is the decision we should make. And this is why these things that are derived from deep learners, you know, the big model transformers like that, that LLMs are, you have no idea where that answer came from.

**[0:12:12] Andrew Sellers:** You can ask it and it'll give you an answer. But once again, that's also just made up. And so I think that one thing for financial services to remember is that these things are still really good, but they're not yet reasoning agents. They're not yet to the point where they can supplant normal predictive models that are bespoke and specialized and for financial services, often transparent. And so I think that one pattern we've seen a lot of our customers use with some success, particularly in insurance and other places, is to actually pair LLMs with more traditional models.

**[0:12:40] Andrew Sellers:** So it becomes an interface. I can use a natural language interface to query the model. I tell the LLM what the interface to the model looks like, and it asks the question. And that allows a human operator to, let's say, evaluate many more sort of claims or to do underwriting at a better pace or to have an underwriter that maybe doesn't have the technical sophistication with the model, the more traditional model. And I think those are the kinds of places where I think that we're finding this resonate with the market so far, is that it's going to help eliminate some of the rote work that humans do, but it's not going to be an underwriter yet.

**[0:13:14] Andrew Sellers:** This is not important actuarial work yet. And I think until we can get these things to the point where there's a better QA stack for them, we can actually run regression testing until we can actually say something about what they meaningfully do. I think by their nature, they're stochastic, which means that if you don't kind of expose the seed, the outputs aren't even replicable. And I think that's a real challenge for people in financial services whose decision making has to be explainable for the good of their customers and the good of their firms.

**[0:13:44] Andrew Sellers:** And so just to kind of wrap up then, so I think that's the first part that you have to address with financial leaders in the ROI. You have to be realistic about what these things can do. And you have to then be realistic about what your data is and how trustworthy it is as well. Because the main defense that we have so far, we see these like RAG, these retrieval augmented generation patterns. The best thing that we can do to protect ourselves from hallucinations is to supply highly contextualized data.

**[0:14:09] Andrew Sellers:** And if we do that, we tend to get better answers. And that's something that, you know, if we have a good data set and we have a good use case, I think that these are these are things that that people are successfully bringing to market.

**[0:14:20] Matthew DeMello:** Absolutely. I'll say in your last answer, I really appreciate you keeping it very, very much, you know, without jargon and easily understandable for the folks at home. All that said, I think you're largely describing black box issues as they get called in the industry. And I don't think that term is going to stay in Silicon Valley or, you know, in LinkedIn chats. I think, you know, like my parents right now, they know what machine learning is as compared to, you know, a few years ago.

**[0:14:47] Matthew DeMello:** I think black box issues and having expectations about transparency when you use applications that are AI enhanced are going to be very much part of the lexicon very, very soon. But I appreciate the frankness of your answer, especially in plain speak. But let me if I can pull apart there, especially because at this moment of new generative AI tools, media hype, the problems I was describing in previous questions, we're running also into this kind of problem of in a very real way, a lot of the latest and greatest capabilities do have solutions for the problems at hand.

**[0:15:28] Matthew DeMello:** But that's a really tough sell to folks who are already skeptical, right? Like the problems of AI will be mitigated by more AI. And you have to build that trust, of course. Just where folks, especially financial leaders, are hearing about generative AI tools being able to help in the data governance process. I'm wondering if we can kind of separate the wheat from the shaft there of in terms of, you know, if they don't if they haven't started in generative AI capabilities yet, you know, where do they where do they need to start?

**[0:16:00] Matthew DeMello:** If they already have AI capabilities or AI initiatives in their workplace, then how are they assessing them to take it to that next level for this new generation of tools? Or maybe to ask more plainly, what do you think generative AI tools offer or rather specifically what they don't offer financial leaders in terms of data governance and better data management practices just to counteract what they might be hearing in the media?

**[0:16:28] Andrew Sellers:** I think it's a great point, Matthew. And so what I'll say is, as we said before, we should be using these LMs and generative AI for these rote work that humans can do, but is a lot for them, and it would be so much faster to automate. Well, it turns out data governance is full of those kinds of tasks. And so we talked at last current, Jay Krebs, in our keynote, talked a bit about some of the things we're bringing to market and really exciting for us in terms of data governance capabilities that are augmented with generative AI.

**[0:16:57] Andrew Sellers:** And so one thing that generative AI can really help with is if I've got, for example, a data set of some kind, we think of them in streams. But if you just think of a collection of data, it tends to have some structure or maybe it doesn't. Well, generative AI is really good at recognizing, hey, this structure is here or it's not. Or here's 50 example data points. What are the three different fields that comprise this data set?

**[0:17:18] Andrew Sellers:** Oh, that's great. Make a description of those fields for me. Here's roughly what the business application of those are. Say more about that generative AI. And so a lot of data governance ends up being this cataloging activity of generating metadata, data about data, data that describes my data set, who owns it, what's its business value of, what these fields are describing, how much is here. LLMs are really good at that. And especially if you pair that with a human operator, a data engineer, it can really save them a lot of time from having to grind all that stuff out themselves or depending on your data governance model.

**[0:17:51] Andrew Sellers:** And Thinservs are more and more adopting what we call a federated governance model, but with domain-driven ownership that basically the people closest to the data that are producing the data ultimately own it in terms of how it's used and how it's governed. Those people have very different jobs as well. And so they don't have time to do all that cataloging. But LLMs can make a recommendation to them. They can just click a button and say yes.

**[0:18:14] Andrew Sellers:** And now we're doing the hard work of data governance in a way that is a lot less painful than it ever was before. And so I think the LLMs are going to be really good at that. As far as their limitations, as we're seeing the things that have been brought to market, I've not met anyone yet that's crazy enough to just sort of let the LLM have at it without any kind of human oversight. Of course.

**[0:18:32] Andrew Sellers:** And so I think that's an important limitation to remember, too, is you're not going to necessarily hand the keys of data governance entirely over to Gen.ai. Yeah, maybe someday. But I think it's really important now that it's not just that there's no technology silver bullet here. Organizations need the right process and they need the right organizations to make sure that they're ready for this new revolution. But if they have those things, then technology can go a long way to building far better and far less expensive data governance than ever before.

**[0:18:58] Matthew DeMello:** Absolutely. And I think it's worth noting, you know, in certain circles of tech, Silicon Valley, et cetera. There's been some talk that this period of hallucinations won't last as long as we think, or it won't be carried with us through the history of AI. It'll be a little bit like ethernet cables for the internet or dial-up modems. It's something we have to live with for now. I know, I believe I saw in some areas of LinkedIn, some people predicting last year that we'd really only see these hallucinations for a couple of months, which just shows you how dangerous predictions are in the area of AI right now.

**[0:19:36] Matthew DeMello:** Indeed. Because, you know, as we've been, like, when we were telling folks who'd come on the show last year, hey, you know, like, we know no one wants to predict, you know, what's going to happen three weeks from now, so don't worry about it. But even then, you know, and that seemed pretty believable, yet here we are, it feels like almost like a year later, maybe like eight months later, and hallucinations seem just about as bad.

**[0:19:58] Matthew DeMello:** I've been using chat GPT for different functions across Emerj. I saw one meme on LinkedIn. They called it mansplaining as a service. And I found that perfect. I found that hilarious. And also it kind of touches on the point of like, yes, these tools are new, but this problem isn't new of working with data sources and folks who think they know what they're talking about, but they really don't. That's a tale as old as time. But hopefully one that we won't have to see at least in this form for very much longer.

**[0:20:28] Matthew DeMello:** Andrew, thank you so much for being with us on the show this week. It's been a real blast.

**[0:20:32] Andrew Sellers:** Matthew, it's been really fun. Thanks for having me.

**[0:20:44] Matthew DeMello:** A very quick point before we wrap up today's episode. I know I made that joke based on a meme right before we ended our conversation with Andrew calling large language models mansplaining as a service. I still think, and maybe this is my English degree talking, that that says a lot more about the hype around these technologies than maybe just a passing joke you might see on LinkedIn or another social media account if you have come across this elsewhere.

**[0:21:15] Matthew DeMello:** But the fact of the matter is that in terms of faulty data sources, misinformation, saying things with confidence or hearing people say things with confidence and assuming they're knowing what they're talking about, that is a problem as old as time. and almost makes you think in the way of confronting large language models as truly a novel technology, what we might be confronting more is our need for absolute truth and really confronting the very question of how do we know if information is real?

**[0:21:55] Matthew DeMello:** I don't think this is really existential. And I think this is going to go for a lot of technologies that mimic human behavior year that we're going to see proliferate in all kinds of sectors, not just financial services, as the AI revolution takes full swing. In that, when we really look at the problem and our expectations, as Andrew discussed, expecting AI to be all things to all people, when we're confronted with those expectations and their inevitable disappointment, we always, always find ourselves asking whether or not we as human beings are prepared to really solve the problem for good because that is what is being asked for in inventing a technology that is that reliable.

**[0:22:43] Matthew DeMello:** It's to solve that problem, whatever it is, misinformation, copying documents, trustworthiness, all of these things, all of these problems ask to be solved for good when we confront them with artificial intelligence. Yet we only have the language to talk about them from a human perspective, sort of a language that always assumes that these problems will always be with us. And I hate to speak in all or nothing terms. There's many schools of thought who think that's a bad idea.

**[0:23:16] Matthew DeMello:** But just to keep this outro short, I think it was wise to paint with a broad brush. On behalf of Daniel Faggella, our CEO and head of research, as well as the rest of the team here at Emerj. Thanks so much for joining us today, and we'll catch you next time on the AI and Financial Services podcast. Thank you.
