301. Using generative AI to conduct outside-in diligence

The more granular your proprietary data is, the more precise your view of the future is going to be. With the benefits of very granular diligence enabled by AI, now could lead to a very precise financial trajectory of the asset with a good degree of predictability for about 36 months.

From McKinsey and Company, I'm Sean Brown and welcome to Inside the Strategy Room. That was David Pralong describing how increasingly granular AI enabled diligence is sharpening the ability to forecast performance, giving investors a new edge. And in today's episode, David and his colleagues will share insights from their McKinsey dot com article titled From Potential to Performance Using Gen AI to Conduct Outside-in Diligence. We've provided a link to the article in the show notes, and now it's my pleasure to introduce our guests.

David is a senior partner based in our Carolinas location and global leader of our transformation practice. He served a broad range of industries around the world on their value creation and portfolio strategy programs. He's also co-led the evolution of our AI enabled value creation platform, Wave. David, welcome to the podcast.

Thanks, Sean. Great to be here.

And Laura Borton is a consultant in our New York office and a member of our transformation practice. She works with clients across sectors to deliver enterprise-wide transformations from initial diligence through to implementation. Laura, welcome to the podcast.

Happy to be here.

Will Bundy is a partner in our Washington, D. C. office and the co-lead of our private capital value creation work. He serves private capital investors, helping them identify and unlock value to outperform the market. And Will is also the author of the book Creating Value and Portfolio Company Operations, which we've linked to in the show notes for today's episode. Will, thanks for joining us today.

Thanks for having me, Sean.

And Chase Covington is a partner based in our New York office and a leader in our transformation and strategy and corporate finance practices. Chase works with clients on large-scale transformations across sectors, including independent diligence, corporate strategy and financial transactions. Chase, welcome.

Thanks for having me.

We're really excited for today's conversation. And David, to kick us off, maybe you could set the scene and tell us a little bit more about the journey from traditional diligence to AI-assisted diligence.

Brilliant. I'll give you a brief introduction, a bit of a background on how we got to this conversation. So, first of all, diligence is critical. We use diligence as a way to underwrite our transformations. But of course, as soon as the diligence will lead to a transactions, the stakes are really high. And getting that not only right, but getting that in a timely fashion represents a real advantage. Now we as a firm got into AI-assisted diligence almost, uh, incidentally.

Uh, some 15 years ago, we started tracking every initiative on a weekly basis for our large value creation, uh, plans, or we call that bankable plan. And 15 years later, we realized that we have now millions of data points about not only the value of each one of these initiatives, but also how quickly these initiatives ramp up in value, the resources that they consume, and some of the red flags in terms of their ability to produce their full impact eventually.

Here, the arrival of AI created an opportunity to make a lot more from this data, including the creation of value creation plan, or what again bankable plan in our language, at the push of a button. The idea sounded great at the beginning. We can today produce these plans very quickly. However, we have acquired a much better understanding of the limitations of doing this, and importantly, the success factors in harnessing data and AI to get there.

Thanks, David. And Will, David mentioned earlier the limitations of generative AI alongside the successes. Where are we in this journey? Has Gen AI already started to deliver on its early promise in this area, or are we still getting started?

Gen AI really hit a breakout moment in 2023, right? It moved far beyond traditional AI and really kind of triggered a very fast adoption curve, um, one of the fastest in in in enterprise history. For context, personal computers, which needed about twenty five to thirty years from the late nineteen seventies all the way to the early twenty tens to hit ubiquity. When we think about AI, it has done that in literally about two years.

Additionally, traditional AI usually handled structured rules-based tasks exceptionally well. Think I need to go and forecast, I need to understand churn prediction, process automation, uh, producing outputs in fixed data sets. The inflection point in 2023 allowed you to do that with an unstructured data set when AI became public and multimodal. Now, when you take a look at the future state, the capability is keeping pace and it's advancing exceptionally rapid. Right now, the edge is around the reasoning, actually taking the inference phase and getting to a point that you actually can have a trusted source and even put brakes into it to make sure that it's not going into a biased or unnecessary action.

Your test time compute is being advanced, right? Test time is where it's actually kind of optimizing that inference phase and actually reducing the overall compute needed. Um, and then it's going into advanced stages of tool usage and citation, which then allows you to actually really gain a bet a bit more trust in the output of the model, right? If you think about 2023 till now, you typically just would type something in to one of the chats and you'd get an answer, but it would not have any documentation.

It would just be the random it would just be the itemized thoughts of of the actual model. Now it, it will actually break down the sites, it'll break down the usage, it'll show you current information that has actually been pulled off the web as well. And then finally, deep research, that's where it actually gets into a deeper understanding. It actually will go in and do the full reasoning, multi-step and multi-step logic associated with the agent and the actual chat that you've put in place and then get you an answer that is fully is much more founded and well researched.

The the edge now is in institutionalizing these capabilities, right, and turning the adoption into systematic judgment and systematic implementation throughout the enterprise. So it is progressing rapidly, right? But the boundaries remain both evolving and mature.

And what about those boundaries? Where do you see the model starting to reach their limits?

They still struggle with accuracy, bias, and complex reasoning. Now, there's ways that you can engineer processes and agents to support that, right? Fact-checking, uh, you know, um, adding in a red or a blue agent in order to made make a more complex algorithm. But it still has its limitations, and it still has to be checked and have oversight, right? Sometimes you, you've seen this, right? With public misfires, like fabricated legal citations, um, election deepfakes and hallucinated, um, financial summaries, right?

Those are all things that you have to watch out for as you're using this technology. The progress, though, is extraordinary. And if you just think about that curve over one to two years, it's it's a matter of time until this gets to pretty close to, um, a fully acceptable and very low, low rate of of concern. Additionally, like better tooling, deep multi-step logic, and it cites sources directly, right? So now you're getting that output and it really you can gain some confidence.

You can even go and double check or triple check what is actually being said to you by the the model. Uh, for investors, what this really means is that the reliability is increasing, right? We're approaching that point where gen ai can support analytical reasoning and not just generate language.

So, well, a quick question that's related to the adoption of gen ai within a diligence context. What are some of the main reasons why companies haven't used it as much as they could, or alternatively, why are they just now starting to use it more and more?

Yeah, the the the big breakthrough, I think, is in the evidence bake based citation, right? And and like for me, when I was using it in 2023, I was hesitant to use it on on diligence. Why? Because I needed to have a bit more founding of what was what was being said to me by the model. Uh, now I would use it almost like, um, back in twenty three, I'd use it basically like an advanced Google search, right?

I would kind of say, Okay, here's what I need to understand, and it would pull out this beautiful dissertation, right? And I've had mass hallucinations, right? Where it, like, literally just gave me a a completely false narrative at that point. Um, when you take a look now, um, when you, as long as you're doing good prompt engineering, we can really start to get to a point where you really loc uh, focus in the model on what you need to answer.

Um, and then it'll actually bring back the actual data, the actual in initiatives. Um, I've actually, we've gotten it to a point now where we're like, if you upload a, uh, a confidential information memorandum, it will actually that it'll actually provide you all the initiatives. Fact check what the management team is telling you with data that we have. So, like, a, and it'll give you the citation of where that data is from and why we're saying it's errant or why we're why we don't believe that what they're currently saying in in that sim.

So that traceability was a big key.

100%, like that is one of the biggest breakthroughs. Now, the other piece is that multi-step logic, where we'll actually kind of break down the model, do some reasoning behind it. Identify the different steps it needs to take in order to actually analyze the problem that you've given it, and then segment it out and actually answer each of those discrete steps. That's another kind of key breakthrough of of the technology.

Okay, one follow up question, Will, on this. So it sounds like Gen AI offers a number of different advantages, but in your experience, what's the biggest one?

The the biggest part for me is it's about the speed. And what right now? The biggest step that I've been using it for is the ideation and getting all the ideas out there that can potentially be taken, right? And the beautiful part is it scans the web, it scans internal databases, it takes a look at the sim, and then can put together the entire list. From there, I can actually launch analyst against these problems. It never kind of fully removes the analyst step in order to really get to a solid value creation plan, both top line and bottom line.

But it actually greatly in. It removes like that first four to five days, and then it also, depending on the output that you receive, can actually remove some of the analytical step needed. Um, depending on the level of conviction that you need to believe a certain initiative and the criticality of that initiative, it can give you a rough order of magnitude on that initiative, uh, within five minutes.

Wow, thanks, well, and Chase, a question for you around implementation. What are the organizations that are building this into their workflows across their entire portfolio of diligence doing differently? What's the secret to success here? And conversely, uh, where, if any, do you see organizations coming up short?

The hint answer is it's cultural. So maybe I'll just start with the results from some survey and research work we've done. Everyone's talking about AI, most everybody's talking a very big game about AI. But if you ask people confidentially, pretty much everyone tells you they're struggling to implement AI. So some facts here, 88% of organizations we talk to have launched some sort of AI or digital transformation effort. Of that group, less than 25% really feel honestly that there's been meaningful progress.

Less than 10% operate at scale using AI. There's a couple of reasons for this, but they all get down to cultural elements. But I'll I'll give you a few more facts on some of the common failure modes here. 75% of organizations lack a comprehensive Gen AI strategy or clear success measures, leading to fragmented experimentation, folks running off and doing their own thing, and there's no scalable impact. 70% of of transformations fail. And And for this reason, we really feel that of every one dollar spent on technology, firms often need five for adoption, workforce readiness to ensure that the technology is used to its fullest.

I'll I'll give you some examples. These some of these failures are are are spectacularly seen, and some are and some, some are are are relatively nuanced, but but, but really preval- prevalent. For example, we've had a client spend money, a ton of money to say, Hey, how do we automate some of the initial activities our analysts do in the in the diligence process? This is just on diligence. Here's how we think about, um, deploying language models, here's how we think about building agents.

But because this requires us to actually change the way people work and change the way people have been ingrained to think about diligence, for example, for their entire career. When you forget the change management and just roll out the tool, you don't get the adoption. For example, uh, uh, another one, this is outside the context of diligence. This is a medical products company, uh, med med tech company. There was a misalignment in their, uh, in the structuring of their AI agent.

Between what the legal team felt was appropriate, what the clinical affairs and medical affairs team felt was appropriate. And because they had that poor governance and a siloed decision making related to the creation of those agents, the output was unacceptable. In terms of the quality of the research that the the quality of the research that it was being being able to to automate, right? So I I think we think about firms struggling with AI. I think the key takeaway is it requires discipline, uh, in particular thinking about how you change your operating model.

Obviously, cross-functional collaboration is important, obviously, robust change management is important, but it's these cultural elements operating model, collaboration, change management. It's the cultural elements that actually more so than better models or better LLMs, that enable the scale, the scale deployment of of AI in any given enterprise.

So, if implementation at scale remains a challenge, what does that look like from an investment lens? And are investors seeing a meaningful level of impact yet?

So we surveyed forty global general partners, um, investment managers, analyze operations across 200 investment teams. This is again the context of diligence, understand really where AI is truly delivering impact. The data shows some early wins. 50% report, uh, some known improvements, faster research, more, uh, expedited drafting of memos for the investment committee, synthesizing data rooms or s i m s, uh. But only six, really, only a handful, describe the results as transformational. Despite that fact, almost everyone, ninety six plus, ninety five plus, expect transformational impact within the next five years.

So there's this widening gap, there's this widening gap between the aspiration and execution. Uh, the problem isn't awareness, it's depth. So most firms have adopted some of these gen AI tools for basic efficiency, basic automation, almost of some of the routine, repeatable activities they often have their most junior associates do. But they haven't really embedded these gen AI tools into the thinking and critical processes of the investment process and really the decision models.

Chase, earlier, David talked about the use of gen AI in tracking and managing the impact of transformations, and you also work in our transformation practice. So how are you seeing leading companies successfully taking that gen AI model and transitioning it into how they actually manage their investment portfolios?

I I think one of the biggest areas we've seen success is the deployment of proprietary data to get to generate an edge in terms of your value creation thesis, but then also seeing that thesis through to conclusion. So what I mean by that, anybody can get a LLM enterprise licensed, and they're not very expensive, and there's many to choose from, and candidly, M, they're almost commoditizing in in a way. Where we see differentiation amongst our investor clients is the ones that are able to take that LLM and build into it proprietary data, research only they've done, access to data because of the companies they own or where they sit in the information channel, data only they have.

That gives them an edge. Not only in underwriting, value creation, or some sort of investment angle, but also seeing that value through to conclusion.

And David, what's your perspective here?

May maybe I'll, I'll just build on what Chase has said. And let's just have a bit of fun here, Chase. So you can keep me honest. The the more granular your proprietary data is, the more precise your view of the future is gonna be. I think the with with the benefits of good data or of past value creation plans, you could literally get now to month by month P and L that reflects the improvements that you have identified at the diligence phase.

Now, there there's a distribution of outcomes around that. Of course, it's it's not a precise science, cause stuff happens in implementation here. But but Chase, I would love your reaction to this idea that the more the very granular diligence enabled by AI now could lead to a very precise financial trajectory of the asset. Uh, within probably with a good degree of, uh, of predictability for about thirty six months. Do you think that's fair?

I I think it's fair, and I'll I'll bring it to life in a anonymized client example that we just had. Ed recently. When you think about access to proprietary data, here's two things we've done at McKinsey. One is, you know, very few, I don't think any firm does as many expert interviews as we do. Many of those we record, log and catalog so that we're able to digest those insights in a very systematic way in one, in our, in our gen AI tools.

Another is, I don't think any firm has done is the transformation work as we do. We anonymize that data, digest into a database and then feed it in as proprietary data to our assessments. You can take those things. You can go from an investment committee meeting where someone says, Hey, I think we can save ten on indirect procurements because I think that indirect procurement organization is immature. You can go from that to saying, for a business like this in this industry, in this context of the capital cycle and this context of the seasonality of the business.

Here are the five to eight specific categories I would focus on to pull to reduce indirect spend. Here's the approach I would take by category, and here's the typical savings range and implementation cost and implementation timeframe we see for each of those levers. So that that's the level of granularity that we have seen some of our investor clients get comfortable underwriting growth from, underwriting value creation from. But it's a different level of granularity enabled by AI versus we have four really smart analysts and they got an Excel file and a data room.

Yeah, a, and this is where we're building back to what Will was saying. That, yes, speed is tremendous with these new tools. But the ability to go deeper into the granularity, using whatever proprietary data that you have, becomes extremely easy with, uh, with with gen ai. And again, we we have data, you know, I'm sure every large fund has data as well. The ability to harness that and have higher predictive, uh, value in the diligence is is what this technology has allowed us to do.

And and again, the i getting back to the point of Chase here. The adoption has been systematically the biggest issue because there's a bunch of professionals who have been doing this for a living for twenty five years and so on. Unless they've created the tool themselves, it's absolutely fair to accept a degree of of skepticism here.

And to that point, uh, that was made earlier about how cultural factors can be a pretty significant headwind on adoption. How do you get people in organizations to actually embrace this?

Maybe I, I'll, I'll, I'll start with one, and I think it seeing is believing, full stop. With with gen AI, and we can talk about data, this and improvements in speed that to actually see it in practice, that's where you will be, where we observe clients adopt. And so you see the use case. Allow my team of analysts to get me an initial answer on the value creation proposal for this investment in two days versus two weeks.

That's quite compelling. When you see an answer that much more granular, much more believable, and much more testable, that is compelling. And so I I think the the name of the game is seeing is believing. And part of the benefit of AI is it doesn't require massive investments of time and capital to be able to build some of these use cases. And so often our advice, my my advice to clients, is start small, one agent here, one agent there, and you'll see how it's improving in an individual's particular workflow, and you build from there.

And what are the implications of this on how firms are reorganizing around using and deploying generative AI and diligence? Are you seeing different organizations, say, with fewer junior analysts and more senior roles, or a different mix of talent? How is this coming through in sort of the way the organizations are evolving?

I I'm gonna answer the question in in two parts because it it depends on the organization. And organizations like, um, a private equity firm or an or an investor, uh, with very relatively lean organization, not not a dramatic pyramid in terms of the the the staffing model. And the work is highly creative and needs a high level of thought and and degree of management, where a span of control is typically lower for, as I said, in spans and layers language.

We see these tools making people more effective and faster, but not necessarily changing the staffing model. That is different, though, than other type of organizations. I'll I'll, for example, tech services, IT outsourcing, where spans of control is higher. Often the work is more repetitive and rule-based. We are seeing a much higher degree of compression of those pyramids. And so for so, depending on the type of organization is the impact of AI on the staffing model. I think in the in the world of investors that this is around b, this is not around.

You don't make uh returns as an investor. Having going from five analysts to four analysts on your investment team, that's irrelevant. But having five analysts make a some, make just a little bit better decision in terms of what price you pay or which deal you pass on, that those are million, hundreds of millions of dollars, billions of dollars of decisions. So that's where you make the the the returns from using AI. Oh, there's Will and and David, y'all should weigh in as well.

Yeah, the the on the only other thing I'd add there, Chase, is there that speed to insight, right? And and the the also discussion is right. How do you actually go and then build the conviction? And how do you focus the investment committee on the areas of concern in the actual, uh, investment thesis, right? This, this, these tools can allow you to do that and get to a finite level of granularity where you can really boil down.

Hey, this is what I believe and this is what I don't believe in this investment thesis. And then then for the junior analyst, that makes you much more compelling at the investment committee. The other piece is you can upload your investment committee document. You can actually go and have the AI bot go and or AI agent go and challenge it and say, what would the investment committee for a private equity fund challenge on this investment committee?

Based on this memo, based on these things around the deal and to give it some context, and it can actually bring out some of those insights. When we put together our value creation ideation memos, that's exactly what we do. We always kind of give you a little thought process on what, what can make you better, and how can you actually go in shadow box to make sure you're as well prepared as you need to be for those senior engagements.

Super, thank you. And and Chase, could you give us any examples of how your clients are deploying AI across the investment life cycle?

One is just how do we quickly digest and synthesize a data room? Typically, it was worked when I was back in when I was an investment analyst. Is you got a data room on Friday afternoon, you then spent the next five, six days trying to digest everything in there. Make sure you understood all the reports, you generally understood two thirds of it, had a bunch of questions. Sometimes the analyst miss a a report that was buried in tab six of, you know, form ten, et cetera.

It can we use gen AI to rapidly digest and synthesize that data, create an environment where we can query the data room, Google the data room to get to produce reports, such as here's trends to gross margin or market share or profit pools. That's a much more effective way, taking the human error out of the process and doing that in a matter of minutes as opposed to several days. It's a really interesting use case, going beyond just the synthesis synthesizing function of gen AI for IC memos.

But how do you digest ma, you know, large data rooms and data sets? Another is on scanning for for for deals. Hypothetical example You have a business that the investment thesis is really around buy and build. We're gonna do a roll up we can buy for eight times, we trade at ten times post synergies, the acquisitions are six times. It's a great thesis, but we need to understand if there's enough runway out there for for uh, uh, uh, you know, deals to do.

And particularly in the geographies and channels and sectors and categories that are most relevant to our business. How can we use Gen AI to rapidly scan to identify? Not just via, you know, Google searching here and there, but to really identify? Hey, here's a list of 500, a list of 200, a list of one hundred companies that may make sense for a deal. Here's where they're located, here's how much revenue we estimate they have, here's who owns that business, here's who runs that business.

This is something Gen AI can pull, help us pull together quickly and make just this thesis around the buy and build much more robust in any investment committee conversation. Really, you know, firms are moving beyond experimenta- in experimentation, excuse me, and those who really integrate AI AI across the deal life cycle are capturing more and more value. As opposed to folks that are, you know, deploying in sort of an experiment based fashion, one or two elements of their life cycle.

And Chase, I'd just like to return for a moment to the point about the importance of proprietary data. Firms are gonna find it easier to move beyond experimentation if they've got a robust fifteen-year data store to leverage with AI. So what do you tell the companies that don't have that fifteen-year data store?

Yeah, that, and then that this is, uh, I think, our, um, characterized conversations with the largest private equity sponsors in the world that have been around since the eighties, that have robust that. That's a different conversation they often have. It. It converses more about how do we access and unlock that data in a way that's useful for us investment committees, a little bit different conversation. If you're the hypothetical middle market investor that's been around for five years or ten years, um, and have a lot of, you know, individual personal experience.

But the robust data sets as an institution, as a firm, uh, may may not be there. Th- th- there's a, I mean, a a couple things there. Th- there's, you know, part of the reason we've done some of the codifications we have at McKinsey around our access to data is to level that playing field with some of those investors. So we do see some of the middle market kind of clients ask us some of these questions.

Um, what? But I I will say a an another one that that's often un untapped is particularly for investors that have a sector focus or a theme focus. How can you bubble up data from your portfolio companies to help you make some of those decisions or get access to insights that inform future investments in a way that's thoughtful? I'll give an ex- example of a middle market investor that does a lot of consumer retail investing. That that that I work with, they are thoughtful around how do we think about the same-store sales for different types of retail environments across our different portfolio companies?

And that give us insight into what might be happening for different categories. And does that inform our our proprietary perspective on where the retail trends may be evolving? That's a very specific example to a very specific investor in a very specific context. But as an example of how to think about creatively using the data that's available to you to build something that can inform your future decisions.

Thank you, Chase and Laura. What's your perspective here?

I think one other guiding principle I'd I'd love to share here is that you really want to buy for speed and build for differentiation. So you generally wanna take a step back and take stock as to what gives your particular firm the investment edge that's not already replicable in the market. So I'm talking about things that are maybe the less sexy things of converting your data into a usable format for AI. That's largely an outsourced thing for most firms.

But now when it comes to applying your individual lens of playbooks and things like that, that's where you wanna build essentially those custom kind of light layers that you overlay on existing solutions like open models to produce then the reports or, you know, artifacts otherwise that really speak to your firm's, um, individual voice.

Thank you. And Will, a related question for companies is whether to buy or build. Um, what's your perspective?

Absolutely, I mean, one of the big things that we want to think start thinking about and and one of the things that we're, uh, blessed with here at McKinsey, right? We have tr- a treasure trove of data, 1,600 transformations, 500,000 initiatives, uh, 250,000 expert interviews, right? And all of that can be tagged, integrated into an AI engine and then retrieved upon demand. Building that is right, it takes forty years to do that.

Um, so there is that discussion of yes, do you buy, do you build? Um, are you always gonna be at a lip, uh, relative deficit to others if you go down the path of building? Over time, that gets upset. But, like, at the same time, the right answer might be to just go and buy what you can and leverage what you can from others. The other thing that you also wanted to think about is when we're having this conversation is one, what truly makes your firm special?

And just like you know, when you're making decisions at a portfolio company level, right? A lot of times, if it's not core to the business, it's something that you outsource and let other people do. There's a large number of entrepreneurs that are building these solutions, building out the data sets through various means, right? You're able to potentially bring a best of breed approach to this and focus what you, what makes you special, or focus yourself on what makes you special and let others do what they can do to supplement your capability set.

For instance, for instance, we don't we. We might have a solution in the background that allows us to book travel, right? That's not something that we're going to build ourselves. But that's something that through AI, and through an AI engine, you can actually potentially make yourselves more efficient. And so take a look at those off-the-shelf initiatives and off-the-shelf products that can make you, that make can make you work. We've actually built out mine, and Chase alluded to it a little bit earlier.

But we're at this point now able to download your entire data, download the entire data room. Leverage all of the expertise that we've built out over those forty years and put together a value creation plan in about ten minutes. It's going to hit the five to six major levers, initiatives, relative sizing, typical timeline to achieve, and actual cash flow impact. And we're making that more robust as we add in additional layers of our data sets that we have here at McKinsey.

And so that's going to continue to scale and continue to improve. It can make peer benchmarking infinitely easier. And I don't know if you guys have used ChatGPT five, but there's different levels of thinking that you can actually have the model go and do, right? There's a baseline, there's a thinking, then there's a deep research. Each has a different layer. Now, it's not exactly the same, but if you think about 2023, when you first put in a prompt, listen, you get listen.

Here's the general people that are benchmarked against you. But then when you actually start layering in that retrieval augmented generation, you start getting a deeper insight, right? It's like, listen, well, this is actually an SMB. Um, so here's some others that you might want to think about, like Dropbox Asana. And then the agents in the reasoning actually says, Okay, listen, here's what you're telling me. I'm going to break this down into discrete parts of this problem.

I'm going to then start to actually take a look at each of these discrete problems. And then it compounds that thought process to the point that you can actually get a very precise data set and allow you to really kind of d um, dial in on what benchmarks you need. And then if you're asking it, listen, what is this? Like, how does an organization compare? Right? Or you upload the headcount census file, right, to a to B, to a, um, to a safe environment, right?

You're then able to actually benchmark the the actual site, you're actually able to benchmark it off of the peers that have been selected. Um, pretty amazing. Uh, to when you start to think about the technology and where we've come.

That is exciting. And Laura, maybe you could take us through some of the practical approaches that people can employ to begin seeing impact with Gen AI. Maybe we can start with prompting.

So it all kind of comes back to the old age old adage, irrespective of use cases where it's garbage in and and garbage out. What we have found in counseling our clients is that there is becoming an increasing kind of over reliance on or over expectations of what Gen AI can do without explicit instructions. And so, you know, what we have found in those coaching opportunities. So there are five really main core habits that we have seen that you can start doing it today, whether you're a casual user, um, of ChatGPT, or, you know, you're attempting to build your own system design so that you can get some more consistently reliable outputs from Gen AI.

First and foremost is around defining role and success. You don't have to go into a crazy paragraph of, you know, you're a diligence analyst, et cetera. But you do need to treat Gen AI like a smart, bright, uh, junior analyst who's great at execution. But only if you're clear about the role, goal and context, being clear a bit upfront as to what you're looking to achieve in the response, will help then the Gen AI capabilities to even then pursue alternate paths.

Um, to arrive at that right answer, even if it's different than what you had initially expected. Second, prioritizing your sources. As Gen AI becomes more and more capable of doing web scrapes, accessing your CRMs and so on and so forth, it doesn't necessarily know what are the top priority sources. All of it is the same to them. And so you can then make that decision up front to say, hey, you know, leverage the financials from, you know, the ten K, et cetera, or take into consideration what's in the investor day presentation, so on and so forth.

To help them narrow the scope based on what you know is the right source of information from you physically doing this over time, that really helps. Then the Gen AI stay focused on what really matters for them, the grunt work, sort of of taking the information from those sources and transforming them into insights. Third is around standardizing, you know, the output. What I have found really helpful in either system designs or just my own endeavors is just to provide examples of what great looks like and the actual, um, you know, format of whether it's a table or what have you?

Um, to help guide the thinking from the model, because that essentially will make sure that your, you know, workflows can be repeatable. Um, and also too, that it can learn from previous examples of what good looks like to kind of anchor then on nuances or different protocols that your particular firm has in place. Next, and this helps really with that trust element we've been talking about so far, is to make sure that you're pairing every number with proof.

So in your prompt, you're asking for confidence intervals, rationale, sources to make sure that you then could take the reviewer role, not the doer role, and say, Hey, does this really make sense? In terms of whether it's a value creation lever, you know, so on and so forth, so that your work and focus is really then where it matters most, where your not your nuance and judgment really comes into play. Where gen AI may not be up to the skill yet without explicit, you know, instructions.

Last but not least is enable clarifying questions. Oftentimes, and we've seen the studies too, where you know open AI models, not just open AI, but you know, other general LLMs are penalized in their training for not having an answer. So if we keep that in mind, what we wanna do is encourage then the model, the system, et cetera, to ask clarifying questions so that it is not penalized then. Or it is only really penalized for hallucinated or well or poorly thought out items.

And so that becomes a really key element to make sure that, again, your output is reliable, and you've already seen this in practice. Oftentimes, if you've used, for example, open AI's deep research function, before you even proceed, it'll ask you a few questions. Cause that's what really helps then set the stage for the the LLM or the system overall to make sure, uh, that it's producing something in line with your with your standards. So, in short, good prompting is really disciplined thinking in text form.

Thanks, Laura. So let's talk about some examples of how an agentic system workflow might work in this context.

So once you've actually decided that, hey, you know, uh, my outside in diligence, deal scanning, et cetera, any of these, uh, repeatable processes, um, I've established is is being, um, important here. Um, you can then essentially, you know, build the actual tech infrastructure and the human component, um, when you're designing the systems. So I'm breaking this up into the two parts. So one is your your system design agentic, you know, component, essentially the the compilation of prompts and decision trees that get you to your output, and also your data and systems.

Now, the data systems and tooling you may think are part of the easiest things to do well, quite the contrary. Your data and systems and all that kind of fun stuff might not necessarily exist right now in a form that's readable and usable by AI. That portion alone, you know, just transforming things that are in SharePoint to a usable, you know, data set by an AI system, could take months, quite frankly, and is often something that is, um, not appreciated by folks when they're embarking on this workflow journey.

And so wanted to just kind of lead with that part where you know, invest the time up front. If your teams are saying that, hey, we really need to take, you know, a little bit of extra time to make sure we're right. Sizing this data for ingestion, that is a responsible step in the workflow. Because the agentic part itself and really kind of translating what a team does physically now today into an agentic flow is actually oftentimes the easier part.

With, you know, skilled professionals that could take a matter of a few weeks, not so much, you know, the data part, but also too when you're, um, attempting an agentic workflow, being really mindful that change management element up front is critical. So not only engaging your domain experts for, you know, the the corpus of knowledge you should use and and so on and so forth. But it's also making sure that you actually have the right feedback loops implemented.

Are you having the right KPIs for usage? How exactly are we gonna make sure that adoption is going to be there? That is a very critical part of the upfront system setup, just as important, if not more so, sometimes, than the tech build. Now, let's see what happens behind the hood now that we've had our agent set up, assuming that we've had our agents right size for the roles that they intend to play. We have those decision trees put up, our data looks good, so on and so forth.

But what do the agents themselves do? Well, they are autonomous, they have agency in the sense of once they're given a particular set of instructions, they're able then to kind of self-organize and be able to break those things down into subtasks and converse with each other to deliver an output that's meets any standard. So in our particular case, um, for this outside-in due diligence workflow, you know, we've defined agent roles as being, you know, like an engineer who's gonna pull down the financials is really skilled in that analysis.

We have a critic that is going to help pressure test some findings and so on, and so forth. So once a user then prompts the system, for say, an outside-in, you know, diligence on company X, it will then essentially organize itself. To make sure that the output is a pressure tested, um, you know, value creation sort of memo. When it comes to, you know, various levers and EBITDA impact, you'd expect, um, learning from prior examples and feedback that's given real time by users.

And that user component is also incredibly critical, cause once every output is generated, it is the user's responsibility to then actually go in and tell, you know, the AI, is that a good response, yes or no, and why? So that ultimately these, um, this workflow can get refined over time, because this is really all V1. All of this technology is new. It is not expected to be perfect in one shot. What's imperative, though, is that the organization then honors their commitment to make sure they're providing the feedback to the teams to develop something, um, you know, even better for the next iteration.

Laura, this sounds like a very involved process. So if a fund is looking to pilot Gen AI with limited technology, where do you have them start? And what would your advice be to piloting Gen AI with a limited tech stack? Perhaps, you know, even if it's just initially,

Yeah, yeah, so it's a great question. Um, and I'd say, honestly, there are so many great off the sho shelf tooling to actually build these workflows for limited to no code. So that's one thing is that know that there are place there, there are sandboxes and whatnot to do that. The biggest thing, though, is is essentially identifying the actual data sources that you want to pull into your agentic networks. Cause, if, honestly, if your data is not super great or up to snuff, then the rest of this is not really going to work anyway.

So it's really, then, you know, one, picking the tools that are going to enable a workflow to be orchestrated. They're quite inexpensive, quite frankly, getting the right tools and to help transform the data that we had talked about again, quite inexpensive. Um, and then it's really just, then, you know, engaging a team that's quite small typically, so then help with the actual prompt engineering behind it. That all kind of goes back to going to, you know, what Chase had mentioned, which are what are your priority couple use cases that you feel like you would get the most value from Gen AI and focusing those efforts on that.

And that's why we see a lot of folks, you know, focus on these diligent spaces because they're repeatable processes. There's already a playbook here. It's now then engaging someone to help make those actual prompts and pull in your data. So it's identifying use cases, taking the time necessary to assess your data and go on. But it's actually, to Chase's point too, earlier, it's much less expensive and time intensive than you think. You can create workflows in a matter of weeks or months with, even with a lean team.

thank you, Laura. And Will had earlier mentioned this notion of traceability in these models. Maybe you could touch briefly on the oversight and some of the key success factors there.

Yeah, um, so happy to do that. So one of the biggest things in terms of oversight is really that cultural shift I had mentioned. Where folks who are now used to or have historically been used to doing the thing as in scraping everything themselves and putting together now need to shift into the reviewer mindset. So it's the trust but verify. So even if you have a greatly tuned system, you're still it's not infallible. So folks' everyday responsibility is to make sure that they are actually then reviewing the outputs, flagging the feedback to the right teams, and also making sure that your procedures are in place.

So that if humans need to be interjected, more so that that is well known across the organization. Additionally, you know, as we reflect on some of the, um, I'll say, necessarily just challenges, but considerations that we have for where Gen AI needs more human involvement, like the security exposure and legal and compliance. Making sure that there are those folks who are then very deep in the weeds of to, as you had as a security work, especially in an investor context.

We have sensitive deal data that's non public information, and so we just need to make sure we're taking those necessary precautions in the design phase, using those key, you know, risk individuals, et cetera. Uh, to help shape then how the system is going to operate overall. So I think it's two parts one structural and one is just as you review the outputs to trust, but verify.

And and Will Laura just talked about that notion of oversight, which begs the question, where does Gen AI really add value in diligence? And where do you still need that critical human judgment?

Yeah, absolutely. I mean, at the end of the day, right? It's it's an enabler. Um, you can't rely on gen AI to make decisions. It doesn't have the experienced judgment that your operating partner team or the investors or the investment committee has. What it can do, though, is shed light into those in that at a very quick pace, which then allows you to step into that reviewer lens that, uh, that Laura really kind of highlighted and really kind of make sure that you're getting what you need out of it, right?

The key thing for me is right, it doesn't capture value on its own. Um, you have to take this output, uh, link it to business outcomes, right? Improve the accuracy, right? Like, there's gonna be things that you just don't agree with, go in and challenge those assumptions, make sure that you have conviction around what's being said. Um, and then also, right, you're you're gonna have to integrate that into your portfolio and your portfolio company. Um, that's gonna be the harder step.

That's what I mean, that's the big move, right, is taking it from that ideation into execution. Um, sharing it and socializing it with your management teams, right? How does the management team gain conviction around an um, a memo, right? That's how you have to have some intervention there to make it humanized, to make it actionable, and to make it understood.

Super. So as we wrap up, David, I'd love to turn to you for your thoughts and where you see this continuing to evolve. What's the trajectory for Gen AI in this due diligence context?

The notion of diligence has judgment at its core, right? Eventually, whether you go with your transformation or you set an aspiration for your own company to go somewhere. All the tools we talked about are merely instruments of sharpening your judgment. Eventually, the none of this is gonna lead to an automatic decision on behalf of the of the company. And the most profound gap still is the trust in those systems. And I see that at McKinsey as well.

We only get to really take advantage of these tools once the senior firm members, those who will eventually exercise the judgment coming out of these again, could be diligence, could be other things. Once those senior members have actually had their hands on tools enough to trust these tools and understand the implications here. And certainly from an operating model point of view, the challenge is that how do you get the senior firm members to actually understand every component of that so that they can exercise the judgment with the confidence of a well calibrated view?

And I think for all of that, once we we get that involvement, I think we're gonna get the full benefit of AI. Until then, I think there's still gonna be a degree of skepticism that that hinders us.

Thank you, David, Laura, Will, Chase, thank you so much for taking the time with us today. This was a great conversation.

Thank you.

Thank you.

Thanks so much.

And thank you to all of our listeners for joining us today. We hope you enjoyed the conversation and we'd welcome your feedback and ideas for future podcasts. Just email us at itsr at mckinsey dot com itsr for inside the strategy room. You can also share your ratings and reviews on any podcast player, with many thanks to all who've already done so. We really appreciate the comments and feedback we receive every week and encourage you to keep them coming.

And if you enjoyed this episode and you'd like to subscribe, you can easily follow our weekly series on any podcast player, or you can also access our entire library of more than 300 episodes. We also offer an inside the strategy room podcast collection page at mckinsey dot com slash itsr, where you can easily browse our prior podcasts across nine major themes and access written transcripts of all of those conversations. And finally, if you'd like to automatically receive our latest publications and insights, we encourage you to visit our transformation practice page at mckinsey dot com slash mt, our strategy and corporate finance page at mckinsey dot com slash scf, and our private capital practice page at mckinsey dot com slash

pc. Uh, we also encourage you to connect with us on LinkedIn, and we've included links for all of those sites in the show notes today. Thanks again for listening. We look forward to having you join us again next week inside the strategy room.