In brief
Enterprise AI often fails on document-heavy work not because the models are weak, but because context gets lost. It disappears at page breaks, across scattered data silos, and between disconnected tools. Sumedh Chaudhary of IBM argues that a coordinated team of specialised AI agents, backed by firm governance, is the foundation that lets AI work reliably in regulated settings.
The central tension is between a small pilot that works and a production rollout that does not. When error rates miss compliance standards, companies tend to retreat to slow but accurate manual processes. Chaudhary's answer is measurable error-rate targets tracked weekly, human oversight, and fit-for-purpose models, rather than rushing to an early win.
Facts
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Sumedh Chaudhary says he has over twenty years in the industry and that the principle of garbage in, garbage out has held consistently.
“I've been in the industry for over twenty years and the one consistent thing I've heard over the years is any garbage you you give into an enterprise system, you'll get garbage out.”
0:08:40 · Sumedh Chaudhary · Data quality
Chaudhary says enterprises have recently had success using generative AI models to analyze unstructured data.
“Uh, using some of the generative AI models to be able to analyze and put solutions for unstructured data.”
0:02:24 · Sumedh Chaudhary · Generative AI
Chaudhary says documents combine images, tabular data and page breaks, which add complexity beyond ordinary unstructured data.
“you've got images in it, you've got tabular data in it. And they are something we call as page breaks, right?”
0:03:00 · Sumedh Chaudhary · Documents
Chaudhary describes an OCR agent whose sole focus is recognising data and extracting a machine-usable digital footprint.
“you have an optical character recognition or an O C R agent.”
0:10:44 · Sumedh Chaudhary · Multi-agent architecture
Chaudhary says vector agents take embeddings from extracted data and store them in a vector database.
“Then you have vector agents that take the embeddings out of that extracted data and be able to store in a vector database”
0:11:18 · Sumedh Chaudhary · Vector agents
Chaudhary describes a splitter agent that preserves context across consecutive pages of a document.
“So you can have something called, we call it as a splitter agent, so you understand the context between page one, page two, page three.”
0:11:51 · Sumedh Chaudhary · Splitter agent
Chaudhary says he has recently seen many users upload documents running to thousands of pages.
“I've seen a lot of folks recently using documents that are thousands of pages.”
0:11:51 · Sumedh Chaudhary · Document scale
Chaudhary describes a matching agent that links references across multiple uploaded documents.
“is what I call as a matching agent. And this is an agent that when you're uploading multiple documents into the system, is able to give references”
0:12:23 · Sumedh Chaudhary · Matching agent
Chaudhary says proofs of concept succeed when scope is small but may fail in production.
“where you have a very successful proof of concept when the problem and the scope of the solution was small.”
0:15:43 · Sumedh Chaudhary · Proof of concept
Chaudhary recommends tracking performance and error-rate targets week by week.
“you should have target numbers for your performance and error rates that you're tracking week by week.”
0:16:20 · Sumedh Chaudhary · Governance
Chaudhary suggests promising a CFO that an AI solution can match legacy benchmarks within three to six months.
“we can achieve the same benchmarks as a legacy solution in a period of three months or six months.”
0:18:07 · Sumedh Chaudhary · ROI planning
Chaudhary says a successful agentic workflow is no longer limited by the number of human users who can handle it.
“You do not have to, you're not constrained by the number of, uh, users who are able to handle that workflow anymore.”
0:18:41 · Sumedh Chaudhary · Scalability
Chaudhary says many automation use cases for data access can be done before adopting agentic workflows, sometimes without an AI agent at all.
“Before we jump into the agentic workflows, there's a lot of automation use cases that can be employed in terms of how you access the data”
0:20:19 · Sumedh Chaudhary · Automation
Chaudhary uses a sedan versus pickup truck analogy to argue that different enterprise tasks need different AI models.
“A small sedan car probably does not need the same engine that you need for a heavy-duty pickup truck, right?”
0:21:57 · Sumedh Chaudhary · Model selection
Chaudhary says enterprises can access thousands of models, though not all are available in every cloud environment.
“an enterprise can have access to thousands of models, and some of those models may or may not be available to you in your cloud environment.”
0:22:30 · Sumedh Chaudhary · AI models
Chaudhary compares orchestrating AI agents to a concert master leading an orchestra of talented musicians.
“you've got this concert master who is trying to run this beautiful orchestra.”
0:12:56 · Sumedh Chaudhary · Orchestration
Chaudhary advises first mapping all data sources and reaching them through a common virtualized layer.
“you spend a lot of initial time making sure you're mapping all your data sources and you have access to that in a in a common tool, work virtualized layer”
0:23:32 · Sumedh Chaudhary · Data silos
Chaudhary argues leaders should not rush into a use case merely to claim an early win.
“And I would not expect leaders to rush into a use case just because they wanna check a box and get an early win.”
0:26:03 · Sumedh Chaudhary · Strategy
Chaudhary says organisations whose AI error rates miss compliance standards may revert to legacy manual systems.
“you might go back to your old systems, the legacy systems, where you were doing the works a little bit more manually.”
0:07:16 · Sumedh Chaudhary · Compliance