Person
Chris Churchman
Chris Churchman is a speaker who appeared on the Goldman Sachs Exchanges podcast to discuss building AI systems for capital markets. He talked about Marquee, Goldman Sachs' platform for institutional and corporate clients, and its tools, including Market View and a visual structuring tool.
He also covered how AI models have changed since ChatGPT, from small context windows to a million tokens, and what firms should build around them. Churchman predicts a phase he calls mandate engineering, and says agents will gain autonomy once an insurance or liability market exists around their actions.
1 episodes · 17 quoted claims
Churchman says Marquee is Goldman Sachs' platform for institutional and corporate clients and that its mission is to help them make decisions under great uncertainty.
“So Marquee is the firm's platform for institutional corporate clients. And really, our mission is to help our clients make decisions under great uncertainty.”
Goldman Sachs Exchanges · 0:02:00
Churchman describes Market View as an ecosystem on Marquee, like a Pinterest for capital markets, where each widget is created by a domain expert from vendor data.
“We've got an ecosystem, fortunately, called Market View, which contains these, you can think of it like Pinterest for capital markets, where each widget is produced by an expert in their domain.”
Goldman Sachs Exchanges · 0:03:11
Marky AI, Goldman Sachs' generative AI assistant within Marquee, was available only internally at the time of recording.
“Marky AI, which is only available internally right now, you ask Marky a question, which is very targeted, is what you care about.”
Goldman Sachs Exchanges · 0:04:10
Churchman says Marquee includes a visual structuring tool, an intuitive interface for building derivatives that also shows scenarios and backtests, which he wants to wrap in an AI agent.
“we have something called visual structuring, right, which is an intuitive visual interface that comprise any derivative, for example.”
Goldman Sachs Exchanges · 0:19:31
Churchman recalls that early ChatGPT-era models could handle only about 4,000 tokens, roughly 3,000 words, in their context window.
“And so you go, right, okay, it can only handle 4 ,000 tokens in the context window, so 3 ,000 words.”
Goldman Sachs Exchanges · 0:06:26
Churchman says context windows grew to 128,000 tokens a year after ChatGPT and then to a million tokens, making work on chunking documents largely pointless.
“And then a year later, the context window goes to 128 ,000 tokens. And you're like, oh, well, that was pointless. And then it goes to a million tokens.”
Goldman Sachs Exchanges · 0:06:52
Churchman says Google solved the 'needle in a haystack' retrieval problem when it raised its context limit to 1 million tokens.
“And so Google, when they raised the context limits and 1 million tokens, which was amazing, they solved the needle in the hate step problem as well.”
Goldman Sachs Exchanges · 0:07:54
Churchman cites a paper arguing that language models hallucinate because training and testing reward guessing over abstaining, like multiple-choice tests.
“there's a famous paper why LLMs hallucinate and it's because they're doing multiple choice tests and they're not rewarded for abstaining”
Goldman Sachs Exchanges · 0:09:03
Churchman describes context engineering as the era after prompt engineering, in which builders assemble memory, data, prompts and playbooks for a model.
“We've gone through context engineering, which is, okay, what if I can cobble together a bit of memory, a bit of data, a bit of prompts, a bit of playbooks.”
Goldman Sachs Exchanges · 0:09:49
Churchman predicts a next phase he calls mandate engineering, which concerns what AI agents may do, under whose authority and in whose name.
“like mandate engineering, which is like, what are AI agents allowed to do under the who's authority in whose name.”
Goldman Sachs Exchanges · 0:10:51
Churchman predicts AI agents will gain autonomy once an insurance or liability market exists around their actions.
“I think once there's an insurance market or a liability market around what these agents do, then I think you'll see the autonomy of them increase”
Goldman Sachs Exchanges · 0:10:51
Churchman says an AI demo is judged on its best day while a product is judged on its worst day, and that closing the gap is difficult work.
“a demo is judged on its best day, whereas a product is judged on its worst day, right?”
Goldman Sachs Exchanges · 0:12:00
Churchman says reasoning models, function calling and the Model Context Protocol made Goldman's earlier reasoning scaffolds and custom tool connections pointless.
“all of those reasoning scaffolds we built, once reasoning models came out, totally pointless. All of those ways that we'd connected the LLM to our tools and our data, totally pointless once you had function calling and MCP.”
Goldman Sachs Exchanges · 0:13:03
Churchman says models do not natively know a firm's entitlements, mandates or how its data is connected, so these are what institutions should build.
“It doesn't know our entitlements. It doesn't know our mandates. It doesn't know the way that we connect our data together, et cetera.”
Goldman Sachs Exchanges · 0:13:34
Churchman credits the 2017 transformer paper 'Attention Is All You Need' with letting every word in a context window attend to every other word.
“it's really the transformer paper in 2017, attention is all you need, which basically said that actually you can have every word pay attention to every other word in a context window”
Goldman Sachs Exchanges · 0:21:26
Churchman says the Kaplan scaling laws and the Chinchilla paper showed that small experiments with AI models can predict results at much larger scale.
“But then the Kaplan scaling laws plus the chinchilla paper basically told you that these things, if you do a small experiment then to see whether it works you can then scale it up”
Goldman Sachs Exchanges · 0:21:57
Churchman says the top priority of AI labs is to automate AI research and development, with running experiments the main bottleneck.
“importantly, in the labs, their number one priority above all else is to automate themselves, which is automate AI R &D.”
Goldman Sachs Exchanges · 0:21:57