17 Sep 2026 · 55m · Daniel Whitenack, Chris Benson, Liam Dunne, Ben Moore · 25 quoted facts
AI search is an expansion of SEO rather than its replacement. Brands still have to do on-page, off-page and technical work, but the priorities change because people now ask long, detailed questions inside chatbots. Liam Dunne and Ben Moore of Discovered Labs argue that visibility depends on relevant content, third-party consensus and a consistent brand message, not on any single citation.
The tension is between what users see and what the model does. Ben describes a multi-step process in which Reddit threads and other sources shape an answer without being cited, and training weights carry brand sentiment. Measuring all this is noisy, so Ben says most of the industry is not doing it rigorously. The guests expect AI agents to move from reading sites to acting on them.
Ben Moore describes four layers: model weights, the model's reasoning, retrieval, and the final response. A brand can influence an answer through retrieved sources that are never cited.
Ben says ChatGPT's retrieval engine once reserved about a third of its slots for Reddit. Most of those threads were dropped at citation time, but they still helped ground the model's answer.
Liam says upvotes did not correlate with more citations. The passages that were extracted were often far down a page with one or two upvotes, because they matched the query's details.
Ben says that where an industry publishes little, such as pricing, a roughly accurate number can be cited because the model avoids guessing. A competitor's silence creates the same opening.
Liam says clients report agents booking demos, and some users arrive through command-line tools where an agent installs software for them. Slow pages or confusing forms may cost conversions because agents will not wait.
No. Liam Dunne says SEO has expanded rather than died, and that the same three jobs of on-page, off-page and technical work remain. What changes is which tactics matter most.
Liam says the AI model is now the visitor: it reads your content and gives the answer to the user, so the click never happens. This is called zero-click search.
Ben Moore explains that when a user asks a question, the model splits it into several related searches and pulls back a set of candidate sources, perhaps 40 to 60. It then checks which of them agree before writing an answer.
Liam suggests three principles: relevant content, consensus from third parties, and consistent messaging. He names Reddit, digital PR, customer reviews such as G2, and YouTube, and says that if he had to pick one channel he would bet on YouTube.
Ben says be cautious. AI answers are partly random, and few in the industry check whether a change in a metric is statistically meaningful or just normal noise.
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Liam Dunne treats AI search as the broad category, with answer engine optimization (AEO) and generative engine optimization (GEO) as the same discipline.
“So I would say AI search is like the broad category, um, and then within that you'd have answer engine optimization. Some people say, which is AEO, some people say GEO generative engine optimization.”
Discovered Labs is an organic search agency offering end-to-end services covering traditional SEO and AI search.
“it's a organic search agency, um, so we provide end to end services.”
Liam Dunne says organic search still consists of three jobs: on-page content, off-page brand authority and technical accessibility.
“We're doing the same three jobs, that's on page, um, so, you know, looking after your owned website. We're doing off page, which is building brand authority, and we're doing technical”
Liam Dunne says an AI model now acts as a website visitor that absorbs content and relays it to users, causing lost clicks.
“the LLM is the new website visitor. They're taking that information, they're, they're chewing it up, and they're spitting it back out to the user inside an Lm.”
Ben Moore says traditional search rankings were largely deterministic except when Google ran a core update.
“unless there was a Google core update, essentially like. The rankings were essentially very deterministic.”
Ben Moore says language model answers have two sources of randomness: floating-point errors in GPU computation and deliberate sampling temperature.
“there's typically two sources of randomness. And the reason I mention this is kind of important. It's like the kernel function, so there's like floating point error in GPUs.”
Ben Moore criticizes brands that buy AI-visibility tools such as Profound and track single prompts without bounding measurement uncertainty.
“they buy profound or some other thing and they're like, Here's my prompt, let's go away, go do this and profound.”
Ben Moore says citation rate, mention rate and share of voice should come with a bounded uncertainty, such as a margin of error of about five percent.
“how do I bound? The you mentioned there citation rate, mention rate, share of voice, how do I bound? The uncertainty on those metrics”
Ben Moore says a model's retrieval step creates multiple query fan-outs and pulls back a set of initial candidate sources.
“creates a a set of, um, basically query fan outs. Uh, you've probably heard heard the term um, and then pulls back in, um, basically a set of initial candidates.”
Ben Moore says a model may pull in roughly 40 to 60 sources and then run a further analysis pass before answering.
“let's say 40 to 60 sources from various places, it's done this. Um, let's say another set of um analysis on it”
Ben Moore says models use a consensus step similar to majority voting to reduce hallucinations.
“Essentially what it's doing there, trying to see, like, well, what is the consensus? So I can basically use like basically majority voting system, so I don't make a mistake or don't have a hallucination.”
Ben Moore says Discovered Labs research found ChatGPT's retrieval engine allocated almost a third of its retrieval slots to Reddit.
“it had allocated like an inse, I think, almost a 3rd of its basically retrieval slots for Reddit.”
Ben Moore says most retrieved Reddit threads were rejected at citation time but still helped ground the model's answer.
“Most of those were actually rejected at citation time, so they basically are kind of used to ground the model.”
Ben Moore says Discovered Labs found traces of Reddit content in model weights, particularly for ChatGPT and also Gemini.
“there is traces of basically Reddit showing up in the weights, particularly from ChatGpt.”
Ben Moore says that where little information exists, such as pricing in secretive industries, publishing roughly accurate numbers can get cited because models avoid guessing.
“if there's blank space, you could almost publish, you can almost get anything cited.”
Ben Moore says Discovered Labs' model rank analysis found Profound had a notably negative sentiment score in the models.
“when we did the model rank analysis, profound has a really negative uh sentiment in the score.”
Ben Moore says models learn brand associations through co-occurrence of entities in training data via the attention mechanism.
“it's all about co occurrence, right? So there is this attention function, basically, where it's basically going along, looking at all the tokens in a certain space and it's associating you.”
Liam Dunne says Reddit offers an edge because many B2B companies do not know how to use it well.
“One of the advantages of Reddit is, um, a lot of B2B companies don't really know how to do it well. And so just like because of that, you, you can gain an edge.”
Liam Dunne says that if he had to pick one channel for search visibility he would choose YouTube because it is a Google-owned search channel.
“I'd bet on YouTube. It's a it's actually a a native search channel, it's owned by Google. I think it's gonna be around for a while.”
Liam Dunne says Discovered Labs saw no correlation between Reddit upvotes and citations, and the strongest correlation was with the content matching the query.
“we haven't seen that correlate with, um, increased citations. What we have seen highest correlation with is the actual um content itself, right?”
Liam Dunne says Discovered Labs clients have reported an AI agent booking a demo on their websites.
“we have clients reporting to us that, hey, a d, uh, an agent, an AI agent, booked a demo last week.”
Liam Dunne says some Discovered Labs clients with tens of thousands of weekly conversions get many users through command-line interfaces where agents act for them.
“we have clients that are generating a lot of conversions, you know, tens of thousands of conversions per week. And a lot of their users are coming via a command line interface.”
Ben Moore says model training cycles have shortened from a new ChatGPT version every nine to twelve months to much more frequent updates.
“the training cycles are actually coming down. So when we 1st started, it was like, cool, we have like a new version of ChatGPT once every like 09:12 months, right?”
Ben Moore says AI developers use fingerprinting to avoid training on their own generated text and the resulting model collapse.
“they're trying to avoid training on like, basically model collapse, training on their own systems. And so they're looking at all this content out there.”
Ben Moore says OpenAI wanted to offload verification of human-written text to third parties such as Trustpilot, which he says it partners with.
“they wanted to offload, uh, the verification of text to third parties like Trustpilot's a good example, they're in partnership with them.”
Liam defines AI search terms and explains that traditional SEO has widened rather than died.
“you won't hear me say that SEO is dead. It's not a it's not an opinion I I hold”
People research inside chatbots, so clicks fall and AI agents become a new kind of website visitor.
“the LLM is the new website visitor.”
Ben argues AI answers are random enough that most metric movements need statistical checks.
“a lot of people waste a lot of marketing hours and a lot of engineering time”
Ben walks through tokenizing, reasoning, query fan-out, retrieval and a consensus check.
“creates a a set of, um, basically query fan outs.”
Reddit threads shape answers even when they are not cited, which explains why citation counts mislead.
“Most of those were actually rejected at citation time”
Liam lays out where brands should focus and why a consistent description of the company matters.
“I think it's relevancy, consensus, consistency, right?”
The guests expect agents to read and write on websites, and expect brands to aim at training data.
“moving from like a, I always describe it as like a. A read only environment”
Welcome to the Practical AI podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work and create. Our goal is to help make AI technology practical, productive and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn X or Bluesky to stay up to date with episode drops, behind the scenes content and AI insights.
You can learn more at practicalai. fm. Now onto the show.
Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack. I am CEO at Prediction Guard, and I'm joined as always by my co host Chris Benson, who is a principal AI and autonomy research engineer. How you doing, Chris?
Hey, doing great today, Daniel, let's how's it going.
It's, uh, uh, it's going great. It's good to see you. Uh, you're visible to me. Um, and and that, uhthat, that's an interesting part of An awkward the topic today, an segue, awkward segue into how, how do things become visible to us on the internet these days, which seems to be increasingly through AI platforms, AI, uh, chat interfaces, answer engines, whatever you call them. And today we're privileged to have with us, uh, Liam Dunne and Ben Moore, who are co founders at Discovered Labs, to talk through some of these things.
Welcome Liam and Ben, great to have you.
Good to be here, thank you. Likewise, thanks,
well, for for those, uh, that maybe are less familiar with this topic in general, around AEO, GEO, answer engine optimization, AI visibility, whatever kind of term is around this, and maybe there are differences between those terms. But for those that aren't as familiar, could could you all give us a context for kind of what those things mean? And then also, like, how how you're involved in those topics day to day, what, what you're kind of doing at discovered labs, and which is kind of the context that you're doing some of the work that we'll, we'll talk about.
cool, yeah. So I would say, and everyone's got a different opinion on this, so, um, feel free to take mine with a pinch of salt. So I would say AI search is like the broad category, um, and then within that you'd have answer engine optimization. Some people say, which is AEO, some people say GEO generative engine optimization. I view those as the same thing and I just call it AEO. Um, honestly, for a very simple reason.
There are a lot of, uh, venture funded companies that have spent a lot of money on that term. Um, and so I'm just gonna, um, fly behind them and and lean into it, um, the background of, uh, so what, how are we involved with this? So we're co founders at discovered labs, it's a organic search agency, um, so we provide end to end services. Now, organic search for us splits into two buckets you've got, like traditional SEO and you have AI search, so traditional SEO.
We want our clients to be at the top of Google whenever people are searching related keywords, AI search. We want our clients to be appearing inside LLMs in a way that they want to be. Um, and so we view that as like the AI search side of things.
Gotcha, and I, I guess, maybe for context, people might be familiar with, like, SEO, search engine optimization, and I is it may maybe help, uh, orient us is like, SEO is. Is that still a thing? Is this topic basically replacing that? Is it kind of, uh, how do the how do the two interact? I I mean, this is always, I guess this is a fluid thing. But if I am a company, um, this, you know, I'm, I'm a company in 2026.
Like, what, uh, what? What is most important? And what? How are people thinking about the effort they're they're putting into? Maybe traditional SEO versus these topics? Do you have any thoughts there?
I do feel free to, but um, I, you know, I can waffle too much, so feel free to ask me questions. I'll go deeper, but ultimately, you won't hear me say that SEO is dead. It's not a it's not an opinion I I hold, I just think, um, the the space has grown. Um, we're doing the same three jobs, that's on page, um, so, you know, looking after your owned website. We're doing off page, which is building brand authority, and we're doing technical, which is making sure agents and Google systems can access your website and understand the information.
It's the same three jobs. For me, it comes down to a matter of tactical priorities, and I think this is where people will, like, get lost. Um, I do think there are some things like, depending on what the goal is right, like to to rank number one on Google for a commercial keyword. You're probably going to use different tactics than if you were to optimize for a surface area like ChatGBT. Again, you're gonna be doing the same jobs, you're gonna be creating content, you're gonna be building the brand of your company, you're gonna be, um, optimizing for, uh, for technical SEO.
But the tactical priorities is where it changes. Like, for example, what does good content actually mean from a perspective of Google versus Chatgpt? Um, so ultimately, SEO is still a thing, it's now expanded, we now have AEO. Those together, I call organic search. Um, we're doing the same three jobs, but the tactical priorities have just just changed a bit. I would say,
could you talk a little bit about? Like, if you are, uh, you know, a business owner out there, and you're, you're, you're marketing, you have your website out there, you got your social. How has this changed? Like, if you were to take a snapshot of the industry? And and a long time ago, I was, I was in, uh, the digital marketing industry for a while. And and you know, that was obviously before AI came a thing.
And you were focused on these things, minus the the AI engine optimization. And so how has that changed? Like, how, how is the thinking of of the marketing department changing? Um, to accommodate the fact that you now have this whole way that people are are going out and searching, and you know, all of us are are AI. 1st. That's very different from maybe 15 years ago. How should, how should business owners, listen, listening, be thinking about that in terms of, uh, how they approach the whole thing?
yeah, so broad topic, so It is sorry, I would say, no, no, no, it's all good. Um, so I would say the 1st thing is buyer behavior, right? So people researching and consuming information inside LLMs has changed buyer behavior, right? And so the symptom of this is people will see that, hey, like, where's all my organic traffic gone? Um, and it's because that organic traffic. A big chunk of it was created by people consuming information on your website.
Um, but if people are consuming information inside an LM, the LLM is the new website visitor. They're taking that information, they're, they're chewing it up, and they're spitting it back out to the user inside an Lm. And so you've now lost those clicks. So I think that's the 1st thing, and so the implications there are well, how do we measure that? And that's really challenging. Uh, like, I'm a marketer, Ben is the the the engineer, um, and measurement has always been like a nightmare for marketers.
Um, and so this is this concept of, um, zero click. Researchers has been around for a while. I think AI has just um, sped that up a bit, you know, we've been dealing with that on social platforms. Um, I think now organic search is starting to feel that. So definitely a change in in buyer behavior. Um, you know, it's, it's. It's probably very similar, as as, like, when the mobile came around, um, I think we're we're seeing some changes there.
Um, and then also I kind of touched on it there. There is now just a new visitor to your website, so websites for the last, however many decades they've been around, have really been optimized around the human visitor. Um, that's why we have, like, user experience. How do we make the navigate? That's why websites look the way they do today, right? Like, how do we, um, shape the navigation bar, like conversion rate optimization, all of these things, um, are for a human visitor.
Um, whereas now, um, it's these agents visiting your website. And so I think there's some interesting considerations there of, well, what are they looking for? And and and what state does your website need to be in for them to be able to access information and and understand you? Um, and so I think there's some considerations there. I don't know where this ends. Um, I know there are a lot of companies out there that are building towards this.
I. E. You have like one version of your website for humans, one website version for for agents. Um, but I'd say those are like the and then then obviously there's a bunch of implications like downstream of of all of those things. Um, so, d does that answer your question? Or do I have to go to deep deeper into any of that?
it's. It's a great start. I appreciate it. Kind of laid, laid the landscape out there. Uh, and I know Daniel has one, but after that I got plenty more.
yeah, I I think my. What triggered in my mind, Liam, as you were talking about that, is is that measurement piece. One of the things that I think I appreciate about your all's approach. And both from a both from a company standpoint, but also a resear research standpoint, is you're interested in knowing actually how the mechanics work around these things and and understanding it at a deeper level. Ben, I'm, I'm wondering if kinda turning to that technical side.
You've published a variety of research over time and you'll have more coming. Um, heard a little bit about that before the interview when we were talking. But um, for example, like citations or mentions in these answer engines or in AI systems. From a technical standpoint, like, obviously, I as a human can go into ChatGBT or or Gemini or Claude or whatever. And I can test out some prompts and see what's cited. Um, how, from a technical stand, like, what's required from the technical standpoint to actually have have a scaffolding and a mechanism that you can track your like.
What does it mean to track your visibility in these systems over time? What are the relevant things? Like? I mentioned a couple things, citations, mentions, like, what are the relevant things that you're looking at? And at a high level, like, what does it take to put the right tooling in place to actually measure and track that?
Yeah, um, yeah, as as Liam said, it's it's a big question, but it, it's a great one. So I, I would say, this is probably the thing that has probably shaken a lot of the SEO industry at its core a lot. In the sense that everyone kind of showed up, uh, you know, in the past with traditional SEO and said, Look great. Let's like, kind of color by, uh, is it like, um, paint by numbers? Kind of thing?
Like, you fill in the colors and we do this and we do that, everything's great, everyone's happy. And no one really actually thinks about, like, what's going on here? Um, and you could go into like, essentially, unless there was a Google core update, essentially like. The rankings were essentially very deterministic. You had a set of keywords and it was like, pretty finite, pretty manageable, all pretty controllable, and again, paint by numbers. And now we've kind of, like, gone into a system.
I can go really deep on this as needed, but um, we've gone to a space where, uh, there are sources of, like, uh, non determinism or STP stochastic elements that people need to start modeling around. And the SEO industry, and we've seen it as a whole, uh, doesn't have like great statistic statisticians, or like great, um, data science literacy. To be blunt, um, and if you're any serious, like statistician or control engineer, you would look at the system you're probing.
And you would say, well, like, what are the requirements on the signals? I'm trying to measure here in order to bound the noise on these these, uh, these measurements, essentially, and the S and the the quantities you're trying to measure, uh, and then you would say, okay, well, then you'd set up your model of the system. You, you like, essentially, a lot of these LLM agents are essentially gray boxes, they're not totally black boxes.
There are research papers out there, there are hints in the the in the network traffic and some of the if you can decrypt the packets. There's also in information there, uh, and there's also like distilled models. So when I looked at the training data, you can also figure out like some information from the distilled models. And so the thing is to set up basically, uh, basically, uh, experimentation, a kind of engine or pipeline, whatever you want to call it.
And then say, Well, how do I bound? The you mentioned there citation rate, mention rate, share of voice, how do I bound? The uncertainty on those metrics, and what do I need to bound to? B Basically, bring that uncertainty under, let's say, five margin of error. Uh, and to be honest, no one in the industry is really doing this, uh, or even really thinking about it. People go on, they buy profound or some other thing and they're like, Here's my prompt, let's go away, go do this and profound.
Aren't interested in it either, they just like to sell you something. Um, and then everyone says, Oh, look, number goes up, how lovely. And then the SEO industry has like an incredible ability to, um, point at numbers going up and claim that it was them. And then has an also incredible ability to point at a number that's going down and claim it's Google core update. Um, uh, so yeah, I just think in general I can go into, like how we think about this and how like the various systems are like displaced here, and and how we think about bounding this randomness.
But in general, um, you need to approach it. Like, how am I gonna model this system? That I know hints abound and with the data we have, and then how do we bound that? Um, that basically uncertainty on those quantities we're trying to measure for, as you mentioned there, Chris, like for the business that has whatever they're selling, um, you know, gardening tools, whatever it is, right? That's that's, uh, that's what people need to be getting to, and no one's really talking about that.
Um, and I said that it does take a lot of time, uh. But I also think we've got to the point where it's like we've got to like stock market dynamics. Like there was a change in Reddit recently, and you know, no one also really knows, is it meaningful? So lots of times we have to come to our clients and say, so, let's say I'll give you a prime example, right? Yeah, there'll be like, you'll probably have CNN on in the background there.
And someone will say like, Oh, or ABC news, whatever it is. And then they come down, they say, Oh, Dow Jones is down five. Everyone freaks out. Uh, if you actually look at the like, basically just standard deviation over the last year, you'd be like, Well, this is not statistically significant. That's like a very basic check, like it's very basic, but we need a print news, so we do that. And the SEO community is a bit bit around that, and also people just wanna like, get eyeballs on stuff.
But again, you know, a lot of people waste a lot of marketing hours and a lot of engineering time around things that aren't statistically significant. That's a very basic thing that we could check. Like, that's not like, that's not rocket science, but there's a very simple example anyway. That's why I think we're like, kind of like, state of traditional SEO, where we are now, and where people are freaking out around it and where we need to head.
I don't know about you. But I am sick and tired of showing up at AI or technology events and immediately realizing that they're just going to be full of sales pitches or things that aren't useful for my day to day. I have more than enough to fill my time and I really need to focus on bringing back value to my company and my day to day work. That's exactly what the Midwest AI Summit is about, October 15th in Indianapolis, Indiana.
We we, as the podcast, are partnering as a media partner with the Midwest AI Summit. And there's just an amazing amount of practical value that you'll get if you attend the Midwest AI Summit. There's actually an AI engineering lounge where you can sit down with folks like me and other AI experts. To get feedback on your roadmap, your architecture, tooling, how you're going about your AI transformation in your company. And there's amazing speakers, great food. This is one not to miss.
This is gonna happen in Indianapolis, October 15th of this year, 2026. We already had one last year. It was an amazing success and you're not gonna want to miss the one this year. Check it out at Midwest AI Summit dot com and you can get 20 off with the code Practical AI 20. So check it out at Midwest AI Summit dot com and use the code Practical AI 20 for 20 off registration. So Ben, I, I wanna ask a follow up question.
Uh, you did a great job at kind of laying a foundation for maybe some of the things that we should be, some of the things that we should be thinking about statistically as we approach this topic. I'm wondering if you could help our listeners understand. Like, if I put in a prompt to one of these systems and a brand shows up, or a thing that I want to show up, shows up, how? What are the mechanisms by which that can show up?
And I, I know you've done, uh, some research across, like citations, uh, and web search versus Reddit versus training data. Before we get into any of those, like, very specifically, could you help us understand, like, what, what do those like, what happens behind the hood? How might I show up mechanically in in one of these systems?
Yeah, uh, so let me get dig into it. Just before I do, I would say to anyone that anyone listening, for sure, do not run down the rabbit hole of being like, this is this one prompt and I would love to rank on it. Uh, you wanna basically model, like, if from like, a basically a topical, uh, domain standpoint, you can get into more of that. But basically, yeah, what happens is, uh, essentially your prompt is fed in, it's, uh, it's tokenized and then ingested by the Lm.
Um, and then there's a essentially a a reasoning stage, right? So basically, uh, the transformer basically has has embedded your prompt. And now looks at all those tokens and starts generating its like reasoning stage based on its training weights. And basically it has a it's running this like forward pass, uh, and there's typically two sources of randomness. And the reason I mention this is kind of important. It's like the kernel function, so there's like floating point error in GPUs.
That's one source of randomness, and the second source of randomness is, um, the actual temperature itself, like the deliberate temperature randomness. Um, that generates the next token. And then the final part of this is like, it's also us. Speaking on the end of that, this is now there's now, uh, fingerprinting on these LMS. So they'll also nudge and create hashing functions in the background to know where the content came from anyway. So it goes through this reasoning phase.
Let's say you say like, a best, um, I don't know, gardening service in, let's say Dublin, Ireland. I'll use that example. And it'll then start reasoning. It's saying, Oh, it'll have a certain set of criteria, it'll like, do some reasoning before it makes any, you know, queries with its retrieval engine. And then it's exposed to a series of tools, and one of them is obviously gonna be the retrieval engine. I'm saying a series of tools. Because when we get to agents, they could be exposed to a whole host of other things that you're not aware of.
Um, then it accesses, accesses this like retrieval engine, creates a a set of, um, basically query fan outs. Uh, you've probably heard heard the term um, and then pulls back in, um, basically a set of initial candidates. Essentially for um to be pulled into the the context window. But before that happens, essentially they look basically there's. It's not always exactly the same algorithm, but DeepMind had an agree style algorithm, it's it's called, uh, and basically it's.
It's a way to avoid hallucinations. And so once they pulled in these, let's say, let's, let's say 40 to 60 sources from various places, it's done this. Um, let's say another set of um analysis on it, which is M is almost across the board. It's very similar to the agree fra algorithm from DeepMind. It's very similar in, uh, um, Anthropic's models, it's obviously it's in Gemini's models, and also ChatGPT. Uh, we've seen this in our own analysis, uh, and also Anthropic moved to it recently when they moved to, uh, Fable and Fable 51.
Essentially what it's doing there, trying to see, like, well, what is the consensus? So I can basically use like basically majority voting system, so I don't make a mistake or don't have a hallucination. Uh, and then it says, Okay, cool, great. This is our kind of like distilled set of contexts we wanna work with now. G Like, let's say, generate the final response, essentially from that cleaned up uh context, uh. And this actually kind of ties back to, like, Reddit, people freaking out about the final citation.
And again, the way I look at like AI visibility, everyone's like, Oh, we'd love to know about citations and mentions in the final like output. But there's all this other part before we got there. So I talked about, you know, the weights, how you're showing up in the weights, and we did some research on this recently. How is the model reasoning? Like, what is what is it thinking like when it? When it makes like core decisions around your domain, or be that bottom of funnel, top of funnel, whatever it is, it, it, it has a, uh, a criteria, it's going through its internal logic, just like a human would.
Essentially, it's not a human, but you get my point. And then finally, you have, like, your retrieval engine and a response. But you really need to be looking across all of those four pillars in order to, uh, dominate your domain. That's how that's how we look at it for all of our clients.
The there there are lots of things to dive into. But before we do that, I'm curious. I actually wanna go back to to Liam for a 2nd. And and like, as I'm listening to Ben explain these things, the thing in my my head was kind of something that you were saying early on about, you know how it changes behavior. And and I'm curious as a marketer, I I like the ability to go back and forth between the the technical and and the the human behavior side.
How does? How is that changing how the human? Beh, you know, that's not now that you have the agent going and doing this, what does that mean for the human? That now has this, this proxy in the agent going out and doing that. And how is that changing the behavior and potentially the transactions, uh, that are following up on that? Um, just to kind of come full circle for a moment,
Yeah, so I guess I'll look at this from like demand and supply side, so from a demand side, the buyer. I think this is just a much better deal, right? Because if we, if we look at search behavior pre LLM, trying to find relevant information on Google was just a nightmare, right? Uh, because, you know, us marketers just ruin everything. Um, and you'd, you'd look for things and like, you just get the list of ten URLs that you then have to go visit and research.
Um, and whereas if we compare that today to like chat based LLMs, it's more conversational. So I could be like, Hey, I'm Liam, um, this is my company, it's in this industry, we're doing this revenue, we're really struggling. These are our core constraints. At the moment, we're looking for this solution. These are the attributes of the solution. You know, we're looking for a piece of software. Must have a free trial, uh, must charge monthly, must integrate with HubSpot or CRM, all of these requirements.
I can front load that and then the LLM does its thing and it comes back with personalized UM recommendations. Now I'm not saying that marketers are, you know, that's our job to influence those recommendations. But it's just a far better deal than like, clicking through all those websites and finding the information myself. Um, like me personally, I just use it for everything, right? Date night, you know, meal prep, like buying software, everything. So from a demand side, I think it's it's a greater deal, and so therefore I think more people are gonna take the deal.
Um, from a supply side, the people who are the the vendors, I th I would say I probably look at this, uh, from two ways. So number one, if people are front loading all of that context. That now creates a really long tail distribution of queries that those models are using. It's no longer just best cold email software, which is what people would search on Google. It's all of these entities and things are kind of hang, you know, um, appended onto the end, right?
The integrations, the the business model, um, all of these unique requirements. So that creates this long tail distribution of queries. Now, the great thing about L long tails is they're less competitive. Um, and so I think this has really been where the opportunity has been over the last year is where, ultimately, I would simplify it down to relevancy. Um, be authority now. Th There's some nuance in there, um, but this is really where the edge was.
And this is where, say, if I'm competing against HubSpot, who has dominated Google SERPs for all the money keywords I'd love to rank for. Well, now I can, um, create relevant content for those super niche queries that my buyer is searching inside an LLM. So I think there's that perspective is the long tail distribution means we can create relevant content. Um, that targets there, and then I think there's the authority, uh, consideration is okay. Well, if everyone has relevant content, how does the model then reason over who to trust?
who does it cite? who to trust? Um, historically, with traditional SEO, you know, we look at page rank, um, it's, you know, um, how we know Google assesses authority of pages and domains. Again, it's it comes down to consensus. Um, it's not as weighted, is my understanding, you know, Google Google have publicly said this, it's not as weighted nowadays. Um, what's like the LLM's version of of that? And this is what seems to be changing a lot at the moment, right?
Because marketers use a bunch of tactics, they work well, everyone starts using those tactics. Law of shitty click throughs kicks in, uh, and so then they diminish, and then we need to find the next edge. Or, like, you know, the these vendors like patch, patch, the vulnerability, basically, this is basically what we're doing is we're exploiting systems, you know? um, SEO is just exploiting Google systems. Um, so I think there's like a whole thing there of like, well, how do I ensure that my brand, my brand's information, is chosen as a candidate?
And and and the position I want, um, is communicated back to the user inside the LLM. Like that, that's how I'm I'm viewing it at the moment from like a marketing perspective.
Yeah, that's, that's really helpful, and I guess to that point, there can be I, I, you mentioned this, Liam, like, or, or maybe it was Ben. Like, trying to get people to think more about, like, oh, here's the prompt I want to optimize around. And I know something that, like, people have talked about to me from various aspects is, oh, you need, like, a Reddit strategy to to be to to do great in AI visibility. And I, we are.
We were chatting a bit about this before the conversation came about. But I know you all have done some research, both earlier research on like what actually gets cited in these LLM systems. But then more recent research around the the weight that a and the weight and the importance of kind of a Reddit strategy. And how that's surfaced visibly versus influencing some of that. Be behind the uh, behind the scenes activity that you were mentioning before, Ben, do you wanna just kinda help us understand some of the research that you've done there?
And maybe that Reddit piece, um, or, or, or other things that kinda influence that, that citation, that are that are relevant in in relation to Reddit.
Sure, yeah, so on the Reddit front, so obviously, um, as I mentioned there, there's obviously the kind of like the 4th pillar of like AI visibility, the actual response. And then there's the reasoning or retrieval stage, and then, uh, we did a piece of research on the ChatGPT uh retrieval engine at the time. And uh, it had allocated like an inse, I think, almost a 3rd of its basically retrieval slots for Reddit. So if it could find relevant Reddit threads for for that topic or that query, uh, it would basically inject that into the context, uh, window at the time.
And then obviously, as it does, like, as it gets those, um, those Reddit threads, let's say 30 of its retrieval, let's say 60 slots, 20 of them are allocated to Reddit. Um, most of those were actually rejected at citation time, so they basically are kind of used to ground the model. And uh, formulate its thinking, but often not actually cited in the final response. And then there's also been so that's, that's one aspect of it, and then there's also, uh, also the long form Reddit.
Like AMAs questions and Reddit training data is also used within the the training data itself. Uh, I don't, I think it's maybe around it's, it's typically around the um, oral HF, um uh stage, where it's like a reinforcement learning from human feedback. Um, and that that's basically golden training data for them, uh, for that. And if you actually see, when we looked at this, there is traces of basically Reddit showing up in the weights, particularly from ChatGpt.
Uh, um, as well as Gemini, um, when we looked at the basically the open source models. It's not like super present, but you can see it, it's, it's, it's there. Um, so with those in mind, I think, uh, you know, like I said, people say, oh well, citations have dropped. It's kind of like saying, um, you know, it's one signal on the engine, I would say, but it's not everything. Um, and ultimately, if you wanna, like I said, if you wanna, um, basically, you know, uh.
Move the narrative for your brand or own in a certain direction. You need to look at all all sides, the reasoning engine, the retrieval side of it, as well as the response and the the weights. Um, so, yeah, I would say, um, it. Reddit is still very much a big player.
Yeah, and may maybe just to check my understanding here I, if I'm understanding what you're saying, this would be like, oh, in that retrieval and query fan out stage. Let's say I retrieve 20 sources and ten of them are from Reddit saying a very similar thing, and maybe one of them is from TechCrunch talking about the same. Like it confirms the ten Reddit things. It could be that the answer engine uses those eleven sources as confirming the same thing, and that's what it's gonna talk about.
But it doesn't cite the Reddit things, it cites the TechCrunch thing. Because it's like, I don't know, uh, uh, for whatever reasons behind the scenes, it, it cites that. Do I have the right understanding here?
yep, exactly, yeah, that's exactly it. So essentially it's like at that, basically, uh, once, once. We retrieve those those that, um, that set of, let's say 20 sources you mentioned, and let's say ten of them are Reddit. Uh, there's a series of processes that go on, like some of them could be like re ranking versus the query. And then there's also there's some, uh, later research around, uh, basically, like domain authority and where ChatGPT wants to send user itself.
Uh, but yeah, that can all of those can basically combine to say, look, we've used Reddit to ground the answer here. We're confident to move forward with this one. Um, but we, we, we haven't, we haven't actually cited it. That's that's exactly what's going on there.
If you're like me, you need a good amount of help. Keeping your website updated, launching new websites, landing pages, et cetera. And you need an actual platform for your company, not just a builder of websites, but a platform where you can launch and continue improving your site. I'm excited to share with you about Framer, our partner and what they're doing to actually enable this sort of work. They have agents integrated into their website platform that help streamline collaboration.
They can build custom code components, they can manage CMS content and much more. They they work side by side with humans. Framer is the pro site builder that's for creatives, teams and businesses that want a professional site and care enough to get every detail right. Learn how you can get more out of your site from a Framer specialist or get started building for free today at Framer.com. Practical AI for 30 off a Framer Pro annual plan that's Framer.com.
Practical AI for 30 off Framer.com. Practical AI. Rules and restrictions may apply.
Then I wanted to do a quick follow up on what you were just talking about. And that's kind of that, that relationship between, uh, retrieval and citation, a little bit. And so it seems like they like, intuitively, I might have thought that there was that, that there was a, you know, a positive relationship between them. And that, you know, um, that retrievals would lead to citations. You just talked about the fact that, you know, that isn't necessarily the case.
Um, could you talk? I mean, it seems like that's a significant thing. Like, if you're, I would guess that if you're a marketer out there, um, and going with that intuitive approach, that that would be kind of a substantial pivot that you'd have to make. Uh, to, to kind of accommodate that. Could you talk a little bit about, um, you know, like, what that means? What does it mean? The fact that the the retrieval and the citations don't necessarily line up that way.
And and and Liam, I'd also love to hear what you have to say about that on the on the marketing side.
So, yeah, so what I would say is, uh, yeah, it is a big, a big change. Cause. Typically it's like, look again, like, you know, traditional SEO, we'd say, basically, look, oh, we're querying an index, we get something back, then it's reasonably stable, minus a Google core update. And now we have essentially, um, yeah, multiple, multiple steps, multiple query fan outs in that. And not only that, we have, like, uh, re ranking involved, as well as like the LLM, basically critiquing the information itself in this agree style.
Uh, algorithm that I mentioned, so, um, yeah, I think basically just to all, just to kind of link back all the possible sources. At citation time, we talk to about about like, basically citation time optimization, to say, like, well, how do we end up in the final citation? Um, and then one of the things is obviously consistency like, or and consensus like. So if I have, if I have this long tail query, basically the one of the query fans we've been retrieved on.
What other supporting materials are out there that are likely to get ranked on that query to support it? So that's like consensus. We have a number of audits we do on this, et cetera. And then the other side of it is so once I pull in that, like even before, so I pull in that that, um, that set of sources that basically add consensus, that'll help. And also you wanna avoid saying, well, let's say, um, let's say there's some like anti consensus information out there.
You don't wanna. You wanna avoid that building up. So let's say there's something about your brand where it's actually like negative, let's say reviews, and there seems to be like it's published in multiple places. That'll be a good example. That's actually one thing we've looked at a lot here is AI perception. And one thing we found is actually quite interesting is that, like, if there's blank space, you could almost publish, you can almost get anything cited.
Uh, so, for instance, like, let's say certain industries are very, very sensitive about publishing pricing, and we found this works like insanely well. If you publish numbers that are even like yardstick directionally correct, um, because there's blank space and the LLM doesn't wanna hallucinate, essentially, um, so that's just a good, concrete example of it. And similarly, like if a if a competitor isn't talking about those, you can also like, do something similar to kind of capitalize on that.
Then the 2nd thing is like, well, even when it basically perceives that information, it's got from your brand, like, how are you? how are you showing up in the weights? So it might have a very, uh, it might have a very negative sentiment. Or around your brand. And if it's looking at data from your brand, let's say, for instance, it's quite funny. When we did the model rank analysis, profound has a really negative uh sentiment in the score.
Um, and uh, essentially that's gonna impact then a citation time. Uh, so like, uh, and this can even come down to simple things like, what is the name of your brand? Uh, so if your brand is like a a word that is like, uh, let's say, already e, uh, a existing English word in the dictionary, for instance, that can be an issue. Or if it's like, has negative connotations with it, or it doesn't align with the like.
Basically, overall, let's say it's a security platform and it's like, openly or something like that, that could have a a negative effect. Um, uh, so there there is multiple points there, both on the retrieval side, like consensus, as well as like how the model is like, basically the sentiment it has towards your brand and also what. So, when these models are trained, coming back to the weight side of things, it's all about co occurrence, right? So there is this attention function, basically, where it's basically going along, looking at all the tokens in a certain space and it's associating you.
So if I have, like, let's say gardeningstools. com, um, what are the other entities I've related to as I've gone through the training data? And then when it comes to citation time, it's like, well, that also influences, like, okay, well, well, this, you know, we're talking about X. So essentially, like you, let's say the the entity shows up, that's related to you, you're more likely to get retrieved again. So it's all like, it's essentially like an embedded form of a knowledge graph.
So typically, most of these indexes from Google, et cetera, are built on knowledge graphs, but now we're in embedded form in this transformer layer. But you gotta try and assess that. So I'd say, basically, there are three areas, well, or primarily two areas the weights, uh, and how it's reasoning on those weights for you, and then the consensus around that and the weights. Embed factual, factual accuracy, uh, sentiment and co occurrence. That's how I think about it.
But, uh, yeah, it can be like, it's not, it's not easy to, it's not easy to, uh, uh, operationalize that. And then like, start weighting that for for like, basically, if we just get profound or something like that,
Liam, I'm, I'm curious, um, so Ben just described, kind of o. Of course, there's this whole chain of things that influences what eventually shows up in the actual response. Everything from the the the model weights to this query fan out to the agreement algorithm. Uh, the the uh, the naming, all all of those things, uh, co occurrence. I'm wondering, like, over time, obviously, this industry of AI visibility is evolving, right? And so at a certain point, maybe it was like, Oh, you need FAQs on your website.
This is how, like the easiest way to to show up. Now. Now you're kind of thinking across all of these stages of, uh, of the process. So how do you when, when you're engaging with new, with new clients, different brands, like, w, what is? Uh, are there any generic kind of takeaways around, like the strategy of where, where you start and what? What is kind of near term and and long term important for your brand? Uh, so that you're hitting all of these things, but also you're able to make progress quickly.
Um, any, any thoughts on that?
Yeah, so, um, I, I would just say three words, and then I'll, I'll, I'll dig into them. So I think it's relevancy, consensus, consistency, right? Um, and all of this you could also put into buckets of like tradit, like again, same three core jobs I I said at the beginning. So, and and it really depends on what your current state is like. Do you have a website today? That's gonna be super important, and like all the foundational stuff really matters.
Um, like the. I used to run a paid sa paid ads agency, and I was like, man, why do some companies, like, really succeed with ads and and others don't? There's lots of factors that go into it, but one of the key aha moments I had is messaging. Is like the companies that really succeed in marketing, they they have, they have a really defined ICP. They really know what their product does, and they're able to map that to clear messaging and they have really good differentiators.
And I think that's really important in AI. Search is like, what are those terms you want to be associated with? Like, what do you actually do? And how do you translate that into a language that people are gonna be searching? And I think that's how you create that relevant content, but you can't create good content if you don't actually know who it's targeted to, or or what you're doing. So I think that's like the relevancy piece.
I think that's how you ensure that during that query, fan out, these LLMs are coming across your content. Um, now there's we can go deeper on that, like, I think you s. You shouldn't just be posting blog content, you need a website, you know, your service pages, like, you know, all of that good stuff. I think then consensus is, and Ben covered this in in enough detail, is basically my content is like me throwing my hat in the ring.
Like, hey, we have relevant stuff over here. Consensus are all the votes of confidence, like, why should we choose this company? why should we? why should we trust them? Um, marketers always ask me like, Well, hey, where should we focus our efforts? And my answer is everywhere, really. What we're talking about here is it. It's a a budget and time constraint, right? Like, if if these, if, if budget and time weren't a consideration, then be everywhere.
If they are a consideration, well, that's really gonna depend on like industry. Um, we're quite bullish on places, uh, like Reddit. One of the advantages of Reddit is, um, a lot of B2B companies don't really know how to do it well. And so just like because of that, you, you can gain an edge. Um, I think also doing just traditional, you know, digital PR, I think building a presence on social channels is super important. Um, if I was to, you know, gun to my head, where should I focus?
I'd bet on YouTube. It's a it's actually a a native search channel, it's owned by Google. I think it's gonna be around for a while. Um, I don't think you should. I, I'm, I'm personally not that bullish on LinkedIn as a search channel, as a social channel. Absolutely, we're, we're on there, I really believe in it. But for search activities, I don't know. Um, I think also building, um, customer advocacy. So we operate in the SAS industry.
That's gonna be your G2s of the world, right? Getting people to say positive things about your brand on the internet is always gonna be a vote of confidence. Um, for, um, consensus and then consistency. This is kind of where it closes the loop. What I kind of opened the thought with is you need to get really clear on what you want to be known as as an entity, as a company, right? So if you go to our website, you're gonna see, uh, we use similar words across the place.
Because I want those to be embedded in in these models and and the world. Um, and so whether it's your activities on Reddit, you're posting videos on YouTube, you're posting content on LinkedIn, even like your company LinkedIn profile. Um, you know, uh, content on your website and on your blog. Always presenting yourselves with a consistent message, right? I think this is like, and so like. One of the the key contrasts here is, if we look at like link building and SEO.
The big check in the box was, Hey, we got a link, we got a do follow link. Um, you know, we're gonna, uh, get some link juice back to our website. In AI search, I actually think the blurb surrounding, even if that link didn't exist, the blurb surrounding you as an entity is far more important. Because, like, who are you as a company? We want that to be really dense, right? Like I want, if if an LLM picked this paragraph and put it back inside to an answer.
Are we being positioned competitively? Like we're thinking about it from that perspective, so that's like consistency. Now we can get into, like the tactics and all that stuff, but it really depends on, like, your current state. Because some people we can talk about Reddit and all these tactics, but some people might not even have clear messaging, right? They they might, they might just have a website with a few paragraphs of text on it, right? And so then your priority is gonna be slightly different.
Um, so it, it does depend on on where you're at.
I, I'm curious with with kind of that, with the focus on Reddit and and the potential there with, you know, like, what? What tends to make, you know, one Reddit thread more likely to surface than another? Are are some of the are some of the signals that you would have that are like, you know, upvotes and comment count, and you know, user reputation. Do those correlate with retrieval? You know, are those, are those con, you know?
And and so when you're looking across different threads on Reddit, you know, does that make a substantial difference? Um, how do you think about that?
yeah, so I'm not gonna speak in absolutes, cause I think these things are always changing. What we have observed is, um, en engagement in terms of upvotes. Um, we haven't seen that correlate with, um, increased citations. What we have seen highest correlation with is the actual um content itself, right? So I I always like, um, to break things into like, uh, uh, easy ways to understand, cause that's how my brain works. But if we just look at this from the principle of user prompts, LLM, LLM does query, fan out, and it's looking for information contained within those queries.
There's lots of things and entities, right? So if I go back to that prompt example, SAS founder, like doing this in revenue in those query fan outs, it's containing all these entities, right? Um, and so I want those to be contained within my content on Reddit. Um, and so we found very often that the passage being extracted from Reddit is like 50 down the page. And the comment has like one upvote or two upvotes, and it's because the passage was relevant.
Um, now I think that might change over time, right? Because similar to how relevancy is super important, but authority is becoming even more important. I think these models are gonna keep similar, like, you know how Google always, uh, releases a core update is, you know, they, it's. It's a cat and cat and mouse game between these providers and marketers who are constantly trying to catch up. And I know there are commercial agreements between OpenAI, Reddit, Reddit, Google, and so maybe, um, things like upvotes, maybe even the history of those accounts, um, things like that might factor into it.
I don't know the complexities of that, um, might change over time, but yeah, just what we we've observed so far is the overlap between what the user is searching and, um, the the content on Reddit.
Liam, you, you started getting us towards kind of, like, how things may or may not change, how how they're evolving over time. As we get kind of close to an end here for for this conversation, I'd love to close out by just asking each of you, maybe circling back to Ben to start e each of you. Like, as you look forward, what, what is kind of top of your mind? Either in terms of like, oh, there's like a huge opportunity here for for people that are willing to to jump into it.
Or something that's like, Oh, there's a really open challenge here. We haven't figured it out yet, there's more work to be done in this area in e. Either of those areas. Um, like, what are you thinking about as you end your days, or as you start out in the morning? That's kind of top of your mind going into this, this next phase of work.
I'll let Liam go, he's ready to go.
so hm, I'd probably say two things, um, so offsite, um, like, how are those 3rd party earn sources gonna influence how, um, I am, or our clients are perceived, uh, inside LLMs? I think there's just so much change happening there, and ultimately brand is the final moat, right? You see this plastered everywhere is like, and so, um, I really think that's where things are gonna change a lot. And so I'm always thinking about that, I'm always thinking about, you know, our clients only have certain budget and time they can allocate, um, and they're on strict timelines.
And so how do we increase the probability that the bet they make is gonna have, um, you know, the highest expected value? So I'm always thinking about that. Um, the 2nd area is agent accessibility, so I think everyone at the Mo, well, for the like, you know, everyone. I'm speaking in such generalities, but um, I think everyone's been focused on discoverability, right? Like, how do we get my content discovered? And I think there's so many edges to gain there.
This is this is going back to my original point. It's like, walk before you can run, um. But we're quickly moving to a place where it's not just going to be agents crawling your website and retrieving information, they're actually going to be taking actions on your website. And we've already seen evidence of this. We have clients reporting to us that, hey, a d, uh, an agent, an AI agent, booked a demo last week. We're seeing things like that happen.
And so then that just again. Websites have historically been optimized for humans, um, humans. If your website takes a little bit to load, or like your form, your demo form is confusing, it's okay. We can figure it out, right? We can, you know, use our intelligence, we can figure it out. It's not fun and it's gonna impact conversions, but we'll figure it out. Agents are they gonna wait around to figure it out? I think there's actually some standards that, um, and we're seeing some movement here from Google, um, and so I think that's gonna really impact how websites look and behave.
Um, and I think that's really exciting, cause I don't think anyone's looking there yet. I know Ben wants to research more in this area. I'm kind of holding him back a bit, cause I don't think it's sexy enough just yet. Um, but I I see that definitely as the next frontier. And you know, we have clients that are generating a lot of conversions, you know, tens of thousands of conversions per week. And a lot of their users are coming via a command line interface.
So they're saying, Hey, I wanna build X, and then the agent is the one going out there and coming back like a private procurement team. And and like, not just saying, Hey, I recommend these tools, they're saying, Hey, I installed these tools for you and they're good to go. I think that expands the surface area that we need to look at. So, yeah, I'd say those two, uh, is is what I'm thinking about.
awesome, yeah, Ben, anything to to add there?
Yeah, definitely, with, as Liam said, with the agents, I just think it's, um, moving from like a, I always describe it as like a. A read only environment to a read write environment is where we're headed. And as Liam said at the very start of this conversation, it's just a better deal. It's a better deal to get all the com, all the context in one place, just talk to an agent. It'll be an even better deal if you didn't have to fill out the form, it'll be an even better deal if you didn't have to take a slot on the calendar.
Yada, yada, yada, get the idea, um, and so that's for sure where we're heading. I think, um, you know, we've seen evidence of this. And if you look at the Google, uh, Web OCP program, um, like this is gonna pro, it'll probably s. It'll be very, very slow, and then overnight go boom. And that's that's what I expect to happen, just cause it, it's human, it's just human behavior. It's just easier, it's just easier. With like, lowest friction path, that's where we tend to, um, and then I would say, um, I, I.
I do think like to date, we've been very focused around, um, basically, like, okay, like, here's some query fan outs and here's the citations. How can I map them up? And we're kind of like playing this, this, uh, a little bit of a like most marketing teams are playing this game of like, here's my content. I'm I now rank for kind of thing, um, but uh, I think people are gonna start focusing more on like the weights and then what's driving the weights longer term.
Because the two, two reasons is that, uh, well, the the market in general is like, sophisticated, people are becoming more aware of how these systems work. The 2nd thing is that the training cycles are actually coming down. So when we 1st started, it was like, cool, we have like a new version of ChatGPT once every like 09:12 months, right? And now obviously these engines are getting trained like on a weekly and biweekly basis. At the top layers, uh, so that even like the top layers are being retrained and being tuned, uh, and then the backbones are being retrained.
Every, let's say, three months or so now, and so they are more influencable. And, uh, if you can get influencer imprinting, those tokens, it's worth m like orders of magnitude more than, um, than being in a in, in a, in a single context window. Having said that, you als ultimately want both, and it's not easy to get into that training data, but that's where I s I see, I see things heading in terms of otherwise. Outside of that, um, yeah, uh, offline is gonna be a big thing.
I think, uh, being able to cause. Essentially, what? J kind of a a kind of overall trend is that Google is having to. Or all these, all these um agents are having to avoid. Basically like dead internet theory, right? They're trying to avoid training on like, basically model collapse, training on their own systems. And so they're looking at all this content out there. And that's part of what the fingerprinting release is. It's part regulatory, it's part, it's probably a part IP play, it's part like multiple plays they have out there.
But it's, it's, it's ultimately, it's one of the reasons I also think they have is they don't wanna train on their own. So if you, if you post a Reddit with like a generated Claude comment, they will know that. They look up their hashing function. Very simple boom. The probability of this being AI generated from our model is very high. Um, um, and they just want they they, uh, they essentially that. I think that's gonna ratchet up and I think there's gonna be a lot of cleaning and basically all these like, uh, the spa, basically the the protection of human level context.
And actually, when OpenAI started out, they had this manifesto online to. They wanted to offload, uh, the verification of text to third parties like Trustpilot's a good example, they're in partnership with them. Um, they'll, they'll keep trying to do that, and I think they'll keep trying to up the ante here. So it'll get harder and harder to basically evade them. So you need to keep climbing the edges around that. Um, so that's where one other kind of like, let's say AI content, bot content, I guess, or bot behavior in general, is gonna become, uh, like, a human signal is gonna become, uh, very valuable there, but also emulating it will, too.
Makes sense? Yeah, well, I'm, uh, I. I was writing down furiously in the background a few things that that you all mentioned kind of throughout. Uh, I need to level up on my understanding. Things are just moving so fast. I really appreciate both of you joining us to, like with your expertise, really helping us and our listeners understand this topic. I would encourage everyone listening to go check out Discovered Labs in our show notes. We'll link some of the research that we talked about here, we'll link their research page, which is which is really great.
Um, thank you so much, Liam and Ben, for joining us. Looking forward to having you back on the show sometime when everything is different.
thank you so much, thanks for having us
Thanks guys.
Cheers bye bye.
All right that's our show for this week. If you haven't checked out our website head to practicalai.fm and be sure to connect with us on LinkedIn X or Bluesky. You'll see us posting insights related to the latest AI developments and we would love for you to join the conversation. Thanks to our prediction guard for providing operational support for the show. Check them out at prediction guard.com. Also thanks to breakmaster cylinder for the beats and to you for listening.
That's all for now but you'll hear from us again next week.