Transforming Search with Perplexity AI’s CTO Denis Yarats

You're listening to Gradient Dissent, a show about making machine learning work in the real world, and I'm your host, Lukas Biewald. Today I'm talking to Denis Yarats, who is the CTO of Perplexity, which is right now one of the most exciting and successful gen AI apps out there. So it was a really amazing opportunity to talk to him about how he thinks about product success, how he thinks about building high quality gen AI applications, and how he thinks about structuring his team and setting up his company for success.

I hope you enjoy this interview as much as I enjoyed doing it. The place to start. I mean, I'm actually a heavy perplexity user myself and is excited to to talk to you. But for those of you, those of those listening that don't know what perplexity is, um, would you mind describing it?

Uh, yeah, perplexity is a uh, first and foremost, an answer engine is is is the fastest way to get a uh answers to your questions, no matter how difficult they are. Uh, so we're trying to uh, combine essentially the advancements in uh, uh, in in search engines in uh, uh, large language modeling to deliver this technology. So two things we kind of like care about for simplicity is just like, how do we make uh our answers to be um high quality and helpful, and how can we deliver them fast?

So that's that's sort of like two things we are focusing on.

And so, like, you know, what parts do you do yourself and what parts are you using, like third party APIs?

Yeah, we started uh, uh, early on, cause we, you know, we didn't know like if if this thing even has a market fit, right? So we started heavily using uh, all the third parties, you know, we initially used Bing, uh, as a as a search engine, and then uh, also obviously open aI models kind of like, uh, hacked quickly something. And then, uh. Once we noticed that people actually like it, then we start investing heavily in infrastructure.

So at this point, I think a lot of stuff, uh, in house, uh, obviously, we, you know, rely on the frontier models, kind of like things like GPT four or like Opus, right? Those those are the best and very few people have access. We're obviously not in in a position right now to train those models. So that's uh, uh, that's uh, that's been get uh, getting from from uh, third parties. But for a lot of other, pretty much all the other models, uh, there and there.

Then there's like tons of models, all all kind of like different, like classifiers, embeddings, you name it, large language models as well. We heavily, uh, rely on open source, um, things like Llama three is is very useful for us. So we've built a a training pipeline, we've built a um inference, and so a lot of traffic is served on on the in-house models. We also, um, um, spend a lot of time and and continue investing in in the in the search engine.

Um, cause we for our service to sort of like work well, we we're not just um, we. We not only need like the uh, the relevant uh documents, but we actually need content and and good snippets. So we kind of like, have to build a content ranking system, which is, uh, um, you know, um, kind of like a different thing that, uh, classical search engines provides. So that's that's what we like invest in. We're in, yeah, obviously, all all the other infrastructure is in house.

So

So you've built your own, essentially classical search engine with like a crawler and indexer and and all of that.

yeah, yeah, this this is still work in progress, but I think, yeah, we, we, we've been investing this for like over a year now. Uh, it's uh. We have a a decent sized index, obviously, uh, not as uh as big as, uh, as we would like it to be right now, but that's work in progress. And we, um, kind of like, going right now after the head of the distribution, right? So we obviously not gonna do well on the on the tail, uh, but that's where actually LLM and like, powerful LLM simplifies things a lot, right?

So, um, that's where, you know, if you, if you don't, if you don't have a um, if you cannot find like relevant documents, maybe like LLM can step in, or or or. Or you can just like, say, okay, so like here, I I don't know like, what's the answer? So like, this is a possible explanation, or this is a possible answer. Like, maybe you can, it can ask, like, maybe you can, like, rephrase the question, stuff like that.

Um, and I think, like for us, as as I mentioned before, it's very important to not only it's kind of like the not only find the documents, but also, uh, scrape and parse them with a high quality. So cause we ultimately need the content. For for for like classical recommendations, for classical search system, you don't, you don't need, cause that you sort of like delegate this responsibility to the user. You just show a link and then user gonna click on the link and gonna read through the documents.

You actually, technically, even though you, you have to compute, you have to get some access to the the content in some form, in some way, uh, just like to compute features, embeddings and stuff like that. But like, this is not being used like, uh, for the user, but like, on our end, this is, uh, something that is very important. And ranking also is, um, um, is is quite a bit different just because we are prioritizing like different things, not, um, maybe optimized, very on like click probability of a click, because that's not what we need.

We're more more optimized on the if this content, if the if this web page was like, whatever snippets inside it has, is it is it gonna be helpful to answer the question? So this is a different target ranking target that we are, um, optimize our ranking model for. And and so that that's kind of like, I would say, like, the main difference between, like, classical search engines and what we're trying to do.

Do you find that? Um, you know, search engines have to combat things like SEO spam because it's so lucrative. Um, do you do you have the same issue?

Uh, yeah, that that's a good point. I think like, uh, I think yes, this is, this is is an issue. Uh, I think there's like two things, uh, going for us, right? So one is, uh, we are because our index is not as, uh, as as large yet. So we we kind of like we s. We spend a lot of time to build this like trust score for like Dom domain or or like a um, a web page.

And so we only sort of like prioritize in our ranking stuff that is kind of like, should be high quality, right? And then, um, LLM also then, um, plays a big part of it, because then the LLM can kind of like s detect if after like, you provide like certain additional signals that can, like, tell, okay, so this is. I would just like use this source because it's, uh, you know, has a higher reputation and, uh, likely gonna have like genuine content rather than the copycat.

So, uh, I think that those are sort of like two things that are we, um, invested into to kinda like fight SEO. But yeah, in, yeah, in. In general, I think like, what the the way people, uh, play SEO, right, is, um, they, uh, uh, try to figure out like the ranking signals that, like, let's say Google is using. Actually, there was like, a couple of weeks ago, there was a a a leak, right? And then so they they would, they would, uh, um, you know, build, uh, um, things to sort of like trick the ranking system.

So maybe it's gonna be like more clicks or or like whatever other, uh, other stuff. So, and because those things are not part of our ranking system, it's it's kind of like that content doesn't go up as as high as, uh, otherwise.

Uh, huh? And then on the LLm side, you, you know, at least with the pro version, you have a choice of model, don't you? Um, why do you do it like that? Like, why not kind of can, can. Couldn't you do, like some analytics to pick the best model? And and just always give you the best thing?

yeah, yeah, this is, uh, uh, I think this is, uh, that's where it's going. I think we're still like in very early stages and where users, um, kind of like, wanna have variety. So we're still like in the in in this, uh, stage where people know what is the difference between, like, GPT four and Claude. I think like, uh, uh, as we as AI sort of like, gets more, uh, uh, wide known and and people start like using product like this, I think the people not gonna care.

But like, this is actually one of a unique feature of our product. Let's say, comparing to just let's say like ChatGpt or or like Claude or Google, because we can offer different models under the same. And some people find actually very interesting people, the the uh, the distribution between like which models people select, uh, is, uh, quite uniform. So this is, uh, um, encouraging, but yes, you're completely right, you know, as as we um, go, uh, into the future, I think, yeah, definitely.

You will have um, a system that's gonna decide, you know, uh, how much compute this query require? And then based on that, you're gonna route it to like, appropriate model. So it's, it's, it's essentially where all of this is going. It's kind of like, uh, it's, yeah, everything is gonna be like, proportional to the compute. Yeah, it's kind of like, okay, so so this query is like, very simple, you know, you can run a simple model.

If it's like something more complicated, requires a lot of reasoning, maybe requires like multiple steps, and there's gonna be more compute involved. Uh, so yeah, this routing is gonna be done.

And uh, what do you mean by everyone? The the distribution of models people select is uniform, you mean, like everybody picks the same model.

Oh, no, it it means like there is a there's a good spread, so like a lot of spread, so there's, uh, uh. It means like, you know, if you have like five options, I think it's kind of like, uh, fairly uniform, rather than like everybody picks the one model. So it's like some people, like, let's say, like thirty I see, percent people using like GPT four, like 30% using Opus and and stuff like that. So so kind of like people like diversity, um, which I think makes a lot of sense.

But uh, uh, yeah, I think at some point, you know those things, as long as you provide like a cohesive product, uh, that is useful. I think people probably not gonna pay much attention to to those details.

um. And I think we found an interview from about six months ago on the new stack, where, um, your CEO was quoted as saying, you know, over the next quarter, we're gonna move everybody completely to perplexity models. Uh, uh, has that happened, or is that still in the works?

Uh, yeah, I think like a lot of traffic, uh, I'm guessing, I don't know, at least like 50% already served in house, uh, and like, yeah, on different models, um, and and as I said, like, oh, oh, uh, this is just the uh, right now, like, we're talking about the main model that, like, generates the final answer. But like, there's a lot of stuff happens before this and all of those models in house. The only, um, as I said, yeah, the the only, uh, sort of like this, like frontier models that are the the most capable models right now.

So kind of like, obviously don't have those in house, uh, and some people, you know, for like, extra reasoning, some people require those. So so that's why we have that as an option. But um, you can, as a as a, as a, as a pro user in perplexity, you can select that there are models or, um, uh, all kinds of other models from different providers.

And and the models you made, um, the are these the same as the or the same genre as the open source models that you've released?

It's, uh, uh, they're like post trained, um, so for example, we take like Llama three, um, like seventy B, that's sort of like the most capable open source model at the moment. And uh, cause the way we use models, they sort of like have to perform like a specific, uh, task, which is kind of like, you know, getting search results and then you're trying to, uh, generate an answer. So those models then go through like additional, uh, uh, stages of post training.

Um, where we teach those models, okay, so like, how do you understand the distribution of the search results that we're providing you, like the snippets? Then, uh, we teach the those models, okay, so like, you don't. If you, if you don't have an answer, if you, if you don't have like a grounding, or like coverage in the in the search results. Like, you don't wanna like hallucinate, instead of like, you wanna like, refuse, um, you know, the.

And there's like all kinds of like, different like use cases that are, uh, product specific. So you know, like, what? What are the weirder things that we're like trying to address? Like, I remember, like, Earl, early version of perplexity, where we put like a a common name of a person. And it would just, uh, find like different people and like, mash this up into like a Frankenstein. So that so we kind of like, try to teach these models too, okay, so this is, uh, information, kind of like maybe contradicts each other, so likely not.

That's not the same person, but different people. And then so just like when you provide an answer, like, try to, um, uh, you know, like, provide description for each person individually, rather than just like combining together. So things like that, you know, like, sometimes we wanna see if it's, uh, um, for like certain queries, you know, you can. You can answer like very shortly. Let's say you ask, like, what is the Nvidia like market cap, right?

Yeah, you can basically just answer in one, in one sentence. But I think it's actually, we find it's it's more useful when you can provide like additional nuance. Even like users maybe didn't have this intent because it's it's actually useful. So those kind of uh things we have like a different vectors and uh, criterias, uh, what, uh, what is a good answer, how how's a good answer should look like? And sort of like, yeah, we optimize for those metrics.

So I would imagine you have one of the biggest embeddings databases in the world, given what you're doing. Is that something custom that you built, or did you take an open source project, or how does that work?

Yeah, yeah, so we, uh, yeah, we have, uh, uh, a lot of data, and this is, uh, I think, uh, this is, uh, actually helps a lot. Uh, I think, uh, because once you have this data, you can do like you can train like all kinds of models. We, uh, uh, started with the uh, sort of like pre trained, like BERT models, right? And then, yeah, on top of that, we like, completely, um, train them on for whatever, uh, we have, like, did all kinds of like embeddings, uh, for like different stuff.

And uh, because, yeah, they they've been using in in ranking, uh, they've been, they've been using like, uh, content ranking, um, snippet generation, like stuff like that, uh, lots of things. And uh, yeah, those are fully based on, um, our in internal distribution of of tasks. But uh, and and same goes for the um, for the uh, old LLMs that we post trained, right? So like, we, we, we know exactly what people are asking. We have a very diverse, uh uh, distribution of prompts, so we have a.

I think every day we might, I don't know, get getting like ten, ten, 10 million different prompts. So we just, uh, need to make sure we utilize them. And and then, and so that's, we like, see which prompts like, uh, underperform. So we take those prompts, like, make a better answers. And and that goes into training. And and the same for true for like all, all kinds of uh, uh, different models, like all, all different, like classifiers, all different, like, uh, auxiliary models, embeddings, uh, ranking and stuff.

Wait, so when you say you have 10 million different prompts, these are user queries, or is this Yeah, the some kind of internal

yeah, yeah, this is, this is, this is user queries, right? So like people, I think right right now, like, yeah, we're roughly getting like 10 million different queries. Sometimes, obviously, some of them like, not useful, so we we don't use all of them. Some people opt out from training, right? So like people don't want it. So but like for for those, uh, you know, we kind of like, clean, clean, clean those up, um, get a amount that we need.

It's it's it's. Actually, it's not about sort of, uh, the quantity, the sheer quantity of the prompts, but it's more like about entropy and like how much information they carry. Um, so that's what are you optimizing?

meaning? Like, how unusual the prompt is, or

Like how, how, how diverse they are, like, uh, how much, uh, uh, sort of like, uh, things they cover. Um, you know, you can, you can, you can get like, uh, you know, a a lot of prompts, but they're gonna be all like the same. Maybe they're like asking about the same stuff. So just like, so that's not super useful.

hm, and how do you think about the quality of the results?

Yeah, so we, um, I've spent a lot of time on this kind of, uh, build, uh, um, internal sort of like taxonomy and and sort of like guidelines. What is a good answer? What is a good answer is? And then we sort of like, you know, right now employing um, we have a internal, uh, team of annotators, uh, we call them like, uh, Lm teachers. We also work with some third party vendors for data. We also heavily employ like LMs to, um, to, to aid humans to verify the answers.

And so, yeah, so basically, the idea is just you are, um, you know, getting to answers and you say, okay, which one is better? Uh, and then, uh, when you decide which one is better, you kind of like, um, utilize those guidelines that we have. So, and then eventually this s signal is gonna is propagating to training. And then, uh, you know, like, we get reward model, and then from the reward model, you train a model.

That is, uh, kind of like, adhered to tho those guidelines. And and the most important thing is sort of like, establish data flywheel here, so you can, like, constantly can take advantage of of new data and and retrain things. So that, uh, yeah, this takes like a lot of time to get right, and I th. I don't think like we fully yet there, but we made a lot of progress.

And and to hear you're retraining the LLMs, I guess, Yeah, yeah, it's or or are you going like, deeper into your? Would you? R potentially like, redo embeddings? Or, yeah, everything, um,

everything basically re being retrained. So like, it's not like, we, you know, train it once and then keep it. Cause like things are changing, we get more data, we get more high quality data. Um, and yeah, this is, uh, like, constantly trying to retrain things.

where do you think? Like, you know, I'm thinking in your sort of like flow, there's sort of like, could be trouble at any point. Like, you gotta find the documents, you gotta return the documents, you gotta like, kind of understand and and summarize the documents to the user. Like, where do you feel like the most opportunity is for improvement? Like, where? Where's the typical, um, error in your system these days? If if there is an error,

yeah, yeah, of course, there is in there. I don't think by no means we, um, nailed it. Uh, honestly, I would say like each and every piece contributes a lot. I think like ranking is a big, big thing. Um, then the uh, how big is the index and how? how many documents you have in your index, right? So like, sometimes, uh, actually, cause like ranking and and the index size kind of like proportional things. So you know, if you have a bigger index, it's, you know, your ranking can be slightly worse, and you're still gonna get a decent result.

And and vice versa. The the quality of the scrape is is very important, you know, we spend like a lot of time to make, ensure, uh, you know, there's like lots of sites right now where you have to, uh, we have to be like custom parsers. Cause like, maybe they have like some um, um, particular like things that, uh, if you, if you don't optimize for those, then then you're not gonna get like the most ideal scrape.

Um, embeddings for the snippets, very important, like, how do you exactly like chunky, uh, like chunk the uh, web documents? Then the obviously LLM itself, like, um, how do you? Yeah, make sure you do, don't, don't like hallucinate, how do you make sure that you don't cite irrelevant results? Um, I think each piece is, uh, uh, contributes to the overall error. So that's why it's very important to sort of like, have an eyes over each individual component and like, be able to debug things quickly and like, understand what's going on.

And sort of like evals, yeah, for y for each, for each step, you need to have like, a um, um, very um, sort of like comprehensive um evaluation. So like, whatever it is,

I mean, so there's clearly like a latency and quality trade off here, isn't isn't there? So, so, from like, a user perspective, how do you think about that balance? Like, do you lock the quality and try to get the latency as low as possible, or lock the latency and try to get the quality up? Like, how do you actually, uh, deal with that systematically?

yeah, so we, we kind of like, yeah, I think like latency is, uh, yes, it, it's exactly right. There is always trade off. Um, you know, more compute you spend, likely you're gonna have a better answers, but it's gonna be ex more expensive and slower. I think we, we, we right now on our product, we have like this two modes. One is just default search, the other one is like, pro search, so and and and that's where, you know, for default search, I think we're like, okay, so try and optimize this is you get like very good answers, very fast.

So, uh, you know, good for like most of the uh searches that you have, most of the questions you have. On on the other hand, yes, when you need like a more complicated stuff, maybe, and and that that's actually, you know, um, usually quite like a good, a good proxy here. Like, just like the length of the prompt, right? The the length of the query, like, if it's something like very few words, um, then, you know, likely people expect very fast answer, right?

So it's kind of like Google, right? Um, but if you if somebody spends, uh, a lot of time typing something in, right? So then maybe they wouldn't. Would prefer to wait a little bit longer, but get a a much better answer, right? Because they already, let's say, like, uh, spend a a minute typing that that prompt. So that's that's a kind of like, very rough, like, uh, class classifier of, uh, kind of like, how much, uh, compute and latency budget you have?

Um, so those things C can be used, but, uh, in general, you know, uh. We're trying to get to a point where the system is gonna be smart enough and it it can like, depending on the complexity of the query, it can like allocate, uh, the compute. And then the the way, the way you optimize it, it's kind of like to to the to users like it, right? So to to users find it helpful or not.

So yeah, maybe there is a always trade off. Um, you know, there, yeah, there is like trade off. Because so they probably don't wanna wait, like they can wait a little bit longer, as long as they don't have to retype the question and and make it better. And they can get an answer. So if if they in order for for them to get answer to the same question on Google, they maybe like, have to type ten different queries and like, analyze stuff.

Um, so that's still gonna be much slower.

Do you experiment with your own internal prompting of your own LLM, or is the need for that gone because you're doing so much, um, post training? And

Yeah, we, I, I mean it, it depends, I think, like the it's still you wanna, you want your model to be quite general. So, um, I think that that that's where I think the diversity of the prompts still, uh, plays important for, uh, role. Um, but you know, for the yeah, when you post train, you kind of like, you don't have to. Um, essentially like, provide like this, um, let's say like complicated system message, cause it's already like, uh, being injected as as part of, like, post training.

that makes sense. What do you think are like the perfect, like, perplex, perplexity use cases, where you really think you'll be better than, um, openai and Google?

Hm, yeah, I think the something where, um, you can get, uh, a unique experience for like, okay, maybe not, not, not on all the types of queries, maybe not as general as Google. But like on on some sort of like verticals, uh, and and and sort of like use cases where you can just, uh, um, outperform them, just like provide like, a better experience. Yeah, you know, so far, we've seen a lot of this is, uh, research, kind of like research type of questions, uh, things where people try to get a unbiased opinion about something.

So kind of like a lot of, uh, a lot of people who use us as like knowledge worker, like professionals, uh. And the the interesting thing is just like those users are, um, you know, a a lot, a lot of our users use perplexity for work, right? So and that that that's very important because this is, uh, something that that's a useful tool. And, you know, and be, and because of that, I think it's also like, has a high potential of being, uh, monetizable.

Um, you know, obviously, we're not gonna be like as good as like when Google says, like, like in like, trying to NBA score, right? So it's like Google, like, nailed it, it's it's just like perfect already. Um, but uh, yeah, there's like certain things where maybe you can answer like much more complicated questions that require, like, you know, multi step reasoning and stuff like that. Um, and and optimized system a little bit different. But also like the way you, uh, position your product and like the tools that you have in your product and the features that you have in the product.

That's is gonna just be like, tailored to a bit better for your work. Um, than, let's say, like Google or stuff like that. So that that that's kind of, you know, um, probably not gonna be like, yeah, yeah, I don't know. I think maybe it'll gonna be like, super general, but I think there's gonna be, uh, opportunity to make it, uh, useful in in some very significant fraction of, uh, use cases.

I in kind of controversial domains, or like domains that are really kind of salient to people. Where, um, you know, the conventional wisdom, if you poll people, would be different than kind of the objectively true answer. Do you aspire to kind of find the true answer and and give that to your users? And and how Yeah, I think, do you? How would you go about that?

yeah, I think, no, no, we don't. Yeah, we are actually wanna be like unbiased. And I think this is something that's gonna be super important, you know, even like this year a alone, like later in the year, uh, but going forward as well. So we're trying to,

Why is that?

uh, you know, yeah, I just, I mean, it's pretty obvious, right? Uh, but uh, I think, yeah, it's. It's kind of like, would be important to provide and sort of like to train your models in a way where it's, uh, can provide like different points of view, uh, and so not not be biased like towards like particular uh, direction. Uh, I think that that's actually, honestly, that's gonna be like the hardest problem, probably to solve, and and make make sure.

So we, we, we've. We've been thinking a lot about this, so from from like, very beginning, that's why, like, one of the, uh, key features. Like, even though it's like, very simple, but like citations and sort of like, um, making sure that people can also, if needed, then ver- verify sources. It's it's very important rather than just, uh, giving them an answer and like, um, be done with it, right? So so those things, uh, yeah, it's gonna be very important.

Do you have to handle? Especially a case where you get totally disagreeing answers? Like, I feel like, usually when I, when I use it, um, it kind of pulls the general consensus for me. But I could imagine situations where you just get completely different answers from different sources. How how do you handle that?

Yeah, sometimes, sometimes it happens. Um, I think, basically, um, yeah, there's like two things, right? So there's like the diversity in ranking. When we sort of like, r um, when you run content, you can kind of like, uh, make sure that there's like, uh, some degree of diversity in there. Um, instead of just like, you know, you maybe have like one piece of information. And then it's kind of like a lot of like documents, um, uh, side, this inform, or like, carry this information, uh, and but.

But it's exactly the same in terms in in amount of like, uh, um, entropy in there. But then there's like some other results, which is kind of maybe rank less, uh, favorably, but like, this is a alternative, like point of view. So you want to that? And that's where I think like LLM important, because they, um, they can understand, okay, so there is this like much content, but it actually, uh, conceptually, it carries only this like piece of information.

But then there's like this another piece of content that, you know, maybe like, less, uh, uh, is not ranked as high, uh, but has like, uh, also valid information, just like different point of view. So, like, LLM can do a lot of, uh, uh, work there. And then finally, you have to. When you pr generate, like, synthesize an answer, you, yeah, just have to tr train your model in a way that it's can, like, um, cover, uh, voice, voice, voice, points, points of view.

So so you, you, you, essentially, you know, to summarize, kind of, yeah, have to retrieve a lot of stuff. But uh, kind of like when you retrieve, maximize the the amount of information, not amount of, not amount of like characters or tokens, the amount of information.

When you think about hiring Lm teachers, do you do you try to, like, get a diverse set of people? Like, you know, sometimes I I used to wonder with Google search results, you know, when they would hire the annotators, like Mo, mostly in Silicon Valley. I kind of wondered if that was influencing, um, the Google search results. Do you do you worry about that?

Uh, yeah, I think, yeah, I think it's very important to, you know, have a first of all, I th. I think it's important for them to have like different skill set. You know, some people may be like, good at, uh, writing, or like, uh, uh, journalism, or or, or like economy or whatever. Some people may be like, good at coding, like more technical. So I think this is, uh, one, uh, very important dimension. But also, yeah, uh, you know, we're trying to hire from like different states, uh, from, uh, across the country.

Not only, um, we, we, we kind of like remote friendly in general. I think like we started company, uh, you know, um, as a as a remote company. And you know, like, yes, we have an office in in San Francisco right now. Like most of the people are, but like, we still have like, a um, a a small office in New York and like a few people working remotely. So I think it makes it a little bit easier to hire like a diverse, uh, group of people.

Uh, and but yeah, I think it's a a very important point.

And and what do you like? How do you respond to, like, the criticism that people won't like, go to the websites anymore, and people won't really be like, incentivized to make high quality websites?

Um, yeah, I think this is, uh, um, I think there has to be probably something where you have to work together with the content creators. And and sort of like, figure out a way where you know, like, like one thing, for example, yes, maybe people go like less frequently, uh, from perplexity than they do from Google. But like when they go, they have like much higher, uh, intent, right? So like in Google, you get a link, you just go there and like, you bounce back.

Because it was just like you quickly find out. It's like, not relevant information. So here, um, you know, you when you click, you like, for sure, are very interested in your document. So it's kind of like, again, trading, uh, the the the number of clicks, but it ins, and with those like more, more like, uh, important clicks, kind of like, like that. So it's it's not just about the count, but it's, um, uh, a a different measure.

And I think, like, on on that measure, yes, I think it's. I don't think like it's as skewed as, as, uh, as it might seem. Just because, you know, uh, yeah, you, you get get a higher chance. Like somebody goes to your website and just like, reads the whole thing rather than from Google. But so that that's one thing I think I would wanna like point out. So um, but the other thing I think, yes, it's.

It's gonna be interesting to figure out, um, what is the beneficial way for both of the parties? Cause, yeah, I think like content, uh, generation is very important. Um, and I think, you know, if people stop producing it, it's, um, then, yeah, probably not good for anybody.

Okay, and I had a totally different note. Um, Twitter tells me you have kind of a interesting story about pitching Yan Lacoon. Would you mind sharing it?

Oh, yeah, yeah, I mean, it was, um, you know, so we like, um, initially, you know, we, we. We wanted to do like search engine, right? But then, uh, when we would go to like, VCs and they would say, oh, like, you know, you guys crazy, cause there's a Google. It's it's just, uh, not not gonna work. And so we initially, um, okay, so we decided, okay, so we're gonna still be search. But in order for us to get money, we'll just do something.

And like, the one of the first thing we realized, okay, so let's Twitter is, you know, very important thing, and like a lot of people like Twitter. And we were just okay. So can we re and I? And like, it was also well known that Twitter search sucks, like, uh, used to, at least like, used to suck, right? And we were like, okay, is there like anything to do there? And like, was the one? So we, uh, spent a a a month like scraping a lot of, uh, content, uh, from Twitter.

And like, we build a, uh, a big database of, you know, uh, of users and tweets and and like connections between them. And and then kind of like, build in this like natural language interface on top. And uh, that turns out worked very well. And then we was like, okay, so, um, trying to like, pg different people. And then we like, went. I wanted to cause I, I was, uh, um, you know, getting my PhD at NYU.

So I was like, on the um, knew Yan before, and we like, Irem and I went to New York. Uh, try to, uh, see if Yan is gonna be interested in, like, give us a check, uh, we waited, uh, and we kind of like, created this like strategy. Okay, so like Yan really likes, um, Twitter, and one of his nemesis is, uh, Gary Marcus, so cause they always like, have the the disputes. And we were like, okay, so uh.

We came up with like a sum of some of the interesting queries about, you know, like things like, oh, you know, um, how many, um, people engage with Yan tweets? Instead of like, uh, Gary's tweets? And it actually turns out like, much better. And he was like, very happy about it, so and he's he's like, got really curious. And yeah, this, um, help us to, you know, uh, get him on board.

Well, wasn't there something about you were like he was? You were waiting for him for a really long

Oh, yeah, yeah, so there

time?

is like, there is a, uh, I I think it was like, a little bit like, exaggerated, uh, maybe. yeah. I think we waited, uh, maybe like for like an hour or so. Uh, cause I think he has like some meeting or something like that, I forgot. But yeah, I don't think it was like, uh, five hours or so,

That's it.

because, you know, it's, uh, uh, makes a good story, though.

Hm, so you, you, you've built, I think, one of the most successful gen AI applications. And you know, people ask me for advice, uh, you know, every day about how to build a good gen AI application, which is kind of a vague question. But I I feel like I should pass it along to you because you would be in much more of a place to respond to that than me. Like, what do you tell people that come to and ask you, how do I?

how do I build a gen AI application? What are the best practices?

Um, well, I think the the the thing that really worked well for us, and I think it was, uh, without it, I I don't think we were, um, we would be where we are is, um, is speed. Um, I think we, basically, and it was like, very tough, uh, it, it, it is still very hard. You have to be, it has to be your life, it it's not a job, uh, but yeah, you have to just do things much faster than everybody else do.

And like, you have to figure out, like, what is the most important? And and try to get the prototypes very quick. And like, see if, if, if there is, they have any feet or not. And then not get discouraged if if they don't work, and just like, try, try next idea. We like, for the first three months, we like been trying. I don't know, like tons of ideas, like, maybe like ten different things. And, uh, we were just like, uh, very quick at like, you know, like, literally, if if we had like an idea, we like, would implement it in like two days.

And then we just, uh, have a prototype and then people try. Obviously, technology was at the at the level where it became possible, primarily like LLM, cause like, was LLM just like spzy up so much, uh, but but I think this is the most important thing, just, uh, relentless execution. Um, cause if you especially wanna compete against the big companies, they have like all the resources in the world, so you just you have no chance otherwise.

Um, but also, you know, uh, uh, as always, there is a selection bias, right? So there's like certain things that, uh, went our way, mainly like chatGpt was a a huge, I think, like for for the whole industry, right? So that was, but we were like, prepared for that, right? So we, we started the company, you know, in in July. And we've been like, kind of like, hoping that things like this is gonna happen. And we, uh, sort of like, anticipating them.

And then they should like, build infrastructure and like, uh, product for that. And then when, when it came out, we were, um, sort of like, in a perfect position to take advantage of it. So

What were the early signs that you saw that let you know it was working? That were different than the things that that weren't working?

yeah, so I mean, it was like, kind of like, and this is the the main reason when we we decided to start a company. It was just the certain things that you would try to do before, and it would take you so much time. And then now you can, like, take GPT, I think at that level, it was like GPT, davinci three or something, Davinci two, code, Davinci, and you would like in a day you can do like something decent, which, like before, was probably like impossible.

And then around, like, um, I guess October, we had the the first prototype of Perplexity, where it was just like a Slack bot and Discord bot, and it's it's been working well. We kind of like, use this internally, it was cool. But then I remember, uh, distinctively, when, uh, that was like a few weeks prior of Chatgpt, when Davinci three came out. And we like, literally replaced the model, um, without like any other code changes, and it was like, significantly.

You can see right away just things like work so much better, like, L, L, like, literally within like thirty minutes. We s we like, noticed this big difference and I was just like, Okay, so this is a uh, there is something here, and um, yeah, so it was very important at that time. And I'm, I'm sure, like the same. It's kind of like, identify an opportunity early on and just, uh, be curious about, be curious about, like, trying different things and like, see where is the current like inefficiency, and see if the the.

And I think like, one of the important thing, and and maybe we got like a lot of, uh, maybe, uh, uh, heat for it. Initially, was like, and people still like, call us like a broker company, whatever, but but I think that was like the the most important decision we've made. I feel like, you know, the the only we realized early on is just like, okay. So if you wanna build something like perplexity, let's say like five years ago, you would have to initially, you know, collect data, train a model, uh, build a product, get the users, and only see like, if there's market fit, right?

It takes a lot of time and then you you realize there's like no market fit. So what GPT like, um, GPT technology and like LLMs in general, like, allow you to do is sort of like, flip this process from from, you know, um, backwards. And you just like, very quickly can prototype product and see if there's market fit. And if there is market fit, then you just, uh, build all the o other blocks, uh, behind. So you just do you.

Because, like, the most important thing that you get is the uh, uh, idea that if there is market fit or not. And then data like data flywheel, and then once you have data, it's just, uh, it's just a matter of execution. And so, yeah, that was a key, I feel like,

And then, I guess my my last question is, well, what has been the hardest parts about scaling up your application?

honestly, the hardest part was like hiring good people. Like, if you, if you get good engineers, I think they'll just like, figure out like these days, infrastructure is, uh, at such a high level. I think if you just have some decent understanding of, uh, what needs to be done, it's very hard to, um, you know, fail, uh, but, uh, you know, I think it was also like, very important to sort of like, identify what, what we care about.

And then just like, always make sure that we, um, optimize for those things. And and for us, uh, uh, as I mentioned earlier, there was like two things speed and accuracy. So we kind of like, we decided, okay, so there's like two m. A lot of things we can do. Uh, you know, uh, let's let's just like, stay focused. So these are two things that are at at the core cause. Like, if we fail on those two things, like, the rest doesn't matter.

And so we're like, okay, so this is two things. Let's make sure we do well here. And like, let's build all of our tools, sort of our evals, all of our infrastructure, to make sure that those things, uh, um, we we can do very well. And that, you know, um, then it's into like, kind of like, part of your culture, then like the the people you bring, they're like, see that the founders, uh, early in place, cares about those things, then they start care about this.

And then, you know, you wouldn't see like now if you somebody pushing like a feature and like, it drastically, uh, reduces, like, increases latency, right? So people not gonna do this because they're gonna like, uh, test for this specifically. And then they and they have gonna have, like all the tools that, um, that are gonna help, uh, those people to like measure the effect of of whatever feature they have. So I think the those things are very important, just basically what, what's what your values are?

Awesome? Well, thank you so much for your time, really appreciate it.

Cool, thank you, thank you for having me, and thanks for the great questions.

Thanks, thanks so much for listening to this episode of Gradient Dissent. Please stay tuned for future episodes.