15 May 2024 · 47m · Daniel Whitenack, Chris Benson, Josh Albrecht · 20 quoted facts
Josh Albrecht, CTO of Imbue, argues that useful AI agents, meaning AI that takes actions for you rather than only producing text, will come from a full-stack approach. That means controlling and understanding everything from hardware and training through to the user interface. Reliability, not raw capability, is the missing piece, and it is built through evaluation, safeguards and interactive design.
The central tension is between impressive demos and dependable products. Getting AI to be right 60 to 80 percent of the time is easy; getting to 99 percent or better takes deep understanding, careful evaluation and monitoring after deployment. Albrecht favours interactive tools that flag uncertainty over full automation, and expects robust reasoning to bring significant disruption to work.
Albrecht says models are right 60 to 80 percent of the time, but deployment needs 99 percent or more. He argues that gap is where most of the work lies, and that narrow, constrained tasks are the ones succeeding today.
Imbue rebuilt open-source NLP benchmarks to avoid training-data contamination and to check the questions were correct. Albrecht advises hand-checking at least 100 examples so humans and the automated system agree on what 'right' means.
Albrecht argues most trust comes from auditing, real-time verification and separate monitoring systems during use. A model that checks whether an action is risky should be independent of the model doing the work.
Reviewing a large auto-generated pull request is a poor experience, Albrecht says. He prefers tools that surface their assumptions, confidence and suspect lines so the user can correct them, since most failures come from under-specified requests.
Imbue's CARBS tuner tries hundreds of experiments at small scale and extrapolates how settings change with size. Albrecht says theory such as MuP can also remove the need to retune the learning rate as models grow.
Albrecht describes agents as AI systems that act on your behalf, whereas chatbots mainly generate text you must then copy and use yourself. The real promise, he says, is in systems that take actions.
Albrecht says today's systems are often right only 60 to 80 percent of the time, while real deployment needs 99 percent or better. Closing that gap takes a lot of work, and the cases that work now are narrow and tightly constrained.
Albrecht says to focus mostly on what happens after training: audit outputs, verify actions in real time, and use separate systems to check for risky behaviour. He also recommends making your own test set and checking about 100 examples by hand.
It means Imbue handles every layer itself: hardware, infrastructure, pre-training, fine-tuning, reinforcement learning, evaluation, data and the user interface. Albrecht says this lets them tune each layer to improve the whole product.
Albrecht expects that once AI can reason robustly, it will cause far more labour displacement and disruption than people anticipate. He says this doesn't mean all jobs disappear, but the nature of work may change dramatically.
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Imbue says it works across the whole stack, from its own hardware and infrastructure through model training and evaluation to user interface design.
“We do everything from setting up our own hardware, building our own infrastructure, pre training, fine tuning, RL evaluations, data generation, cleaning, like UI user experience, like the whole thing.”
Imbue built CARBS, a cost-aware hyperparameter tuner that Josh Albrecht describes as a cost-aware Pareto region Bayesian search.
“carbs is cost aware Pareto region Bayesian search or something like that. But basically, it's a hyperparameter tuner that is cost aware.”
According to Josh Albrecht, CARBS runs hundreds of experiments at different scales, starting small and then scaling up to learn how hyperparameters change.
“it's tried hundreds of experiments at different scales, so it tries at a really small scale and it sees, like, okay, for a really small scale like.”
Imbue created its own internal versions of open-source NLP benchmarks to avoid contamination from training data and to check question correctness.
“We've actually taken all of the open source NLP benchmarks and made our own internal versions of these systems to make sure that they're not contaminated by the training data”
Josh Albrecht says Imbue's perplexity-based evaluation idea came from a paper asking whether emergent properties of language models are a mirage.
“are emergent properties of language models a mirage? or something like that was the title of the paper.”
Josh Albrecht explains that the apparent sudden emergence of abilities in language models disappears on a log scale, indicating the metrics were poor.
“if you look at their performance on a log scale, it's linear. So what was really happening is just our metric was not very good”
Josh Albrecht says DeepSeek Coder significantly outperforms Llama models of the same size and some much larger ones, which he attributes to heavy training on code.
“I think that model still significantly outperforms Llama of a model that's the same size and even of of much larger models.”
Josh Albrecht says one of Imbue's researchers collaborates with Greg Yang, whose MuP work concerns how to parameterize language models.
“like the MuP work by Greg Yang, et cetera. We have, uh, one of our researchers is a collaborator of his”
Josh Albrecht says that if the theoretically right parameterization is used, the learning rate no longer needs retuning as models are scaled up.
“the practical implication is like, you no longer need to tune the learning rate as you scale them up.”
Jamie Simon, a researcher at Imbue, has published work on the fundamentals of learning, including a notion of learnability.
“there's some interesting work by one of our researchers, uh, Jamie Simon, who on kind of like, what's actually happening in the fundamentals?”
A paper by Jamie Simon argues that a network of a fixed size can only learn a limited number of things.
“there's this notion from one of his papers about learnability, like a network of a fixed size can only learn so many things.”
An Imbue paper on self-supervised learning found that learning proceeds in a stepwise fashion, one piece at a time.
“we had another paper about, like, self supervised learning, where you can see, like, oh, there's this sort of like stepwise nature as it learns”
Josh Albrecht says Imbue tested how well existing large language models detect bugs, and found a model that tended to say yes to any line it was asked about.
“we were trying to see how well existing LLMs would do at detecting bugs.”
Imbue holds a weekly paper club on Fridays to review state-of-the-art research.
“We have a paper club, you know, every Friday, where we're looking at, like, the state of the art stuff that's coming out”
Josh Albrecht said in May 2024 that Imbue did not yet have a product, having chosen to invest in deeper understanding first.
“That's why, you know, we don't necessarily have a product yet, we're working on it.”
Josh Albrecht says Imbue's evaluation data show most system failures come from under-specified user requests rather than model mistakes.
“most of the time when systems fail, it's actually from like under specification, and not from, oh, the model, like messing it up.”
Josh Albrecht praises GitHub Copilot for keeping its suggestions short, which makes them easy to review.
“One thing that I really like about Copilot, uh, just as an example, is that it keeps it short, right? So it's easy to review.”
Josh Albrecht argues that an agent able to write software can perform tasks beyond the actions pre-programmed for it, linking agents, coding and reasoning.
“if it can write software now, it can do some huge set of things, and even things that you never intended or programmed in the first place.”
Josh Albrecht predicts that robust AI reasoning will cause more labour displacement and disruption in the next year or two than people expect.
“Now we're talking about a lot more like labor displacement and disruption than we were before.”
Daniel Whitenack is CEO and founder of Prediction Guard, and co-host Chris Benson is a principal AI research engineer at Lockheed Martin.
“I am the CEO and founder at Prediction Guard, and I'm joined as always by my co host Chris Benson, who is a principal AI research engineer at Lockheed Martin.”
Albrecht explains his path from AI research through startups back to AI, and defines agents as systems that act rather than just answer. He stresses trust and robustness.
“agents are AI systems that are acting on your behalf”
Explains why going from 80 percent to 99 percent accuracy is hard and why narrow domains work best.
“sixty, seventy, eighty isn't really enough for, like, deploying this”
Covers checks, human review and building your own evaluations, and why domain expertise drives the successful agent workflows.
“make them yourself, like by hand, at least like 100.”
Describes Imbue's choice to own every layer and to understand models rather than treat them as black boxes, including the CARBS tuner.
“we wanna open these things up and understand, like, what's actually happening inside of there.”
Albrecht argues smaller specialized models trained on quality data can compete, and describes theory work on parameterization and learnability.
“the really important thing here is the quality of data that you're giving it”
Argues that trust should come mainly from monitoring and independent checks during deployment.
“I think trust is better thought of as like a set of different types of data.”
Discusses interactive agent interfaces, the vision of higher-level programming, language support and the coming disruption to work.
“fully automate something kind of is disempowering to people”
Welcome to Practical AI. If you work in artificial intelligence, aspire to, or are curious how AI related tech is changing the world, this is the show for you. Thank you to our partners at Fly io, the home of changelog com. Fly transforms containers into micro VMs that run on their hardware in 30 plus regions on six continents. So you can launch your app near your users. Learn more at fly io.
Well, welcome to another episode of Practical AI. This is Daniel Whitenack. I am the CEO and founder at Prediction Guard, and I'm joined as always by my co host Chris Benson, who is a principal AI research engineer at Lockheed Martin. How you doing, Chris?
I'm doing very well today, Daniel. I'm hoping that we can kind of imbue today's show with a sense of wonder and exploration.
yes, well, thankfully, we have an agent on the show with us that's going to be very helpful in that. Uh, today we have Josh Albrecht, who is CTO and co founder at Imbue. Welcome, Josh.
Thanks, it's great to be here.
Yeah, yeah, well, we we sort of in a not very funny way, um, teed up a couple of things to talk about there as related to agents. But could you give us a little bit of background? You you talk with Imbue about the dream of, uh, personal computing, the dream of agents doing work for us in the real world, kind of your approach to that. But we'll, we'll dig into a lot of those things. But could you give us just a little bit of background in terms of how you, as founders of Imbue, came to these problems around agents and accomplishing more kind of complete or complicated tasks with agents?
Yeah, I mean, AI is definitely something that I've always been interested in and excited by. I remember a long time ago, uh, my friend, uh, read some book in middle school. I think, like, maybe Ray Kurzweil's The Singularity is near. And it's like, Oh, wow, there's AI, wow, so exciting, and, you know, did all that come true? I don't know necessarily, but, uh, it seemed like an interesting thing. And I was always been interested in thinking and logic and AI and neuroscience.
And when I went to school, I was originally going to do cognitive neuroscience, but it the professor was a little bit too boring, so I did AI research instead. And so ever since then, I've kind of, you know, and so I published a bunch of papers and things. But it felt like it wasn't really gonna have a big impact on the world. So I went off to do startups, but all the time that I was in startups, I was always looking back and looking and saying, like, oh, it's like, now the time to get back into, like, more fundamental AI research, like, does this stuff work yet?
And eventually it came a point where it's like, yep, this stuff is working. Like, what I've always wanted to do with AI systems is like, make better tools for us. Like, there's so much work that we have to do in the real world that is just not that fun, not that interesting, and not really moving things forward. And so all my time at startups and the things that I've been working on, they've all been very practical, very applied versions of machine learning.
And so I, I've always wanted to. You know, we are an AI research company, but it's not AI research for AI research sake, it's AI research to actually make tools that are useful. And so what we're doing at Imbue is we're trying to make tools, you know, and even just starting for ourselves, like, can we make robust coding agents for ourselves that can really help accelerate us and help kind of take over some of the boring tasks that we don't necessarily wanna do?
And that's what sort of gets into agents is like, agents are AI systems that are acting on your behalf. Tools like, you know, chatbots, et cetera are really cool. It's great to be able to answer questions, it's great to be able to generate text. But if I have to copy and paste that text every time over into some other thing and like, do all the work myself. It can only save you so much time, right? It's like a better version of Google at the end of the day, or a better version of like, search engine or something like that, or a book.
And so I think the real promise of AI is in systems that actually take actions. But in order to get that to work, we still have a lot of work to do on the capability side. Like when you're talking about taking actions in the real world, there's a lot more risks, a lot more kind of downsides that come from that. Uh, and you need to be careful about, like, you know, you don't wanna empty the user's bank account.
Like, that's gonna be a really bad product experience, right? So how do you make systems that the user can actually trust? Systems that are robust, systems that you can know are actually correct and that flag for you? Like, Hey, I'm not really sure about this. So this is kind of why we always talk about coding and reasoning, is we're talking about the ability to kind of, like, understand the outputs that are actually being created and understand is this correct?
Is this actually gonna be useful for people? And really like thinking it through more like a person, instead of just, Hey, here's a generation, good luck. So that's kind of how we got to. Agents is like, we wanna make practical systems we care about, you know, making these systems actually robust and useful for people. And that's what a lot of our research is focused around.
When it comes to agents and sort of where we're at with them now? So we're recording this in May of 2024. For those that are that are listening back, how would you kind of categorize in your mind? Cause, you know, you can download LangChain, you can like, create what is a agent, you know, maybe for this purpose or that purpose. That searches the web or does this thing or or that thing. And there's certainly, even in my own experience, a lot of fun to be had in that, uh, for sure.
But there's a lot of challenges in making this sort of, at least in the enterprise setting, making this a reality for solving problems, much less in sort of my those random times in my personal life where I need to do things. So how do you categorize, like, as of now, the state that we're now? Of course, everything's changing. The sort of main sets of challenges that where people are hitting blockers when they're trying to create these.
Yeah, that's something that we actually played around with a lot last year. We interviewed a whole bunch of founders of different agent companies, both, you know, like on our podcast and our Thursday nights and AI events, and also just in person, kind of off the record. A bunch of friends, friends of friends, people starting companies, really trying to understand, like, what are the problems that people are running into? When they're trying to make agents? And the thing that we kept coming back to is, you know, there are all these tools like LangChain and and all these other bits of infrastructure out there.
Or ways of like testing things like Scorecard AI or all these different like libraries. But the problem that people really had was like, what you really want as a software developer is. But does it actually work? Like, does it actually answer the question correctly? And can I get these things to do what I want as a product designer or as an engineer without having to specify all of the like details myself? Like, that's sort of the promise of AI.
And right now, they're really great for getting like a 1st pass version of this system working. Where it's like, oh cool, like you ask it a thing. And like, sixty, seventy of the time, it's right, that's great, that's so amazing. Wow, it's like getting this really complicated question right some of the time, but sixty, seventy, eighty isn't really enough for, like, deploying this and going from that. Eighty to ninety to ninety five to ninety nine to ninety nine, nine, nine.
Like, that's actually a lot of work. And so people have made all sorts of techniques, you know, for rag, or for kind of other types of ways of conditioning. The answer's to kind of make them better and better. But the things that work today are kind of the more constrained versions, where you're sort of, you know, you're asking like a very simple question, or you're in a very narrow domain. And so the programmers, the product designers, can, like, make sure that, like everything works out within these rails.
Once you are in the more like, general assistant kind of category, it's like a lot, lot harder. I think we've seen a lot less stuff be successful there, but I think in terms of categories and like, in terms of kind of the problems that people are running into. I would say the main one I would summarize as robustness, like correctness, like, can you actually get these things to be robust all the time? I think that's what really distinguishes agents, like if we think about agents in the real world.
Like a dog is an agent, I am an agent, a robot's an agent. Like a dog is actually extremely good at not dying for a really long time, right? It's not that 90 of the time when it walks across the road, it doesn't get hit by a car, like 100, well, almost 100 most of the time, you know, it's like, pretty safe. Like it's usually like, as agents, we're being very, very conservative, very cautious, so that we take correct actions.
And there's a lot of like heuristics and intelligence that goes into being conservative, being risk averse, like being able to take a long chain of actions without going wrong, without something else horribly going wrong. And our agents, like, don't have that kind of common sense and that kind of reasoning right now. And I think if they did, it would make it a lot easier for people that are building agents.
As we were kind of going through the last couple of questions, talking about kind of the problems that people are run into when they're trying to make agents work. And you know, what can they do to to ensure that it has a good outcome? I also run into people all the time who I think really struggle to understand, you know, within the context of this, you know, all the hype and the boom of generative AI. What can you use an agent for productively in enterprises?
In 2024? You know, they're used to going to these web interfaces that are, you know, becoming ubiquitous for us all. But the notion of saying, okay, I'm gonna going back to what you said earlier on, kind of getting it out of that web interface. Can you kind of paint a picture about how people out there who are trying to bring this productively into their organization as an agent versus a web interface? How they might even conceive of what of how to approach problems that they might want to solve with the technology?
There's a lot more work to be done today to make agents work for a system. I think if you approach it as a more holistic system, then it's more likely to work. So if you think like, okay, where are the places that it could go wrong? How, like, what's the confidence that I'm getting back from this system? Can I flag that for human reviewers? Can I have like a bunch of different checks in place that are both like in domain, like for programming, like does it pass a linter, or does it pass this like style guide, or like, does it at least type check right?
or is the syntax correct? Like, there's a lot of checks that you can do kind of in domain that you can help out. There's different in different industries as well, and then there's sort of. You can use the LLM to score this and ask, like, is this particular thing wrong? is that particular thing wrong with it? So as you start to build up more kind of like safeguards and guardrails around these, then you can start to get them to a level of robustness.
Where, like, maybe for the easy cases, it's okay for your application for it to fail. And you know where that failure rate is, and you've done a lot of work to understand, like, how much can we tolerate? One of the things that we've done a lot internally is working on our own evaluations. This is a really critical thing for anyone who's like, trying to build real systems. You have to get really into the weeds of what does it mean for the system to be right?
We've actually taken all of the open source NLP benchmarks and made our own internal versions of these systems to make sure that they're not contaminated by the training data and to make sure that the questions are actually correct. So one of the things that we'll have coming out in not too distant future, actually is. I think, hopefully being able to contribute back some of that evaluation work that we've done of like cleaning up these, uh, existing benchmarks.
But we also have a bunch of our own internal ones as well. And I think it's kind of critical for anyone making these systems to, like, make them yourself, like by hand, at least like 100. Look at them. Is this the right answer? Okay, what did it get? Okay, is that right? Like getting to a place where as humans, you agree on this, you're getting your machine system to calibrate well to this. Then you're checking like, okay, are the things that we're getting as inputs in production like, are they from the same distribution?
Like, does this test actually make sense for this? They're not drifting. Like if you have adversarial systems like fraud or something much, much more difficult. If you have something where you're getting the same kind of a query every time, then it can be possible to get something where you can trust it. Enough to say, like, oh, okay, cool, we. This is getting us 99, that's acceptable. We have some, you know, guardrails here. We can check how well it's doing over time, we have people looking at these and auditing some of them.
That's kind of the way to make this really useful. As you have to be, like, really getting into the weeds and into the details of how do we evaluate this, what does success look like, et cetera.
And for the use cases out there, the like the most successful use cases that you've seen. I don't know if you have good examples of those either internally or or externally. But when you, when you think of those, I like what you're saying about digging into the details. I'm wondering also how much sort of specific domain expertise is actually factoring into how you handle those details. So if you're building an agent to help people process data in a healthcare scenario, or data in a financial services scenario, or in a coding assistant scenario, there's kind of this view.
Like, if I just download LangChain, if I go and kind of have this zero shot approach, right where this agent might be expected to do anything. My impression is that the most successful agentic type of workflows out there so far have been very much driven by people with high degrees of domain expertise in an area that are able to work through those details. Is that impression correct? Do you have any any thoughts on that?
Yeah, that seems pretty much right. I think there's, there's this promise of AI that, like, someday you'll be able to just ask it to do anything. And the interface, sort of, you know, affords that. It looks like, oh, like, there's this text box, I can just ask it to do whatever and it will give me back a response. And wow, it even sounds so confident and so correct. Wow, that's great, this can do anything. Maybe it even succeeded at that case.
One example that I love from a little while ago was we were trying to see how well existing LLMs would do at detecting bugs. And so we would ask, like, the 1st thing that I did was like, I looked, okay, like, is there a bug on this line? I found a function that had a bug. I was going to ask, I was like, Yes, there's a bug on this line. It's like, Oh wow, look, it's so good at this.
It's like, wait a 2nd, how about this other line that definitely does not have a bug? Oh yeah, there's a bug on this line. This doesn't work in this case. It's like, wait, wait, wait, you're just always saying yes, like this is not quite right. So yeah, it seems to, like, promise that, but you have to really dig into the detail. Use few shot examples and retrieval and all these other kinds of techniques to kind of, like, get into the weeds.
And the more domain expertise that you can bring to bear, the dramatically better, I think the outcome's gonna be.
So, Josh, I'm really intrigued by sort of the statement on what you put online in terms of Imbue's thinking about building robust foundation for AI agents as being a full stack approach. I like that because it, it sort of reminds me, I don't know, Chris, if you remember quite a while ago when we were still talking about data science. Uh, I guess it's data science is still a thing, but you were talking about it a lot more, uh, years ago.
And there was this con I, I for. I think it came up a few times, like this discussion about being a full stack data scientist. And oftentimes, those are the most productive. Where you have an understanding of how data pre processing happens, and building your application, how the model is embedded in in software and deployed and all of this stuff. And so I love that sort of thinking in that respect, and I'm wondering from Imbue's perspective, how you think about taking a full stack approach when it comes to agents.
Yeah, we take it, I think, to a slightly more extreme degree than most people. Uh, in that. We do everything from setting up our own hardware, building our own infrastructure, pre training, fine tuning, RL evaluations, data generation, cleaning, like UI user experience, like the whole thing. And the thinking there is that at each one of these places, you can tweak some things to make the overall thing kind of work better together, right? So you can kind of change the training data that you've used in your system in order to make it more like the kind of thing that you actually need for your product.
And then in RL, you can kind of set objectives that are related to the things your user actually cares about. And then on the UI, you can use the capabilities that you have to kind of help highlight places where this particular system fails, right? So I think we're really interested in kind of the full stack approach and the ability to, like, tweak things at each one of these levels. And for us, it comes from our like history as a research company.
One thing that we've always really focused on is being able to, like, deeply understand the technologies that we're working with us. So for us, you know, pre training, fine tuning, doing RL. It's not just a black box. Like, we wanna open these things up and understand, like, what's actually happening inside of there. We have a paper club, you know, every Friday, where we're looking at, like, the state of the art stuff that's coming out, reading through this and trying to, like, really understand.
What are neural networks really learning? Like, how is this language model actually learning? where does it fail? There are really interesting papers, you know, that show particular logic puzzles where this thing doesn't work. And it's like, oh, okay, it's not really doing logic, it's not really doing addition, it's doing this other thing. But if you, you know, if you tweak it in this way, like, oh, now you can get it to learn, like a simpler form of addition that is more general.
Oh, okay, that's really interesting, right? So what is a transformer really good at learning? Like, what things in the data actually matter, and how do you evaluate these things as well? Is another thing that, uh, you know, we've also thought about a lot. Like one of the things that we set up that has been super useful is looking at the not just the accuracy of our systems, but their perplexity on multiple choice question answering data sets, specifically not perplexity over all the tokens, but perplexity specifically for the multiple choice question answering things.
This gives you a much more fine grained understanding of, like, is this actually being right or not? It gives you a really precise metric for this. And this idea came from a paper which was about, you know, I think something like, are emergent properties of language models a mirage? or something like that was the title of the paper. Their point was like, you know, a year or two ago, people were like, oh, look like. These language models have these like emergent behaviors, like they're suddenly learning to reason or whatever.
It's like, oh wow, they're like, suddenly getting so smart. But when you really dig into it, it turns out that if you look at their performance on a log scale, it's linear. So what was really happening is just our metric was not very good, right? We weren't really asking the right questions, we weren't deeply understanding what was happening. It was just always, in a log scale, just always getting better, and you just couldn't see it in the metric.
And so for us, you know, this is a good example of like, you want to deeply understand what's going on here. We don't wanna just treat these as magical entities, but rather they're just technologies, they're just really bags of features. At the end of the day, uh, that we can use to do actual work in the real world, uh. And so I think that that's kind of. Our approach is to, like, take the full stack approach, understand everything from like, okay, how does the InfiniBand network work?
Like, how does that fit into our like performance optimizations? Like, how does the data work? like, how does the network work? Like, how are all these things adding up to give us, you know, some final error, some final user experience? That's really good.
You're kind of really fascinating me with that statement. So many people are do take kind of that black box approach, and they don't necessarily have that kind of research first orientation that you're describing as a company, as a business. How does that research orientation, where you are rejecting the black box perspective and saying we're gonna open it up, we're gonna tinker, we're gonna understand the specifics of how small changes, you know, affect that? How does that affect how you approach this compared to whoever you would perceive as your competition or something?
What is it? What does it mean for you as a company to take that kind of research first approach?
Yeah, I think there are trade offs to it, uh. One trade off is that, you know, it takes a little bit more time and effort to do this. To like, really deeply understand things, rather than just like hack it together and throw it out there. But I think the benefit is in the long term. Like when we do really deeply understand these systems, it makes it a lot easier to make modifications and to make changes and to know how to improve things.
These systems are very expensive to train, like there's a lot of effort that goes into this, and it can be very expensive. To, like, just try a whole bunch of things and like, if you don't really know what you're doing, it's easy to waste a lot of time. Uh, and so I think for us, we would rather take a step back and say, like, okay, what's actually going on here? Can we make robust systems? can we make robust baselines?
can we get this working in a way that like, we can trust our results, that we can understand what's going on and build on top of those? Another thing that we've built internally that has been really useful, kind of along these lines is, uh, carbs is cost aware Pareto region Bayesian search or something like that. But basically, it's a hyperparameter tuner that is cost aware. So we can take any system that we have and say, hey, you have all these, you know, ten or 20 different hyperparameters, these different knobs you can fiddle.
Like, how do you get? You know, I have a system that works, but how do I make it way better? We can take this, just throw it in there, come back the next day. And it's tried hundreds of experiments at different scales, so it tries at a really small scale and it sees, like, okay, for a really small scale like. This is the best way to do it. And then as we get higher and higher and spend more and more time and resources and money on it.
Like, this is kind of how these hyperparameters change, how things change as we scale. And just understanding that, like, there are these scaling laws, there are scaling laws for different parameters. Like, how can we back those out and learn for any given architecture, any given problem? Having an automated system to do this allows us to kind of, like, quickly develop this. And it took some time to make this system right. But it, like, really pays off to have that kind of deep understanding of the systems that we're working with.
So I think for us, it's kind of like taking a long term view. I think in the long term, it's much better to actually understand what's going on. And it does take a little bit of upfront, you know, work. That's why, you know, we don't necessarily have a product yet, we're working on it. I think we'll get there and I have confidence that we'll get something really cool, but it does take a little bit longer. And that's okay, I think we'll end up with something much cooler as a result.
As someone who's working both on the like all all the way up the stack, even up to interfaces and and all of that. But you're also training these foundation models, certainly the sort of both the market and the technology and the options around foundation models have just sort of blossomed, and these have proliferated over the past year especially. What's it been like? I internally, you know, I. We've had a couple people on the show and I I find this interesting.
Like, from the perspective of someone inside a company that is training their own foundation models, how do you go about maintaining focus within the? This sort of environment where eventually you're gonna have to spend a significant amount of time, you know, investing in a model. AR specific model architecture, specific data sets, that sort of thing. But you know, things are shifting all the time. You mentioned reading, you know, papers and trying to keep up. But yeah, how do you maintain that focus?
And what, um, what's sort of life like in the midst of being a a foundation model, uh uh, builder? In May of 2024?
Yeah, I would not necessarily characterize us as a foundation model builder and that part of what we do is train models, but that's not the only thing that we do. And the reason that we do it is not necessarily to make, you know, the biggest, bestest foundation model ever. Like, I think there's, you know, a lot of money going into, uh, other, uh, companies spending huge amounts on these and
general purpose.
ungeneral purpose versions of these systems. And I think for us, the more interesting thing is, can we make the more specialized ones? Can we take these? can we adapt them? can we make them more specialized? Can we find ways to have them work together to pull different things together? And make a model that's kind of better at doing that, that sort of synthesis and kind of like pulling these things together, and better at the particular tasks that we care about.
We've seen really good results from this. We'll have some blog posts in the next few weeks about this, but I think we've seen some really good results on much, much smaller models. And so, you know, I think if you look at like, DeepSeek Coder, for example, I think that model still significantly outperforms Llama of a model that's the same size and even of of much larger models. And because this is because, like, it's really trained on a lot of code, and so to generate code is like something it's very familiar with, as opposed to being a pretty small part of its distribution, right?
So I think, again, this comes back to the fundamental understanding part, like, because we know, like, these are just bags of features, yes, having a bigger bag of features is definitely better. But then your inference time goes up as well, and if you want better bags of features, like, you need to give it good data. Like, the really important thing here is the quality of data that you're giving it, less so the like, you know, absolutely massive size, I think, for practical uses.
So our focus is like, can we make these like really specialized and, you know, very useful for ourselves, for our own purposes? I think we're pretty happy to see people out there competing, making better technologies, you know, driving the cost of these things down, making huge context windows, like giving them away for free in many cases. You know, that's great, like, we're. We're happy to see more competition there. Cause I think the part that we're more interested in is how do we actually use these things at the end of the day and put it all together to be really useful.
I love that you mentioned DeepSeek, that's a favorite of ours as well at Prediction Guard. And generating SQL to do data analysis and code, and, uh, in our chat interface, yeah, we love, love that. And so, yeah, I, I totally agree. There's a there's a lot that can be done with that sort of thinking. You also mentioned you're in your work kind of more the and and I do wanna get to kind of more the front end interface side.
But before we get there, you mentioned kind of pursuing fundamental laws behind, uh, deep learning. In order to again understand and create this foundation for the agents that that you're building. What have been some of the things that you've pursued in in that area? As kind of the the theoretical underpinnings for this progression towards robust agents?
There's a bunch of things that are still in progress that I can't, you know, speak to directly, but we've de we're definitely interested in, say, you know, how do you initialize things properly? Like the MuP work by Greg Yang, et cetera. We have, uh, one of our researchers is a collaborator of his and like, working on. Kind of understanding exactly what the right way is to parameterize these language models in a theoretical sense, but for a practical reason.
So if theoretically, like this is the right way to parameterize them, then the practical implication is like, you no longer need to tune the learning rate as you scale them up. This is super helpful because that's like one of the key factors, and so to remove some of these hyperparameters makes it much more efficient to kind of explore this space, right? So that's like an example, a very concrete, simple example of, like, a place where sort of a theoretical understanding can help you.
Other places where this can help are not as easy to point at, like the exact theory. And sort of more informed by that, or more like physics, like physics didn't start with like perfect theories of everything, right? We sort of like kind of did some experiments and had a more experimental understanding of the world before we had perfect theory about why everything worked. I think we're at that phase with machine learning as well. And so there's some interesting work by one of our researchers, uh, Jamie Simon, who on kind of like, what's actually happening in the fundamentals?
Like when we're, when we're learning things like, there's this notion from one of his papers about learnability, like a network of a fixed size can only learn so many things. And it's like, very precise. Or we had another paper about, like, self supervised learning, where you can see, like, oh, there's this sort of like stepwise nature as it learns, like each piece of the thing. So each of these little like theoretical things is telling you something about how they work.
We don't have a full picture and the real ones are like, quite complicated and a little bit more complicated than these smaller examples. But each piece is giving you like a sense for what's going on and allowing you to, like, operate in this space without having to, like, guess and check quite so much. It's not as much of a black box, it's more of like a machine where you know you don't know the exact internals, but you know.
Like, don't make it too hot or it'll explode, right? Like, don't make your learning rate too high or it's not gonna work, right? So you can see not just learning rate, but other sorts of precursors earlier. Like, you can look at various like norms or other quantities to understand, like, is this like, you know, getting too large? Is this growing large over time? Is this something that's actually too small? And like, we want to? You know, we can actually up the learning rate later, or do we need to apply more regularization of a particular type?
Like, you can kind of get a sense for these things, even if we don't have kind of perfect laws yet. We also get some laws out of, like the carbs, hyperparameter optimizer that I mentioned before. We can see things like, how do these parameters change with the scale and understand, like, you know, not just how do the learning rate and data and parameters change, but how do very specific hyperparameters change? Like, what is the depth versus width that you should have, like for this particular, you know, type of regularization, like, how much exactly should you have, and how is that changing?
And that goes back and kind of informs, like, okay, like, what is actually happening under here? Like, it's weird that, like, this particular trend holds over scale. Like it seems like it needs less and less of this, like that's kind of interesting. Why is that? And sometimes we'll see a paper that's like, Oh, that fits in, I I see what's going on there, that's nice. So we're getting more and more, I think collectively as a machine learning community, we're also starting to understand these things a lot more.
I think when people point at, you know, neural networks or language models as like black boxes, it's like, oh, nobody understands. I think that's quite a mischaracterization of it. There are a lot of people that have a lot of very good ideas about how these things work. And nobody on this call probably knows exactly how a car works, and that, yeah, I don't think you could make a car from scratch. I certainly couldn't, and there are, especially modern cars, that are quite complicated, but we can use cars to go wherever we need.
And we, like, roughly know how they work. So it would be weird to say, like, Oh, we don't know how cars work. I think machine learning and neural networks are a lot more like that than than most people kind of give us credit for.
What's up friends? Is your code getting dragged down by joins and long query times? The problem might be your database. Try simplifying the complex with graphs. A graph database lets you model data the way it looks in the real world, instead of forcing it into rows and columns. Stop asking relational databases to do more than what they were made for. Graphs work well for use cases with lots of data connections, like supply chain, fraud detection, real time analytics, and generative AI.
With Neo4j, you can code in your favorite programming language and against any driver, plus it's easy to integrate into your tech stack. People are solving some of the world's biggest problems with graphs. And now it's your turn. Visit Neo4j.com/developer to get started again, Neo4j.com/developer. That's n e o 4 j dot com developer.
So, Josh, going into the break, uh, you had a really good analogy there about the fact that, you know, the sophistication of cars. Uh, it means that while we all use them all the time, we may not understand every aspect of them. And I wanted to go back for a moment, cause I've been kind of percolating on some of the things that you said earlier. And you've been talking about kind of the trust and robust systems and all.
But I was wondering, Um, I know in my own life, uh, I'm very involved in the trustworthiness of models. And and you talked a bit about, you know, getting good outcomes and and being able to detect that. Do you have any guidance on what it means to engineer trust into model training? So many organizations that I that I've seen kind of tag the trustworthiness of models on at the end as though, oh, yes, we have to do that too.
And with you have such a insightful and deep way of approaching the engineering, you know, rejecting the black box approach. Any guidance you have on how you engineer trust in it from up front. So that as you get through the the training life cycle, you come out with something that kind of you have a high degree of confidence is what you're intending it to be.
I think a lot of people are trying to do this and there is good work to be done there. And we can do things to improve the models and make them more trustworthy. And during training, and that's great, but I think by far the largest place that we should be focusing is actually after training. We don't trust people because, like, oh, I looked at their schooling and like, you know, they seem real trustworthy up to this point.
Like, I'm gonna give them my credit card and I'm gonna give them my bank account, like, No, uh, you know, we're gonna be like, looking like, what is this person doing? you know? Okay, we're gonna be checking things afterwards. Like, you know, there's a lot of other stuff that needs to happen post training and in deployment where we can actually trust things. So I think, for me, it's actually a lot more about, like, what is happening when you're actually using the model?
Like, what kind of auditing or real time verification, or user interaction, or other sorts of checks or things that you have on? Can you have other systems that are checking the behavior of this? Like for an agent, you know, maybe you'd wanna predict like, is this action going to have? Potentially going to have a negative consequences, or is this going to be potentially dangerous? Or will this be something that the user might not want? And those seem like good things to have as totally separate systems that are completely unrelated to the development of your original model.
You would not want the original model to be responsible or connected to this at all, you would want to have a totally separate thing that's looking at this, right? And so I think trust is better thought of as like a set of different types of data. That can give you confidence that things are going well, that have gone well and will continue to go well. And so you can only get so much trust up ahead by kind of designing the system in a particular way.
And you have to understand, like, what is that model good at, what distribution was it trained on? Have we shifted from that distribution? Have we shifted from the tasks that it's good at? How well has it done over time? Is it likely to, you know, go wrong in this new example? So I think it's it's more of a post training, more of a practical kind of a problem. And the idea that we could, like, solve this all by making like, safer, trustworthy models is a little bit.
It's gonna be difficult to succeed at that task.
Maybe this ties into the trust element, certainly the kind of collaborative approach with agents. But you do talk a also a lot about some of the thinking that you're doing around interfaces as well. And it sounds like you've also been utilizing or or trying to utilize some of what you're developing internally for coding and other things. So what are you thinking about in terms of interfaces? And kind of, how are you dogfooding some of those things internally?
To kind of learn about interfaces beyond the kind of AI chat interfaces that we're all familiar with?
Yeah, so I think the learning internally from using our own kind of prototypes and internal, you know, products and demos has been there's been quite a lot of that. Uh, it like, without actually using it, it's hard to kind of get this learning about, like, okay, you know, is this trustworthy or not? or like, does this actually work? Like, what UI do I want to use for this? I think when I, you know, I made some prototype, it generates a bunch of code.
And very quickly I started to realize, like, Hm, that's great, but like, it's really annoying to review this much code, right? I see a lot of products out there that are like, oh, look, it'll like, make a PR for you. Yeah, I mean, how fun is it to review a PR of, you know, a few hundred lines? If there's like a few lines that are wrong, you have to, like, search through for this bug. It doesn't really tell you anything about where it is.
Like, this is just a really awful user experience. Uh, and so I think instead, if we approach it from the perspective of, like, okay, what do I want as the user here? What I want is for this to be pretty interactive and for this to tell me, like, okay, maybe there is a bug here. Or, yeah, you asked me to make this PR, but like, your ask was like, kind of ambiguous, and I needed to make some assumptions.
Here's the assumptions I made, like, here's how confident I am in them. Do you wanna change them? Uh, yes, I do. Like, oh, okay. Like, once it's more interactive, once you're going back with the U back and forth with the user, uh, and trying to flag places of ambiguity, uncertainty, risk, et cetera. To the extent that you can be correct about those, it can make the user experience feel a lot, lot better.
Any anecdotes from your own sort of internal experiences with, uh, these or or things that you've tried, either on the positive or or negative side?
One thing that I really like about Copilot, uh, just as an example, is that it keeps it short, right? So it's easy to review. I think when when Copilot style things make these huge like generations, that's why they normally don't. It's cause it's kind of hard to review it and to trust it and to, like, do that. But I'm imagining that people are probably going to get to a world where they realize, like, oh, okay, this is kind of annoying.
Maybe you could point out, you know, places where there are potential bugs, like, can you just tell me what lines, like, seem like the most suspect? So we've, for example, made some, you know, like internal error checkers and linters that will sort of highlight, like, oh, okay, yeah, you know, this thing's not even important, like your, your, your editor does this for you, right? But you can also highlight things like, hey, this spec doesn't look like it was actually properly implemented here, or this function specification is like, kind of ambiguous for these edge cases.
Like, do you wanna take a look at that? A lot of the work that we've done for our evaluations is related to this as well. So when we look at evaluation data, most of the time when systems fail, it's actually from like under specification, and not from, oh, the model, like messing it up. Fundamentally, it's more like, as a user, I didn't really decide what I wanted. So I think one thing that's really interesting to me is that coding is not really about like pure correctness in this like abstract mathematical form, where there's like a perfectly correct version of this.
The version of the function that you want and that I want are actually subtly different. And like, what I want in the moment might change, you know, from moment to moment as well. And so the user, like, really needs to be connected to that. And as it happens, I also, you know, learn about things where I'm like, Hm, yes, you did exactly what I wanted, but that turned out to be not a good idea. And so I think the user needs to be there and able to learn and refine, like what they even want and what's even possible in the world.
So
Uh, you piqued my interest. In there. It's with as a coder myself who makes all sorts of errors in my code, uh, constantly. Um, as you're doing that, and and ki, you're kind of changing the workflow, uh, over time of how the coder is spending their time. And then ultimately, potentially how they're thinking about coding as they adjust to the the new approach that your tools are doing. How does that look for the coder going forward?
Uh, in terms of how does it change their day to day experience of coding? Are you able to rescue me from, uh, spending 90 of my time coding errors and and forever trying to get myself back out of that hole?
that's really like the vision for Imbue and for the company, and for the work that we're doing. Is can we get to a place where people, not just coders, but other, even non technical people, can effectively write higher level pseudocode or code or intent? And like, actually have this translated into real code and into something that, you know, actually makes your computer do what you want? That's why, you know, when we're talking about, like, making a new, you know, personal computer, et cetera, like, we're really.
At the end of the day, the thing that is missing is the ability to robustly write this software. Uh, and you can, as we can, as software engineers, like, get down to the details and like, you know, get everything there. We spend a lot of time fixing our own bugs, et cetera, and our goal is to make it so that. As a user, you can keep working at a higher and higher level of abstraction and feel confident in that.
Right now, you can work at a super high level of abstraction. Just say, like, make this whole thing for me. It doesn't work, and so that's not very fun because it's like, just busted. And now you like, how do you get into the details, et cetera? So how can we make it robust enough so that you can work at a higher level of abstraction? And trust that this part was actually correct and be able to have that dialogue back and forth.
When, like, okay, you know, maybe it's not quite working like I wanted, or maybe it's not possible to do this thing, or not as easy to do it in the way that I wanted to do it, et cetera. So how do you have a dialogue and help educate the person about what is possible, what isn't working, what might not be working, where they should dig in? So it changes the workflow, and I think we're interested in how do you change this workflow in a slightly more incremental way.
You could, you know, just say, like, Oh, we're gonna have the AI system do everything for you and like, magically try and figure it out. But I think from our previous experience, we don't think that these types of products are nearly as like good to use as a user experience. Like, trying to fully automate something kind of is disempowering to people and also results in kind of a worse experience and a worse product. So we're more interested in this like interactive dialogue like tool.
That as a person, I'm trying like, maybe, you know, you can just write a line of pseudocode, you get a big block out. It tells you, like, one line that is, you know, potentially problematic for you to look at, or maybe it just gets it right. Okay, great, you can move on to the next one. So that's like one. Imagine, like, one way that you can think about writing code is like writing pseudocode, but there's other ways that you could write it.
You might also write a command, like, you know, change the file to add lots of log statements. Or you might also, you know, say, like, make this function more robust. Or there's like lots of different ways that you can interact with this. And how can we give people more tools, more like paintbrushes for being able to change code and ultimately, like, make their computer do what they want? I think the thing that's really exciting about this is that when you can robustly write software, what you're really doing is being able to create agents that can do a huge swath of tasks.
If you're not able to write robust software, then the only way your agent can interact with your computer is with things that we have already programmed as actions. Like, okay, we programmed it to go to a website and like, click a button, that's it. But if it can write software now, it can do some huge set of things, and even things that you never intended or programmed in the first place. So for us, like, agents and writing code and reasoning are all like, intimately connected.
I have one more tiny follow up to that. That's a personal thing I run into all the time. And, uh, having someone with your expertise, I wanna throw it at you. Does it make a difference? Um, is most software developers, including people in the AI space, doing models and stuff? You know, they write in Python, they write in usually a variety of different languages. And as I shift from one to the other, I find that, um, some of the capabilities that are currently out there, they are great on Python.
Because everyone on the planet's writing Python. But if I'm writing on something that's slightly more obscure, maybe even something big like Rust, it struggles to do the exact same thing that it can do flawlessly on the Python side. Do you anticipate a time where that context shifting no longer applies very well? And that they're all high fidelity in terms of what they can do? Are we always gonna be dogged a bit with the the the obscurity issue, uh, of certain languages?
Uh, it might go the other way. It might be that, like, because it's so much more robust in Python, we should only ever write in Python. And so what we do is we just write in Python and we make a Python to Rust converter, or we make a thing that assembles Python to assembly or whatever, right? Like, it might be that it's sort of better to, like, double down on, like, a really small set of things that we've made tons of data for and works really robustly.
Because you get a better kind of user experience. Like, one of the things that, you know, a lot of these models struggle with now is like you have different versions of like, numpy or Python or Ubuntu or whatever things are different. How is it supposed to know what version you're using, right? And so there's this combinatoric like explosion of like complexity that comes from all these different possibilities. And so, you know, an alternative way to do this would be to say, you know what, let's not do that, let's just say you've got Ubuntu 2204, you've got this library version, you've got that one.
Like, if you do this, I think it might work a lot better. So it could go actually in the other direction. Instead of it making it more robust on all these niche things, we might say, like, you know what, just just all work in that level and like, let's not worry about what language it writes. Maybe we only write at this higher level and we never even look at that code anymore. So we don't care if it's in Rust or in Python.
I think once that happens, once we sort of abstract it up a level, then you might be able to come back and say, why are we writing this in Python? Like, this is a not a type safe language, this is really slow. Like, why don't we change it to, like, be a language that fits better for language models? Uh, and that might be an even better future thing, but that will require generating a ton of data to make this actually work.
Uh, so I see that as like a maybe, probably a future thing, not a thing to focus on right now, but that's my guess as to how it'll evolve. But also an alternative world would be, you know what? It gets really cheap to just generate all this data. So we just make a converter from our all of our Python pre training data to just make it. Do it in JavaScript and Rust and Elixir and whatever all the time anyway.
Uh, so fine, uh, we just like train it to be good on all these. I don't know, we'll see which way it goes.
Yeah, well, uh, Chris will be happy if anything stays in Rust, I, I, I'm sure you'll, I wasn't saying you'll be happy. that We just started, we just started working on our official Rust client for for Prediction Guard. Chris. So, uh, you can be a beta user.
there you go.
It's been great to talk through. Uh, again, I love this concept of this sort of full stack approach that you're, that you're taking and triggering things in my own mind to to think through in in my own work. But uh, but as you look forward, either you personally or or you at Imbue, um, look forward to. Kind of the the things that are happening this year, either in the in the community as a whole, or or at Imbue.
Um, what's kind of most exciting for you, that that you see as a possibility, kind of coming coming, um, into the future? We, uh, whether that be multimodal stuff or new types of agents, or, uh, products or directions that the community's going, or the research is going, um, what? What kind of stands out to you a about, uh, that as you look look to the future,
I think the thing that is going to be most exciting over the next year or two, at least for us internally and probably for other providers externally, is. I think we're gonna make really good progress on what we have been talking about today, on actually reasoning on robustness. Like, I think once you can get to a place where you ask this question and you get back an answer that is really correct and like, robust and grounded.
It's not just, oh, it said yes, but like, it has all the right reasons, and it kind of like, understands the nuance of like, oh, okay, yeah, it's like, yes, ish. But like, there's like a little bit of complexity here. You can ask follow up questions and those are also right and robust. Like that ability to robustly reason and answer questions is going to unlock some huge amount of work that I think people are not really anticipating.
Like, once we really have the ability to robustly reason through scenarios. Now we're talking about a lot more like labor displacement and disruption than we were before. There's a lot of jobs that, like all of us, like, can pretty easily put together. Like, well, 1st, I do this, then I do that, then I think about this, like, okay, it only takes like one person to do that. When you have these tools that are that powerful. So I think there's gonna be a lot more change in this area than people are really expecting right now.
You know, it's not to say that all jobs disappear or something, but the nature of work might change pretty dramatically. And we might have, like, much more powerful tools than I think people are anticipating.
right? Yeah, well, uh, we were really happy that Imbue is thinking deeply about those things as we look to the future. And and at uh, a really practical and useful way, as as we look forward. So thank you for doing that, thank you for your research and and for taking time to join us. This has been great.
Yeah, it's been, it's been great, thanks a bunch guys.
All right, that is practical AI for this week. Subscribe now if you haven't already. Head to practicalai fm for all the ways and join our free Slack team, where you can hang out with Daniel, Chris and the entire changelog community. Sign up today at practicalai fm community. Thanks again to our partners at Fly io, to our beat freaking residents, Breakmaster Cylinder, and to you for listening. We appreciate you spending time with us. That's all for now.
We'll talk to you again next time.