Orchestrating agents, APIs, and MCP servers

Welcome to Practical AI, the podcast that makes artificial intelligence practical, productive, and accessible to all. If you like this show, you will love the changelog. It's news on Mondays, deep technical interviews on Wednesdays, and on Fridays, an awesome talk show for your weekend enjoyment. Find us by searching for the changelog wherever you get your podcasts. Thanks to our partners at Fly.io. Launch your AI apps in 5 min or less. Learn how at fly.io.

Well, welcome to another episode of the Practical AI podcast. This is Daniel Whitenack. I'm CEO of Prediction Guard, and I'm really excited today to dig a little bit more into Gen AI, orchestration agents, coding assistants, all of those things with with my guest, Pavel Veller, who is chief technologist at EPAM. Uh, welcome, Pavel, great to have you here. Thank you. Hello, hello, Yeah, yeah, well, I I mean, I. There's a lot of topics. Even before we kicked off the show, uh, we were chatting in the background about some some really interesting things.

I'm wondering if you could just kind of level set us of. People may or may not have heard of of EPAM. I I think one of the things that I saw that you all were working on was this Gen AI orchestration platform. Dial. Maybe before we get into some of the specifics about that and other things that that you're interested in, maybe just give us a background of, you know what, what EPAM is. I know you mentioned even in our discussions that some of what you're doing right now maybe wouldn't have even been possible, uh, you know, a couple of years ago.

And so things are developing rapidly. Just level set, uh, the kind of background of this area where you're, you're working.

sure, yeah, yeah. So EPAM is a professional services organization. We're global, we're in 50 something countries, 50 zero. People globally work with clients. We have been for, I think, 32 years to date. So we do a lot of different things, as you can imagine. And, uh, what I was mentioning about doing things that would not be possible is doing things with Gen AI. Today. We do a lot of work for our own clients, uh, we also do work for ourselves, applying the same technology, cause EPAM historically, as a company, has been running on software that we ourselves built.

The philosophy has always been that things that do not differentiate you, like an accounting software or like CRM, you would go and buy off the shelf, things that differentiate you. Uh, how we actually work, how we operate, how we execute projects, how we, hm, hire people, how we create teams, how we deploy teams. All of that software has always been our own since as early as late 90s, and we keep iterating on that software for ourselves.

And that software today is very much AI first. And a lot of things we do, we do with AI, and really do, because AI, in its current form, exists.

Interesting, yeah, and and how, how, I guess, does you know? Uh, I think when we initially kind of was was, we were prompted to to reach out to you. Um, part of it was around this, this orchestration platform, so talk a little bit, maybe generally, not necessarily about the platform per se, although we'll get into that, but just gen AI orchestration generally. So you talked about, you know, some of these things that are becoming possible. W Where does orchestration fit in that, and what?

What do you mean by orchestration?

You probably think of dial, you can google it. We do a lot of applied innovation in general as a company, and this is one of the good examples of applied innovation to AI. The best way to think of dial would be, you guys all all know, chatgpt, right? Chatgpt isn't an LLM, it's an application that connects to an LLM and gives you certain functionalities. Can be as simple as just chatting and asking questions, it can be a little more complex, uploading documents and speaking to them, like talk to my documents.

It can be even more complex when you start connecting your own tools to it, right? We see our clients not only do this, but also want something like this for their own business processes. And this orchestration engine becomes, how do I make it so that I don't have 20 different teams doing the same similar things over and over again in their own silos? How do I connect my teams and their AIS and their, you know, thoughts and, uh, results into a consolidated ecosystem?

And it's likely because of gen AI, and because of what we can do with conversation and text becomes sort of conversation. 1st, you can think of conversation first application mashups, almost right, like you talk, express a problem. What comes back is not just the answer, maybe what comes back is UI elements, buttons you can click, forms you can fill out, things you can do, as well as things that are done for you by agents automatically. So dial, in that sense, is, well, by the way, it is open source.

You guys can also go look, you know, download and play with it, but it it is a this like chatgpt like conversational application that has many capabilities that go beyond. Uh, we have dial apps, um, they predate MCP, but the idea is that you so dial itself has a contract, um, an API that you implement. You basically come back with a streaming, uh, API that can receive a user prompt and whatever you do, you do. And you come back to dial with not just text.

It's a mu much more powerful payload with UI elements, interactive elements and things that dial will display for me, the user, to continue my interaction. And dial becomes this sort of center mass of how your company can build, implement, integrate AI into this, uh, single point of entry, and then dial goes well. From day one, dial was a load balancing model agnostic proxy, right? So every model, every deployment has limits, you know, tokens per minute, tokens per day, whatever requests per minute.

You'll likely, if you're a large organization with large HM, different workflows, your AI appetite will go well beyond a single model deployment. You'd like to load balance across multiple, and then you'd like to try different models, ideally with the same API for you, the consumer. So dial started like that, started like load balancing, model agnostic proxy, single point of entry. We can log everything that is prompted in the organization, we can do analysis on that separately, because that's very helpful to know what kind of problems your teams are trying to solve.

And then it evolved into this application hosting ecosystem. Now it's evolving clearly towards what MCP can bring in, cause now you can connect a lot more things to it through MCP. So I think it's running at like 20 something clients by now.

so just a couple of follow up questions. It's been in the news a lot, but just, uh, just so people, um, understand if they maybe they haven't seen it. What are you referring to with, um, mcp, and kind of how that, uh, relates to some of this, uh, API interface that that you're enabling?

Well, the easiest is to google it, you can google it, you're gonna find it, it's on Claude. Uh, I let me tell you how I think about this, cause not what it actually like. that's helpful. Yeah, Yeah, I think about it in a very simple term. So MCP allows to connect the existing software world to LLMs in a way. Think like, and I'm gonna. I don't wanna hype it too much, cause it's not yet a global standard or anything.

It's very early, early, early days, it's mon. It's been months, right? But what, let's say HTML and browsers and HTTP, they enabled to connect, uh, well, us people to software all over the world. MCP does that, but for LLMs. So today, if I want, if I, if I. Today want to be able to prompt my application that is in front of an LLM to do things with additional tools. Let's say I want it to be able to search file system based on what I prompt it and find a file and something in that file, right?

So my application needs to be able to do that. My option is what I can write that function. I can then tell my Lm, Hey, here's this function you can call if you want to call it, I'm gonna call it for you. Great, that's one function. What if I need to do something else? I wanna go talk to my CRM system and get something out of there. I'm gonna write that function. If I'm gonna write all the functions I can think of, it's gonna take me years, probably hundreds of years.

So instead, what I can do today, I can say, Hey, my Lm application, can you talk? A protocol? Cause there's a protocol called MCP. I'm gonna bring you MCP servers that other people have built for my CRM system, for my file system, for my CLI, there are MCP servers for everything. IntelliJ exposes itself as an MCP server to do things that ID IDE can do. Now you can orchestrate those things through Lm, so you connect all these MCP servers through an MCP client.

This application in front of Lm to Lm expose the tools to Lm. Lm can now ask the client to call a tool, and through this MCP protocol, the client calls the server, the server does the function that has been written in that server, and boom, Lm gets results. So it's this connective tissue that did not exist three months ago, three months ago. Everybody was writing their own, and right now, everybody, as far as I can tell, writing MCP servers, and those who talk to LLMs, they consume MCP servers.

yeah, and maybe just give um e even uh. So I I like the example that you gave of sort of searching file systems. What are just to kind of expand people's understanding of some of the possibilities? What are some of the things that you've seen maybe implemented in in Dial as things that are being orchestrated? You know, in in general terms, what are kind of some some of these things?

Let me give you a higher level and much more sort of fruitful example. Okay, we have our own agentic developer, it's called AI run code me, uh, cause AI run has multiple different agentic systems. Code me is specifically coding oriented, we have others oriented at other parts of SDLC workflow. By the way, you guys can go to SWE bench and look at verified list. I believe code me as of now is, uh, takes 5th. It's number five on the list of all the agents who compete for solving open source defects and stuff.

So code me as an agentic system has many different assistants in it. Dial as a generic front door, as a chatgpt would like to be able to run those assistants for you as you talk to Dial. And until MCP, it really couldn't other than, Hey, code me. Implement an API for all of your assistants, let me learn to call all of your APIs. Now the story is, Hey, code me, give me an MCP server for you, which is what they have done.

Dial as an MCP client can now connect to all of code me features, all the assistants, expose them as tools to an LLM and orchestrate them for me. So I come into the chat, I ask for something. That something includes reading a code base and making architecture sketches or proposals or evaluation, right? And Lm will ask code me assistants to go and read that code base. Because that there is a feature in code me that does it.

And Dial needs to only orchestrate, but doesn't need to rebuild or built from scratch. That's the idea. So this is the example.

yeah. Could you talk a little bit? I I'm asking selfish questions, cause sometimes I get these asked of me, um, and I'm always curious how people, how people answer this. So, one of the questions that I get asked a lot in respect to this topic is okay. I have two, you know, tool or function, or assistant one, and then I have assistant two, and then I kind of have a few, right? And it's fairly easy to route between them because they're very distinct, right?

But now, if you imagine, okay, well, now I could call one of a thousand assistants or functions or something, or, or, you know, later on, 10,000, right? How does the sort of scaling and routing kind of actually? How is that affected as you kind of expand the o o the the space of things that you can do?

So that I think, and again, I can't know, and I don't know, but I think that is still the secret sauce in a way that is still why. There is all of all of this coding agents in SWE bench. All of them work with, let's say, Claude Sonnet 35 or Claude Sonnet 37, or GPT 40 or LLM is the same, and yet results are clearly different. Some score ten points higher than the other. You go to cursor, uh, ID cursor, you ask it something, it does something, you switch the mode to max.

Uh, they've introduced very recently cursor on on Sonnet 3.7 and now on Gemini 2.0. I think they have a max mode, which is pay per use versus their normal monthly plans. Because Max will do more iterations, will spend more tokens, will be more expensive, will likely run through more complex orchestrations of prompts and tools and whatnot to give you better results. So how you build the pyramid of choices for your LLM, how you cause? Yeah, you will not ask Lm, you will not give it a thousand tools.

If you as a human look a thousand options and you lose yourself, you know, hundred options in it. I again, I don't know. I expect LLM to have the same sort of oops, overwhelmed effect. You don't wanna give it a thousand tools, you wanna give it groups, you wanna say, hey, you know, pick a group and then within that, so you wanna do this. Basically, a like a pyramid, like a tree. But how you build it and how you prompt it, and how you, how you do this now, that's still on you.

This is the application that connects the MCPs, the tools that it itself has, the prompt that the user has given, the system instructions and building the some of the chain of thought Lm can build. And this is gonna be a very interesting balance. What do you ask Lm to build? How much of this sequencing of steps will be on you in your hands versus how much you're gonna delegate to Lm and ask Lm to come up with a sequence of steps?

And from what I, from what I've seen over the last year, you're better off delegating more to LLMs, cause they get better at it. So the more you control the sequence yourself, the less, um, the more sort of hm inflexible it becomes. You're better off delegating to Lm, but you don't expect it to just figure out from one prompt. Daniel, I can give you that example that I gave in the beginning, if you want, about the failure.

Yeah, go for it So I use AI, so I built with AI, right? But I also use AI as a developer. So I'm on cursor as my primary ID these days. I use the AI run code me that I mentioned. I play around with other things like as they come up, like cloud, cloud, code and things. But um, I also record what I do, little snippets, 05:10 minutes videos for, uh, my engineering audience at EPAM. For the guys to just look what it is that I'm doing, learn from how I do it, try to think the same way, try to replicate, get on board with using AI.

So I set out to do a task. I wanted to record, I wanted to on record, get a productivity increase with a timer. My plan was I'm gonna estimate how long it's gonna take me, announce, let's say 2 h. Do it with an agent. And I always pause my video when the agent is thinking, cause that's a boring step, but the timer's gonna get ticking. And at the end, I'm gonna arrive at Let's, let's say, an hour, maybe 40 min out of two.

Boom, that's the, you know, productivity gains and 30 min in. I completely failed. I had to scrap everything that the LM and the agents wrote for me and start from scratch. And my problem was I over prompted. I thought I knew what I wanted agent to do. There were like, there was three steps, like, copy this, write this, refactor this, and you're done. And it did it, it iterated for ten minutes. It was the code me agentic developer that we have.

When I scrapped it and started doing it myself, I did half of it, stopped and realized that the other half was not needed. It was stupid of me to ask. So the correct approach would have been to iterate, do the first half, stop, rethink, and then decide what to do next. But the agent was given the instruction to go all the way, so it went all the way. And this is the other thing with thousand instructions, right?

You don't want an agent to be asked to do something that you think you know, but you only really will know as you iterate through.

in these cases as well. So, like, I find your experience with the you know, balancing how you, how you prompt it, you know how far the agent goes. All of this is intuition that that you're kind of learning. One of the things that was interesting, we just had, um, Kyle, uh, the CEO of of GitHub on. And we were talking about agents and and coding assistants. One of his thoughts was also around the orchestration. After you have generated some code, right?

It's one thing to create a project, create something new, but you know, most of software development kind of happens past that point, right? And I'm curious, as someone who is really trialing these tools day in and day out, kind of as your daily driver and UTL utilizing these things, I think that's a on. People's mind is, oh, cool, like I can go into this c, you know, tool, generate, you know, a new project, that maybe whatever it is.

Uh, you know, you always see the demo of creating a new video game or or whatever the thing is, right? But ultimately, like, I have a code base that is very massive, right? I'm maintaining it over time. I'm, uh, you know, most of the work is more on that operational side. So in your experience with this set of tooling, what? What has been your learning? You know, any insights there, any any thoughts on kind of where that side of things is heading?

Especially for, you know, you're dealing with, I'm sure, real world use cases with your customers who have large code bases, right? So

Well, that's, uh, great that I'm. I'm so glad that you asked, cause what I do is actually that latter aspect. I have a mono repo of like 20 different things in it that could have been separate repos of their own. So I have a large code base that I work with. And I actually saw our own developer agent occasionally choke because it attempts to read too much and it just chokes on. Like tokens and limits and things that it can do per minute or per hour or something.

So that's one thing. But what I find myself doing with cursor, for example, I actually pinpointed very actively, very often, cause I want to work with these files. When it's something specific, I'll just point the files at it. And I'm gonna ask, I'm gonna prompt it in context of these three or four files, and that limits how much it's gonna go out. But really, back to your question. To me, it's not about code bases that much.

I don't think it's gonna be well. Maybe if I do something greenfield and funny, it's gonna write it, I'm gonna run it, and if it works, it's all I need. Like, it's correct, it works great today, and it's still a mental shift. It's still early. I'm still looking and thinking of the code base that I write with my agents as code base that will be supported by other people, likely with the agents, but people still so correct by itself is not good enough.

I want it to be aesthetically the same, I want it to follow the same patterns, I want it to make sense for my other developers who will come in after me. I want it to be as if it's the code that I have written, or at least more or less that I have written, and that slows me down a little bit. Clearly, I'm sure, uh, but the other thing is, I am the bottleneck. An agent will take minutes, small digit like single digit minutes, if not less, to spit out whatever it spits out.

And oftentimes in code bases, it's not a single file, it's edits in multiple places. Then I have to come in and read it. Here's the difference. When I write myself, my brain has a timeline. I was thinking as I was typing, as I was thinking, as I was, I know how I arrived at what I have arrived at. I may decide that it's bull, you know, scrap it, start over. That happens. We're all developers, but I know how I arrived at where I am.

When I look at what agents PR agent produced for me, I have no idea how it arrived at where I am. I need to reverse engineer, like, why, what did it do? It takes time. I tried recording it, uh, and I can't, cause I can't speak as I think at the same time. This is the bottleneck, literally, so just this is the bottleneck. The other thing is, when I was doing that video with the timer, I sort of I expected certain outcomes, but I also knew that if it works, I'm gonna say this at the end, I'm gonna say, guys, look.

It took me 20 min, let's say 30 min out of an hour, so it's two x, right? Literally two x productivity improvement. Amazing, isn't it? But here's the thing within the 30 min that I've spent. The percentage of time I spent critically thinking was much higher than normal. Percentage of time I spent doing boilerplate is much lower. Cause the agents did this. I really critically thought about what to ask, how to prompt, and then analyzing what it did, thinking what to do next.

Do I do I edit? do I re prompt? Can I sustain the same higher percentage of critical thinking for the full day to get two x in a day? Probably I can't. So what probably is gonna happen? I'm gonna get two x, but I'm gonna use the time in between as agent work to do something else. My day will likely get broken down into more smaller sections. My overall daily productivity is likely to increase. I'm likely to do more things in parallel.

Maybe I'll do some research, maybe I'll answer more emails, right? But it's gonna be more chaotic, also likely more taxing. I don't think we've learned yet, I don't think we've had enough experience yet. I don't think many people talk about this yet, people talk about this. Oh my god, look, what I've built with agent. Ah, I will. I wonder how they're gonna talk about how they've worked for like six months with the agents, and how six months that they've done with the agents is better than six months without.

And how they feel at the end of the day and think think about in the zone, right? We all, I hope, like as engineers, we like to be like, you know, disconnect all emails, whatever, get the music on IDE in front of you. You're in it for like, 2 h with agents, you just can't. You prompt an agent, it goes off doing something. What do you do? Do you like, pull up your phone and then your productivity increases one way, your screen time increases the other way.

It's not a good idea. What can you do? Like, what do you do in this minute and a half or three? And you don't know how long, right? You? Well, you can see the outcomes coming up, but the agent is still spinning, it's still spinning. Like, so, I'm sorry, it's a long answer to a question, but that's what I'm thinking about, like, constantly, and that's what I don't yet have answers for. But I really hope to eventually, through experiments and recording and thinking, arrive at at least what it means for me, cause I cannot even tell you what it means for me yet.

yeah, yeah. I mean, um, I. I experienced this yesterday too, because I'm I'm preparing various things for for investors. Um, you know, do updating some competitive, uh, analysis and and that sort of thing. And you know, I, I just when you have whatever it is. I think it was 116 companies. And I like, Oh, I'm gonna update all of these things for all of these companies, like, I'm, you know, obviously, I'm going to use an AI agent to do this.

This is not something I want to do manually. Uh, is put in all of these things and search websites. So so I did that. But to your point, it was like I could, I could figure out how to do a piece of that and get it running. And then I see it running. And I, you know, I realize that this will take however long it is right, 10 min or or whatever the the time frame is.

And then you context switch out of that to something else. Um, which, for me, I think was email or whatever. I'm like, Oh, this is gonna run, I'm gonna go answer some emails or or something like that, which in one way was productive. But then I had to context switch back. And I'm like, Oh, uh, why did I output all these things? or, or, you know it? It happened to be that I wasn't watching the output right.

And then, and in one case, when I ran it, I was like, Oh, well, I really should have had this. It output this column or this field, but I, I didn't think of that before, and I wasn't looking because I turned away from the agent back to back to my email, right? So, yeah, I think this is a really interesting set of problems. That is more of like, a new, uh, yeah, it's a new way of working that hasn't been parsed out yet, right?

and I tried not to do it like I tried. But then you sit idle, like you literally sit idle. It's like, and it, it, it doesn't feel good, it feels, it feels like, Oh my god, why am I not doing anything?

yeah, it's it. It's an interesting dynamic, that's that's for sure. And I've definitely seen, uh, people that show, you know, having multiple agents working on different projects at the same time. And that when I see someone with two screens and things like popping up all the place, I, you know, there's no way I could in my brain sort of monitor all of that that's going on, right?

It must be very taxing. 1st and 2nd, half of those merge requests, pull requests from the agents will be, let's say, subpar frustration, and you will rise, too. Like you would think, man, I would have done it already myself, much better, like, what is this like? Emotionally, it is a very different way of working emotionally. And I really, I, I can't. Well, I keep thinking, I can forget I ex. I advise people also to think, not just think about productivity gains, not just think about delegating to agents and enjoying the results.

Think about how it changes the dynamic of your day and how you think about it. A afterwards, right?

yeah, yeah, that's interesting. Um, so I know we're circling kind of way back, though. Is interesting discussion, but I I do wanna make sure people can kind of find some of what you're doing with, um, with Dial. You mentioned kind of the open source piece of this. What's sort of needed from the user perspective to kind of spin this up and start testing it in any, any? For those of those that are out there that are interested in, like, trying thing some things with the project, what would you kinda tell them as a as a starting point?

And like, what the process is like to kinda get a system like this up and running?

I actually not sure I can tell, uh, for dial specifically, I have nobody's running local dials. It's not something you run locally, gotcha, it's something that you run sort of centrally in an organization of size. Can be different, but you expose it to your people through like a URL that they all can go to and use. Like, sort of use AI through dial and do things through dial. O. One of the apps we built as an example earlier, it was last year, uh, was like, talk to your data.

But if you look at analytics like Snowflakes of the world, they all have something like this today. Like semantic layer, which you work on, and then through semantic layer, through prompting and through some query conversions and connectors to data warehouses and data lakes. You get yourself a chat with your data, like analytical reports, graphs, tables. So we built that that was built into dial, so you go to dial and then again, imagine chatgpt, imagine chatgpt that allows you to choose what model you talk to, right?

Not just openAI models, but all the all the other models that exist, as well as applications. So go to this chat GPT, which is, in our case, dial. You select this data heart AI, we call it, which is our talk to a data, and you start talking to it. And this is still your dial experience, but you're really talking to an app that then talks to Semantic layer. Then it's, you know, builds queries based on your questions, runs them, gets data back, visualizes it in dial, because dial has all these visualizations capabilities.

I explain how it's not just text coming back, builds you charts and you can interact with it. So, but again, you don't run dial locally. If if you want to explore what it is, I hope, I expect that if you go to, I think it's, uh, rail dash epam, com, uh, or epam dash rail, yeah, oh, I gotcha, dot com, epam dash rail dot com. Thank you. And you're gonna read about what it is, and you're gonna find all the links, uh, to hopefully documentation, how to, you know.

But also most HM, most companies who we work with, they want more than just, hey, how do we install it? they want. And now we want to build with it, and that's where we come in with professional services. Uh, we can build them things for their dial so that they can do the AI that matters to them in their context, with their data, with their workflows, with their restrictions on things they can and cannot do. And yada yada, yada,

yeah. And I'm wondering for this kind of, if you think about this zoo of underlying applications or assistants, I'm wondering because you've obviously been working in this area for some time. Do you have any insights or learning? Uh, around kind of easy wins for, you know, underlying, uh, functions or agents that can be ti tied into this sort of orchestration layer. Uh, or maybe like more challenging ones. Things that you've learned over time in in developing and working with these things.

In terms of, you know, things that you could highlight as, you know, easy types of of wins and things that. I mean, you mentioned the workflow stuff around some of what isn't yet kind of figured out, but more on the orchestration layer and the function calling. You know what? What are some areas of of challenge or things that might not be figured out yet? That are that you think are interesting to to explore in the future?

let me think so, um, cause my 1st thought was to. So, you're asking about connecting tools and functions to an LLM, and which of the functions, or what? What type of connectivity sort of is easier? And

yeah, yeah. Is there anything that's, uh, out of scope or more of a challenge currently, or is it fair game for? Kind of, you know, I guess it's whatever you can build in that function, uh, in the assistant, but yeah, what? What limitations are there, or challenges in that kind of mode of development of developing these underlying functions or tools?

I see, um, so it's kind of a twofold answer. If you take the technicality aspect, like, how do I build a tool that does X, the complexity is really in X. Like, if you wanna go and query a database, how hard is that? Well, not hard, right? I mean, connectivity to the database, if you have a query, you run it, you get results back. So it's not hard to do the technicality of querying a database. Making it useful and making the result useful in context of user's prompt and conversation is a lot more challenging.

I had this, so I'm running a service you can. AC. It's actually has a public webpage called API EPM.com. It's our own, so you, you will not really go past the front page, but you'll you'll understand what it is. It's a collection of APIs that we built, my team has built that exposes a lot of data. Remember, I said EPM runs on internal software, so all of those applications, they stream their data and their events out into a global data hub.

Think big, big, big Kafka cluster, but that's Kafka, so you can read data out of it as a Kafka consumer. But if you wanna have, like, more modern, you know, API search, lookup, this, that, so we have a an API service, all of that data. And somebody came to me today and said, Hey, have you heard of MCP? I'm like, Yes, of course, I have. Why don't you guys build MCP for API? Repom.com? My answer is it is easy to build API repom.

Com speaks RSQL. I can build a server that will take your query, create RSQL. Um, Lm will be able to do that easily, run it, give back the data. But I said, it's not gonna be useful, cause this is single data set APIs. Your questions are likely analytical, you likely wanna ask something that expects me to do summary by month. This, this, this and give you like, a which like? That's a very different question. So you ask me about MCP to an API, easy to do, make it useful for your actual use case, much harder to do.

I likely need to do a lot more than just connectivity of tool to an LLM. I need to understand what you're asking, figure out the orchestration that I that is required, maybe custom apps, maybe something else, and then and then you start hitting authentication, legacy apps, all the other roadblocks. And in a way, the talk to your data is an amazing prototype that we built, and I have a video about this. Um, but we sort of stopped because we clearly sensed how steep the curve is to get it to like actual cause.

What we wanted to do, what we envisioned we could do, was analytics democratized. So you don't have to go to analytical team, ask them to build you a new Power BI report and them spending a week doing so. You can just come into dial and say, Hey, you know, show me this, this, this and this. And yes, we technically can do it, but to be able to do this for all kinds of questions you can ask about our data, that's a much harder thing to do.

So,

yeah, yeah, and it it also, yeah, to your point. Underlying systems might have limitations, I think, in analytics related use cases that we've encountered, um, with our customers, you know, uh, often I'll just ask the question around, Hey, if you, if you gave this database schema, or whatever it is, to, you know, a reasonably educated, you know, college intern, or or whatever that is. And you ask, you know what columns would be relevant to query based on this, you know, based on this natural language query?

You know, you can pretty easily tease out. Well, I look at all these columns, I have field one, fifty, seven and custom underscore new underscore field. You know, you. There's no way for for just someone off yet, you know, to to know anything about that. And so it's not really a limitation of what's possible in terms of the technicality, like you said, it's more of, you know, you're not always set up for success in terms of utility, like you mentioned.

and for data, that's where semantic layer comes in. So if you have descriptions of your columns, of your tables with business meaning, then connecting that semantic layer with some data samples to LLM will allow it to write the query that you thought was impossible to write. Because it is impossible without the SEM semantic layer, sort of can explain the data that you have in business terms, in the language that the questions will be asked of your assistant.

And that's what allows us, allows us to do this. Talk to your data analytics.

yeah, well, I, I know that. Um, we've talked about a lot of things. I think you. You are probably seeing a good number of use cases across your clients at at EPAM, and also your own experiments with Dial and other things. I'm wondering, as you, as you kind of lay lay in bed at night, or whenever you're thinking about the future of of AI, or or maybe it's all the time, or maybe it's maybe it's not, uh, at night, but uh, yeah.

As as you kind of see what is, uh, to your point, just bringing it all the way back to the beginning. You see what is possible to do now, which even six months, a year ago, whatever it was, you know, is was not possible. What kind of, uh, is most exciting for you, or, uh, most interesting for you to see how it plays out in the next, you know, six to twelve months? What? What is kind of constantly on your mind of where things are going?

Sounds like, you know, the how we work with these tools is one of those things. Uh, we already talked about that a little bit, but what else is, you know, exciting for you, or, or, um, encouraging in terms of how you see these things developing?

My answer may surprise you when I think about it. I don't, you know, think or anticipate any new greatness to come. I actually mostly worry, and I worry because I know that my thinking is linear. Like most of us. Uh, even though looking back, we know the technology has been evolving rather exponentially, our ability to project into the future and think what's coming next is linear. So I am unlikely to properly anticipate and get ready for, and then expect, right, and wait for what's to come.

I am sure to be surprised, and I guess, as everybody else, I'll be doing my best to hold on to not fall off. So I worry seeing how the entry barriers rise, it's harder for more junior people to get in today. When I'm asked about skills I recommend that people focus on as far as trying to be better prepared for the future, I always answer with the same things. I always say fundamentals and then critical system thinking.

Then fundamentals you can read about a lot, but you really master them when you work with them yourself, not when someone else works with them for you. And not having them is likely gonna constrain you from being able to properly curate and orchestrate all these powerful AI agents. And when they get so powerful that they don't need you to curate and orchestrate them, then what does it do to you as an engineer? And maybe that's not the right thinking, but this is what Hm, I think about at night, like you asked, Uh, when I think about AI and what's coming?

I am excited as an engineer, I like using all of this. I just don't know how it's gonna reshape the industry and how it's gonna change my work, you know, in years to come.

Yeah, well, I I think it's something even in talking through with you. Kind of some of the work that you and I have been doing with agents and how. That really has triggered a lot of questions in our own mind of what is the proper way of of working around this. And I think there is going to be a W, you know, that is a widespread, uh, issue that people are gonna have to navigate. So, yeah, I think it's, I think it's very valid.

And uh, we'll, we'll. We will be interested to see how it develops and would love to have you back on the show to, uh, to have your learnings again in, uh, in six or twelve months of of how it's shaking out, uh, shaking out for you. Um, really appreciate you joining. It's been a great conversation.

Thank you very much, it's been a pleasure.

All right, that is our show for this week. If you haven't checked out our changelog newsletter, head to changelog.com. News. There you'll find 29 reasons, yes, 29 reasons why you should subscribe. I'll tell you reason number 17. You might actually start looking forward to Mondays.

Sounds like somebody's got a case of the Mondays.

28 more reasons are waiting for you at changelog.com news. Thanks again to our partners at Fly.io, to Break Master Cylinder for the beats, and to you for listening. That is all for now, but we'll talk to you again next time.