GenAI risks and global adoption

Welcome to the Practical AI podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn X or Bluesky to stay up to date with episode drops, behind the scenes content, and AI insights.

You can learn more at practicalai. fm. Now on to the show.

Welcome to another episode of the Practical AI Podcast. This is Daniel Whitenack. I am CEO at Prediction Guard, and I am joined as always by my co host, Chris Benson, who is a principal AI research engineer at Lockheed Martin. How you doing, Chris?

Hey, doing great today, Daniel, how's it going with you?

It's going great. I traveled a bit over the weekend to run a a a mini marathon, which was fun, but I am back safe at home. And, uh, of course, safety is is something that hopefully we can talk a little bit about today. With our, with our guests, who are, uh, Rick Kobayashi, who is co founder and CEO at Citadel AI, and Kenny Song, who is co founder and CTO at Citadel AI. Welcome.

Thank you for having us. Yeah, thank you. Well,

Yeah, it's great to have you both here. Um, good to get introduced to you both. I'm I'm really excited about this conversation. Because, of course, security, uh, and safety related to AI is is very close to my own work, and so it's it's always great to connect, uh, connect with people in this space. And yeah, I, I'm also interested to hear a little bit. And I, I even saw some things on your on your website, you know, talking a little bit about some of the AI in Japan, some of the, I guess, regulations or guidelines for for businesses that have come out there.

So maybe before we get into the the nature of of what you're building and what you're doing. Any thoughts on? For those out there that might be in the US. or in Europe? And and be constantly exposed to the things that are going on in AI in those jurisdictions? Any thoughts about you know what is the same, or what stands out about what's happening with AI in Japan?

so, yeah, anyway, thank you very much for joining us. Uh, and uh, uh, my name is Rick. And uh. As for the Japanese conditions, basically, I'm afraid to say it's around one, two years behind the US. situation, basically up to now. But, uh, on the other hand, surprisingly, as for Gen AI, I think that Japan is one of the most advanced countries, not the development of the foundation models, but the utilization of the Gen AI applications.

And probably it might be related to kind of Japanese, kind of animation or cartoon. So, you know, like, there are lots of, say, robot, or some of the say animations, which specify some of the AI or robot technologies. And, you know, Doraemon, or there are lots of animation over there. And people are very, say, getting familiar with such a kind of a say, talking with robot or some of the say advanced computers from their childhood. So in that sense, I think that many people do not have any hesitation to work with or talk with like a chatbot or any Gen AI.

So in that sense, the hurdle to introduce LLM or Gen AI in Japan is very low in that compared with, uh, US. or other other countries. So, in that sense, development side or technology side, frankly speaking, uh, Japan is behind the US. situation. But as for the uh usage of such kind of Gen AI, I think I hope, uh, Japan is one of the most advanced countries.

Yeah, that's, that's really interesting to hear the sort of perception, uh, side of things, um, and of course, with more, uh, maybe even more adoption or adoption. That's ahead of maybe other places in the world. It could be that, you know, users or usage of the technology has hit some, uh, bumps along the road, or or has hit some problems. Um, which I, I, I know is is a lot of what you all are are working on.

What has been the situation in Japan around kind of regulation and business usage of of AI? Is it, is it on the sort of more regulated side or less regulated side in terms of the the government? And and I guess, uh, regul regulation in terms of security or safety, privacy, these sorts of things,

I think Japan is in the middle between the EU and the US. situation. So they say that a soft approach, so there is no strict say regulation in Japan, but there are lots of kind of guidance in Japan. And people are, say, have already noticed some of the issues, I mean, security side or safety side of AI. And on top of that, because they are say trying to introduce LLM applications for like, uh, uh, contact center applications.

So they are concerned about more, say, kind of reputational risks rather than security risks. So in that sense, uh, they understood the importance of the safety of the AI or trustworthy of the AI, but more say, concerned about reputational risk rather than security risks.

Uh, I'm curious, as we're, as we're kind of talking about the adoption and the environment around that is, you kind of mentioned that that in Japan, it's very, you know, people are really getting into, uh, the utilization of LLMs and generative AI. Do you have any any, uh, kind of thoughts around? You know, what is it? a, what is it that's driving that, you know, compared to other countries that you've observed? If you're looking at that kind of adoption in Japan versus the US.

or versus Europe or whatever, um, any thoughts around that? Because I, I had, I had observed that as as, uh, someone in the US. that Japan seemed to be, uh, implementing. And I was kind of curious if there was, uh, you know, what was the what was kind of the driving force there? That that got the utilization up, uh, especially a among just, you know, your typical, average, everyday folks, they are not, you know, outside of the AI industry.

Uh, would love your thoughts about that.

I'm not so sure the the background reasons why, uh, Japan is more aggressive to introduce such kind of LLM applications. Apparently, as I said, people has lower, say, risk of the per perception, I may say. In some cases in Japan, like a chatbot or kind of say, new technology might be a kind of friend in that sense. Uh, in US side, such a kind of a new AI is kind of an enemy of a human.

yeah, so there kind of a a little bit of a cultural difference in terms of how that that openness to adoption and stuff. So, yeah, that would make sense, that would make sense to me.

yeah. And I guess sometimes your your friends can potentially hurt you, even if they, even if they're not not trying, which I know. Um, you, you all are are kind of involved in both, you know, the evaluation of these systems, you know, running metrics against these systems and kind of building trust. Um, in in a very real way, Kenny, I'm wondering if we bring you in here. And and maybe just, you know, safety and security around AI is such a broad topic now, with so many people addressing it from so many different perspectives.

I'm wondering if you could help us zero in on on the may. Maybe the kinds of problems that you all are are are exploring, and how those fit kind of more generally into the landscape of, I guess, security risks or or threats as related to to AI.

Definitely, yeah, so thank you for having us on the podcast. My name is Kenny, I'm the co founder and CTO of Citadel AI. So what we do at Citadel is we build software tools to help organizations test, monitor, and govern their AI systems. And in the world of LLMs, when customers come to us, they usually have some proof of concept that they've been developing internally. So they've been using some foundation model, they're building some chatbot or some agentic workflow.

And they come to us when they want to make that proof of concept production ready. Um, and um, usually their main problem is they want to mitigate some of the risks that they see right. So things like hallucinations, um, toxicity, or users trying to prompt inject the system, um, they're. They're looking for a solution to these types of problems, um, and that's where we come in, um, as a tool provider.

Yeah, and how does that? I I guess? How are you seeing? When you're interacting with your customers or they're coming to you with these problems? What is what is the impact of those of those problems, like hallucination or injection? Is that something that is causing real problems, or something that they've sort of heard about that they're concerned about, but it's maybe not causing a real problem? Or, y, you know, what are you seeing there in terms of, I guess, the the impact?

Let's say I just wanna throw caution to the wind and ignore these things. What's what's the bad side of this that could happen? Um, were I to kind of take that more, uh, more loose approach, any thoughts?

Yeah, I think it's usually a mix of both, um, and it also depends on the the sort of the risk appetite of the company, um, that's developing the system. And typically, our customers are larger enterprise companies, both inside of Japan and outside of Japan, um, and so they have very mature risk management practices. And before they, you know, launch these POCs into a production service, whether that's internal facing or external facing. Um, they want to make sure that they, um, you know, have appropriate controls in place to manage the risk, and they've, um, cr properly identified the potential risks, reputational, um, data security and and so on.

So, yeah, I, I, I think for the customers we talk to, um, it's generally a big concern for them. And they come to us with the UM problem of mitigating some of these risks that they've already identified.

Uh, I'm curious, as as you've kind of identified, you know what, that kind of the strata of customers that you're looking at there, what, what is kind of? Is there? Is there something that definitively kind of separates that kind of mature, larger organization from some of the smaller ones? Do they have a different set of problems that they're coping with? Um, or maybe just haven't gotten far enough along, uh, in terms of maturity and risk management.

Um, what, like, at what point do you see the uptake kind of falling off within maybe smaller organizations? You know, what's what does that look like as you as you move from that upper strata into the mid tier?

I I think my my general perspective is that there there's two ends of the spectrum of like, startups to enterprise company, and for startups. Um, the the risk is fairly low, cause you don't have a brand to protect, you don't have an existing business to think about. Um, and so you can just launch these POCs out to your customers very quickly, and you can iterate quickly. For enterprise customers, uh, they tend to be more guarded, uh, they they have a closer eye on potential risks to the business.

Um, and uh, those are the customers that sort of really want pretty robust testing before deployment, and also monitoring and potentially real time guardrails, sort of filtering, um, after deployment. Yeah, I think it depends a bit on the size of the company.

Yeah, I, I'm wondering with that, like, as you're kind of looking to those types of organizations, I'm I'm curious a little bit of the backstory of of Citadel. Um, m. Maybe Rick, you could tell us a little bit of kind of how you came to the these problems. Cause you seem like, um, you, you have customers now, you've developed some of these things. W When did that happen? How early was that in? Kind of? Was that before gen AI was that, you know, as gen AI was coming up?

And and how did things develop in terms of your your thinking around these problems? And how you would bring something to the to the market in terms of addressing them?

uh. So, in that sense, it was before Gen AI. And actually, the person who came up with such a kind of a idea is not me, but Kenny. So Kenny is worked in, uh, uh, Google brains, and he was one of the core member of the TensorFlow team. So he's taking a leadership to develop, uh, most advanced AI technologies in Google. And found that there should be a lots of risk about the trustworthiness or say safety of the AI.

And, uh, he believes that probably he is the best person to explain by himself. But but anyway, he reached the idea that, uh, uh, uh. Some of the tools which protect such a kind of a safety or security issues would be, say, should be become popular, especially in the enterprise companies where they may not have, say, a bunch of the uh, ai engineers inside. So that is a background. And, uh, our company name Citadel itself is, say, shown the our what we are focusing, so Citadel is, as you know, say, like a vault or castle.

So we are the company to say protect such a kind of risk, I mean human from AI risk, such kind of a say concept is, say, uh, based on our uh with our company name.

Yeah, anything to add there, Kenny?

Yeah, that that's a pretty good overview of the background. Uh, when we started the company, it was at the end of 2020, um, so it was before the era of large language models. And we were initially really focused on helping organizations monitor their traditional predictive AI models, so tabular models, vision models, that kind of thing. And about two years ago, we started getting a lot more interest in our from our customers in LLMs. And how do we, you know, reliably test this new technology?

How do we integrate it into our workflows and our, our business applications? And so these days, I think a lot of our new customers come to us with, um, LLM types of of questions, and then we still have a lot of existing customers that work more on the predictive AI side.

So, Kenny, I'd like to to follow up. As you guys were were kind of developing the idea, going back to that a moment for the company. Um, and Rick mentioned that you had been there at, uh, Google Brain and that you were on the TensorFlow team. I'm, I'm curious, um, uh, ha, as a, you know, for me, as a as a someone who's used Tensorflow a bit. Um, and and so that was like, oh, wow, you know, one of the people who who helped put that together.

I'm curious what parts of the experiences you had there led into the formation of Citadel. In your mind is is, uh, as you had some of the the insights that you might have developed in your previous employment. Did did any of that? W You know, how? How did that lead to what Citadel does? Um, how did that kind of bring forward when you and Rick started the company up?

Sure, so, um, a bit more about my personal background. So in 2017 to 2020, I was working at Google Brain as a product manager, and, uh, my team was responsible for building machine learning infrastructure at Google. Um, so this included TensorFlow and also other platforms like TFX, TensorFlow Extended, and some of the work on Google Cloud's AI platforms and so on. So sort of the software foundations that power a lot of Google's machine learning applications. And I think Google tends to be pretty ahead of the curve, um, in AI adoption, and I think back then it was more called machine learning rather than AI.

But basically, what we worked on was, um, making models at Google more reliable at Google production scale. And that usually meant building, um, these pretty sophisticated pipelines that not only target training the models, but also serving them in production. Um, so we had pretty robust um systems for monitoring data drift and validating data that enters production models, um, monitors the output of these models. And, you know, at Google, you can afford to have hundreds of platform engineers build out this kind of infrastructure to use internally, um, but for most other types of organizations, they can't make that level of investment in internal platforms.

And, uh, we felt that there was an opportunity to, um, sort of build some of these model monitoring and data validation tools, um, for other companies, uh, and so that's where the idea of Citadel AI kind of started. And when we, uh, started the company originally, we focused, um, a lot on adversarial attacks. Actually, back in 2020, since it was a very hot research topic. So basically, you know, designing noise and images and and other types of input data to trick models into making the wrong predictions.

We sort of found that after a few months, um, this wasn't really a problem that companies were interested in. It's like a very interesting research problem, but less interesting commercially. Um, so after that, we pivoted towards more observability and testing of these predictive models, and these days we focus a lot more on LLM testing and monitoring.

And maybe you could just give us, you know, G going going back to you, Rick, maybe you could just give us a few examples. That stand out of kinds of companies and the kinds of models that they're running and the way that or or the some of the things that they would be interested in tracking or detecting or observing. Uh, a few exam concrete examples might help our our audience kind of grasp some of the the cases that you're working with.

One of the largest customers in in in Japan is the financial industry, surprisingly so, like banks or insurance companies or security companies are very aggressive to introduce LLM applications into their core operations. Uh, such as like a contact center or internal applications. And, uh, when they say they can go through POC stage, and when they get into, uh, commercial operations, they start caring about some of the kind of safety issues. So because those financial industry is governed by the the government itself, so in that sense, different from regulation related AI, they have already been kind of regulated by traditional, uh, say, financial regulations.

So if, say, LLM applications, uh, behave badly, that damage their core businesses. So to introduce such a kind of a AI application is to differentiate or or advance their services compared with their competitors. But by introducing new technologies, if their core damage, uh, core businesses damages, that would be very, say, huge negative impact. So, and also, uh, if the AI behave badly, as I said, their say. The Japanese government may say, in some cases, scold or punish them.

So th that is one of the most largest risks for them to introduce, uh, LLM Gen AI applications. So what we are doing is to making sure that such kind of a say bad behavior, uh, will not happen in the application or not, and making sure that they can safely introduce, uh, say, go into commercial operation. So, uh, some of the leading the bank company or insurance company is our say our customers.

Gotcha, I'm curious if I could follow up on that for a 2nd with when you're. We've kinda talked about the the notion of risk management in general, that that industry is is dealing with. How do they, given the risks, uh, for a large, you know, organization to implement LLMs? Do you do you have any insight into how they're making the J the risk management judgment? Um, on the benefits of implementing maybe a new chatbot versus the potential downside, where, you know, as you pointed out, could be damage to brand, damage to core operations, damage from a regulatory standpoint.

That's like, as I'm listening to you, that does sound very risky, you know, from that perspective, how do they make? How do they evaluate that? Like, say that they know that they wanna come to Citadel and get that help. What do you have a sense from your customers? What a typical balance on that is of the benefit of the LLM utilization versus the downside of if things go off. How do they? What are they thinking when they come to you in that way?

So, in that sense, they fully understand that balancing is very important. So to protect the risk, uh, so much may say, uh, delay their say the service advancement. On the other hand, say, uh, without having any security or safety, uh test, uh, they may get into bad situation. So they need to how to balance such a kind of risks and benefit. And, uh, they have or set up several like, uh, internal organization to manage such a kind of a risk inside.

So, yeah, as you can easily imagine, financial industries come, they they are very structured company, so there are some of like a risk management department inside. And, uh, in some cases, uh, there are like a AI governance team inside or something like that. So they jointly work together and try to manage, uh, such a kind of a risk. And at the same time, they try to say, introduce the advanced system so that they can differentiate themselves from others.

And especially as for the contact center, probably the same situation in the US. But, uh, when the the consumer end user try to reach out the banks or insurance company and call it the free, free dial. In many cases, they can't easily to say reach to the contact center person, they have to wait 30 min or something like that. So, uh, because of that situation, they are so so aggressive. Introduce such a kind of a applications into contact center and internal purposes.

I and I. I'm just thinking about this scenario where you have the contact center, you're introducing whatever it is, a chatbot, a voice assistant, that sort of thing. And then I'm thinking back, um, to what Kenny was talking about, the about, kind of how, the how. The company started evaluating models that, m, you know, were not Gen AI models yet. And I'm wondering, um, Kenny, if you could help us think about, you know what, what needs?

Cause, if I'm understanding, you know, part of what you all are doing. Part of it is evaluating the risks of a particular model or system, and part of it is observing those and monitoring those in in real time. And if I think about like a traditional model, let's say a model that detects tumors in medical imagery, right? I can have a very nice sort of ground truth data set. And, you know, uh, maybe it's hard to get because there's some privacy concerns, but I, I can still get it.

I need to get it to train my model. And I, you know, have very specific metrics right around whatever it is, accuracy, f one score, et cetera. I, uh, can sort of grasp what the, you know, performance of that model is, maybe even compare it to human performance with something like a a call center. In some ways, people might struggle to connect that to real metrics that make sense, right? Because it's like, oh, well, people could say anything right to to the to the chatbot.

How do I? How do I know on what's gonna come in, you know, either from a normal usage or malicious usage, or or whatever. And how do I connect that to any sort of metric around a model? So I think sometimes people struggle with this idea of metrics and Gen AI models or Gen AI systems. Could you help, maybe clarify, like, what? What are some of the relevant metrics that that people could think about in terms of these systems?

That might help them understand, you know, how the how the systems are behaving.

Sure, yeah, that's a very spot on question, I guess. Before I talk about specific metrics, I'll just take a step back. 1st. And I, if if we sort of think at a high level, what is the same between predictive AI and generative AI? Um, I think the structure of how you maintain reliability is basically the same, right? You need testing before deployment, and you need monitoring after deployment, and it's also very similar to, like traditional software applications, right, where you have automated tests and automated monitoring.

Um, and so I think that part is the same, but the part that's much trickier for generative AI is that. Usually, as you mentioned, you don't have ground truth in the same way that you do for a classification data set, for example. And so the metrics that you use for evaluation are not as well defined. So you can't measure accuracy, you can't measure precision or recall. And the output of a generative AI model is also much more complex than just like a probability score.

And so in that environment, it's it's very hard to determine. Um, how do we actually evaluate these things in a quantitative and objective way? So the approach that most of our customers take and most of the industry has gone in is, um, basically, um, using LLM as a judge, um, so you can craft these evaluation prompts that ask an LLM to evaluate some quality of some generated text. So it could be, you know, a very simple example is sentiment, so you evaluate the sentiment of of of some text.

Um, you could do that with traditional, um, like a sentiment classifier as well, um. But there are more sophisticated metrics, such as, you know, uh, detecting hallucinations against some ground truth document, or measuring the relevance of the answer relative to the question. Or you might have more, uh, we call them custom metrics that are sort of designed to be domain specific. So if you have, like, a refund chatbot, you can design a metric that, uh, measures if the chatbot adheres to your, your company's refund policy.

And so these metrics, they're very flexible because you can design the evaluation prompt in natural language. Um, and in our tools and our open source libraries, we have a set of built in metrics. You know, it's like a library of metrics you can choose from. But for many of our customers, they also extend those built in metrics to customize them to fit their business applications.

So, uh, Kenny, you, you were mentioning this sort of idea of LLM as a judge, which is, you know, using a a model to evaluate the model. W in in some sort of axis of of performance or some some quality, which definitely seems like a flexible UH option. But also some people, you know, some people might be thrown off by this sort of circular. Using a model to evaluate a model. Also, there's sort of this, you know, you're.

You then have a model that's evaluating the model, so how do you evaluate the model that evaluates the model? And and you kind of kind of get in in in this. How have you all navigated that side of things? Both kind of in terms of using using the larger model, making sure that the evaluations are are sound and also kind of, uh, transferable, you know, one model to the other. And, you know, maybe benchmarking the system over time, cause also the models you might wanna use as evaluators might change over over time.

Yeah, also a very good question, and it's a question that we get from our customers quite a lot as well. And the way that we generally approach, um, the the evaluation workflow, which includes designing these metrics, is not usually human judgment. And taste is treated as the gold standard. But the problem with having humans evaluate every experiment with your LLM system is that it's very expensive and it's very slow. Um, and so, in the ideal world, you would design these LLM as a judge metrics that can mimic human preferences.

Um, and so in our software tooling, this is what we, um, design specific workflows to help users to do. Usually, when they when a customer starts on a evaluation project, they'll, you know, of course, think about the evaluation criteria. That's important, but then they'll also have humans do a small set of that evaluation. So maybe like 50 to 100 of these manual annotations. And from there, you can design LLM automated metrics and measure their correlation and accuracy against a human judgment.

You usually need to iterate a few times to get that, um, custom LLM metric as close as possible to the human judgment. But then once you have that, it's very powerful, right? You have this automated metric that, um, is a very good proxy for human judgment, and it's automated. Which means you can run it at scale, you can deploy it during evaluation, but also, um, in monitoring as well. And you can also potentially use that as a production guardrail.

Um, in our firewall,

Rick, I was wondering, I know, in, you know, we've kind of alluded to it that, you know, the the two sides of the equation in terms of testing the models and then monitoring. Could you talk a little about Citadel Lens and Citadel Radar? Uh, how you bring them to customers and and h, and what the relationship is between those two, uh, products that you're bringing to your customers? And and you know, how do you? how do you go to?

how do you present them? When? When somebody is is interested in in being able to bring this level of security to the models that they're interested in.

Sure, the 1st of all, now we are say merging radar UH features into lens, so lens can provide both testing function and the monitoring functions together at this stage. And, uh, as for the the balancing or difference between testing and monitoring, as Kenny mentioned, especially in the case of LLM, how to customize human, say, uh, say, setting back. So so our system is in the concept that the system should follow the human rather than human has to follow the system.

In that sense, uh, we set the human annotation or human judge as a 1st priority and try to our metrics to be customized to the human judge. So that is, I say, very important point. And to make it happen, uh, we need to, or the customer need to go to go through testing phase 1st, so that all the say metrics should be aligned with the human judgment, so that and we can make use of the same custom metrics during the say.

The monitoring phase or fire firewall stage. So, in that sense, even though final goal might be a monitoring or firewall, but the to. Before going to that stage, how to test and customize the metrics is very important, I mean, very critical, uh, to protect the safety and security and reputational risks. So we recommend, strongly recommend to start from testing phase 1st. So testing phase is not just testing, but customize your metrics into your say professional, the persons, so that is a testing phase, and after that they can go into monitoring phase.

So that is, uh, our approach to the customers.

And I'm wondering, uh, you know, either one of you could answer this. But why is it? And this may be obvious to maybe more of the software engineering type crowd, but maybe less so to some others outside of that crowd. Why is it important to, once you've tested your model, to actually monitor it online? For maybe the things that are are potentially problematic inputs, whether that's a a security thing like a prompt injection, or, um, maybe something outside of the, you know, so some type of input that you wanna filter out, like IP going in or or something that doesn't fit your policy.

Why is? Why? Is it important to have that monitoring piece and not just the testing piece? Cause if I test my model and I convince myself that it can't be prompt injected, which I'm saying that sort of in jest, cause, um, you know, I, as you all know, every model, there's no perfectly aligned model, there's no every model is vulnerable to various things. But let's say that I convince myself of high performance in in one of these areas, why?

Then is it use useful and necessary then to monitor that over over time or in real time?

Technically, probably Kenny is also again the best person. But, uh, even if, uh, the customers, uh, goes, can go through the testing phase. The market condition or, say, kind of, uh, say, the human reaction may change over time. So if we were safe right now, but, uh, if anything, say something new happens today, what we say guaranteed today may not apply tomorrow. So so it's it's a very general things, but, uh, say, in that sense, the keep monitoring is very important to to protect our customers.

Uh, even if the market condition or the world condition change, or economy condition change.

yeah, and just to give a concrete example of why you may want monitoring, so we we really view them as complementary, and you really need both if you want to make a system reliable. Uh, so, for example, if you have, like, an answer quality metric that, uh, measures how how high quality an answer is. Um, you should, of course, use that for testing to make sure that it meets some, like, you know, eighty ninety bar. But then in monitoring, you actually want to measure quality of the real answers that your chatbot is giving to real customers, right?

Um, so, from a quality perspective, it makes a lot of sense from like a safety and risk reduction perspective. Um, another example is that, you know, uh, as you mentioned, in, um, during testing, you might test a a bunch of prompt injections against your system. But then in in deployment, you have real users. Some of them are adversarial, some of them are actually trying to prompt inject. They may do it in creative ways that you haven't tested before, right?

And so you may want some, uh, some kind of guardrail that will automatically detect those attempts and filter them out. Um, even if you're sure that the model, um, is robust to, you know, 90 of these attacks.

Uh, I'm curious, uh, you guys have some open source available, uh, out there, uh, I know one of the tools, uh, is LangCheck. Can you talk a little bit about your approach to open source?

For context, LangCheck is a is our open source Python library, um, that contains, um, a a suite of metrics that are built in that you can use for evaluating the quality of text. One of our motivations for creating this library is that I think we launched it in, uh, October of of 2023, roughly. And around that time, um, there weren't a lot of sort of industry standard metrics and practices for evaluating text, particularly in non English languages as well.

Um, so there was some focus on, you know, these metrics in English, uh. But we work with a lot of customers that have Japanese text, or, um, Chinese or German, and and these other languages. And we wanted to make a library of these metrics that anyone can use. And we view this as a as a pretty good starting point, right? So if you're, uh, if you just need like one or two metrics, you're comfortable writing code.

You can use LangCheck and integrate that into your test pipeline or your monitoring system. But then if you want something production scale and you want an easy workflow to design custom metrics and test them against manual annotations, um, that's where our commercial product comes in.

Makes sense, yeah. And as we get a a little bit closer to the end here, um, there there's so many things and this is such a, uh, in, you know, so many in depth areas to go in. Which is is why it's great that there's, you know, wonderful people like yourselves exploring, uh, the topic. But I'm wondering, uh, if we could maybe talk just a little bit as we close out here about what you're excited about about.

Kind of, yes, uh, citadel, but maybe the kind of general ecosystem that that you're a part of. As as you look to the future, you know what's what's exciting to each of you about how the ecosystem is developing, what's becoming possible with the technology. Yeah, what, what's inspiring to you, or, or what are you thinking about in terms of the the future? Maybe I'll start with you, Rick.

Okay, so, yeah, now the Chatgpt five is released. And but when we say, look back at today, uh, from say, five years later, I believe that, oh, this is a very premature, say model or something like that. So in that sense, the, uh, technology advancement in the AI field is so rapid, and in that sense, yeah, there may be a lots of risk coming in. But on the other hand, there are in mostly infinite opportunities. So you can find a huge, say, variety of possibilities.

Not only just, uh, say, the uh, ai, directly, uh, related technologies. But, uh, I believe some of the material, uh, products or machines or anything will change, uh, maybe within five or ten years. So in that sense, uh, we are in the midst of the say period where anything can change in that sense, especially technology related. So there are lots of, say, risks may coming, and we like to protect such a kind of risk as a company.

But on the other hand, uh, people can find many possibilities, opportunities for you to try. So I strongly believe that, uh, we. So, even though there are lots of issues in the world, but, uh, people can enjoy, or say, say, uh, can, let's say, make best use of this opportunity. Everybody. Yeah,

Yeah, that's great. What, what about yourself, Kenny?

Yeah, I think as a consumer of AI, um, in both my personal life and work, uh, from a consumer perspective, it's. It's really exciting to benefit from all the advancements in in these new AI tools and and models. I really loved O3 as a model in in ChatGpt, and I love using cursor, and I'm excited for these tools to become more and more agentic over time. And I think that's sort of the trend that you see, um, if you just look at ChatGpt.

Um, originally it was just like three, five and GPT four, that, uh, just answer a question based on a forward pass of the model. But now these models will search the internet and it'll sort of reason and think about what to search next. And as a result, the the outputs have become a lot better. Um, so really excited for that to improve even more from a consumer perspective, and then, um, from a business perspective. I'm really excited to help bring these capabilities to our business customers and help them use AI more reliably and more effectively in in their business.

That's great. Yeah, well, thank you both for taking time to to chat with us today. Um, and thank you both for the work and thought that you're putting into the tools that you're building and the open source projects that you're putting out there. It's a great benefit to the to the community and to the business world, of course. Um, so thank you for the work that you're doing. And um, yeah, we'll look forward to, uh, to keeping, keeping, uh, an eye on on what you, uh, what you evaluate and and protect us from next.

So, uh, appreciate you both and hope you have a great evening. Thank you for joining.

thank you very much.

Thank you for the conversation.

All right, that's our show for this week. If you haven't checked out our website, head to practicalai fm and be sure to connect with us on LinkedIn X or Bluesky. You'll see us posting insights related to the latest AI developments, and we would love for you to join the conversation. Thanks to our partner Prediction Guard for providing operational support for the show. Check them out at predictionguard com. Also, thanks to Breakmaster Cylinder for the beats and to you for listening.

That's all for now, but you'll hear from us again next week.