25 Mar 2026 · 29m · Michael Krigsman, Pradeep Mannakkara, Ben Mayrides · 21 quoted facts
Traditional security tools were built for predictable software, and AI agents that reason, delegate and act on their own break those assumptions. Cvent's CIO and CISO argue that governing agents takes technical controls (identity, scope, observability), shared decision criteria and cultural change together, using the AWARE framework from Glean's Work AI Institute. Their aim is to approve projects quickly without giving up control.
The central tension is speed against safety, personified by a CIO who wants to move fast and a CISO who must keep the company safe. Cvent resolves it by encouraging experimentation in sandboxes, filtering ideas through return-on-investment checks, agreeing on criteria in advance, and treating every go or no-go call as provisional. That approach produced over 6,000 agents, and they admit only about 1,300 are actively used.
Ben Mayrides says existing identity and access controls assume deterministic systems, while agents plan, reason and delegate. Their reasoning is also opaque, which makes it hard to reconstruct what an agent did after an incident or for compliance.
Answering the same five AWARE-style questions up front replaces gut-feel objections with specific requirements. Builders can design to those requirements, so projects get approved faster.
Cvent encouraged employees to create agents freely to build AI skills, knowing many would go unused; about 1,300 of over 6,000 are active. Moderation and usage metrics arrived from Glean within three to four months.
Ideas must show business value with finance before reaching security, and experiments run in sandboxes without production data. This filters demand and gives security, legal and privacy time to vet the survivors.
Because agents, use cases and delegation chains keep changing, a go or no-go call must be revisited. Cvent's security team called Claude Cowork too risky for now rather than rejecting it outright.
Ben Mayrides says traditional controls assume predictable systems, but agents plan, reason and delegate tasks on their own. Their reasoning is also hard to trace, which makes identity checks and incident reconstruction difficult.
It is a framework from Glean's Work AI Institute for agent governance. The host lists its pillars as intent, context, guardrails, risk scoring and blocking, and ecosystem observability. Mayrides calls it a technical control structure, unlike broader organizational frameworks such as the EU AI Act or NIST.
Pradeep Mannakkara says it was deliberate: the goal was to get people using AI and understanding what is possible. They expected a mess and added moderation and usage metrics later; only about 1,300 of the 6,000-plus agents are actively used.
Cvent first filters ideas through its project office and finance for return on investment, so weak ideas never reach security. Approved experiments run in sandboxes without production data while security, legal and privacy review the risks.
Mayrides advises taking inventory of your agents, then instrumenting them for observability and identity. He says if you can only focus on two parts of the framework, those are the two to start with.
Stated by the speakers. Each quote matches the transcript, which shows what was said, not that it is true.
Cvent is a hospitality and event software company with about 30,000 customers and 5,500 employees, and has been part of the Blackstone portfolio since about 2023.
“Cvent is a leading SaaS hospitality and event software company, about 30,000 customers, 5, 500 employees, over 6,000 agents, and since about 2023, we're part of the Blackstone portfolio.”
Glean's Work AI Institute developed the AWARE framework to address AI agent security, governance and compliance.
“Glean's Work AI Institute developed the AWARE framework to tackle agent security, governance, and compliance.”
Ben Mayrides says much of an AI agent's reasoning is opaque and hard to reconstruct, and that existing observability controls do not integrate well with agentic architectures.
“a lot of agent reasoning is opaque, um, and it's difficult to sort of reconstruct those those elements as well, and we need that.”
Ben Mayrides says the ability to reconstruct agent actions is needed for compliance with regulations such as the EU AI Act.
“We need that for compliance reasons with things like the EU AI Act.”
Ben Mayrides says existing frameworks such as the EU AI Act and the NIST risk management framework are mainly organizational governance frameworks that do not go deep into technical controls for agents.
“the frameworks that are out there are predominantly organizational governance frameworks. So, when you think about the EU AI Act or the NIST, uh, risk management framework, again, they're great”
Pradeep Mannakkara says Cvent's pilot and rollout of Glean bought the company about nine months of time by satisfying business users' needs for self-service.
“one of the things that we realized with Glean as we kinda did that pilot initially and then rolled it out was that actually bought us probably about 9 months of time, candidly speaking”
Pradeep Mannakkara estimates that only 15 to 20 percent of Cvent employees had taken AI training before the company mandated it.
“we haven't baselined our employees. We've maybe we have 15%, maybe I'm being generous, I'll say 20% that, for whatever reason, have taken training.”
Cvent's CEO attended the first session of the company's mandatory AI literacy training.
“Actually, our CEO was in the 1st session. Right? Just to kinda tell you how important this was.”
Pradeep Mannakkara says Ben Mayrides's security team evaluated Claude Cowork and concluded that the risk was too high.
“Ben and team kinda looked at this and said, Risk is too high.”
Ben Mayrides says that for agents, risk assessment must focus on what an agent can do, access and decide, rather than only on model security such as training data and data retention.
“in an agentic architecture, it's completely different, right? It is about what that agent can do, what it can access, what decisions it's making”
Ben Mayrides says answering five questions related to the AWARE framework gives security and business leaders shared success criteria and reduces gut-feel objections.
“if we can just answer these, let's say, 5 questions related to the aware framework, um, if we can agree on”
Cvent created an AI intake flow with its project management office and finance organization to evaluate AI ideas.
“we've also actually partnered with our PMO office and actually created an AI sort of flow, which thanks to our partnership with our finance organization”
Pradeep Mannakkara says new AI investments at Cvent must show some return on investment in partnership with the CFO organization.
“Now, partnership with the CFO organization, you've gotta have some sort of ROI if you're talking about some new investment”
Pradeep Mannakkara says Cvent's test-and-learn stage uses sandboxes that are not full production, do not use production data and are not integrated with the wider ecosystem.
“it doesn't mean full-blown production, it doesn't mean production data, it doesn't mean integrating it with the rest of our ecosystem, which has higher risk”
Ben Mayrides says on-device inference might slightly reduce the risk of data exfiltration but could cost visibility into and control over what an agent is doing.
“on-device inference, that's awesome. Uh, maybe, um, from the standpoint of it's going to maybe slightly reduce the risk of data exfiltration”
Pradeep Mannakkara says Ben Mayrides's security team and other teams at Cvent have created AI bots to perform part of the risk assessment.
“Ben's team, among other teams, have actually created AI bots to kinda do some of that risk assessment”
Pradeep Mannakkara says he is requiring a task-level catalog of what each agent does, which privacy, legal and security teams can query.
“a task level, activity level catalog of what is an agent doing, which the business or tech or whoever can essentially document”
Ben Mayrides predicts SOC 2 will add control criteria for AI agents within about a year and a half to two years.
“I can see us seeing a control criteria in SOC 2 within, like, say, the next year and a half, 2 years, uh, we need to be ready for that.”
Pradeep Mannakkara says Cvent's agent sprawl happened because the Glean platform enabled employees to create agents.
“the sprawl, so to speak, happened because of the Glean platform, and it enabled them.”
Pradeep Mannakkara says Cvent asked Glean for moderation capabilities and they arrived within roughly three to four months.
“we also worked with Glean, said, Hey, we need a way to moderate this stuff, cause it's going to be a hot mess. And I call it within a 3, 4 month time, the capability came in.”
Ben Mayrides argues that applying the same level of governance to every agent will not scale, so risk processes should be tiered.
“if you're applying, you know, governance to, uh, the same level of governance and control to every agent or every AI system out there, your your governance, uh, capabilities are just they're gonna crumble”
Ben Mayrides explains that identity and observability controls assume deterministic software, so agentic systems need new governance and education.
“the security architectures that we're all used to are really focused on deterministic systems, right?”
Using a Salesforce example, Mayrides shows how the same agent can carry very different risk depending on whether it reads or writes data.
“That's a completely different scope, completely different risk profile”
Mannakkara explains how a platform with fine-grained security let business users self-serve and bought IT roughly nine months.
“that actually bought us probably about 9 months of time, candidly speaking”
Cvent gave everyone foundational AI literacy training, with security and legal contributing, and the CEO attended the first session.
“we're gonna give mandatory training to everybody in the company”
The Claude Cowork example shows how security can say not yet, with specific reasons, rather than a flat no.
“Well, and it's risk is too high for now, right?”
Ideas must show value with finance, and experiments run in sandboxes, so security reviews only what matters.
“will naturally filter out before it even has to get to Ben”
Agreed criteria give both leaders a way to discuss tradeoffs such as on-device inference without relying on gut feel.
“Gives us that shared language where, you know, a user comes to us”
Changing agents and delegation chains mean decisions must be revisited, so risk review must run at speed and scale.
“So, the go no-go decision is not a one and done, is my point.”
Mayrides recommends combining AWARE with larger frameworks and starting with an inventory plus identity and observability instrumentation.
“inventory your agents, um, instrument them for observability and identity.”
Mannakkara admits only about 1,300 agents are active, and says the sprawl was intentional to build AI fluency before moderation arrived.
“Now, we knew that this was going to be a bit of a mess”
AI agents are making decisions in the enterprise, but who's governing, managing them? Glean's Work AI Institute developed the AWARE framework to tackle agent security, governance, and compliance. Pradeep Mannakkara, CIO of Cvent, and Ben Mayrides, their CISO, explain.
Cvent is a leading SaaS hospitality and event software company, about 30,000 customers, 5, 500 employees, over 6,000 agents, and since about 2023, we're part of the Blackstone portfolio.
Ben, when we look at agent governance, agent security, this is a proliferating bundle of issues. What are some of the core challenges from your perspective as a CISO?
Number 1 is education. The space is moving so rapidly, it's evolving so rapidly, and understanding not only what these tools can do, uh, for us, but also understanding what they can do to us, I think, is incredibly important. Um, so education, awareness, uh, from the board level on down, I think is is is a big challenge. Uh, and, again, it's just the rapid pace and the eagerness with which organizations, um, mine included, uh, are adopting these technologies.
So, uh, that's, I think, challenge number 1, but m maybe to get a little bit more technical, um, I think that, uh, when it comes to some of the technical controls, the security architectures that we're all used to are really focused on deterministic systems, right? And as we know, AI is all about non-determinism. Um, so the architectures are broken. So, when you think about agent identity, for example, agents are planning, they're reasoning, they're they're they're, uh, and and they're delegating actions, and our existing IAM controls, um, aren't necessarily architected and built for for that right now.
Um, same thing for observability, for example, uh, you know, a lot of agent reasoning is opaque, um, and it's difficult to sort of reconstruct those those elements as well, and we need that. We need that for compliance reasons with things like the EU AI Act. Um, uh, but also in the case of a security incident, we need to be able to restr- uh, uh, you know, restructure and understand what actions an agent took, and so on.
But, the existing security controls that we have for observability aren't yet attuned, and they don't really integrate very well in an agentic architecture, right? So, we need that's where we need purp- uh, purpose-built, you know, governance, we need education, um, and we need these technical controls to mature.
Pradeep, Ben made a really interesting comment. He said, Agents are valuable for what they can do for us, but also to us. Maybe from a CIO perspective, dig into that for us and to us.
Whatever we implement, we need to make sure there's adequate safeguards in place to manage risk. And I rely on Ben, our legal team, our privacy team, to kinda help sort of shape that aspect of it, but at the same time, move fast. Now, your question around what it can do for us, I look at in terms of the value that it can bring us, right? Whether it's taking action, whether it's finding information, whether it's suggesting all those things that Ben highlighted.
I I I think we've all seen where agents can do really well, and we actually have demonstrated ROI proof of that. The things that you can sort of take to us, and if I necessarily consider it negative, it just means we need to make sure we've got some mitigating controls in place, and that can happen in a variety of ways. Uh, one of the challenges go back to your last question we have is there's a lot of vendors, partners, solutions that are out there.
You multiply that times the number of employees we have, and now we have a a lot of, uh, ideas, amazing, great ideas coming to us, and what we've gotta figure out is how do we review these things? How do we look at the value of each of these things? And as we try to do this in a meaningful manner, one of the things that we also look for is to make sure that there's financial safeguards as well in place.
Um, and then the the 2 things is, you know, I rely on Ben and other teams to make sure we're protecting ourselves. That's how I look at the 2 part.
You spoke about the various ways of looking at governance, looking at protections. The Work AI Institute's AWARE framework covers 5 governance challenges for agents. Intent, context, guardrails, risk scoring and blocking, and ecosystem observability. How do these principles map onto your decision-making as you're deciding what to work on, what not to work on, what to hold back, and so forth?
It's very relevant, of course, uh, to any sort of agentic architecture that we have. So, you know, as we actually have been a adopting the platform, it's been taking a look at, uh, alright, what is what is an agent actually? Uh, what is the identity that it might be assuming or that might be delegated, uh, to it? Um, and it's the it's the who and and what is taking that that sort of action. Um, we need to actually identify that, eh, in advance, like, what, in addition to, what is the scope that that agent is actually gonna run in, what are the data connections, uh, or the data, uh, sources that it's going to run in, uh, and the context
is incredibly important because, like, say an agent that is, um, maybe going to Salesforce and reading a a customer record for the sake of summarizing that record for a salesperson versus an agent that is accessing that record to maybe update it, um, modify it, or maybe write, uh, you know, uh, summarize that, and maybe write some of the sensitive data to another resource. That's a completely different scope, completely different risk profile, so, stepping through the AWARE framework, we're able to actually break that down and then make sure that we have the right kinds of controls in place, right?
So, it's essentially a comprehensive set of controls and sets of issues you need to look at.
In my view, it's a technical structure, it's a technical control structure because, you know, the frameworks that are out there are predominantly organizational governance frameworks. So, when you think about the EU AI Act or the NIST, uh, risk management framework, again, they're great, and they have their purpose, um, but they don't go deep enough, and they don't actually, uh, tie into, sort of, the technical controls and the the technical considerations that you need to build in to an agentic security architecture.
And I agree with Ben, so, which is why, sort of what we look for is try to figure out what's a good pave road of technologies that we can provide to our business. I'm gonna separate our call it our CTO organization, our product development organization, cause there we have a, I'm gonna use word, a mature framework in place with actual appointed people and their roles, and they work within this framework that we, uh, put in place.
On the business side, we see a lot of desire to move fast, right? I see some heads nodding in the room. And what we look for is, there's a lot of demand that's coming to us to say, get us something quickly. So, one of the things that where we look to look for is technologies or platforms that have this sort of inherent within that framework or that platform or tool that allows us to then deploy something quickly, cause we are here with Glean, one of the things that we realized with Glean as we kinda did that pilot initially and then rolled it out was that actually bought us probably about 9 months of time, candidly speaking, which satisfied a lot
of our business users' needs, meaning they didn't wanna rely on IT to do things, they wanted to kind of go and experiment and do things on their own, and we, as IT and security, didn't have a facility to easily enable that within a corporate, uh, ecosystem meaning, can I get access to my email, my Slack messages, my Box files, Salesforce with that fine-grained security controls and things like that. So, by us having the number of platforms that allow our users to achieve that, and then training, which you brought up earlier, uh, which then also security has sort of blessed the sort of the walls around it saying, Hey, if you do it within this space, you don't have to come
to us. We know the controls exist, off you go. Um, actually gave our users a ton of capability, uh, to the point where we had a AI council with numerous requests that we would have to, uh, somewhat custom do, those all sort of went off on the wayside. Now, there's sort of the next evolution of that that's coming, which is, I want workflow. But
Ben, you mentioned the term training. Pradeep, you said, surprisingly to me, that culture is part of your role as CIO. How do you make the framework come alive, which seems to me to be a a cultural aspect, and it's more than just training, right? It has to be more deeply ing- alright, so so talk about that. How do you bring it alive?
We're very bullish on adopting responsibly using AI, you know, with with all those other things that you talked about from a governance point of view. And as we're going through that, I'm like, we're very bullish on this, however, we haven't baselined our employees. We've maybe we have 15%, maybe I'm being generous, I'll say 20% that, for whatever reason, have taken training. They've learnt it on their own, they're using it in their lifestyle, but my belief is, you know, you might have a third, a third, a third, third that are interested, a third that's like, I don't believe in this, a third is like, let's see where this thing goes.
But one of the decisions we made was, we're gonna give mandatory training to everybody in the company, the foundations and a literacy, and in there, security has a piece, legal has a piece, everyone has a piece around the education that kinda talks about some of the pitfalls, uh, with AI as well. So, again, it's not a one and done, but it sets the sort of guidelines. Actually, our CEO was in the 1st session. Right? Just to kinda tell you how important this was.
So, Ben, it sounds like this is fundamentally a different kind of training than we're rolling out a new tool, and here's how, you know, here's how to use the tool. This sounds different.
It is a big mindset shift, I think, as, uh, many would probably already, uh, understand and and and have experienced themselves. Um, but I think in addition to what Pradeep was just saying in terms of that education and that cultural shift, you also have to have a structured ri- risk process, right? So, as ideas, um, you know, this mindset shift happens and ideas, new ideas are coming up and new proposals are coming up from all over the business, we have to have a structured risk process to be able to take those ideas, evaluate them, yes, across a number of different dimensions, including security, um, uh, and we have to have clear success criteria, right, for evaluating and building into those
projects, building into those deployments, and building into the operation of those those AI systems.
Also, the respect that, sort of, the this CISO organization earns, like, it has to be earned, that trust. And I think we had a recent example, right? Claude Co-work is out, right, still in sort of the it it's not GA, but it's like, Hey, it came out, people hear about it, they're like, We wanna test it. And
And Claude gives you messages when you open it up. Hey, Claude is here. You should try this now.
1,000% and they also tell you, Don't use this in a regulated environment, and so on. However, at the Yeah. Read end of the day,
the fine print.
it's a fine print.
Oh, and by the way, we're not responsible.
A now, at the same time, this is where the CISO, CIO role, I'm like, We should m have a framework to evaluate that, to understand capability, but not at the detriment to the company. Ben and team kinda looked at this and said, Risk is too high. And and and not just risk is too high as a generic statement, but here are the reasons what what decisions
Well, and it's risk is too high for now, right? Because, um, you know, there were s there, I think, a a a challenge that, um, security folks run into is that sometimes, um, maybe we're not looking at, uh, the the business, let's say, is not looking at all of the risk factors. Maybe the, you know, traditionally or up to now, um, it's been very common for the business to look at model security, right? Um, where it's about what what data is that model trained with?
What is the way to access that model? Is that model provider actually gonna retain my data, for example, and train and use it to train their their model? Uh, in an agentic architecture, it's completely different, right? It is about what that agent can do, what it can access, what decisions it's making, what what actions it's delegating, uh, what, you know, what users it is impersonating. It's all of those things, and so, whoa, let's kinda it's it's so it's not no, it's let's take a look at this and let's be deliberate about it.
Let's be thoughtful, and, um, again, you know, kinda coming back to the, uh, aware framework, what I like about it is that, you know, it sets that c- clear success criteria, um, for, you know, me and and him in particular, but also other business leaders that are really keen to adopt these tools, we can say, Well, look, if we can just answer these, let's say, 5 questions related to the aware framework, um, if we can agree on and we can get to the same answers, um, it eliminates a lot of the, uh, c- this guy gives me a hard time all the time about, Well, w- why do you not like it?
And I'm just like, Ah, just, you know, i I it's a gut thing, and that's not an acceptable answer, right? I mean, I would figure, um, e- That's a therapist, You know
what I tell my about that messaging. It's it's how you message that same response can Yeah, but be interpreted.
those those answers get to a certain level of specificity, and we've agreed on the r- the the rules. He he hates it when I s I say rules, but, Guidelines. um guidelines. Frameworks. Requirements, um, uh, perhaps,
Helpful hints.
right? Um, but if we can agree on what the answers of those questions ought to be in advance, gets us to a level of specificity, removes the ambiguity of the, you know, s and and and that, from my perspective, gets me a little bit closer from and out of the the the realm of being a gate and maybe just being a part of a business conversation and a business decision.
If I can paraphrase or simplify your roles, correct me if I'm wrong. Pradeep's job role as CIO is to move fast. Your job
Like I said, is to make sure he doesn't trip while he's got the scissors in hand, so we can okay right? So, so we so we yeah.
can agree. How does a framework such as Aware help manage this really fundamental tension? I I am sure you must have users coming to you, and they say, We gotta do this, you know? We got now, we gotta do it. How how do you manage the tension?
I think it's along the lines of what I I I was just trying to get at, um, which is that it gives us a I I mentioned a little bit earlier, having a structured risk process and and clear success criteria. It gives us a set of questions that if we answer those questions, then we can, uh, you know, when it comes to being able to greenlight a project, um, in particular, everybody knows what those what those, uh, criteria are in advance.
They can build it into the design and to the deployment and the operation of those things, and we can get to yes a lot faster.
Pradeep, when your users come to you, and they say, this, we need this now, what do you tell them?
Similar to the sort of aware framework, we've also actually partnered with our PMO office and actually created an AI sort of flow, which thanks to our partnership with our finance organization, says, Hey, there you know, we we if you wanna go and demo something, by all means, go talk with the salesperson, go see a demo, understand what the capabilities are, love it, actually, and inform us, right? I don't have to come up with the idea to implement something.
We are actually we'll learn from each other. Now, you wanna take it a step further means that, ooh, there's value, and and and I love value. So that means if we're gonna help add to the top line or take away from the bottom line, however it is that's gonna drive value, I'm in.
So, we can do whatever we want, in other words?
With our partnership, 1,000%. So so, what where that comes into is great. Now, partnership with the CFO organization, you've gotta have some sort of ROI if you're talking about some new investment, and if you believe there's value, we're gonna listen, and I've actually devoted resources specific to some of these AI, uh, initiatives, so that we don't have those backlog with the rest of what's happening in the business, right, in order to move fast. So, that doesn't have to do with security, that's just reorganizing ourselves so that we can be much more nimble to get to these different gates, which is just to understand, Hey, is there an ROI?
Is there value? Is this thing real, cause there's a ton of vendors and solutions that are out there. So, that funnel within the business teams will naturally filter out before it even has to get to Ben or some of our teams. Stage 2 of this is, I love to use the word test and learn. It doesn't mean full-blown production, it doesn't mean production data, it doesn't mean integrating it with the rest of our ecosystem, which has higher risk, and and and, you know, when you look at the checking the boxes on that framework.
So, we're like, Hey, if it's a sandbox, let's create a a workspace, maybe that's not connected to something to test that out. Let's make sure we have proper t criteria. So, not to kinda drag on, but what we've done is created different buckets based on the type of proof of concept or technology that you want, and once you get past that, that gives security time, legal time, privacy time to really vet these things based on those frameworks around risk that Ben and team have.
Ben, does the shared language that emerges from a framework like Aware make it easier to work with your counterpart?
For sure, right? So, again, it sets And and that criteria.
to ease that tension?
It well, I don't I don't know. I'm not sure I'll go we'll go that first. Right, uh,
I'll go first. far.
when when we're both under pressure or, uh, you know, to to deliver and to to help the company turn transform and adopt and and sort of reimagine itself, uh, leveraging AI, that that that's a lot of pressure, I think, for anybody. Um, but, yeah, it it again gives us a that that sh- I think I like the the how you phrase that in the question. Gives us that shared language where, you know, a user comes to us and says, Yes, we've gotta adopt this tool now.
And then I can say, Well, look, yeah, so, on-device inference, that's awesome. Uh, maybe, um, from the standpoint of it's going to maybe slightly reduce the risk of data exfiltration, um, but there it opens up other cans of worms, and I can actually refer to the Aware framework and say, Well, you know, w- problem is is that I'm gonna lose potentially some observability, the ability to monitor, the ability to actually sort of reconstruct what might be taking place with an agent's reasoning, uh, for example, if all that stuff is going on, like, on an an end user's laptop, we've gotta figure out, uh, also, not only if we can monitor that and, uh, say, if there is an action that
it that agent might be planning to take and we don't want it to take and we wanna intervene before it actually executes anything, um, you know, we need that observability to be able to enable that that kind of thing. So, again, it it helps me to just have a sort of a logical way to step through and, um, you know, along with, uh, Pradeep and other technology partners in particular, be able to talk through these issues, um, and I I like sort of the independence of it, cause it's not just Ben thinks, it's there are a bunch of other smart people out in the industry, uh, have given th- this some thought, and, you know, that, I think, as a
security practitioner, if there is any in the room or or, uh, you know, online here, um, sometimes you have to sort of, say, Well, look, this is the way that other companies do it, um, and this is the way the industry does it. That that does have, uh, you know, it it is a little bit of a credibility, uh, boost.
I'm half joking, but if Ben's bot would skip that answer, it would actually be like, Oh, okay, the bot AI bot told me it was too risky versus versus Ben, I I tease, Well, I'm but there's
on I'm on camera, so, you the know, I gotta
the there there is, uh, little bit of, you know I I call it half joking, but people do look to the responses that AI will generate, and, actually and Ben's team, among other teams, have actually created AI bots to kinda do some of that risk assessment, which I think is one of the components of that framework as well.
We've just spoken about how you overcome the fights. What are some of the real challenges when you're working together to make a go no-go decision about a model, a tool, what have you?
when we're evaluating a go no-go decision and and to greenlight something, um, is a tendency to look at, um, only part of the problem, um, and you wanna take a as holic- holistic an approach as you can, right? So, I mentioned, like, some people, they they only know model security, they don't know the agentic, uh, security, uh, and and the implications of that. Um, the other thing about go no-go decisions is that I think especially given how fast the, uh, AI space is moving, um, how the the volume at which we and other companies are going to be adopting and building agents and integrating them together into, you know, massive workflows, a and all that stuff.
The the technology is gonna evolve, the use cases are gonna evolve, the agents are gonna change, there're gonna be new use cases for them, there're gonna be, you know, uh uh, you know, new delegation chains that are gonna pop up. So, the go no-go decision is not a one and done, is my point.
It's iterative.
It's iterative, and it has to be streamlined. Um, and, you know, so he and others that are the builders and that are really focused on delivering, they need to understand that they need to y- they need to come back to this risk process, and this is where he's like, Oh, man. But that's That was but my job good. That was something is to
to say.
make make sure that I can I can do that risk process at speed and at scale. So we have
So, you so you have to And with clear to. So, you criteria. heard what he said, I mean, that you have to go back actually, and
we've got a better path, which is we're gonna basically have a catalog that he can
there you go.
query. Right? So, this is how do you simplify to Ben's point, Ben and Legal and Privacy have a need to know, Hey, what is going on in terms of your organization, your ecosystem with regards to AI? So, there is a documentation step that is an important piece if you ha- especially if you're regulated and things like that, that's going to become more of a requirement, especially if it's customer-facing, what type of classification it is. So, a lot of those elements of that aware framework are actually going in.
So, one of the things I'm pushing, or actually requiring, is not at a software level. We already have a software catalog if it's AI or not or whatnot, but a task level, activity level catalog of what is an agent doing, which the business or tech or whoever can essentially document, which then enables privacy, legal, security to query or have agent go against it, and then, based on risk, do what they need to. And if it changes
Ben, what's your advice for CIOs, CISOs to put a framework like Aware into practice?
Consider the the bigger frameworks, the the more monolithic organizational frameworks that I mentioned, um, and synthesize a a a framework like Aware with them, um, inventory your agents, um, instrument them for observability and identity. If if if you can only do, uh and focus on any part of the aware framework, it would be focus on those 2 things first, and then go from there.
This is great. Uh, Pradeep, how do we put this into practice? What should we do?
people need to partner together, but you also have to simplify the messaging. Like, the business folks wanna know, Yes, no, can I do it? Can I not? Why not? How quickly? So, you've gotta Yeah, we business have
people can't understand a whole lot more than that.
and to some extent, nor should they. Right, so so, that's the middle ground I think that's important for CIOs and CISOs, and as we work together, how do we help translate, simplify that message, give a framework that says, here's what you can do, here's what we need to work together to enable.
How is this agent governance landscape going to evolve over the next year or two?
So, from a compliance ban uh, perspective, I can see us seeing a control criteria in SOC 2 within, like, say, the next year and a half, 2 years, uh, we need to be ready for that.
I would love to see a uh uh vendors playing in this governance space, so, I don't have to work with Ben and team to configure that every single time we have different frameworks. I want an overlay, kinda like you do with security that can work across different enterprise grade players, so, enterprise grade is gonna be important.
Great. Do we have time for questions at all? Or When you started, you said how many agents? Twenty, 600?
Yeah, we have over 6,000.
6,000, okay. So, when you scaled them starting from 10, 200, what were your scaling challenges as you scaled to 6,000?
A lot of these now, we have different platforms where agents exist, which, you know, Ben's gonna make sure they're all, uh, good to run, but the a lot of these agents, uh, the sprawl, so to speak, happened because of the Glean platform, and it enabled them. Now, the Glean platform is one of the the all the different data sources and things that we have, because of the fine-grained security controls, we trust it, and the users are trained to run.
Now, one of the things should we talked a little bit about this is when we s when that started, we knew we were gonna have more agents, now, even though we have 6,000, how many are actively used? It's probably about 1, 300, candidly speaking, but what we wanted to do was we need to change that culture. We wanted to get people's behavior to understand the art of the possible. So, we encourage people to go ahead and create agents, figure out how your life could be different.
So, they went out and started creating agents, started tweaking agents, learning from them. It was purposely done. Now, we knew that this was going to be a bit of a mess, but the purpose was different. The purpose was get people interacting. Now, meanwhile, we also worked with Glean, said, Hey, we need a way to moderate this stuff, cause it's going to be a hot mess. And I call it within a 3, 4 month time, the capability came in.
We can also see metrics and what's being used, what's not. Now the moderation is in play, right? So, part of this is not to say, How do we stay one step ahead? But it's what's the goal at this point in time? It's a journey. Part of that journey was get people to create them and understand what's okay and get them excited. Now, which we may be a long-winded answer here, but, uh, what we're getting now asked for is people wanna create workflows.
They're like, Hey, great, I get the output. Now I want it to take action, and they're very hungry to go to that next stage.
So, you have agents, and then they have kids, and they move out to the suburbs, and then, before you know it, you have agentic sprawl.
Yep, great analogy. You know, the space is so dynamic right now. Again, you have to have a a good development process, a structured and scalable, uh, you know, tiered, um, uh, risk process as well, because I think if you're applying, you know, governance to, uh, the same level of governance and control to every agent or every AI system out there, your your governance, uh, capabilities are just they're gonna crumble, right? They're not gonna scale.
Okay, and with that, Ben Mayrides and Pradeep Mannakkara. Thank you, Bill.