Herman Errico Agents today, they really take action. They don't just generate text anymore. They are actually changing code in production. They're sending you emails. They're acting on your behalf.. I wanted to propose a system category that would allow builders and practitioners to align on, "This is a problem, and this is a new class of security systems to solve that problem." So with my team at Vanta, we decided to donate AARM to the Cloud Security Alliance. We're gonna be creating a benchmark to allow companies to not compete on marketing communication, but compete on execution.

Allie Howe Welcome to another episode of the Insecure Agents podcast. I'm your host, Allie Howe, and today I'm super excited to be here in San Francisco at AI Council with Herman Errico from Vanta. He's a technical product manager there. He's also the creator of ARM which is a new framework for AI runtime systems that defines what an AI runtime system must do an AI runtime security system, that is. Herman, would you like to introduce yourself?

Herman Errico Yeah, of course. Well, my name is Herman. I'm a product manager at Vanta, and I also created ARM which is a kind of system category specification for agentic runtime security, and I'm super excited to be here. Thanks for thanks for having me.

Allie Howe Yeah, no, super excited to have you here. I saw this come out, and I was just super excited to chat about like, you know, why, why was it needed in the first place, but also, like, we'll dive into some of the policy that you've got built into, you know, how to, how to runtime security solutions, sort of evaluate policy, like, on the fly- Yeah And what's needed there to make a decision on whether, what an action... an a- a- agent can do or not. So I guess let's start with what inspired you to create ARM or ARM as we're calling it. Yeah. I know that people are calling it different things now.

But what did you see? Was it at Vanta, somewhere else, like in the ecosystem? Like, what was missing, and why ARM?

Herman Errico Yeah. That's a, that's a great question. If you think about the security industry is nominated by a large set of problems. And they will vary in intensity based on your role, your organizational industry, but everything really revolves around security categories. Once you identify a problem, let's say you have a malware problem, what you would wanna do is implement an antivirus, implement an EDR. Or if you have I don't know, an access problem, you might wanna implement single sign-on and different types of, like, identity and access management policy. I was, like, evaluating the space as part of, like, a technical research at Vanta. I was really looking into the details of like, okay, what is that system category for agentic runtime security?

And the more I dived into the, the topic, and then I really found that as of today, there's no system category for agentic runtime security. I know some categories are emerging. Think about AI gateways- AI firewalls agentic runtime security and detection. But those are really borrowed frames from different security tooling that don't really work in in the agentic runtime security setting. 'Cause agents today, they really take action. They don't just generate text anymore. They are actually changing code in production. They're sending you emails. They're acting on your behalf. We, we, we've seen, like, you had Peter Steinberg the other day. Yes. And, he really revolutionized the kinda, like, agent adoption with OpenClaw, or I don't know, what is the latest name today?

Yeah. OpenClaw. I think it's still OpenClaw. OpenClaw? Yeah. Cool. There's, like, NanoClaw, now TrustClaw, all these different claws. Yeah, yeah, yeah. So in that context, we really have a gap. And I wanted to propose a system category that would allow builders and practitioners to align on, this is a problem, and this is a new class of security systems to solve that problem.

Allie Howe Yes. Amazing. So this is something you saw while at Vanta? Yeah. Amazing.

Herman Errico Yeah. As part of, like I'm a product manager at Vanta. I build frameworks, and as part of that, I will have to go and find out what is the, what are the security practices that people need to put in place in terms of, like, building their security program, and that's where I kinda, like, identified this problem.

Allie Howe Amazing. Is there any security vendors out there today that are solving this or have, like, the ARM methodology, like, in place? Or-

Herman Errico Yeah. Yeah. That's a great question So this type of system category is slightly different from the traditional ones we're used to. Traditionally what we see is there's some industry trend that happens. Back in 1987 a guy wrote a paper on the computer virus. Fast-forward 18 months, five big vendors started building the antivirus category. One of them was McAfee. And that kind of like defined the system category for like antivirus. In our case, what we did is we build this specification, and then we also defined a set of conformance requirements. Right now in our builder registry on ARM.dev, we have as of this morning, 53 vendors building in this space aligning to the specification, but only four have completed a conformance review-... to actually validate that their solution actually solves the problem.

And those are Noma, Formal, RunLayer, and Operant AI. So we could say- Yeah... or we could argue-... that as of this morning, there are four companies that have built a solution that actually solves this agentic runtime security problem.

Allie Howe And this is something the CSA just adopted too, or you donated it from Vanta to the CSA.

Herman Errico Yeah, exactly. So, obviously Vanta's a vendor, right? Yeah. We are building solutions in this space. We are thinking about expanding in certain areas of agentic control plane, a- agent governance. Obviously we wanted to be some- in, in some ways independent-... from like the creation of this category. So the only direction for us was really to donate or find a the right home for for ARM. And, based on a conversation that I had with with Jim and Daniele, which are respectively the CEO and CTO of Cloud Security Alliance, we then really aligned and understood that the objective that they had in terms of like defining and building the agentic control plane-... really aligned with where we wanted to go with ARM.

So with my team at Vanta, we decided to donate AARM to the Cloud Security Alliance, and now it's a project that is like formally being part of like their research organization.

Allie Howe Amazing. That's super cool. Yeah. What will ARM be able to get at the CSA that maybe it wasn't before, like more reach or more people working on it? What do you see, like the next six months in terms of like adoption timelines?

Herman Errico That's a great question. If we already look at a trajectory of ARM and where it started I published the first ARM paper on the 9th of February, and three months after, it's already been adopted by Cloud Security Alliance. There's 53 companies adopting this framework. So I think adoption in terms of, like, the total number of vendors solving in the space will probably stall, 'cause obviously you don't want 1,000 vendors solving the same problem. Competition will obviously play a key role here. I think where the industry's gonna go, or, well, at least my hope or the direction that we would like, is some level of we're gonna be creating a benchmark to allow companies to not compete on marketing communication, but compete on execution.

Something that we're already working on, w- it's gonna be released in the next couple of months, so that ARM companies can first do a conformance review to validate their product actually solves the problem, and then we're gonna be applying a benchmark to allow them to validate, "Hey, my detection rate of X amount of threats is X percent." And that gives the buyers a tool to better evaluate which solution fits, fits my need and the complexity of my organization.

Allie Howe Amazing. Yeah, and I imagine if you're building AI, as most people are, and then selling their products, you could adopt an AI runtime solution that is conformant to ARM. Yeah. And that's something that you could show and maybe put in your Vanta Trust Center.

Herman Errico Yeah. Yeah, that's definitely a, a, I think a an interesting idea to show even as part of your SOC 2. Part of... One of the sub-services organization for your security services is a ARM solution that helps you keep your agent running in a secure way. Because if, if we really go back to the fundamentals of the problem, agents, well, LLMs are non-deterministic, solutions. Now, if we then go into the details of what that non-deterministic means in terms of number, we have a hallucination rate that goes between 12% to 36% to 50%. So we really have to imagine that four in 10 of the decisions being... or the actions being taken by an LLM will most likely be some level of hallucination. model alignment will solve that at some level, but in a runtime, the hallucination rate does present itself, and if you don't have a solution that intercepts the action pre-execution, evaluate the action based on session context- and then decides can this action occur, if you don't have that type of solution, then you will most likely have... put it in numbers.

If you have 100 million actions-... running every day, which is absolutely possible, you'll have 12 million actions that are most likely wrong malicious or going in the, wrong direction.

Allie Howe That is an insane amount of wrong activities happening inside an organization. And where one of those wrong activities could be drop a database or, like, you know.

Herman Errico Yeah. S- and even if I say I wanna be very, kind of, like, conservatory-... and say, let's say that 0.1% of those actions are very disruptive, you still have a large amount of problems happening at an organization that normally they would not happen. Yes. So, that's, like, the, the reason why model alignment plus agent runtime security are really the, kind of, like, two cornerstones of how you would deploy agents safely at an enterprise scale.

Allie Howe That makes sense. I think what has been difficult for people to nail down, whether it's in products or frameworks or new protocols, is, like, how do we define an action as bad? Like, what's the context that we need in, in order to determine, okay, like, you know, this agent has gone off the rails, or this action shouldn't be allowed. I was really interested in a lot of, like, the policy that you had within ARM, the different, like Yeah... performance requirements. Like R4 caught my attention, for example, around there's all these different, like, authorization decisions that you support. Yeah. It's not just allow and deny- Yeah... but it's also modify, step up, and defer.

Can you talk about what those mean?

Herman Errico Yeah. In a binary world where either the action is allowed or not allowed that works with traditional machines because you can define a very static world where if user is, does not have admin access, then don't allow, right? But then in this case, we're really talking with, with machines that are really operating like humans. Yes. The human spectrum of, like decisions is not, it's never black or white. It, it is really varied, and the reason why we, we defined those, like, five dis- different classes of actions, it's because in some cases, some actions are obviously allowed because of, like, the agent has access to certain systems, it's performing in the right direction, therefore it's allowed.

Or obviously denied. the agent has tried to delete a database. Great. That is certainly something that you should not do. Then I'll touch on, how we can make that decision. There are cases where you need a human, you need human intervention. Therefore, you need some level of, like, human approval- to decide can the agent do X, Y, Z. the system maybe doesn't know or is not sure or really requires approval, then you need to step up to the human. Modify is where we talked about hallucination. Yeah. So there's a trap called confused deputy where the agent starts on a, direction, then gets additional context, gets confused, and goes in a different direction.

Now, sometimes it's, advantageous even from an efficiency perspective to if the agent is deviating from its original goal, rather than just blocking the action or sending back, you can, give additional instruction to say, "Well, your original goal was to, update a copy change. Now you're trying to change those additional files. That is not the direction I wanted to go." Therefore, that modify action really gives you that flexibility. And I think the defer is we want to build some flexibility into the systems. The... agent starts to perform an action, we start evaluating them, and then it starts to get flagged as this action is potentially problematic, but I'm not sure yet, so I'm going to continue to analyze the action, but I might need context.

So the action is currently blocked, and it's deferred to a second moment where you might either you'll get more context, the agent is gonna go and review the action. So we want to like to, wanted to create different patterns rather than just like it's either blocked or allowed.

Allie Howe That makes sense. How do you know the context that you have and to make those decisions is trusted context? I know like today it just seems like the context layer is so dynamic in terms of where agents can get information r- from, whether it's internal sources, external sources. Yeah. I just read this paper called like AI Agent Traps from DeepMind, and they talked about how something, you know, it's not very obvious, like it's not like a true like prompt injection, but if you get into that context layer from multiple sources saying, you know, X product or X person or whatever is like the industry standard, that you can evoke like a cognitive bias within the model- Yeah To like prefer that over something else.

It's kind of like the golden gating like thing that the Anthropic talked about when they changed like the model weights or like showed how if you change model weights then you could get a model to elicit like preference towards one thing versus something else. Is this like example where like, you know, the, A- ARM could help potentially with helping us decide like what context can we trust or how do we like... I guess, like what's your strategy, sorry, like for ARM around like evaluating context and determining what's safe to use to make those decisions?

Herman Errico This is the most important topic of ERM and even the most complex. I wanna differentiate a couple of topics. The first one is ERM the, the objective of ERM is to help both builders and practitioners to align on a problem and define buckets of solutions. The actual implementation on how you do it from an architecture standpoint or how you go about building your secret sauce in terms of like evaluating context will be still left at the actual impl- at the actual builder level. I know of different implementation that I've observed with the different conformance reviews that we run. Obviously, without getting into the detail of like what what is the secret sauce of certain companies, but We need to get to the organizational domain.

To really being able to say, "Is this context trusted or not?" Companies will need to start defining, for example, a l- skill registry. Skills need to be centrally defined and centrally organized. And then if the context f- that the agent has gathered comes from the outside, you need to treat it as untrusted. Yes. And then put in a reasoning layer on top to evaluate that context. Obviously, you're not gonna be able to, like, m- make a perfect system from day one, but obviously there are ways to r- narrow down the scope of analysis. The first one is user access management. Once you start reducing the scope of actions or even MTP tools, skills, and actions that the agent can do, you immediately have reduces re- you've reduced the risk.

And then you'll define sensitivity layers of like this agent acts in a very untrusted environment because it's an agent that goes online, might get false information, might get prompt injection data. So therefore, this agent has a level of sensitivity that we might increase the level of scrutiny on what that agent does and receives. But in other cases, we will apply a different level of sensitivity because maybe the agent is essentially gets information from very trusted sources, only the skills that the access to are just like organizational skills, et cetera. So that's how I'm kind of like seeing it. So to summarize, it's really a matter of like the builders will define what they the, their kind of like, their secret sauce.

It's a matter of like, it's an organizational problem. And then you need to narrow down the scope of what the agent does to like have a complete solution.

Allie Howe That makes sense. Yeah. I keep seeing the parallel where like agent security very, feels very similar to human security, where it's like, okay, I reasonably trust my coworker to make these decisions, you know, like, you know, 99% of the time. But again, like they're human, they're fallible. Like we should check that their actions align with like the larger mission- Yeah... or the larger like policies of the company like in hand. So I feel like that's kind of the similar like thing that you're describing.

Herman Errico 100%. And one additional observation that I've made is We're building solutions that in some ways will dictate the behavior of machines that maybe at some point will be sentient. It will have some consciousness. So maybe we are ne- we're no longer in the kind of like nor- traditional security space. It's more of like a nature space. Human resource security, if you like. And another area that we, we've thought about is defining tenets. 'Cause obviously you can say, "Hey, these are a million policies that are either allowed or not allowed in your organization." Obviously, you're not gonna be able to cover all the use cases. That has been a an issue with data loss prevention.

Data loss prevention to have a great implementation, you need to probably invest a year defining policies, fine-tune them. Takes a long time because it's, like, very deterministic, and it's, like, all regex based. In this case, we're more talking about something that is non-deterministic. You need to define rules. An example could be a skill that teaches the agent of, like, it is never cool to delete a database. Yes. And that's how you... I have a small child, and to me, it's kind of like I will never give my child a million of instructions. I will teach him principles on how to behave in certain scenarios. And then as the, my, my child grows up, he's gonna start learning on how to be kind, how to, like, respect other people, and this is what I think we should teach at runtime to agents as well.

Allie Howe Absolutely. 'Cause, I mean, you can never predict at inception of that agent that all, all the scenarios it's going to get into, all the things it's going to need to solve. So you can't just sit and have, like, write down a list of all the different ways- Totally... that it could go wrong. Which brings me back to kind of like policy. 'Cause in terms of, like, writing good policy around what an agent should or shouldn't do, how do we write good policies such that we're reasonably confident that this agent's going to do the right actions in whatever scenario that it ends up in? 'Cause I think, like, take GitHub to make it a concrete example.

Like, if I've got my agent and I don't want it to ever take destructive actions in GitHub, I have to go and enumerate, like, the different ways that it could do that. Maybe that's, like, delete. Maybe that- it- that's taking private contacts from a private repo but publishing it in a public repo. Humans have historically just had a lot of challenges, like, thinking through every single use case or edge case there and, like, well, how agents could go wrong. So how do you create good policy such that you're reasonably confident that you've got all the bases covered? And is it fundamentally, like, impossible to cover every edge case, even with, like, an LLM or a human?

Herman Errico Yeah. I do hope that ERM companies do build solutions that will get us to a 97, 98% detection rate. However, that path to get there is gonna be very long. Yeah. How you write good policies is You just don't write good policies. And, that is really a counterargument to say, if you don't want an agent to take destructive action-... you should block that action from even occurring. And that is, you need to have strong interception in place. An example is In, in the ARM implementation, there's four different layer in which you can apply you, you can apply the kind of like interception of the action.

It could be done at the kernel level, SDK there's Harness. There's, like, different ways to, like, intercept the action before it occurs. There is also SDK. You wanna create a, the deepest possible layer of interception-... so that when the agent is traversing a network and going and performing a certain task, you can just block it. That's the best way to really apply certain policies. Because what we have seen-... is that agents are smart. Yeah. They realize that there is a there is a guardrail in place, there's a security control. They are goal-oriented. They will try to go and circumvent that goal. So for example, in certain a- at application-level control will be bypassable because the agent can just say, decide independently to go and build an API and call a terminal and start building something that goes around the interception.

So I think going as deep as possible with the implementation is probably gonna be the right direction to take. And, that's how I kinda like thinking about how you can, like, build strong policies.

Allie Howe That makes sense. And then in R5, in Arm, I thought the adaptive policy- Yeah... Was really interesting concept where there's this concept of, like, semantic distance tracking where you can figure out the similarity between, like, an action that's occurring versus its, like, stated intent, and be able to adapt a policy dynamically in real time. Yeah. That's, like, super, super interesting. How is that done in practice?

Herman Errico There's two companies that have that have done this. I don't wanna mention who they are because I don't really like to take part in terms of, like, this company's better than the other one. But what I've observed is that certain companies have gone the extra mile to, like, develop models to really statistically evaluate where, what were the type of actions that have been occurring in the organization that took a certain pattern. And then they do really pattern recognition to say, if you were making a copy change, you were expected to take these steps and call these tools. Right now you're making a copy change, but you're trying to do these additional things.

That and then you can really calculate the distance between these are the patterns that were taken so far, this is you're taking a different pattern, and then you can measure the distance between those patterns and say, well, this, even if it's within policy, you're actually doing something that it's not expected. And then maybe that's where you can apply human in the loop-... or you can defer the action, or even just modify the action to go, go back to your original goal. I'm gonna give you an example. Yeah. Friday, I was... Well, I should not be making production changes on Friday, but hey. I was making a simple copy change on a feature that we're building, and my, I was using Cloud Code.

My terminal had disconnected, and then- When it did reconnect, it obviously didn't bring in the recent change that I made. So when I did make some additional changes, then Claude realized that there was a diff between the current version that I submitted and one that I just made. So rather than flagging it to me... Well, it actually did flag it to me, but I added on auto-approve because I was just making a copy change. So what Claude did is it replicate all the changes that happened in the last 45 minutes, which was roughly, like, 241 changes and 10,000 lines of code. So it did replicate 10,000 lines of code and create a pull request, which was enormous for a copy change, because it wanted to reconcile all the changes, changes that it did observe.

Now, obviously, nothing happened because that pull request was closed, and that's perfect. But you can see that agents, sometimes they will get a different interpretation of what's going on. And sometimes with auto-approve w- we have, like, we have this, like, approve fatigue, where we have a lot of, like, approval to make, and sometimes we just go, okay, y- we go YOLO mode and- Yeah just approve everything. But, without certain, like, patterns of, like, if we had distance tracking, we could reobserve. Normally, if you want to make a copy change, you're expected to change up to six files, as an example. In this case, you're changing 241 files.

That action should have been blocked by a system to not allow you to go and create that pull request. Yes. So that is, like, a concrete example on how semantic distance tracking could work.

Allie Howe That's so interesting. And I bet that same system could probably capture, like, okay, it's stepped up to a human. You approved the same thing, you know, maybe 10 times. Do you want to consider, like, allowing this in the future?

Herman Errico Yeah. Potentially, it d- a system, an ARM system, if well configured, could really remove the vast majority of approval fatigue that we have today, which are which are creating two problems. The first problem is that people are just auto-approving everything, and that's a huge mistake because we've seen what happens when you auto-approve. You either get 240 change, you notify a bunch of people, and you're gonna, like, you know, you look like a fool. But also, you can have disruptive action, which is something that you definitely don't want.

Allie Howe Yes. Oh, for sure. Yeah, I think we saw something, not necessarily related to, like, consent fatigue, but I saw the, the Railright incident where the, an agent, like, dropped a database. Yeah. An ARM system would have definitely, like, been super helpful there, I feel like.

Herman Errico 100%. I do still think that there is some basic level of security that you can put in place-... before even going, I need an ARM solution. I see a lot of, like hype in the market of, like, implementing all these fancy new solutions, the newer tool, the most advanced one. But sometimes even... An example is in, in that scenario, obviously I don't wanna go and critique what happened because I don't know the exact data. But obviously having different layers of approvals would have probably reduced the risk of that occurring. So sometimes just, like, putting, putting basic security controls in place will definitely reduce the scope of those disruptive actions.

Allie Howe Amazing. Yeah, that makes sense. Where do you think that runtime security generally, like, is heading in the next, like, 6 to 12 months? Like, is this, is this ARM system something mostly for security vendors to go and adopt and share with the world? Or is this something that, like, teams that are trying to secure their AI internally can go and implement?

Herman Errico Okay. I, I think it's very hard for someone to, like, even give predictions- Yeah... in this market because in six months everything could change. We might have deterministic AI and maybe that kind of like removes even the need for... I don't think it's ever gonna be the case because, like we've seen, like, security is pr- is a problem that has never been solved entirely. Every organization that is deploying agents that are taking actions should absolutely consider to have a agentic runtime security solution, such as an ARM solution, in place. 'Cause otherwise you're just running at full speed with a car with no seatbelt on with your eyes closed.

Totally. That is the reality. Yes. So I definitely see Two things: commercial adoption, but consumer adoption as well. I had to install an antivirus on my dad lapso- la- my dad's laptop because he was kind of, like, browsing around in the internet, right? And then he was get- always, like, getting some level of malware. I was downloading stuff from the internet. So I really had to go, "Okay, I'm gonna buy him a good antivirus." And then I think at some point my dad, my dad uses ChatGPT. He loves it. But at some point my dad is gonna probably gonna go agent mode, and then he's gonna need some level of, like, consumer level ARM solution in place, which might be either embedded into OpenAI, for example.

Yeah. Or just, like, install on his local machine like you would install McAfee or Sophos Antivirus.

Allie Howe That makes sense. So this could be something maybe, like, the labs implement or create, but also, like, maybe vendors- Potentially... or also just, like, anyone building, like, AI agents could- Yeah... you know, implement it themselves or make sure they have some solution in place.

Herman Errico And you, you might absolutely see the implementation of ARM in a Waymo. Yes. Or in your robot that does your house chore, 'cause obviously imagine that the robot starts walking the dog and then the dog is kind of, like, stuck in a pole and just, like, continues to pull. That is, like, a action that unless you, like, build that runtime protection in place- it always, like, creates that risk of a autonomous action adoption. That's why the objective ARM was not to frame it around agentic but around autonomous action. We wanted to, like, keep it broad because the use cases could be multiple and hopefully that's gonna be the case in the future.

Allie Howe Absolutely. 'Cause I think the reality today is most organizations aren't even at that, like, full autonomy level or anywhere close to it.

Herman Errico No. No. I'm starting to see some very interesting signals from the industry. AWS Bedrock runtime has created a secure enclave where you can run your agents right in locally on this, like, secure server. NVIDIA has built a runtime solution as well. So the industry is definitely going in the right direction. We're absolutely, I'd probably say 1% within autonomy 'cause, like, until we have, like, full AI employees that are, like, taking actions and no one's instructing them-... they're, they're being given a set of responsibilities and tasks, we're, we're very far from autonomy.

Allie Howe Yes, no, we should, we definitely are. 'Cause yeah, agents today I feel like are lacking autonomy and capability to actually go and do meaningful work, and it's usually because they don't have the security controls in place. So hopefully something like an ARM solution or implementation will give agents what they need to be able to actually go reach their full potential.

Herman Errico Yeah, there are really a lot of smart people actually building these solutions. So I do think that they will be able to build something in the space that will increase AI adoption. They're absolutely well-funded. I think roughly ARM companies have gotten a quarter of, of billion in funding so far. So there's definitely like some level of, like, investment in the space.

Allie Howe Amazing. Super cool. Yeah. Was there anything else you'd like to share with us about ARM or Vanta or anything else before we wrap?

Herman Errico Yeah. Well, for sure, like for every person that is interested in getting involved with ARM, we have a working group available at Cloud Security Alliance. The co-chair of the working group are myself Akul from Microsoft Diana from Noma, and then Chris from Zenity. So we've, like, created this, like, group of people that are actually building in this space. So you're all welcome to join and participate. If you're looking for an agentic runtime solution, just go to the builder registry on arm.dev and you'll see some amazing companies in that space.

Allie Howe Amazing. Super cool. Yeah, definitely wanna continue to stay in touch and follow ARM. It sounds very helpful and something companies should definitely go take a look at. So thank you so much for coming on the podcast today.

Herman Errico Thanks you for having me. Really appreciate it, and hope to keep in touch.

Allie Howe Thanks, Herman.

Herman Errico Thank you.