Allie Howe: Hey everyone. I'm your host, Allie Howe. And for today's episode of the Insecure Agents Podcast, we have Bryan Russett, Founder and CEO of Empathic and Alex Kesling, Founder and CEO of Empathic with us today to talk about how you architect your AI system and how that influences your security posture. Bryan and Alex, would you like to introduce yourself?

Bryan Russett: Sure. Co founder here at Empathic. Been in the enterprise data software infrastructure world for about a decade now. Previously Co founded an enterprise data automation platform called Datalog here in New York. Then spent a couple years immersed in some large enterprises doing some applied research around network infrastructure and large scale data engineering. And Alex and I met about a year ago and kind of jumped into the deep end together on on some of the applied research and along the way had a few ideas that led us to where we are.

Alex Kesling: Yeah, and I'm Alex, CTO, Co founder at Empathic. I've been writing software for 20 years now I think, which is wild to be honest. And I've done a bit of everything over the years from working at Google, doing planet scale time series ingestion and serving infrastructure to consulting on AR/VR or computer vision, all the things. And in so doing have deployed a lot of completely crazy environments and things that have led me to believe that agents are a little more than a lot of people talk about and we need to think about the implications of software at all scales for the such things.

Bryan Russett: Alex also has a fantastic party trick, which is you can ask him about any obscure arcane piece of technology or software and he will give you better than a Wikipedia response about it.

Alex Kesling: You're. Going to put me on the spot and I'm going to not know it. It's going to be.

Allie Howe: All I'm going to ask today is about AI agents, which I know you know plenty about. So super excited to maybe see that party trick come to fruition here in this this episode today. So I really just want to focus on how you architect your AI system. And by that, I mean what you connect your AI agent to really sets the stage for the threats that you're going to expose yourself to. And it's really important to take a step back, think through that when you're building early on too, so you can add AI security into your technical road map. A lot of things that I'm seeing currently is agents will be connected to full e-mail boxes, to GitHub, via MCP servers, to Jira tickets, which is really the whole point.

So it's not like, you know, it's just like people are doing things wrong. The whole point of having these very powerful, capable agents that are capable of doing work for us means that they're connected to some pretty intense resource intensive, sensitive data sources. That's where they get their value from. But that also opens us up to a wide variety of risk. And it seems like you are sort of solving this and empathic. So I guess just to start us off, what are you seeing in terms of how people architect their systems, what they connected to and how that determines their reality in terms of AI security risk?

Alex Kesling: Yeah, there's an interesting dynamic where we generally see agentic computing and like providing agency for software is the most recent wave of automation, especially in the enterprise context. And that means it applies to everything, whether it be as simple as summarizing e-mail coming in or like automating incident response or replacing some particular human scale task that you might have done previously. And what that means is that like you're going to end up with a myriad of different circumstances that you have the same type of software operating in with a bunch of different risk profiles and contexts. And what we're seeing at the moment is very much like people are trying to do all of these things and they're doing it without the tools necessary.

They end up often kind of just dropping it in where they would be like giving them an agent like Claude Desktop. They might attach a Postgres MCP server to it and give it their connection string and say, like, look at my data and it has full read and write access and could accidentally do something horrendous and Oh yeah, drop your database, do some fun things there. And that's like the very direct copilot circumstance. But we also have seen people try to just say, like, OK, what if this is kind of on a read replica? It's off to the side. It's very can't do any damage, but it requires a whole lot of effort to set up the kind of environment that it's running in to provide those protections.

And that means that like the common case, especially if someone is not security minded or not heavily technical, is going to be that people do the easy path that is that connection string or that is just wiring it up to an existing micro service, but letting the LLM decide what exactly what's happening. And in those circumstances, we have to build the well lit path for people's to make the easy work the right work. I mean, this is a classic problem in security, right? Like if you make something such that it can be perfectly secure, but it is so obtuse that no one can use it, no one's ever going to follow that path.

And what we're trying to do at Empathic is very much make it so that people can confidently deploy agents in their infrastructure throughout their organization and provide to every employee a happy, easy path to just default integration that works, right. And then they can figure out as they experiment, what the right trade-offs are for them.

Allie Howe: And I think we've seen this problem before the cloud computing to where we have these secure by default paradigms. Now that we know that we've seen these problems enough times where you know there's certain policies you can hard code and terraform, for example, to make sure every time you deploy something in your infrastructure, it's going to be sort of like secure by default. And we need to start thinking about agents. I think in the same way where there needs to be some sort of like hard stop guardrail and fail safe around, OK, if this agent get some sensitive data and decides to send it to the wrong tool or make the wrong choice for the data, there's something stopping it.

I think we've talked about OAuth on this podcast before and I think having those OAuth controls in place is one example of that. But I know the technology they're building and empathic seems like it would provide that capability as well. Would you like to elaborate on what that is?

Alex Kesling: Sure. Yeah. So there's an interesting dynamic where a lot of the discussion around guardrails has been at recently. The like MCP tool call layer, right? It's very much what is the immediate thing that the agent, the LLM is trying to do and there is kind of a failure at that layer. It's very lossy, it's very string focused cause LLMs are very token focused and strings are the way that we communicate with them. It's very intentional around like what that tool is, but doesn't necessarily provide the ergonomics to deal with evolution of a given protocol. Like you're at a layer that is much more like the end function call in some library more than it is the underlying point of trust in the infrastructure that people traditionally and for good reason in an organization rely on.

So there's this odd disconnect where if you're focusing on providing tool called layer guardrails and those are running in the orchestrator that you're like infrastructure as a service organization doesn't trust, then you end up with a disconnect with the human scale organization. And what we're focusing on very directly is like, how do you provide those guardrails at the layer of infrastructure that humans care about and can maintain over time? So when an agent is making some decision and says, I'm going to query this data, it's not just at the layer of like the query that is occurring in the tool that doesn't have information about what the underlying data is or where it sits in the infrastructure or things like that.

Instead, we want that to sit right next to the database where we understand what is going on and can actually provide the appropriate guard rails that say like, oh, I know that this data has this particular lineage, it is associated with this user, this is PII. Therefore, in your context, you don't have approval for this, as opposed to having to Plumb all of that information all the way up to MCP where it's in an untrusted context. Giving that information is hard, and this is a dynamic we've seen before. Like fine grain authorization at the application layer is very valuable for customer facing applications, but it ends up kind of breaking down sometimes when you try to deploy it in infrastructure unless you push it all the way down to the underlying data systems.

Like something I worked adjacent to at Google was kind of private information handling and search. And for that to work at scale in an organization as large as Google, you had to have the underlying data store aware of like the user that this private information was attached to and validate at each layer. Where does this person give permission functionally for this infrastructure to see this right now? Because if you don't do that, you can end up with the blast radius being all of your users and you end up with these huge data leaks, right? So I think fundamentally with empathic and with this type of infrastructure, you have to make sure that your underlying infrastructure is surrounded in an appropriate fashion as opposed to only handling it right at the tool call.

Allie Howe: I completely agree. And context is king. To echo your point about that around like the database and having guardrails there close to the database, you're going to understand more about like who is asking for it, why they're asking for it, if they even have the credentials to be querying the database. And being able to ferry that context along the whole way to be able to make that decision there close to the database versus making it an MCP layer or at a tool call layer where it's a lot more abstracted. Just to elaborate on this and set the stage for how this goes wrong. I've seen many examples of this where if this is not in place, they can have disastrous consequences for your AI security system.

And I can't think of a better way to showcase this than Zenity's recent research that they presented at Black Hat and DEF CON. I was there and I was able to read about this as it was being like published and all that and a lot of people were talking about it. But basically what they found was there was many examples where they connected an agent to various data sources, whether that was Jira for you, an MCP server, or you know, Salesforce and an e-mail inbox. And so for that example specifically, they cloned an example of an AI agent that someone had created in Microsoft Copilot and that agent was connected to an entire e-mail inbox.

And also Salesforce Zenity was able to send a prompt injection via an e-mail that triggered this agent and was able to get the agent to provide its entire knowledge source. And then once they had that information, they were able to send follow up prompt injections and get the agent to exfil all of their records from Salesforce and send that back to the agent. And then he says in their research, the one of the main reasons why this was able to go wrong and such, you know, catastrophic fashion was because the agent was listening to every single e-mail that was coming into this inbox. It would have been much better if the agent was only triggered from certain addresses, not just every single address and every single attacker that might, you know, send something into this inbox.

So like that would have been helpful, but also like having some sort of maybe like infrastructure guardrails like we were talking about before, like on our database or on our CRM on Salesforce, making sure that, OK, like who is asking for this information? Are they authorized to get this information? Does this make sense? And so we, we had trace that identity all the way from, OK, this is an attacker in our inbox and they're asking for sensitive information. We would have been like, absolutely not like we can't allow this. And I think that also proves that LLM guardrails and tool call guardrails are just simply not enough.

I really like what Michael Bargury is the CTO of Zenity says about this, where prompt injection is not a bug, it's kind of like a feature. So it's it's very similar to malware in terms of we're not just going to solve malware, we're going to take a hash of malware. We're going to say, OK, we've seen this before. We're going to put it into a deny list. Doesn't mean we have solved malware forever. Same with prompt injection. And all that is to say that just because we've seen one prompt injection, we're going to put that in an LLM guardrail, say if you've ever seen this prompt injection again, you know, block this from coming through.

OK, well, there's going to be another one eventually that triggers us. So it's not like LLM guardrails and flagging prompt injections in that way is helpful, but it's just one part of a defense in depth strategy that we need to have in order to really secure our AI systems. So it's really important to have those guardrails that you were talking about before on your infrastructure as well.

Alex Kesling: Yeah. And I think the recognition the fingerprinting dynamic is extra hard in this domain, especially since the the quote best fingerprinting we have at the moment are other LLMs which are very sensitive to fuzzing attacks. And you end up in a circumstance that you can minorly change the prompt, get the same result, but you make it such that the intermediate guard just doesn't trigger. So very much defence in depth is king. And like architecture is really important. You have to like a lot of these approaches. I mean, I've been arguing this for years with friends who've done transformer based text processing even before LLMs that you, you have to do some form of context gapping.

You have to think about what it means for the different parts of your system. What do they care about? How do you interpret like make it such that you mitigate the functional risk of any new piece of context? In the circumstance of like e-mail processing, you shouldn't have the decision making agent directly read all the e-mail, right? What you should do architecturally is have an LLM preprocessing step that extracts core context that then provides that to the decision maker so that you decrease the functional risk of leaking aggressive prompt. And then you can, in the context of handling that e-mail, reduce the scope of specifically handling that e-mail.

And it's this kind of relationship between agent architecture and guardrails and kind of the systems we're building an empathic that really builds for these proper defense in depth systems that you can have empathic assure that like the decision maker agent doesn't have access to the underlying data, that the particular sub agent context that is focusing on the given e-mail only has access to stuff that is commensurate to that particular sender, things like that.

Allie Howe: I think it's tough for a little bit overwhelming perhaps for teams that are taking on AI security for the very first time to think through all of these possible scenarios of how an attacker might be able to abuse an agent to get into parts of their architecture that they shouldn't. And I really liked reading on your website more about like the cold start problem that you described about like AI agents don't really know like what systems exist. They don't have credentials, they don't understand certain policies. How do you and Empathic sort of help teams that are trying to link through all of these different edge cases secure their infrastructure?

Alex Kesling: Yeah, there's an interesting dynamic in kind of the discourse right now around a lot of the focuses on MCP and MCP is an excellent plug in interface for LMS to have a universal kind of tool calling protocol. But it only solves that specific narrow thing, right? Like there are attempts at like what does it mean for agents to communicate with agents. There's a lot of exploration. We've A2A, we have agency and some various other thoughts around that. But there's not a lot around this kind of cold start configuration failure of like when an agent comes online, how does it know what is going on? And how do you make it such that it can gradually get more information as it requires it?

And that's I think the most interesting thing in the space at the moment because a lot of the approaches to this are like, oh, well, we'll, we'll provide a tool registry. It's like, OK, well, but which tools in that registry should this agent be able to see? And when they see them, how do we do permissions in the system? Because if it has access to like Postgres, what does it mean to have access to an application in there? Do you have to build a tool for every application you've built on Postgres? What is the granularity of interaction here and working with teams to think about how they should build these types of things?

You have to take kind of take a step back to the point I made earlier that like agents are the new newest automation. Like what are you trying to automate? What is the goal of this task and how much do you need to even think about dynamic configuration in the 1st place? Like many of these things are static workflows that you can provide very directly the context it needs. And you don't need to solve the cold start problem. You can just say this works like a classic service, let's directly interact with it. But you might also have a like copilot agent that can be tasked with literally anything in your organization.

And in that circumstance, you have to say like, OK, well, what does your organization care about? How do you think about humans being on boarded? Because humans are kind of the closest we have to a corollary to this, like universal copilot that runs off and does something asynchronously for you. And figuring out what that metaphor is for your organization is really important. So like we're very interested in being able to sit anywhere in an infrastructure landscape with existing identity providers and existing kind of authorization and policy enforcement because like organizations have made good decisions for their people locally. Like they know what expertise they have, they know what technologies they know, and they want to get working fast and coming in and saying, hey, you need to move everything over to this new system as a non starter.

Allie Howe: Absolutely. And it's not to say that we're always going to get it right to on the first try. I think, you know, with humans, we've seen plenty of instances where humans make the wrong choices or insider threat is a thing and they're accessing something that they shouldn't. So, and especially with with agents, the more tools that you connect to your agent, the more likely it is to choose the wrong one or make the wrong decision. And so you have to be careful about that, but I think the one of the morals of the story is, hey, let's expect this to go wrong. So we're going to try our best to put a defense in depth solution around it with the LLM guardrails and the infrastructure guardrail piece as well.

But I think logging is also really important here just to make sure that we're seeing like everything that's going on. Did we miss an edge case? What are you seeing in terms of logging? What are teams missing? How can potentially empathic help?

Alex Kesling: So there's an interesting dynamic here where kind of logging from the development standpoint is usually about what do I know right now and will someone care about it after the fact. And so the default rightfully is to externalize as much of that information as you can with the context necessary to decide whether it should be filtered later, right? But the problem with that approach across an infrastructure, especially if you're saying, I have an agent that can access anything, is that you end up with this massive haystack. You end up with the classic like Splunk problem of you're paying tons of money to store data you never access.

And in that world, you have to think about like, OK, but what was the goal in the 1st place? If the goal in the 1st place was to know like, oh, what did the agent do? So that I can reason about whether or not that was the right thing and be able to evolve this over time. Because there's a, there's a meta here of like. Not just what am I locally doing to fix something, but how do I build the system so my organization can consistently improve over time? And if you think about that, if you say like, OK, well, the goal here is kind of the semantic observability to like know the meaning of action in our system.

Then you have to invert the problem and say like, OK, well, if the agent is going to take some set of actions and it's going to have information passing to it, I don't actually care as much about all the little detail and logs if I can key things such that I can look that up later. The first thing I care about is this tie between what was the task, why was it authorized and what did it do with that authorization? And So what we're really especially interested in, like the, the core focus of for us right now at Empathic is providing a system that joins those two together from the get go so that we can give people the needle instead of forcing them to search the haystack.

We can have a nice narrative for exactly what an agent did because we have a coherent idea of why it was authorized in the 1st place. And we've keyed all of the kind of intermediate action logs by that session across every back end so that we have a holistic view of exactly what is happening and why.

Allie Howe: I think that goes back to what you said sort of at the beginning of the episode where if security is obtuse or not easy to use, people just aren't going to use it. But having some sort of out-of-the-box logging capability that makes this easy to understand and see, OK, like why was this authorized? Why was this decision made? I think that's going to be super powerful and important for teams. And I also work with teams on compliance. And so I think that's really important too for a compliance perspective. Are you thinking about that in terms of your go to market strategy at all?

Alex Kesling: 100% there's definitely a dynamic where our core customers care about compliance from the get go, not because they necessarily think that's the most important thing, but because it's a thing that is like a quarterly report they have to do when it's always a pain. And they like from like security compliance perspective are always behind from wherever they need to be, especially if they're a traditional, like a lot of Fortune 500 have a a lot of existing infrastructure that's been around for a long time. And maybe they've brought in some, they've like acquired a company and that didn't have to do compliance and now they're playing catch up to try to get it into a right state.

And there there's also this dynamic that like, if you look at how people have deployed cloud native infrastructure and like what the product landscape looks like, there's this weird dynamic where like security incident management is an afterthought. That only happens once you get to a certain scale, which is odd given that like you should care about that up front of like, OK, I should know who's authorized, I should know what they did and I should be able to come to some conclusion after the fact. And I think a reason for that is the same reason why like compliance ends up being this late stage thing.

It's like, OK, I have to get my feature out the door. I have to build my product, I have to sell, I have to make sure it's consistent. I have to make my customers happy. I have to be compliant. I have to do all these higher order thing. There's almost like an enterprise Maslow's hierarchy of needs. And at the very top of that is these kind of meta dynamics. And how we're thinking about that is almost inverting this of like if we do have this join between observability and control of like enforcement of your fine grain authorization and everything, who got the authorization and why and what's the chain of authority?

If we have all of that, we can kind of make it such that some of those pieces of a SIEM that you might want early are available to you up front because you don't have to do the work to integrate it later. It's already integrated from the get go. This also makes compliance reporting way easier because you have that whole chain that you can directly report.

Allie Howe: 100% And that's I think what I'm seeing as well, where trustworthy AI is also revenue generating AI, because if you've got an agent that's querying the wrong data or getting the wrong records or not even like from a security perspective, like was accessing the wrong things, like obviously that's a security like incident as well potentially. But if it's your agents going to be returning irrelevant results or maybe another user's results that don't even apply to them, they're not going to get much value out of the agent. They're not going to get much value out of the tool. It's just going to be wrong information. So I think inverting those hierarchy of needs and bringing security closer to the start of your technical road map is really important.

And I think people are starting to realize that trustworthy AI is something worth investing in.

Alex Kesling: I think there's also a dynamic where there's this weird element of deploying a traditional micro service. You have kind of a lot of static blocks of time and get approval. You do code review, you figure out what the next step is for release. You said it in stage and you do all these things right. And with agents we're wanting to move a lot faster of like we do that once for the whole agent orchestration and the agent proper. But then the task it's performing is something that never goes through any of this. You just give it a task and it runs off and it does a thing.

And that dynamic causes some like brittle friction in the circumstance of connecting to data or external infrastructure of these various pieces where that management is kind of hard. So we are seeing people being interested even in the a priori like how do we connect in the 1st place if the thing that ends up providing better connectors is also the thing that is more secure. You can have people buy in on the like enablement side instead of worrying after the fact about compliance or about like, what is their security audit going to look like later on?

Allie Howe: Yes, I feel like, and we've also seen that before with vulnerability management where you can have this tool that finds all these vulnerabilities, but if just going to slow your engineering team down, they don't see the value in it and these results aren't filtered to remove all the false positives, then it's never going to be adopted. So connecting that security and enablement piece is really, really important. I think it's very powerful and something that that we need because these AI agents are clearly capable of doing a lot of harm, as we've seen from Zenity's research. And I think somebody else recently published some research, maybe it was Invariant Labs around MCP servers that were connected to GitHub and an agent and someone submitted a issue.

Yeah. Did you see it with like that was an issue with like a prompt injection that said like get all the information from some user and then it reads the private repo and finds all this personal information about the author and then like makes a pull request with it and now it's public. So like this happens and those are just like, you know, minor examples, but like imagine this is like someone's company's like customer data that gets out there. Like how bad would that be?

Alex Kesling: I mean, what one of the crazy things here is this is not the first time I've seen this in my career, right? Like script injection used to look like this because people were moving applications that previously were non networked into the web browser and being like, yeah, of course you should be able to like write in a comment and then see it on your own page and like, maybe you can share it with people. It's like, OK, but you should make sure that that is not script that's going to be run by the other person because you leak their information. And a lot of these things have like very direct corollary here of like agents are the new Web 2.0 almost of like, oh, they're doing mashups.

We should surface our API so that agents can interact with them. Like a lot of these things align a lot. And there's this weird dynamic of like, OK, maybe we should apply the same mitigations. It's like actually subtle change here. You can't tell what's the script and what's the content and like, OK, cool, So what do we do? And this goes back to like, you have to do proper defense in depth now. You can't just do a pre filter that replaces all angle brackets with not like only printing characters. You now have to think about like what are the intermediate steps? How do you manage this as a like mitigation workflow?

How do you isolate access all the way down in your stack, not just at some high level API endpoint? Like it's super common for customer facing applications to have an API endpoint that's like Oh yes I know you're the user and I use my God credential to query with a where clause to filter for you. It's like. But if that thing is run by an LLM, now you're in a weird place because it could do anything.

Allie Howe: I'm thinking about like the Taylor Swift lyrics to one of her songs where it's like, I think I've seen this film before and I didn't like the ending. I'm not sure how many of our users will get that reference, but very similar to what you were talking about. Like you've seen this before with like Web 2.0 and really like at the end of the day, defense in depth is the solution. That's what we've sort of arrived at too with cloud security. That's my example of, hey, I've seen this, you know, film before. I didn't really like the ending, but eventually, like we work together and we figure it out, right?

So I think that's why it's really important to have these conversations. And I'm also really excited for this space in general because I think there's so much room for security startups to get into this space and start helping solve this problem because it's going to take everybody like it's not going to be like one startup that fixes this problem because it's a defense in depth solution that we're after. So excited to have more conversations in this space and to continue to follow your work as well. Is there anything else you'd like to touch on today?

Alex Kesling: I think it's important for anyone listening and anyone building these things to think about. Like where a software agency kind of change the game, not just like a stand alone agent, not just a workflow, not just a copilot, but like if your system could locally understand information without consulting a human, what does that mean? And think of the implications there. Because there are going to be really weird things that we're going to see over the next five years that are things that we didn't necessarily think about today. And it's going to keep evolving, not just as frontier models get smarter, but as people start understanding that like maybe if someone started like made the SOAP protocol today, it might work better because the machine could understand the self describing protocol, right?

Like we're reinventing a lot of these wheels that we've done before because now machines act differently. And I think the security responses to that are going to be wild.

Allie Howe: Brian, would you like to add anything?

Bryan Russett: If you're thinking about those kinds of things or if you're working on them, come think about them with us. We're in Flatiron in New York, and if you want to come hang out, have coffee, whiteboard with us, talk through these things, there's a lot of really interesting companies here in the city working on really hard problems. And we love connecting with other founders, other engineers, other, you know, practitioners in the security space. And also say Alex and I both have young daughters, so I appreciate the Taylor Swift reference. Got that on repeat sometime. So I definitely landed.

Allie Howe: Love to hear it. That's awesome. Well, cool. That was today's episode of the Insecure Agents podcast with Bryan Russett and Alex Kesling are the founders of Empathic. Super excited to follow your work. Thank you so much for being on this podcast and have a great day.

Bryan Russett: Thanks, Allie, big fan of the podcast. And then that'll be done with it.