Insecure Agents — Episode 28: Ian Webster, CEO & Co-Founder of promptfoo
Podcast: https://insecureagents.com
Listen: https://podcasters.spotify.com/pod/show/insecure-agents/episodes/Ep--28-OpenAI-Acquires-Promptfoo-Ian-Webster--RSAC-e3habnn
Transcript (auto-generated, lightly edited for clarity).

Allie Howe: Hey, my name is Allie Howe. I'm the host of the Insecure Agents podcast. Today we have Ian Webster with us, the CEO and Co founder of Promptfoo, which was recently acquired by OpenAI. Super exciting. We're in the middle of RSA here in San Francisco and I'm super excited to chat with Ian about the recent acquisition and what made Promptfoo successful. Ian, would you like to introduce yourself?

Ian Webster: Yeah. Thanks, Allie. Great to be here. Co founder, CEO of Promptfoo, also the original author of the tool. So started the open source project at this point over three years ago and happy to answer any questions that that you have about the the journey and, and, and how we got here.

Allie Howe: Yeah, absolutely. Super excited to talk about the journey. But just to recap, so it closed like a week ago.

Ian Webster: That's right. Yeah. So the the deal closed a week ago. I think we, we announced it slightly beforehand the previous week at the, the, the team is at OpenAI now just working on the integration, keeping the the work going on Promptfoo and so forth.

Allie Howe: That's super exciting. I think I read that you're going to be using your technology in Open Frontier or Frontier AI. What was it called?

Ian Webster: Yes, OpenAI has an, an, an initiative called Frontier, which is really like like a product stack or or packaging that, that puts together a lot of the great things about the OpenAI platform for the enterprise. So obviously when you're thinking about a vertical platform for deploying AI in the enterprise security compliance, government, like all those, all those questions come up. That's a big part of what my team is is going to be working on.

Allie Howe: Amazing. Yeah, 'cause you Promptfoo'd it a lot. You're bringing a lot of technology to OpenAI from. Looks like the a little bit of like ML SecOps or like model provenance, like being able to understand what models are safe to use. But then also you did the LM red teaming evals and then also helped on like the guardrail side as well. So like a full suite of tooling for the entire life cycle for AI application development it seems like. Are you bringing all of that to OpenAI?

Ian Webster: Yeah. So OpenAI bought the whole company, the products, the IP and so forth. So I expect to like, I, I think we'll be contributing to, to pretty much everything under the sun there. I will say our bread and butter with Promptfoo was and is red teaming the the adversarial testing or offensive testing as well as we have a huge open source community around evals, which is not in the security space, but just general like product quality functionality, that kind of thing. So that has always been our wedge. And if you ask me, I would say we have always done that much better than anyone else.

The other aspects of our platform like guardrails for example, or some of the code scanning and so forth, those kind of followed from like what do we see customers want after they have performed the the sort of offensive testing.

Allie Howe: Amazing. And then you said you started with an open source project. Was that just red teaming or what did that do?

Ian Webster: So I started the open source project while I was at Discord and at Discord my teams were responsible for launching a bunch of different Gen. AI features. And we actually first started using Promptfoo for general purpose evals like product quality evals. This was in like the GPT 3. 5 era. So it was like, is the AI behaving? Is it doing what what you wanted to do, that kind of thing. So Promptfoo started as a project, not really in the security space, but more just in like, I guess you could call it MLOps or generally now just like AI development. It quickly became clear to me at a Discord that like AI security was going to be a huge problem.

And like Discord is a great company. It's relatively small and nimble. Like even at a company like Discord, we faced a lot of questions about like, hey, is this AI going to be secure? Is it safe for for the end users? Are we like complying with policies, that kind of thing. So it's really like if there was that much difficulty at Discord, I can only imagine what my bank is going through or like my hospital or that kind of thing. So that's when I started thinking about like what is the most valuable type of eval that can be run and landed on on like those adversarial evals and that that's really where like red teaming and the whole security go to market angle came about.

Allie Howe: Amazing. Yeah. I think that's a really key part of your story, how you started with the red teaming, but not just from a security perspective, but from a performance perspective. So I think a lot of people and companies obviously want the adversarial guardrails, but they also want guardrails around, OK, like I'm Ford, I don't want to recommend Tesla. So how can I test for that? And so if you're appealing to not only security, but to maybe product leaders or engineers, I think that's a really unique part of your story. And maybe that's, you know, one of the reasons why that you've been so successful is because not only are you appealing to security, but people that care about performance.

Ian Webster: Yeah. I think, I think that's accurate that that is something that we we've approached this a lot differently from other solutions in the space where we started with that wide, I say evals, which is a bit of jargon, but on the AI side, evals is just like a fancy or short for evaluation is just kind of like a fancy way of saying we're going to send a bunch of inputs and look at the outputs and we're going to do this systematically and in a data-driven way. So I think that that under the hood, any good red teaming solution in the AI context is going to have to take that approach until like we managed to get that right just from a framework perspective, from like like an open source and community perspective.

And that gave it a lot of momentum that that meant that hundreds of thousands of developers were using this for for for their evals in AI development. And you know, inevitably when and the security team is like, all right, so you know, like did secure, how are we going to get this to production? Promptfoo was a tool already in their toolkit.

Allie Howe: Amazing. Yeah, that makes sense. I think appealing to developers is like a secret weapon for security companies that they should really lean into more. And we're at Snyk Space today, filming out of there. I really appreciate them giving us the space and I think that's probably a key to Snyk success as well. A security company can't just be a really great security company. It's got to be usable. It's got to something that developers don't hate, that they want to use, that they see value in. And the things that developers care about might not be the same things that security teams care about. So if you're able to tell a developer, hey, if you use this tool, not only are you making your code more secure, but you're making it more performant, that's a huge one.

I think that leads to it like a lot more adoption. And I think you did a really great job of also educating developers. You guys saw you had that Promptfoo evil MCP server that you all created to show the community, OK, here's all the scary like security concerns MCP has, like rug pulls and tool poisoning, all of these things. And so how would you say that contributed to your success, that education aspect?

Ian Webster: Yeah, I think so in, in the earlier days, I say early days, we're talking like two years ago, one 1 1/2 years ago, like promptly remained an open source project. So I started three years ago. It was open source for like a year and a half I guess at this point close to two years ago. And then and then I began to commercialize it. At that point about two years ago when I was commercializing it, I was still having to explain to people what is the concept of AI red teaming. I was having to define evals for people. Like all this stuff that I mean, it's still very much a moving target, but all this stuff is very new.

So there was a ton of education and this is evident in like the details and the documentation that we released. And then I also experienced a lot of that, like when people would come to us and say, Hey, I'm interested. Like there was, there's kind of this whole upfront educational component. I would say one thing that worked really well is we we just have very extensive documentation examples. We've tried to do as much as we can open source and that's the only way that I know how to do things. But it turned out OK from that perspective.

Allie Howe: Amazing. Do you credit the wide adoption of Promptfoo to the open source community or was there any other like go to market strategy like events that you deployed?

Ian Webster: Yeah, no, I, I wish I knew how to do event marketing or I wish I were good at networking and, and I don't know, cold emailing or like whatever it is. I, I don't know how to do any of that. Our pipeline was almost 100% inbound really attributable to open source. I think the reason why that worked so well is there's a lot of snake oil or like smoke and mirrors in both AI and cybersecurity. And when you put those together, there's just so much noise out there that having something that's open source, that developers love that that security engineers can, can like take off the shelf and start using and getting value out of that speaks volumes.

And I think that, I mean, my, my approach here was just to, to, just to like, do it and, and kind of like show, don't, don't tell. And that's, that's what made it like a that that really helps set up the trajectory on the business side as well.

Allie Howe: That makes sense, and it seems like it's very easy to try Promptfoo as well. You're exactly where developers already are, and like MCP agent frameworks, the CI/CD like that's very easy to try and see value quickly.

Ian Webster: Yeah, it is. And there's, there's a single command that you can run with, with, with NPM or NPX and most developers will have that installed. So we tried to make it as easy to get started as possible. And we would, we would measure and optimize the time to basically the time to the aha moment where someone can see a result that makes them say, shit, maybe I should take this AI security thing more seriously. I mean, looking at it there, there's still a bunch of places in the product where especially on the configuration side like that is too difficult and too slow. We did put a lot of effort into improving that and just trying to make it easy to onboard.

Allie Howe: Yes, that makes sense. I think your platform's like particularly powerful because the red teaming insights can influence the guardrails in real time. In that feedback loop, I think is what separates, you know, one LLM guardrail company or a red teaming company from another because there's a lot of like snake oil, like you were saying, like some people will just have very rudimentary red teaming that just, you know, checks for the same like hate speech or like routine issue that I think that's very well commoditized. But what a lot of people aren't doing is allowing for those like custom guardrails around I'm Ford. I don't want to recommend Tesla.

Can you test for that? But also can you take my red teaming results and have that influence my runtime security?

Ian Webster: Yeah, I always tell people that's kind of the Holy Grail. Regardless of whether you use Promptfoo or you, you prefer some other solution, you're going to want your offensive testing to feedback into your your defensive setup. And you want that to be as seamless and frictionless as possible. And the reason why, like in this space in particular, that's so necessary is these these AI's are essentially black boxes, which means that the only way to there's no really like static scanning solution to AI security. There are parts of it where static scanning makes sense, but you actually do need something that uses live inference in order to say, hey, is this, is this safe or not?

That's something that I learned very early on back at Google when, when we would test these, these black boxes. So, you know, I think you, you do that testing and then you'll have a bunch of findings and ideally incorporating those findings into your guardrails is, is turnkey and it's a lot easier said than done. So the way that Promptfoo works, at least on the commercial or enterprise side, is it will recommend remediations. So that's not always guardrails. It kind of depends what the issue is. Sometimes that could be be at the prompt layer, sometimes that's at the API, like something is fundamentally broken, like you screwed up access control.

You just obviously need to fix that. But guardrails are are a big component. I wish there were a standard guardrails kind of like definition or whatever, but there isn't. So we built a bunch of integrations for like these common guardrails and then we also created our own guardrail offering that would just like like learn from the findings and get automatically tuned from the findings. I think ultimately there, there are like two types of guardrails. There's kind of the generic out-of-the-box classifiers. That's what that's what most guardrails are that are that are just trained to like recognize hate speech or, or certain types of PII or whatever.

And then there are guardrails that are adaptive and can actually be tailored toward your business context or your use case. And it's kind of that second bucket where there there's actual, you know, like substantial value there and in getting that right. And I see more companies struggle with that.

Allie Howe: Yes, absolutely. Yeah. I feel like a lot of companies just kind of like don't know where to start when it comes to AI security. So if we're able to offer this story, I feel like around your entire AI life cycle from like looking like the static code scanning to like the act of like red teaming once you have something built and then the runtime guardrail like influencing that with the results of the red teaming. That's a really compelling story of, hey, we've got you, we've got your back all the way from shift left to runtime. One question I have is where do you think is the right time nowadays to be like inserting this continuous testing, whether it's the scanning or whether it's the LM red teaming?

Because with something I'm noticing is a lot of people are trying to create like software factories where the coding agents will write all of their software, they will review everything and then deploy it. So it's very like highly autonomous and humans are in the loop as little as possible. So now I feel like security companies have the ability to, using an agent skill or MCP that you can insert yourself into the agent harness so that if you do have some sort of autonomous like software factory, agents can use your tools to the scanning or the red teaming, take those results and then automatically use those results to make their code better.

Is that something that you're saying teams want to do?

Ian Webster: I sold mostly to like large enterprises, so none of them are are quite going the software factory route yet. We're also kind of blurring the lines here between just like like traditional additional security vulnerabilities and AI security vulnerabilities where Promptfoo was like very explicitly focused on what are all the ways that natural language can screw, screw with your AI apps. And there's things like prompt injections with direct and indirect and jailbreaks and stuff like that. I would, I would say, I mean, in general, my philosophy has always been these things need to be built securely from the start. I don't think that's an uncommon view.

What I've seen in AI security specifically is everyone started all the way on the runtime side just because it, it looks the most like what they are used to. Like we we're all kind of used to the concept of, of, of like a WAF and you have something that that checks the input and output or whatever. I think that is the least effective part of the AI security stack, especially if you're not doing that, that tuning of the guardrails from from the the red teaming outputs. So a lot of people are starting from the outside and going in, which is like fine if you come from that direction, but then you actually have to go in.

So for Promptfoo, about 30% of our overall usage, remember this across hundreds of thousands of developers, about 30% of that usage was in CI/CD. I think that's, that's like a great kind of landing spot for Promptfoo or something like like Promptfoo where you conduct that, that sort of like smoke test or regression test or you, you can, you can gate it according to your security preferences. I wanted to move like so. So we were landing there. I would, I would wanted to move even further left toward, toward developers like like Promptfoo has a very robust CLI that, that like many developers like and use and we actually recently launched IDE plugins.

So I, I just want to be where the code is written or in the world of software factories, like I want to be hooked into there as well. And I think that that is going to be, that's going to basically become a requirement in AI security once the architectures get more complex and we're dealing with agents and not just chat bots and RAGs. And I think a lot of the existing AI security solutions are going to fall over flat or basically just become performative if they are not code aware as well as telemetry aware. Like during the actual testing there, there needs to be some sort of observability hook in order to understand what is going on under the hood in in the agents.

That was a long answer, but basically, like philosophically, yes, it should be early in the development cycle, but also very soon it will have to be early in the development cycle.

Allie Howe: No, I totally agree. Yeah, I think early makes sense. And then also just start from the inside and not just from the outside working in because I agree that I think a lot of people in 2025 was like, oh, all I need is an LLM guardrail and from what whatever company and I don't need to care like how strong their like red team is on the back end that like influences guardrails. I'll just pull anything off the shelf and I'll use it and I'll be secure from prompt injection. And like that's, that's not something that we saw in a very like motivated threat actor. We'll still continue to get through no matter how like good the models get because adversarial attackers can use the models themselves to write better prompt injections.

One thing I really have liked that I've heard him and let my network is this concept of like you need to count on the agent to do the wrong thing or you need to count on like it being compromised by a prompt injection like it will happen. So in the event that it does, what are your hard boundaries that you've given your agent to make sure that it's not authorized to access this information? Or you could revoke A credential and shut down a rogue agent if you had to. I think different identity infrastructure can do that. Well, I think maybe just certain sandboxing for high risk functions does that well.

Is there any other like how it hard boundaries that you're saying that you think are particularly effective nowadays?

Ian Webster: Yeah, I think, I mean the best boundary is to not build the thing that that like kind of checks the boxes that that that will turn it into a risk. We talked a little bit about education. A lot of what I talked about with security teams in in large companies nowadays is, is like this concept of a lethal trifecta where you have an agent that is hooked up to unsanitized user input, has access to sensitive data, and then it has some way to speak to the outside world. It sounds obvious when you say it out loud and, and you put it that way, but a lot of times like the, the fact that the data is, is like like untrusted or the, the input is untrusted or the fact that something can be exfiltrated like those, those possibilities are obfuscated in a complex application.

So I think that sort of lethal trifecta analysis or having control of over the, the tooling, the tools that the agents are hooked up with, I think that that's also really important to be able to detect and prevent certain combinations that that would enable an attacker to, to like misuse an agent. In other words, I think we're basically saying that the same thing, but we we want to make it impossible even in the worst case, right? Like in a world where you cannot trust the agent, it should still not be possible to to do bad things.

Allie Howe: Absolutely, yes, 100% yeah. That's what I was trying to get to also. And can Promptfoo's like code scanning for like define AI vulnerabilities or the red teaming help you find architectural misconfigurations or bad design patterns that make you more. More likely to have the lethal trifecta occur.

Ian Webster: Yeah. So on the red teaming side, we definitely like detect those like we the agents specifically go after those cases in order to detect them. This becomes really effective when you hook up your code base. So that Promptfoo actually has an understanding of of your architecture, how the code is built. In a lot of cases it can just identify like the lethal trifecta issue upfront and then create a proof of concept and then at Test time. So Promptfoo will receive your applications telemetry and in most cases, like it can actually look at the tool calls and like the the tools that are available. And that's, that's another way to detect that we have definitely or like I I've definitely wanted something lighter weight that doesn't require like you to spend live tokens on this.

I think the, the red teaming part is necessary. But like, yeah, as, as I mentioned, we recently came out with, with, with code scanning that that lives like, you know, in GitHub or even in your IDE because I, I want to flag these potential issues earlier. So there's not a perfect solution to this, but it's just like, you just want to get, you just want to put the tools in the developer's hands or, you know, put this in front of developers. So it just becomes something that is that is like better understood and kind of like flagged and looked out for and that kind of thing.

Allie Howe: Amazing. Yeah, I feel like we've talked about a lot of great insights here around like if you want to get acquired, if you want your AI startup to be successful, clean appeal to developers, that's important. And then also, if you are building AI applications, start from the inside workout, make sure your red teaming and your guardrails have that feedback loop. LLM guardrails might not be just like everything you need, like focus on hard boundaries like you were talking about and make sure that you are not vulnerable to the lethal trifecta. How you architect your agent matters. Is there anything else that you'd like to leave our audience with in terms of what's the one or two like, best things they could do or they should care about, especially in the next like year as AI security is evolving that they should focus on?

Ian Webster: I guess like forward-looking, I think the main thing that I would call out which I did mention earlier, but I think AI security looks very different over this year than it has in the past two years. And it's mainly because it is because the architectures are getting more complex, which means it's no longer sufficient to just try a bunch of inputs and look at the outputs. And that's what, but that's kind of like the default method of every tool out there, including Promptfoo, right? So that, that's, that's why we are investing more in kind of like the, the ability to, to integrate the code, the telemetry, etcetera.

That's why I'm kind of a, a broken record on, on that is that I know everyone's like, oh, AI is like moving so quickly, blah, blah, blah. But in this case, I actually do think something needs to look very different on the AI security side. And if you don't understand that, if if you are just going with a naive red teaming solution that you know, just try some inputs or like has a list of prompt injections or like slightly mutates those prompt injections, you are not actually doing AI security. You are doing like a basic adversarial evaluation, which I think is not enough. So it's kind of my two cents or you know, how I'm thinking about this.

And then for me personally, like I'm thinking, how do I leverage being on the inside at a foundation lab and having access to the research, having direct access to like the models and inference? Like what are the new ways that we can come up with it in order to solve this problem specifically for agent security? So that's what's on my mind for the next year.

Allie Howe: Amazing. Yeah. Super excited to get to need to follow Promptfoo and all of the ways that you'll make OpenAI security stronger. Where can people go to try Promptfoo's open source still available? And what's the future look like for people that want to try Promptfoo?

Ian Webster: Yeah, the open source is is definitely still available. We are building and maintaining it, you know, making commits and improvements every single day. Best way to try it out would be to go to promptfoo dot dev and just follow the the instructions from there.

Allie Howe: Amazing. Thank you. And then where can people connect with you if they want to join in on the conversation?

Ian Webster: That's, that's a good question. So it kind of depends like what part of the conversation do they do they want to be part of? I think for anyone who is, who wants to be like developer and open source focused, the GitHub and the Discord are a great way to do that. We, we like actively monitor both of them and depend pretty heavily on the community for changes and suggestions, that kind of thing. And that that's always been the case. We have this point close to 300 contributors, like people who have actually written code for Promptfoo, but probably 1000 other people who maybe didn't write the code, but who helped shape Promptfoo in some way from an open source perspective.

So for folks who are interested in Promptfoo or maybe just like getting your hands a little dirty in open source for cybersecurity, definitely check out the GitHub.

Allie Howe: Awesome. Thank you so much and Congrats on the acquisition and thank you so much for coming on the podcast.

Ian Webster: Thank you.