Allie Howe Hey everyone. Welcome to today's episode of the Insecure Agents Podcast. I am your host, Allie Howe, and today we're thrilled to have Samuel Colvin, Founder of Pydantic with us today to talk about how the framework we use for building AI agents relates to your AI security posture. And what's next for Pydantic. Samuel, you want to introduce yourself?

Samuel Colvin Thank you so much for having me. Yeah, I'm Samuel. I originally started Pydantic the library way back in 2017 and then started a company around it in beginning of of 23. Yeah. And then here we are.

Allie Howe Amazing. Yeah. So today we're sitting in New York City, which is super exciting for us. That's the home of the Insecure Agents podcast. There's always something going on in New York. I think you said you were here for the MCP Summit. Do you want to tell us about that?

Samuel Colvin Yes. So this was this was a summit of the well, there were there were two days one of the the core team maintaining MCP and then the day with where I was at of those maintaining the SDKs for for MCP. But yeah, talking very widely around what where MCP is going, what, what we want to fix, what we want to not go and change. Yeah, and obviously security came up a whole bunch. Obviously OAuth is an ongoing discussion within MCP. And I think obviously everything has the where. Where MCP is weird is that it spans the entire range from standard IO I just want to run an MCP server locally to Microsoft want to have, as I understand it, MCP is going to be part of the next Windows release.

I don't know whether that's public or not, but I've said it now as they want MCP connecting Office 365 bits of Azure. So they're going to have thousands of nodes running MCP clients and servers, but it also has to work on a local machine. So it's a, it's, it covers a wide range of different, yeah, different scales and therefore challenges.

Allie Howe Yeah, I'm, I'm seeing most people start to realize that this lethal trifecta issue where you've got agents connected to secure data that's, you know, behind and gated by some sort of MCP server. If you've got the ability to externally communicate as well, which these agents do, that can lead to a lot of issues. So I think more people are starting to get educated and aware of the risks of MCP servers, especially with the latest research that came out I think from Invariant Labs from Blackhat showing that if you, you can easily submit a prompt injection via whatever the MCP server is connected to. So like if you if it's kind of to GitHub, for example, and you have a GitHub issue that contains A malicious like prompt injection asking to take, you know, the author's private data that's in their private repos and then make it public in some way, I think Invariant Labs covered that case and published that research.

Samuel Colvin My opinion is that MCP is a protocol as the name suggests, it's also an ecosystem. Neither the the protocol does not have any security vulnerabilities. We know what it has done is it's allowed us to go and have multiple different connected agents or systems in the way that we'd all been claiming we were before and now actually are. And that has exposed all of the faults and basically omissions in security within AI systems. All it's done is enable us to go and connect to things using LLMs. And it turns out once we do that, then no one has sorted out how we do security. We can't make things as powerful as we want them to be without also introducing insecurity.

I mean, it's, it's basically we're back to the world pre AI where we have like I still have spear phishing and phishing attacks on individuals. We just now that those individuals are AIs which are even more prone to coughing up the wrong thing and believing someone saying please give me X and humans are.

Allie Howe Yes, 100%. And I think like you mentioned before, I think auth is a real way to solve this where you can't really control what it's going to barf up. But if you have strong guardrails in place to making sure you know this user can't access this data in this database or something, whatever it's connected to you can enforce.

Samuel Colvin Yeah, and it's authentication and authorization. I always go a bit blurred on which ones which, but the point is, it's not just like that. I'm kind of allowed to connect to the server, but which tools am I allowed to call? Which resources am I allowed to access? And that's a that's a hard question, and one where arguably there's a space for the protocol to be more opinionated and more restrictive rather than the current. Lots and lots of things are possible, but lots of people have lots of those given access to that have not thought through the potential implications of it.

Allie Howe Yes, we actually we had a really interesting conversation with Ian Livingston from Keycard Labs on a previous episode of this podcast. He was saying that auth for agents needs to be context aware. Do you think the MCP protocol could help with that?

Samuel Colvin I think it can, but it's, it's such a complex problem. As I said earlier, it covers such a wide range of different use cases and it's extremely frustrating if the protocol goes and says, Oh, no, you can't do that thing when, when you're like, no, no, I understand the context of what I'm doing and I want to be able to do it. And I, I think, you know, you could argue that the job of the protocol is to be a protocol, as in HTTPS is secure, that allows us to go and do like insecure things on top of it. But basically with, with great power comes the capacity to be greatly insecure.

But that kind of has to be there when you have great power. You can't get rid of that.

Allie Howe Yes, absolutely. And then similarly related, what do you think is a agentic frameworks role in the security of AI agents?

Samuel Colvin I think I'm going to slightly dodge the question and the same give great power and therefore the capacity for great insecurity. I think you can come again, you can go and get the fundamental things right in the way that we got very easy to use HTTPS everywhere security. That doesn't mean you can't go use the API which allows you to ask anyone, but at least we don't have someone else in my coffee shop. You know your traffic and what size I'm looking at and what traffic being transferred.

Allie Howe Yes. No, for sure, absolutely. It's interesting too, for me to think about how far we've come since, you know, the beginning of this year at the and AI Engineer Summit in New York City. And in February, 1 of the talks mentioned that you should not start with an agent framework. You should start from scratch. And then once you know, like what you need, then you could add a framework on later before you have all this abstraction. Like what's your take on that? Does that still sound

Samuel Colvin I? Think it's bullshit I think that lots of people got burnt by certain frameworks that I shan't name being overly abstracted and stepped back from Let's use a framework to let's do it all from scratch. I think that, you know, if we, if we think in web years, we're roughly around 98, two, 1002, something like that in AI. And I'm sure that if you went to Silicon Valley in those days, it would have been, you would have found people with beards saying I don't need to use a web framework. I'm going to parse my HTTP headers myself because that's going to be much more performant.

And why should I pay for the overhead of building a mapping for HTTP headers when I know I only need to access these two HTTP headers? Fast forward even five years, let alone 20 years. No one thinks it is credibly a useful thing to do to write the code to parse my HTTP request myself. We're going to go and use a web framework, and we're going to choose our web framework that is not too opinionated and does the right stuff. And we're going to go and say, for this thing I want to use Flask. For this thing I want to use FastAPI. For this thing I want to use Django.

For this thing I need to go and use Hyper and Rust. But we're not going to go and start passing those headers ourselves. And I think it is the job of the framework to be just opinionated enough provides the fundamentals that are that everyone wants to go and use where it is easy to make security mistakes, but not be too overbearing and come along and tell you what paradigm you want to go and use on top of that. And I think we're like one of the things that has, you know, end of last year people were saying everything was about agents in the sense that the agent was the system that this agentic loop solved everything.

I think we are past thinking that the agentic loop solves everything. What I think has happened more implicitly and people haven't kind of acknowledged is sure I have to go and tether together 10 or 50 of those agentic loop. But the agentic loop is incredibly useful because basically anywhere where I'm doing structured data extraction, if I get back to the wrong data, I want to pass it back to the agent. And that is like the simplest agent loop. So everything becomes agent. But instead of the agent being just a start up in one agent, a start up has 10 different agents. And so it needs to be really quick to go and build them.

They need to be type safe to make to allow you to refactor them quickly, etcetera, etcetera. But I yeah, I think the don't use a framework enthusiastic case is purely a reaction to LangChain.

Allie Howe Interesting. Yeah, that makes sense. I remember too, I was doing some research and I was looking at different security vulnerabilities that certain frameworks had versus others. It's amazing. Like some, I think Palo Alto's Unit 42 published some pretty, pretty bad vulnerabilities honestly, for some other framework such as server side request forgery and critical prompt injection where whatever the user submitted that could have been converted and used to run Python code. So that's basically like remote code execution essentially. That's pretty severe.

Samuel Colvin Yeah, there's been some quite serious, but I mean I think there was a point where certain other frameworks, I shall name where the basically the default way of getting access to variables within context within tools was go and put it in the system prompt. And then you can pull it out the system prompt. And the, the hopefully the LLM will take care of passing your Postgres DSN into the tool. And like sure, Pydantic AI doesn't stop you from doing that. If you are dumb enough to do that, we're not going to hold your hand and stop you doing that. What we are going to do is have a type safe convenient way of passing dependencies into tools so that you can do it the correct way.

So that's where I think you can make it a bit harder to go and do the insecure thing or a bit easier to do the secure thing. But still, giving freedom to make your own choice would be nice. Freedom to to treat yourself if you say way.

Allie Howe Yeah, OK, interesting. And I know Pydantic also offers some features that in my opinion sound like, you know, security advantages. There's a reason maybe you'd want to use Pydantic. So the MCP run Python server that you created for running Python code securely.

Samuel Colvin Yeah, yeah. I mean, I think if you're going to ask an agent to run, run Python code, the idea that I think certain other other frameworks, I think smolagents for Wilds like we're going to try very hard cut off bits of CPython that that might do dangerous things, but still basically run CPython And your main interpreter. That seems to me like it's great first and local development trials, but you would never want to put anything like that into production. I think even MCP run Python has some like very edge, a security vulnerability that that what's funny when people get and copy the things I've done, they often, you know, they copy my mistakes as well.

And that that's always quite amusing to see. But you know, even there you can, if you really try, you can get access to the bit of the file system inside node modules, which I don't think represents the like remote code execution, but it definitely could allow you to somewhat take the environment. So you need to be even there. You need to be careful about what you do. And I yeah, I think, I have had a like sort of fascination with running insecure code for years and years. When I ran the previous company, we wanted to allow, it was a SAS platform. We wanted to allow the developers to run, use Jinja templates, e-mail templates.

And even back then that was like 2014. I was sort of shocked. It wasn't the go to way of just like run, I think go to the box and that still is unsolved. I think Pyodide and running it in a browser environment as it were gets us a long way. But even that doesn't fully solve it. And if you look at all these companies now, what are they called E2B? Is it E2B? Whatever that company and Daytona as well, who who offer remote code execution. You know this offer offer like running code. They've all actually gone and used Docker containers that used basically infra isolation because running untrusted Python code is basically a fully unsolved problem still.

But yet Python is really the lingua franca of what models want to run, right? When the OpenAI ChatGPT system prompt got leaked, it basically said if you're doing any calculation, use Python to do that calculation. And so we're moving for a situation where the way that models any form of calculation is runs in Python code. And so I think that still remains a bit unsolved currently was basically throwing the problem, throwing resources at the problem and spitting up the containers. But I would love it if there was a a safe, fundamentally safe like lower lift way of doing it than docker containers.

Allie Howe Makes sense. Does TypeScript also suffer for these same problems?

Samuel Colvin Not really because JavaScript you, I mean we have, we have isolates right? As in we have, we have had 20 years of browsers needing to be able to run someone else's code and not have access to system resources. So Deno, for example, has bulletproof isolation because fundamentally it's built on V8. And ironically what we're using in MCP run Python, it's Deno to give you that same isolation. And then we're running Python inside JavaScript, basically to go inside the JavaScript WASM environment to get us isolation.

Allie Howe Amazing. I know that's a great tool for anyone that's trying to take, I guess, a user input and maybe run it as code. Do it securely in isolation.

Samuel Colvin Then I actually think Deno is an awesome company. I'm really impressed by what they've done. I'm surprised that they haven't done a better job of jumping on this sudden demand for or run untrusted code. As far as I know, that's not something they've been pushing and they absolutely should because they could be killing. It

Allie Howe yeah, It's interesting you say that. I feel like users are starting to become more savvy and aware of the security pitfalls that are at play and different solutions for them.

Samuel Colvin I agree. I think one of, so I was, I was saying this to someone the other, I think one of the problems we have in software development in general is that we have one, one word or a few different words. Coder, developer, engineer, which covers an enormous range of different people. If you think about it on a in, in construction, we have the guy who carries bricks around the building site and we have the structural engineer who designs the Burj Khalifa and they are completely different names. It's very obvious to anyone, however involved in the construction industry that they have very different skill sets in software where it is much harder to see it from the outside.

We and the the main history is terrible. We call those both of those people devs. And so we and that and that automatically that causes us all to kind of even even as developers, we we bunch them all into one group of people. And there is maybe the the lovable end of the spectrum is starting to be so clearly different that people are thinking of lovable developer or whatever you want to call those people as separate. But like for the vast majority, you know, from people in their first week of coding through to experts, we still basically one set of words. And so we therefore we bunch them all together and we have we think of the security concerns as bunched together when they're actually very different.

All hard for all those people, but but different.

Allie Howe Yes, that makes sense. As you are able, are you able to see if users become like more savvy over time or they have different problems thanks to Logfire and the data that you're capturing there?

Samuel Colvin Yeah, I think that the Logfire is, I mean, for me it is AI mean and obviously I'm very biased, but it is an amazing view into your application. I mean, I've been using Cloudflare Workers for years. The first time I hooked up Logfire to Cloudflare Workers and I suddenly was able to see the latency between like I didn't know this but like the latency of a KV store request when the key doesn't exist or the value doesn't exist. It's like 5X does exist. I never knew that. And suddenly you start thinking about building your application difference. Now that that in that case it is about performance, not about security.

But I, my friends at Prefect have just released FastMCP Cloud, then they want to work with Logfire to do the observability. So whatever it was 2 days ago, I hooked up Logfire to FastMCP Cloud immediately saw, well, I won't say what they want, but I mean, there was like 2 days after I've launched it. So this is not a criticism of them, but some stuff in the environment that shouldn't have been there, but that, you know, that was like trivially obvious to see for me. But, but it's not obvious, don't have the observability. So I think it is incredibly full, like being able to just have that view into what your application is doing.

Just just to be clear, FastMCP Cloud is awesome. I, I'm really looking forward to, to both using it and building with them. Yeah.

Allie Howe Amazing, so they help deploy MCP servers?

Samuel Colvin Yeah, so, so FastMCP is a library of its of its own for building MCP servers in Python. And they've now gone and built a cloud service for deploying it, which is I think best in class right now for trying to deploy MCP servers.

Allie Howe Amazing. Yeah, that's really cool. That makes sense. Are they helping at all with like scanning MCP servers for like tool poisoning or anything?

Samuel Colvin And not doing any of that, Yeah. I mean, they've done an amazing job. They went from an idea to an open beta in eight weeks. So like what they have right now is deploying your server at an audit log and you can connect Logfire for observability and they're doing the registry stuff. I think they're really interested in that security side of things and moving into it. And I think it's clear that there is not a like VC scale business to be made out of getting individuals to deploy their MCP server. So obviously the, the, the business here is an enterprise and that's exactly the kind of solution they need to go and solve.

But as companies like Vercel have demonstrated, starting with anyone can rocket and go and deploy it quickly is a great place to start for enterprise, for start up, for individuals, for everyone.

Allie Howe And circling back to observability, I think having something like Logfire or observability in general in place is important not only for performance, but security as well. To understand what your agent's saying. It seems like there's starting to be these infrastructure type platforms that are starting to merge of like they're going to help you deploy your agent into production. Because I think a lot of people are finding that's easy to create a, you know, proof of concept agent. But launching into production now we're talking about observability, we're talking about security, we're talking about evals, we're talking about these last mile problems that are very difficult to solve.

Do you see frameworks like their role in that to be helping build those types of infrastructure in place or no?

Samuel Colvin Yeah, I think absolutely. And I think you know, FastMCP Cloud is a good example of that. I think we have toyed with the idea of having deployment for Pydantic AI. I think MCP is the obvious way to expose it, A2A being the other one. And so I'm, I'm a fan of FastMCP Cloud and I'm constantly looking for things I don't have to build. So I think for the foreseeable future, that's going to be our go to if you want to deploy a first Pydantic AI agent, you use FastMCP Cloud. But I mean, yeah, I think there are enormous numbers like if a lot of it is similar to the kind of Heroku and the Heroku successors.

But there is, I remember thinking when I was, you know, very inexperienced 2014, first deploying stuff onto Heroku and discovering that there was, there was no security help. Like if you wanted to, someone took our SAS platform down by leaving their phone on their F5 key and it like held the F5 key down the input the screen they were on, but made a bunch of different database queries. Our system was down for half an hour until we rang to like stop pressing F5 and there was nothing built in. Go and help you with that. I think the Cloudflare is a big step in that. And, and it's amazing that systems like Heroku didn't have a Cloudflare equivalent like reverse proxy in front.

But I think that as as the as the, the bit of user code gets smaller and smaller and therefore we deploy more and more of these services, the importance of built in security and built in auth and stuff like that only gets more and more important. And yeah, auth is something that FastMCP Cloud goes and installs on day one in a way that that like the HTTP platforms never did.

Allie Howe Yeah, I think that's really important for the future of trustworthy AI having some sort of secure by default in place. That's really interesting that they're adding auth in as well. How did they do that? Or.

Samuel Colvin Well, it well, they're lucky because auth is is built into MCP and so there is a there is a protocol for it. Sure, technically there's basic auth in HTTP, but no one wants to go and use that. You can't go and be like, oh, I'm going to add auth into my HTTP PaaS platform. I think the other, the other similar case is in security within the call to the LLM. And that's something where we've been talking to a number of security companies about how there's way more that we can do there. If we can intercept at the LLM call and either block it right there.

And then as in we're, we're talking about building very small models that can detect security vulnerabilities as the basically during the request. So you've got, in theory, if you've got a request, it takes 3 seconds, you've got 3 seconds of window in which you can go and do your own check. Was the prompt malicious and then basically cut off the response or quite often, no one actually manages to hack a system in a single prompt. So if you can, if you maybe you've got 5 minutes until they've actually hacked it to go and look at their requests and be like, this process is obviously malicious and then go and, and block them.

And so we're working with, we have some products we haven't announced yet that will be the infra for that. And then we're working with security companies to solve that exact problem because it, it is completely unsolved at the moment. I think presumably the the big foundational models are trying to go and solve it. But if one of them has a good solution and you want to use someone's else, someone else's model, you're out of luck. So I think there's an enormous space for gateways that are intelligent enough to do security. There's been lots and lots of noise around model routing since GPT-5. I think it is.

I think it's mistaken. And security is more important than routing. And with that's really where we should be, should be pushing. Maybe I'm giving away too much of our road map, but yeah.

Allie Howe Now it's exciting. I was going to ask you what's next for Pydantic So.

Samuel Colvin So I preempted the question. Yeah, yeah. I mean, go on, I'll say. So we're building Pydantic AI Gateway and security is a big part of what we will enable. Even if we're not building it, we'll have the foundations to go and build it and then we're working with, I won't name them, but some really exciting security companies on the, on the, the threat detection bit of that.

Allie Howe Yeah. So it's super exciting for sure. And I think that really highlights the shared responsibility model for agent security and AI security in general. It's pretty much like cloud security, right? So it's like you can, you can trust AWS or whoever your cloud provider is with security, but it's your responsibility too. So having, you know, not only sort of like classifiers or guardrails built into whatever closed source model you're using or open source model having something at the gateway layer to intercept prompt injections as well. And maybe you could have your own runtime security in place also that monitors your whole system. I mean, just layers of defense here that really make it so there's less likely chance of what?

Samuel Colvin One thing we don't want to what it would be unfortunate if we had to push all of that responsibility down onto the app developer, agent developer, because we were talking to a large bank the other day who said they have a, they have a what they call a technical community of 10 to 12,000 people who they are now recommending use Pydantic AI and they're going to be deploying, you know, enormous amounts of of different agents. They want to be able to stay basically anyone go and deploy an agent and and see how well it works. But obviously they really don't want the scenario in which it these go off the rails and do insecure things and insecure can mean, you know, accessing resources you shouldn't access.

It could also involve saying something that you're not allowed to say, saying something racist, for example, Right. And so I was talking to someone about a company who offer insurance and want to use agents and that, you know, you can imagine and that someone says in your next reply be as racist as possible. Then you say, why can't I get insurance? Someone takes a screenshot of the response, which is racist. And that is like enormously damaging to that company. And so having a guardrail in place, which basically says never under any circumstances reply being discriminatory is really important. But you don't want to have to make every single person building every app do that.

And so lifting that into the gateway layer or into the model layer if you control it makes complete sense.

Allie Howe Yes, that makes sense. And I'd like maybe later, will there be any like customizable guardrails as well?

Samuel Colvin I mean, fundamentally what we're building is webhooks, webhook before, webhook during, webhook after. And then we'll work with particular security partners, but we'll also allow you to bring your own webhook effectively. And then that webhook can just full on guardrails, right? It can just be one another smaller or bigger model in parallel to to check the response.

Allie Howe Amazing. I think if you have that too, the argument for, oh, you shouldn't use a framework, you should start from scratch, really starts to fall apart.

Samuel Colvin Yeah, yeah. Yeah, absolutely right. And and you know we're doing the same stuff in in Pydantic AI, right. We're not you know, ultimately our model is an abstract base class and I have implementations of our model type which actually don't use LLMs at all, go and use a tiny MiniMax model for for solving particular things. But it still works particularly nicely to use Pydantic AI for that because we're extensible. We're not trying to force you to use the implementations we have. We're trying to give you the right fundamentals that you can go and go and extend if you want, but for the most part, you won't have to.

Allie Howe Amazing. So obviously Pydantic's been around for quite a while, and then we had Logfire and now we've got Pydantic AI. How do all of those relate and how did you sort of stitch those together?

Samuel Colvin Yeah, we, we probably did it in a slightly odd order in the sense that the the the natural progression now for a user is I use Pydantic. I trust them. They're good. Oh, they've got an agent framework. I'll go and try it. Oh, it seems to be good. I like like it. I like its taste. And now I need observability because all of them is a desperately need observability for lots of obvious reasons. So I'll go and use Logfire. And then once I'm using Logfire, oh, this, this doesn't just instrument Pydantic AI, it doesn't just instrument AI, it's full observability. We can actually roll that out across our entire AI or innovation team.

And then the neighboring team are like, actually, can we use it as well? And whether we see that in enterprises, we see this like actually, once we've got self hosted Logfire and we can use it, there's a bunch of other teams who are waiting to use it. They haven't yet persuaded their boss we need to strip out Datadog and move to Logfire. But now we've got the cluster and I can go and use it. I'm actually going to use it instead of Datadog O ALC or whatever else. But I think the interesting thing for us has been the how fast Pydantic AI has grown. We're weird as I think we are very bad at getting GitHub stars.

We are very good at getting downloads. So to today, even even till now, I think Pydantic AI has like 12 K stars or something. I mean really not very many in the scheme of things and yet we've gone from releasing in December to now having overtaken all of the agent frameworks apart from LangGraph and LangChain stayed ahead of the new agent framework. And I think pulling away from Google ADK and OpenAI agents, you know, there is whatever trillion of market cap building agent frameworks. I'm not saying that to be anything they're doing, but there is serious interest in that space. And our approach of no frills, build the right layer of abstraction, do that well, try not to move too fast and build, only build things when they're good seems to have appealed to people.

And I think that is a reflection on the fact that 2025 has been the year that AI hype has turned into AI. There's still a lot of hype, but suddenly we've gone from some product manager has been trying something through to the engineering team. We're told to put this into production and the engineering team find Pydantic AI superior to the alternatives is what is suggested by the download curve.

Allie Howe Yes. And that makes sense. And when you get into that adoption piece, those cracks start to show. And if you've got good, solid fund fundamentals, then you're in a good place.

Samuel Colvin And I think we saw a lot with other agent frameworks that were much more expansive in terms of their feature sets, but limited in terms of how well it worked. When you started to scale both the size of the application and and traffic, people actually wanted to go back. And I suppose in some sense Pydantic AI, going back to your earlier question about should I just use the SDK? Finance AI is kind of probably more minimal spec on top of just use the SDK than some frameworks where they'll like, we'll tell you how to do rag, we'll tell you, yeah, we'll tell you how to do all of the vector search.

You know everything else. Here is our way of building React agents. Here's our way of doing deep agents instead. Here are the right fundamentals. You're an adult, can do it yourself.

Allie Howe Yes, that makes sense. One of the things I was really curious to ask you is what's it like as a founder of an agent framework? How do you know like what to put in your road map, what to chase with like new protocols that come out, new technologies? How do you know what's going to stay and what's worth chasing?

Samuel Colvin I was. Thinking that actually on the walk here and thinking how it as a founder, you, you have a view of the world. And the more opinionated your view of the world is, the more you succeed when when you get it right and the harder it is when you get it wrong. Because if you're like 1° off, but you only have 1° of width in your view of the world, you have nothing. And obviously the hard thing when your view of the world doesn't match reality is am I ahead of my time? Am I going to be right? Or am I just completely wrong? And I think that there was a situation a year ago where my view of the world, the AI was still engineering and the same security concerns and the same scalability concerns and the same engineering concerns applied didn't match reality.

And people were saying, oh, I can go and do use this thing that in my engineering judgment was obviously wrong. But it seems to be working. And I'm, it feels great now that my view of the world seems to be more closely matching reality. As in people are now taking those engineering concerns and those security concerns really seriously rather than Yolo deploy. And now we have a million revenue in a month and everything's great.

Allie Howe Amazing. It feels very similar to when people say, oh, you need to have taste when it comes to evaluating like AI outputs and verifying them. Having an opinion as something that's hard to do because sometimes you get it wrong and sometimes people don't share those opinions. But I think just in general, being able, being willing to be opinionated allows you to be successful. So that's really cool to see.

Samuel Colvin If you get it right. But if you if your view of the world is fundamentally behind rather than ahead or just outright wrong, you're in lots of trouble. So that's the scary thing, right? When when your view of the world doesn't match reality, do you? You have to do one of two brave things. Either you have to realize you're wrong and change, or you have to stick to your guns even though everything doesn't agree with you. Those two things are both quite scary.

Allie Howe That's very scary, that's true. But no reward without all the risk, of course.

Samuel Colvin For the most part, running a company is, you know, incredibly high upside and in the scheme of most people in history, pretty low downsides. So I think, you know, for the most part, it's a luxury.

Allie Howe So in my work, in my day-to-day, I've talked to teams, I help them with AI security, and in doing so, I've really become to try to understand what is the recipe for trustworthy AI. There's different technologies in different companies that I think play a big part in that. I think frameworks also play a part in that as well. How do you see Pydantic AI contributing to building trustworthy AI?

Samuel Colvin So I think presumably you can. There are multiple different dimensions of trustworthiness in AI. There is the security component, which is I guess is where your main interest is, but there's also just reliability. Does it? Does it do the right thing? I think the first step to trustworthy AI is using AI in the right domain. If you are trying to extract structured data from an e-mail or a PDF or something, you can do that. If you are trying to use an AI to give people medical advice, you're going to have a much harder time because it's a fundamentally more difficult problem. The evals of does this app make someone stop smoking fundamentally take 40 years of whether did that that person ever smoke again.

And so the first step is use it in the right domains. And there are, there are actually you can, you can boil an awful lot of AI down to a relatively small set of cases. And even within them, the ones that work and don't work, I think that is the main thing. And then it is building the right primitives in that space. So if you wanted to take the like extracting structured data from emails, you want to use vector extraction, whether you use tools or JSON schema or structured outputs in some of the models, you want to be able to do schema validation and reflection on that.

Those are basically primitives at this point that you want to use a framework for Pydantic AI does that. Well, we're not the only one, but I think we do it better. I think that's a big part of it. And then it's, again, it's all just engineering. It's like having good documentation that was written by a human who understood the thing, who probably wrote the code, is going to make it a lot easier to build a trustworthy AI system if you're not trying to work out what happened or read the source code to work out how it all supposed to work.

Allie Howe Yes, I like that mention of reliability. I think that's really important. And AI that is not on topic is also like not only security concern, but it's of course just a performance issue. Like if your AI is not performant, then no one's going to use it. It's not going to be revenue generating. So I think trustworthy AI is something people should really be interested in because trustworthy AI is revenue generating AI.

Samuel Colvin Yeah, absolutely and and non trustworthy AI. We've all had instances of like some GitHub did it a bit where they like put AI buttons all over GitHub. Half of them did nothing or didn't work properly, not did nothing, but like actively didn't, were not trustworthy. And so that that damaged the reputation of AI within GitHub because people stop pressing the buttons. Now Github's AI review is actually usable, but I think it probably clicked significantly less than it would otherwise be because they like spammed AI buttons all over the place without making them good. And so non trustworthy AI harms AI adoption among other things, because people get, I mean, one or two times and then they're like, am I going to use this?

I know for example, PagerDuty has an AI button. I have not clicked it. And that is not to say it's good or bad. It is that I just assume it is no good. And that was because it came out around the same time as those GitHub buttons that was not working. I don't have a particularly high opinion of PagerDuty's UX. I assume that AI functionality is not going to work. So yeah, it's compounding. Basically, if you build trustworthiness in AI, people will will go on and use it.

Allie Howe Is there anything else that's coming for Pydantic that you wanted to highlight today or anything? We didn't talk. About

Samuel Colvin I think we're we're pretty flat out so so Pydantic AI V1 I think that's really important that that is whether I don't actually know whether we got around to releasing V1 last night or whether it will be over the weekend on Monday morning. But we we had we've we've been a bit slow. We promised it in July and we're now going to have V1 by end of August or like the 1st of September. But I think having, I think we will be the first of the major agent frameworks to go V1 and at Enterprise in particular, having that confidence 1, you have the confidence in Pydantic because of, you've used Pydantic for years and you probably trust our engineering, you've probably got some trust in Pydantic AI by now that we've, we've that our taste in not being overbearing is right.

But now you have the trust that we have a, we have a stable release. We're not going to go and break things. If we break things, you get to shout at us and be like you broke that thing and we will try very hard not to whilst continuing to add features. I think that will be really important. Whether you call that security or whether you call that a good engineering practice is you know, kind of not the point. Yes, so Pydantic AI V1 coming, coming now full issue support in Logfire. So that flow of basically building up different consistent exceptions into one issue is going to be really helpful.

Something people have been got used to and that we're adding now properly. And then evals functionality, full evals functionality and Logfire. We've had some, we've dipped our toe in it before. We build a few different things. Nothing really works. We now have full support for offline evals and we're working on combining that with the online eval stuff, which is to, in my opinion, I know we're not, this is not, I could talk for an entire podcast about evals, but like I think that evals are too complicated at the moment and we need something easier to do. And I think that's why we're going to get to next is the online evals point and click at whether or not the agent did well or badly.

And now we have a golden data set. Now we can run the evals and they obviously can crossover with security as well.

Allie Howe Yes, absolutely. Yeah, evals are very close to security. They can be used for, they can help implement your guardrails too if you find the this continuous cycle for sure. Well, so that's a very exciting. So we've got improvements in observability, we've got more guardrails, we've got big changes for Pydantic. I'm excited for you. That's awesome.

Samuel Colvin Yeah. And the gateway that that is as yet, well it's announced now, but we were hoping to have a public beta in September of that.

Allie Howe Yeah, amazing. OK, where can people follow Pydantic to stay up to date on those releases?

Samuel Colvin Oh, we're really bad at at being being good at that. Well, we will try and blog about the main things and I'll be very vocal on Twitter and LinkedIn. So so there.

Allie Howe Amazing. That sounds great. Well, thank you so much for coming on the podcast today for everyone. That was Samuel Colvin, founder of Pydantic. And that wraps up for this episode of Insecure Agents. Thank you.

Samuel Colvin Thanks. For having me.