Allie Howe Welcome to today's episode of the Insecure Agents podcast. I'm your host, Allie Howe, and today we have Kwindla Kramer, CEO of Daily and creator of Pipecat AI with us today. Kwindla has spoken at AI Engineer and is well known for the Pipecat framework which makes it easier to build voice agents. Kwindla, would you like to introduce yourself and tell us more about your work?

Kwindla Kramer Sure. But first, thanks for having me on. I love what you do with the podcast. I love talking about this stuff, so it's going to be fun. I'm Kwindla Kramer, I'm Co founder of a startup called Daily. We make infrastructure for real time audio, video and AI. We predate the current Gen. AI boom. Our customers traditionally did things like telehealth and education applications, using our network infrastructure for the audio and video. When GPT 4 came out, it was just an amazing moment for us because we realized you could talk to a computer now in some of the same ways you can talk to other people.

And we have the infrastructure that powers a lot of people talking to people over the Internet. So we started to build people talking to LLMS, tooling, experiments, all the stuff you build them, and there's some new technical possibilities. And do you love the engineering? Pieces of this. And we realized that there were a lot of really interesting hard problems, some of which feel like 2026 agents problems, some of which felt like the traditional like networking problems that we specialized in, like how to move stuff around at extremely low latency and build very responsive systems. And as we started to solve some of those problems, we thought we could help the whole ecosystem cohere if we open sourced all that layer of tooling.

And that's the project that became Pipecat, which is now the most widely used way of building voice agents for things like customer support and outbound calling to remind somebody that they have a doctor's appointment or answering the phone. For small businesses, we probably have 200 customers that are building phone, answering for little niche verticals, but making huge differences in how those small businesses operate. And you know, now it's been about 2 1/2 years we've been building this talk to an LLM stuff, and voice agents are a big thing.

Allie Howe Amazing. Yes. Now I heard a little bit about voice agents last year, but I feel like this year they'll become even more of a thing when we have like more multimodal systems. His models have gotten better. And does Pipecat help with other different agents well or just specifically voice?

Kwindla Kramer You know it's funny I talk about voice because that's where the market has started to grow just hugely in the last six months. But we are a video and audio company and Pipecat is really about the real time multimodal and multi model stuff. One of the very first things I personally built with the tooling that became Pipecat was before GPT 4o could do vision. I built a little piece of software for some sort of experimental stuff on my Mac machine that used GPT 4 and a really great open weights model called Moondream for screen vision processing at high frame rate so that my voice agent knew what was going on on my Mac screen.

That is a little less mind blowing now. And then it was, you know, right when GPT 4 came out. But I just tell that story to illustrate the fact that from the very beginning we've thought both multi modality and multi model agents are the way you're going to build this stuff.

Allie Howe Perfect, No, amazing. I think for the last like little bit around the Insecure Agents podcast, like our theme we started off with was security, making sure that, you know, agents are secure and chatting about all the ways they can be insecure. And I think in this year, in 2026 with this podcast in my vision mode, I'd love to like focus on it's not just security, but just like agents that work in an insecure agent is one that's, you know, probably has security issues, but also maybe is unreliable, not trustworthy. There's so much that goes into an agent building, an agent that is reliable and trustworthy.

And thinking about that, it really brought brought to mind the game that you had shared at the Daytona event that I saw that you spoke out of like a week or so ago. And you showed this space game where this is long running agent and you can talk to this agent about the moves that you've made in the game, get it to calculate new moves for you. You can chat with this agent essentially about like all of the history and all the interactions you've had with it the entire time. And to keep that agent running may be very responsive and natural to talk to, very fast, low latency, and also be able to query that history in real time and have that such a large memory context for it to pull from.

That is just a huge engineering challenge. And I'm starting to get into this myself because I'm starting to build a multi agent product myself for like my startup that I'm building currently. So I know this is absolutely non trivial. There is a huge divide between a POC agent and something that works actually in production, especially long running stateful agents. So what are the common engineering challenges that you've seen with that and how did you build an agent that worked that well?

Kwindla Kramer So many fun threads to pull on there. So there's a game you're talking about? Started as a tiny side project because I wanted to experiment with Claude code. Like, literally, I had this idea. I knew I was never going to, you know, actually do it. And it was my nights and weekend, actually just occasional evenings thing with Claude code as side projects sometimes do. It got out of hand and the main reason it got out of hand was a. It was so much fun, but also. Every problem that I had in this multi user, multi agent, completely LLM based kind of game world and that Claude was writing for me was a problem that we also had in enterprise voice agents.

And I spend my days talking to people at big companies that are trying to build production. Quality voice agents and and the whole suite of stakeholders. So the engineering teams, the product teams that you know, the AI Tiger team, the C. And IT people, they all have different perspectives. They all are worried about different parts of that elephant. And then we're trying to help them solve, as you said, the agent problems from a reliability, an architecture that they can build on over the long term, a security perspective. And it was so great to have this open-ended canvas to experiment with these problems without actually having to deliver something to a customer.

And it was amazing how much the. How much the architecture stuff for the game mirrored the architecture stuff we had spent the last couple years kind of figuring out for voice agents. So you have to compact the context and do lots of other kinds of context. Engineering for a long range running agent, you have to think about what happens when you're shoving a whole lot of metadata into the agent as or structured data in addition to like user input like the LLM is. Super unreliable in different ways when you're doing structured data stuff than when you're just doing conversational stuff. And in production agents, you really have to do both.

You have to have structured agent, structured data and like natural human conversation because really that's why you're using these LLMS. Because they can do both of those things and we've never had computer tooling that can do structured and open-ended stuff at the same time in the same sort of envelope. You have to think about things like prompt injection. So it's a multiplayer game world and you can send other players messages and that's a prompt injection surface area. And so all of these sort of architectural things. I just had this this like way to experiment with that I didn't have a way to experiment with when I was doing customer facing work.

But it was just the same. Very hard problems, lower stakes, same problems.

Allie Howe You know, that's a great way to experiment, too, because then you can bring other people's opinions in and see how we solve this in real time. Yeah. I feel like we haven't had established best practices for how to build agents. So I'm wondering now at this point in early 2026, do we have the right sort of like frameworks and standards in place with like MCP and like different protocols? I know like we started off with like server side events for MCP and then they moved into streamable HTTP. Like we have, we figured out like from an engineering perspective, like, OK, it's this protocol streamable HTTP maybe of like how we actually have like the conversation between a client and a server and an agent interaction.

We use MCP to distribute tools. Like do you think we're there? Do we still have a long way to go?

Kwindla Kramer Oh, we have so long to go, but the fact that we're all building stuff is the way we get there. I was super grouchy about MCP when it came out. And, and I, I'm really optimistic in general. Like it's why I'm an engineer. I think I share that with a lot of engineers. But like, I was just super grouchy about MCP. And so I had to take a step back and be like, well, why am I grouchy about this? And what I realized is. I was the, I'm now the old guy who has lived through a few kind of Cambrian explosions in tech, including kind of the birth of the, you know, consumer Internet.

And I remember being a baby programmer and being like, we have this thing called HTTP and the people who are as old as I am now in 1995 were like, that's a terrible protocol. And I was like but. I don't well, AI don't know why and B, it doesn't really matter because it unlocks all this stuff you can do. And I really think a couple of things happen when we have these kind of amazing technology transition moments. One is the standards aren't built by standards committees. They aren't the perfect thing. They're the things that we're all building together, that land at the. Right place at the right time that then unlock, you know, more things that we can all figure out together.

So from like an architecture astronaut perspective, none of these things we're building are perfect, but that's OK and it has to be OK. It's the only way we can make progress. The second thing is every first implementation in a new context is terrible, like it just always is. I I remember reading the Node JS networking code in the first, you know, relatively early versions of Node JS. And I was so appalled I had to like. Go for a walk because I've written that code a bunch of times for a bunch of systems. But of course it doesn't matter because node is everywhere now and you have to build something and then we spend the next 20 years, you know, patching all the mistakes we made.

I really loved your podcast with Dex and Ian and Aaron on the OWASP Top Ten stuff because I thought it covered so much ground and. One of the things that Ian said during that podcast that I was like, Oh yeah, is. Authorization and authentication used to be, you know, mostly authentication. Then we did a tiny bit of authority and now because of the needs of agents, we have kind of flipped that and, you know, maybe do a little bit of authorization, but all authentication, but a lot of authorization. And I think we see that pattern over and over where like half of what we know is applicable and half of it we kind of have to throw away and rethink or rebalance or reprioritize.

The only way you figure out which half, though, is actually by building a bunch of stuff.

Allie Howe Yes, 100%. And also, like, new primitives are needed. Like agent sandboxes, I feel like have really taken off in terms of like, how do we run code quickly?

Kwindla Kramer Yeah. You did a great podcast with Ivan from Daytona.

Allie Howe Yes. Oh, and that was eye opening for sure, because like, I was all, you know, keyed in on sandboxes for the Security benefits. Like, oh, this makes total sense. Let's make the blast radius smaller. Like let's run Claude code, for example, in the sandbox, but the performance implications, which I mean, I would was included in that because I wasn't really, you know, deep in the weeds like building. So I didn't see those problems. But apparently a lot of people on Kubernetes have a the agents don't work as well. They're they've all these performance challenges and sandbox has solved that for them too.

Kwindla Kramer The Kubernetes is such a great example of like a core technology layer that we basically have to reuse in this era because it's support on say the hyperscalers is so fundamental now. But probably much less than half of Kubernetes is actually what we really want. If we're going to sort of run these stateful long connection but lightweight sandboxes. It's sort of just kind of the wrong shape. We'll fix it, we'll figure it out, but it's such a perfect example. I've actually been working on sandbox agents for the Gradient Bang game and I would just. You'd throw out a third thing. So you talked about security and also performance, developer ergonomics.

So one of the things we're doing with the game is kind of breaking apart the pieces. So it's not just a game you can kind of play as a person, but you can bring your own LLMS in to play with you or to perform, you know, tasks, long running tasks for you in the game. The only way to extend a world is to give people some kind of protocol to, you know, hook up to that world. And so I did build a Pipecat MCP transport, despite being grouchy about MCP. And so you can run these little MCP containers inside a sandbox and your game world hooks up to those MCP containers.

And it's a developer friendly way to kind of bring new capabilities into a system.

Allie Howe That's exciting, but I'm still like security brains going. And of course now we've got like larger like surface area and more things to bring the table A. 100% so. That's where that yeah, the AuthZ comes in of like, let's figure out like how do we put like a hard boundary or a box around this agent making sure it does the right thing? You know, not only users to the actual like game as a vector for prompt injection, but now you've got agent to agent communication.

Kwindla Kramer Yeah. And I mean, what is the metaphor even? Is that agent acting as you or is it tasks to do a specific subset of things for you or is it some entirely different thing we don't have a great, you know, name for yet?

Allie Howe Yes, exactly. And it's like, what are all of, like, the possible use cases are allowed, like questions. And one of the very first podcasts that I did was around the Darth Vader NPC that got released into Fortnite. So that was like, one of the first examples of, like, voice agents that I saw that actually, like, worked. Yeah. Like, but when it first got released, there was like, all these, like, bad things that it was saying, like, like racism, like, whatever. Like, I'm showing off the rails. But then like, within like, 2448 hours, like, I was like, I was looking at this, like, OK, this is another Microsoft Tay chatbot.

It's gonna get shut down. Like, we tried again. It didn't work, but they fixed it like very quickly and it stuck. And I was like, we're super impressed. I was like, OK, like how we figured out like guardrails then like what was some of the secret sauce that was going on there? I have to imagine, I feel like they finally thought through like all the different like areas of like these are the questions that we expect to have Anything like outside of that should just kind of be like shut down as like a like a validation layer around like the question like if it can take contains these things or it's not related to this, then like we're not even going to send it to the LLM.

Kwindla Kramer Yeah, I mean, I I think the answer is always defense in depth for security, right. Yes. And you know, way more than I do about security stuff. But whenever I'm thinking about how to take any of the stuff we build and put guardrails around it, the answer's never, I'm just going to put guardrails, right? It's there's a prompt, there's how you allow access to the external data. There's guardrail stuff, which you can implement in multiple ways. There are network egress and ingress gateway checkpoints. And you really ultimately in a production system, you will have all of those. It's just sort of getting there step by step.

Allie Howe Yes, 100% no for sure. Like I definitely believe in a defense in depth strategy for AI. And that's something that the product I'm building is supposed to like help people with is we would create like an architecture diagram like automatically of your code base or whatever, sort of like, you know, your PRD, whatever you can context you can send us and then we show you in that architecture map. Like here's sort of the areas where AI security risk exists. And then here's like some soft boundaries like Cardinals you can put on, but also some hard boundaries to like context, wear off sandboxes, other things as well.

So yeah, definitely like that's super needed. I feel like, and, and your game system that you built like today, like how did you add like hard, hard boundaries into that system?

Kwindla Kramer One of the fun things about that is we don't really have to, but we feel like it's a low stakes environment. And so actually that's great for seeing what people actually do and what happens. And you never really know how to build any system properly and especially securely until you actually have that real world data. And I do think that Darth Vader example you're talking about is a is a really good example. You know, you never know. I mean, it's different for different systems when it's safe enough to roll it out to production initially, but you can't ever get to security without some production experience.

And you know, so responsiveness of your development team is actually part of the security envelope, right? And as you said, you can take stuff down, but it's actually better if you can just keep patching it. So we learned so much from actually building this system than having real world people do fun stuff in them. I do have a hot take about hard boundaries that I really am curious what you think about. I've been in so many rooms over the last year with big company people who are talking about both security and reliability because they, they very much overlap, right? Especially in like a healthcare context where an LLM hallucinating is a patient safety issue.

And that's a, you know, broadly speaking, that's a very important part of security in, in, in healthcare. Lots of examples of that also with lots of people building security products that they are trying to sell to enterprises and then teams building these agents that are trying to get something out there to actually solve a very pressing need. Because once you see what these LMS can do, not shipping something is a is a big cost, just like shipping something bad is a big cost. My favorite example of that is I don't think anybody, including me, expected healthcare to be an early adopter of generative AI. And I was in lots of rooms with healthcare people who were blocking roll outs of generative AI two years ago and saying, we can't trust this anywhere in the patient provider work flows.

We're not going to build generative scribing. We're not going to build No, you know, we're we're just not going to let our healthcare providers use any of this stuff until we've locked down all the hallucination issues. But what happened was healthcare providers just started using ChatGPT because it saved them so much time and improved the quality of their work that they could deliver to patients. And people who are in healthcare are in healthcare because they want to help patients. And so if they have some tool that's going to be a step function improvement in the quality of care they're giving patients, including by saving them a ton of time so they can actually spend more time with their patients, they're going to use it.

Even if it's maybe not HIPAA compliant or maybe not blessed by IT. Like if it's a big enough benefit, they're going to use it because they care about their patients. And that dynamic within three months, that dynamic forced IT and security people at big healthcare companies to totally change how they were thinking about, you know, the bar for rolling out blessed solutions that were secure enough to roll out and, you know, develop a responsiveness and all that stuff became the most important thing, not getting all of the boxes checked the way you might have for rolling out a new database in a, you know, less than a year ago.

So that was super interesting. And I think my lesson from a bunch of these conversations, and I'm not 100% sure I believe this, but is we need to stop trying so hard to put deterministic wrappers around these non determining, non deterministic systems. Because actually you can't build these systems to be deterministic. And all the time you spend trying to build deterministic wrappers around them is just blunting your ability to actually get value out of these new systems. What we need are new safety mechanisms that we are comfortable with that are themselves non deterministic.

Allie Howe Yes, that makes sense for sure. Yeah, it's the same thing. I think when people were scrubbing like the data they would send to like LMS, it's like, oh, we can't trust like where the data goes and which I totally understand that, but then you're sort of like limiting the benefit of using that in the 1st place.

Kwindla Kramer Yeah. And maybe Waymo is a good example here. I'm. I'm consistently surprised by the number of people I know who have a like I'm not going to ride in a self driving car reaction. Yeah. When? To me, it's just so clear that these systems are better than the median driver and that is already saving lives. And if we rolled this stuff out faster, it would save really massive numbers of lives. And it's just a difference of perspective. And I'm not saying I have the deep understanding that that that the perspective I have should be the right one. I think we need everybody's perspective. But it's it continually strikes me as sort of a very different reaction to these new technologies.

And the Waymos are necessarily pretty non deterministic, right, But they are really good.

Allie Howe Exactly, yes. I feel like in order to like creep up that like autonomy curve where we get to the Waymo example, having those non deterministic guardians figured out and in place, like gives us the confidence level to say, OK, I understand this system will fail. In fact, I completely expect it to fail. Like I'm going to assume breach or something that Michael Bargury, CTO of Zenity, has said in his research time and time again, like prompt injection's a feature, not a bug. We can't fix it. So let's just let's let's admit that like even if we build it perfectly, it could still go wrong, which is something that Aaron Stanley said on a previous podcast with me.

And I think that's so true. So we have to assume that like these systems will hallucinate or will fail. There will be some sort of like instance where it doesn't work the way we want to, because then let's assume that like, how are we going to handle that from like a hard boundaries perspective? And obviously the stakes are higher with like a Waymo versus like something else. But I think it's a, you know, it's a worthwhile like conversation for security and whoever at the company wants to use that product, for example. It's a worthwhile conversation to have because we're going to do more help than harm by rolling it out.

It makes sense. Like the patient provider example you gave, I think there was some study, I wish I remembered enough to quote it, but it was some study that said there's this like human fatigue problem and doctors don't get every diagnosis or every X-ray scan read correct. And while totally yes. And while yeah, the AI like isn't perfect either, like it's actually was better than the doctors.

Kwindla Kramer Yeah, it's a nice way to reframe it that prompt injection is a feature, not a bug. I mean, obviously, as long as you think hard about it and understand what, you know, what the reframing is trying to do for you. I mean, another way to put that might be prompt injection is all we're ever doing. So the stuff we're labeling as prompt injection is just the stuff that we didn't mean for the system to do or didn't mean for the system to react to in a certain way. But all we're ever doing is like sticking this, you know, input into the LM and getting some output.

And that is a useful reframing. I mean and maybe in the Waymo example it's that non. Determinism is all there is. And once we wrap our heads around that, that frees us up to do the incredibly in depth engineering that the Waymo teams have done to make those systems extremely reliable. And I think maybe one of the reasons I'm more comfortable in a Waymo than, you know, some other people I know is not only have I written it a bunch of times and gotten used to it, which is the natural human psychological condition, but I know a bunch of people who worked on the Waymo engineering, you know, systems.

And I know to some degree how much engineering has gone into those cars and how much iteration and layered stuff there is that makes them work. And you know that that work has paid off. And if you have a sense of that, you know, if you have a sort of an intuition about how much work has gone into making these things reliable, it sort of helps. I remember in when I was in grad school, the focus in AI was on deterministic systems. It was on rules based systems. And neural networks were not super in vogue like they were around. There was work on them, but there was a real effort to make AI basically to take the rules based path and extend it, you know, into what we would now call AGI.

And we tried very hard to build those systems for many years and they got better and better and better, but they didn't get us where we are today. And the you, I think we all would probably agree now you can't, You could not build a deterministic Waymo. So wrapping back around to the prompt injection is a feature, not a bug. The non determinism is a feature, not a bug. But it does require new kinds of engineering. Work.

Allie Howe That makes sense. And then like, I guess what happens if the Waymo encounters a scenario that it wasn't trained on exactly or does something different? Like does it take the training it has and we feel confident that it's going to handle that situation correctly because of that? Or is there so much testing that goes into the Waymo that you think like every sort of thing it would ever see is covered already?

Kwindla Kramer That's such a great question. And I mean, the machine learning engineers talk about it as, you know, being in distribution or being out of distribution and have lots of interesting kind of tools, both, you know, mental tools and actual technology engineering layer tools to think about that. I think that the rest of us who maybe are coming at this not from the machine learning perspective, but from, you know, what, you know, Swix calls the AI engineering perspective, are making a mental shift to understand is these are black boxes that we cannot at least currently examine and debug in the same way that we examine and debug, you know, kind of traditional software.

And that mental shift is really important to make because if you don't make that mental shift, you are kind of handicapped in how you build with these new systems. I, I gave a talk at an event last year about this, a small model I've worked on with with some friends in the open source community and some colleagues at Daily that does turn detection for voice agents. And turn detection just means when should the agent respond to the person. If you think about a conversation like we're having, we do that in some super complicated, subtle, interesting ways when we talk to other people, you know, LLMS or the the full systems that we call voice agents that use LLMS are not as good at that yet as people.

And a bunch of us are trying to work on pieces of solving that problem. So there's an open source open data sets for training, open training code, open inference code model and the Pipecat ecosystem called the smart turn model. And I showed a demo of the Smart Turn model and talked about how we trained it and showed how you can contribute to it if you want to sort of get up to speed on small audio native models. And then a lot of people came up afterwards with questions, as people do. And one of the people who was really, really wanted to ask questions was all these questions were of the form.

But does it work this way? And I just had to keep saying, try to find a different way to keep saying, I don't know and it's impossible to know. And he kept being like, well, we could test it this way. And I'd be like, that's awesome. Let's build that test framework that will teach us something. It will not teach us how it works. It might help us find gaps that we could hill climb on. As the machine learning engineers say, it will definitely be a useful exercise if we're trying to build intuitions about this model. But if what you're trying to do is build an intuition about how it works and your analogies are how humans do that job, no.

Like, these models are black boxes. They are trained on a bunch of data. A bunch of stuff happens inside these very large connectomes. And, you know, trying to find the points at which you can instrument and debug what's happening inside those black boxes is totally a research project. It is not something you can do as an engineer. So that is a very important mental shift to make or you're just going to get stuck.

Allie Howe Absolutely. And then I, I deep dive in like model vulnerabilities at one point and then you can there's hidden back doors that's going to be in there. And so, you know, it's like Winter Soldier type thing. You say like the certain thing, the LM and all of a sudden it's saying something that you wouldn't have expected it to say or is completely different. This is sort of like trained away and hidden underneath the surface. So it's like, yeah, they're total black boxes. Can't you can't control the model at all. You can only control like what you're going to, how you're going to handle the output.

Essentially.

Kwindla Kramer I'm at root a network engineer. I fell down the rabbit hole of like large scale networks, low latency networking, distributed systems, programmable networks when I was in grad school. I sort of never really emerged from that rabbit hole. That's awful. And I actually think that was probably in, you know, in a small way, a useful prep for this era we're in now, because I used to have arguments with the theoretical CS people about what we, what we're talking about now. This is deterministic system stuff. And I would be like, I love this type theory stuff. Like, I find it super fascinating. Like I want to be better at type algebra.

But it, you know, you're, you're actually building this model of this system that we're working on together. And there's a network layer. And at that network layer, we actually can't sort of type the responses. Like it is not, it is not predictable what will happen. Especially when this server is serving lots of different requests, we have to think about these systems more. Like we would think about maybe an anthropologist, you know, looking at a super. Complex social system and then trying to build some categories, but also being humble about the fact that you don't really know how this thing works that you're kind of theorizing.

This is not a math problem primarily, this is an anthropology problem. And I, I feel like, you know, that set of tools where you're tracing your instrumenting, you're, you're building like massive numbers of test cases. That is what you need to do to like build secure envelopes around these models. You can't just write some equations and be like, this covers the the state space.

Allie Howe That makes sense. Yeah, I feel like some of the best like AI security platform tools that I've seen, they don't only do this like shift left thing where like they're looking at like model scan model security AI BOMs in the beginning. But then like the build phase or the test phase are also looking at like doing red teaming continuously. And they're taking the results from that red teaming. They're applying them to their runtime security, so their guardrails. So they're constantly evolving like, OK, like, you know, we saw this pattern now we're going to prevent these sorts of questions or we're going to check the output from the LM, make sure they're not saying anything like this.

And you couple that with like, a more like harder boundary that has that AuthZ piece, and all of a sudden you've got an agent that is a lot more reliable at least than what you started with.

Kwindla Kramer Yeah, no, that totally makes sense.

Allie Howe Super cool. And then one other question I had for you was around how you handle, like, long running, like context and like, memory for that, like the game agent. I found when I've, like, interacted with like, cloud code for long enough and like other agents, like you run into this compaction problem. How often does that game agent have to do compaction?

Kwindla Kramer The real answer is you don't really know because we're still experimenting, but you definitely do have to do compaction. And so the pattern that we use for compaction in the game agent, which we also use in some of the long running voice agents, is there's a voice loop that you can never block. And that voice loop is, you know, voice input, context management, LLM inference, voice output. There's a lot more pieces in, you know, production agents, but that's the basic voice loop. The voice loop really needs to respond in under 1500 milliseconds. Ideally, you know, more like a second. That's what humans expect in conversation.

And so no matter what else is going on, you really want that voice loop to not be blocked in parallel. You can run a second inference call every time you trigger inference. And most of the time that inference call is a no op or you even gate it and you never even run it based on, you know, a guardrail rule basically. But when you actually do run compaction, it runs in parallel, not blocking the voice loop. Basically you mark your place in the context and then the compaction agent can compact everything it's seen so far and it can compact it based on the current stuff that's going on in the context.

You can prompt it in different ways. You could be arbitrarily sophisticated about how you do the compaction and at the end it can take as long as you need to. And at the end of the compaction process, you just swap out everything before that marker you made with the summary and then change that context for all of the agents that are sharing that context, right? So the voice loop gets that new context between turns and you just keep going. That gives you infinite context, which is great. It's lossy, of course, and there's lots of interesting research questions about how to do that compaction well. But the critical idea for the voice agents is you never block that main voice loop.

And we use that kind of pattern a lot where you're doing long running tasks or you're doing parallel inference and trying very hard never to block the voice loop. And I actually wish Claude code was built that way from the ground up. And I love Claude code and I spend like 10 hours a day in it. But I, but because I write a version of this stuff, you know, myself professionally, I'm like, why didn't you know why? Why does it not compact asynchronously? And I, and I actually think every human facing, every agent with a human facing surface area is going to sort of migrate in this direction where you have like AUI loop that you try very hard not to block.

And you figure out how to do almost everything else that you can't do in, you know, a couple 100 milliseconds in some kind of async process that shares part of the context that can inject into the main context. There's a bunch of patterns for that that like we, we tinker with and, and now sometimes ship to production

Allie Howe Super cool. So when you do that thing where you mark the place of the voice loop and then it's doing the inference and while it's waiting for the inference to complete the parallel loop, like what is the voice agent doing? Is it like it's just continuing the conversation? Like it, it doesn't even really care that a compaction is going on.

Kwindla Kramer You use this pattern for things other than compaction. So sometimes you want to tell the voice loop incrementally, hey, this thing is going on. I mean, you can sort of imagine doing that for compaction, for like UI Polish. If you somehow could know that, you know, we're 20% done with compaction, you might stick that metadata event back into the voice loop context periodically. And then if the user can be like, hey, you're, you're getting a little slow. Are you compacting your, your voice loop would have the context to say, Oh yeah, I'm already compacting, I'm 20% done. But that's really just kind of UI Polish.

Fundamentally the voice loop just keeps doing what it's doing and it doesn't have to care about compaction. And when the compaction is done, it just transparently to the whole rest of the system sort of does surgery on the context.

Allie Howe I see. So it's like when that 2nd loop was like yielding some sort of event like that gets into the context and then the voice agent can. OK, interesting.

Kwindla Kramer Yeah, exactly. And so the fun. So I think, I mean, there's a bunch of different metaphors people are are are sort of starting to think through for multi agent systems. There's like agent handoff and state machines. I tend to think a lot about state machines, but also just about this sort of parallel loops idea. And the thing that that unlocks for me is that metaphor highlights the fact that context coordination is a hard problem. And if you have a bunch of different LLM inference calls happening in parallel loops, then what one of the things you have to sort of think hard about is, well, they probably share context, but they probably don't share all context.

And that's a powerful thing because it lets you specialize these agents. It lets you give them just the right kinds of tokens. And like the decks context engineering metaphor, but it's a hard problem. Like what? What do you share? When do you share it? How do you keep the context separate but not not completely separate, all that stuff,

Allie Howe Yes. And then how do we secure the context window such that doesn't get poisoned? And I don't know that I've seen a good way to solve other than like don't put any sort of untrusted context within the context window. So as soon as you poison your memory, like you're not getting it out, it's getting there forever.

Kwindla Kramer That's right. Yeah, totally. So yeah, first, don't put bad stuff in. It's a really good rule. And as you were saying about MCP, like the MCP spec is evolving and one of the ways it's gotten better is there are now good sort of boundary points where you kind of don't you don't pollute the in context as much from the MCP context anymore with kind of state-of-the-art use of MCP. And we're all figuring sort of this stuff out in parallel.

Allie Howe Yeah. So that was super cool, I think. Didn't that come with like the code execution thing they did? Yeah, Yeah, that was a really good like security benefit as well.

Kwindla Kramer I do think the code execution stuff is a super interesting thread to pull on too. One of the things I like about the Gemini models is that they were trained from the ground up to to write little bits of code as part of their execution loop. That seems super powerful, but also something that is not completely leveraged yet in those models or in the API layer they give us to access those models. You can see it though, if you kind of look at the traces and you look at how the Gemini models do tool calling. Traditional tool calling looks very much like structured data operations because it is it grew out of, you know, models gave us structured output.

We started abusing it to do tool calling. The labs were like, oh, that's actually a real use case. Let's let's train tool use in as a first class operation that these models know how to do. The Gemini models were pre trained with a little bit of a different idea, which is that it's all just writing little bits of code and executing it. Again, not fully realized yet, but sort of I think it's actually sort of the next step there. But what I want as an application developer is I want more API surface area that lets me control that. I want to be able to use the fact that the model can write little bits of code more directly.

And you see with the amazing things you can do with something like Claude code that if you have that at the harness layer, that's incredibly valuable. Like the model and the harness both sort of know about code and, and you know, you the, the, the model writes a lot of code, the harness executes it. It, it's super, super powerful. One of the trends we see in all AI engineering is that the things we do in the harnesses, when they're really valuable, they migrate down and get better support in the model. And I think that's one of the things going to happen in 2026 is sort of we we figure out how to fine tune, how to specialize smaller models on code execution loops.

Allie Howe Yeah, that makes sense. I feel like coding agents are really just becoming everything. Agents.

Kwindla Kramer Yeah. I mean, Claude, code is my operating system now in the same way that like for 20 years Emacs was my operating system. Like it runs everywhere. It's super extensible. So I can do lots of things in it, and the difference is that it can kind of self improve in a way obviously that Emacs never could. I had to write a lot of Emacs Lisp to make Emacs do what I wanted to do. Now I tell Claude, hey, that thing we just did make a skill. And it, you know, maybe takes a little nudging or alliteration, but it makes a skill. And then next time I want to do that thing, I just use the skill the Claude wrote.

Allie Howe Yes, for sure. I know it's getting so powerful. Especially just figure out different like paradigms at work. Like I think people got, you know, really hyped up on this like Ralph loop that came out like earlier this year, wouldn't come out earlier this year, but it got really like well known in the last like few weeks or whatever. But it's only so powerful. It's like the put stuff you put in the harness. Like for example, like one of the things I put in like in my Ralph loop was like a, like a hook for it to run, like socket, for example, to check if anything this coding agent's downloading.

Is it a secure package? Because the more I can trust this loop to just run and be secure, the more I get hands off like, great, I'm gonna go to the beach. So you write the code, we're good.

Kwindla Kramer Totally. The Ralph loop stuff is so powerful and. I think it's a divide between people who have sort of really started to wrap their heads around how do you leverage the capabilities of this new stuff versus people who just haven't spent enough time on it yet because it's controversial on Twitter whether that's a good approach. And I just think that's crazy that's it's not a good as we've talked about in sort of other areas here, it's the only approach. I mean, the thing we figured out as a community in 2025 is that an agent is an LLM and. Loop at the beginning of 2025, everybody said 2025 is the year of agents, and I was like, yeah, maybe there's a lot of interesting stuff.

That's one way to think about it. At totally 2025 was the year of agents. Like that actually came true both in the, you know, bleeding edge people experimenting and in the Enterprise. That idea that an LLM needs to be wrapped in a loop and it can do a bunch of stuff if you do that right, transformed everything enterprises are doing with workflows. That gave people a way to think about like, oh, what do we, what can we and should we be building if we're the kind of enterprise AI team at a Fortune 500 company? Totally changed how we think about what we can do with coding agents.

The fun thing for me was that we already thought about agents that way just randomly because that's the only way you can build a voice agent. A voice agent is an LLM in a loop. It's voice input, context management, inference, voice output. There's no like literally no other way to build a voice agent. And I think that is true actually for everything. And I think that's what we figured out collectively together in 2025 S The Ralph Wiggum thing is just like a very clear explanation of that for writing code. And then it opens up that whole thing we were talking about where like half of what you know is super applicable and half of what you know is like you just have to throw it away.

In theoretical computer science for 70 years, we've had like, how do you do loop termination? As I like a first class, you know, mathematical level problem. This is the problem in Ralph Wiggum stuff. It's the problem in voice agents. When you're on the phone with a customer support voice agent, one of the things we have to test for is does the voice agent know when to hang up the call? It's loop. It's loop termination. And in Ralph Wiggum stuff like you have the security stuff and the context engineering stuff, but you're also like, when are you done? And one of the first demos I saw of Ralph Wiggum stuff like maybe October or something last year, the person giving the demo highlighted the fact that they weren't very good at writing the Ralph Wiggum stuff to stop.

And when they would work closely with it or look at the traces, it would do the plan, do the plan, do the plan, do the plan, do the plan. And then when it ran out of stuff to do on the plan, it would just hallucinate features. Oh, wow. And that's obviously a problem from a lot of respects, including a security perspective, right? Yes. I think the demo, the whole room laughed so hard at that demo because they set it up really well. They were showing all the features in the plan and the Ralph Wiggum loop sort of knocking down the features one by one. It got to the end of the plan and it was like, I'm going to add enterprise OAuth.

Of course, you are right. Like you've trained on the whole Internet of product specs. Enterprise Oauth is in every product spec at some point in the maturity of that product, right? So the loop is like, OK, that's what I'm going to do next.

Allie Howe Yes. No, I I've seen that too, like called like an optional at the end of like the plans of like, we could do this and it's like we could, but we're not going to like, we're going to stop at this condition. Yeah. You really have to be like thoughtful of like, this is where I want it to stop. This is like the different like expectations I have around if it's going to download the package and needs to make sure that it's like a secure first before it does this. So yeah, having those guardrails in place for those boundaries is really what unlocks the power of the Ralph Wiggum loop.

And I feel like that's the biggest thing that I learned between 2025 and 2026, like 25 was the year of agents and 2026 is the year of enterprise agents that actually work. It's like, OK, let's take the time to really think these loops through, give the loop the and the harness what it needs to be able to do this securely, performantly and effectively. And that's how we unlock like so much value. I mean, now I'm writing code like 10 times faster with this loop. So much So.

Kwindla Kramer I mean, you were talking about best practices. Best practices are so unevenly distributed right now. So often a big company that's using Pipecat comes to us for, you know, some kind of help or commercial support or whatever. And because Pipecat's totally open source and vendor neutral, like we've often not known that they've built a bunch of stuff already. And part of what they're coming to us for is like the kind of expertise that I like to think we have about like very low level, you know, hardcore engineering stuff. But at least half of what people are coming to us for is what are the best practices?

Like we have this POC, it's clearly promising, but how do we get from POC to production? And I think another way to frame, you know, 2026 is the year Enterprise agents might be right now. It's amazingly easy to build a POC for all kinds of different agents. We've got like good foundational kind of building blocks in, you know, the large models, open source tooling, all that stuff. It's actually quite hard still in general to go from POC to really where you need to be for production. And there's a little bit of cognitive dissonance there because it's not usually quite that big a gap in technology between, oh, I can build the POC and oh, I can get this to production, but we're so early and all this stuff.

But actually at the point where the POC's are like super, super compelling, like genuinely compelling, but too early to have the best practices kind of fully embedded in everybody's experience in the tooling and you know, the work, the work processes, all that stuff.

Allie Howe Yes, absolutely. No, I totally agree with that. I think that's a really good take around the fact that the best practices are not evenly distributed. And just hearing you talk about having the two loops for the voice agent of one that's doing the parallel and like compaction or inference and you've got the other ones like a constant running like loop like that's. And then like, OK, like I'm going to go try that now. Like that makes like total sense. And I'm not sure that I've seen that built into like frameworks I've tried or it's really not.

Kwindla Kramer And I think I mean, again, we got to that, you know, earlier than other people in this narrow domain of voice agents just because we had no choice. And now I think it is useful in basically every kind of agent. I, I was talking to a very large financial services company about their POC and we were talking about latency and they were talking about how important latency is and how they're a long way from where they want to be. And I was like, yeah, latency is always really important. And like I people, you know, there people who know me have like a, a bingo card thing, you know, where every time I say latency, they like whatever, you know, because I'm always talking about latency.

And then I looked at their prototype and they, and it's a text agent and they were saying, you know, sort of next step is voice agent. And they showed me the prototype and they were like, yeah, the, the turn latency in the text chat is 45 seconds. And I was like, I thought we were talking about 3 seconds, you know, because usually our challenge getting from like a text agent latency to a voice agent latency is going from three seconds to going to under a second or going from 5 seconds to under a second or whatever, 45 seconds. And they were like, and we're going to roll this out to customers in a, in a like early, like small release.

I was like, no, you're not, because even with a text agent you have 45 seconds is not going to work. They were like, well, we're multiplexing, you know, human customer support agents in text chat sometimes 45 seconds. I was like, yeah. That's true. So I can't argue with that. So maybe this is better, but you can't, I mean, my perspective, you can't ship a text chat agent with 45 second latency. So why is it so slow? And they were like, oh, well, we have 7 layers of guardrails. Yes, yes. And that's, you know, back to your world of like security and, and, and the, the hard boundary stuff.

It's like, well, I, I think we can fix some of that. You know, I think we can put some of that in parallel inference loops because just like compaction should have an async, some guardrail actually should happen async because you can align the timings where you know, the, you know, the, the, the threat model is that you don't care if the guardrail takes 5 seconds to run. It's OK. And so that's a lot of what we do is sort of pull stuff out into these parallel inference loops these days.

Allie Howe Yes. And that's here, Jack. I feel like so many different providers started providing like guardrails. And that was such an easy concept for people to like understand like, oh, I could just add the same like take this SDK, like add it to my code and like there's a guardrail and like I'm good. Like I've got security now. It's like, yeah, when we talked about a defense in depth solution, that's just one piece. But then the latency issues that arise from that, it's tough.

Kwindla Kramer Yeah, it's really, it's really tough. I mean, you will laugh at me, but I think maybe 2026 is the year of latency, more people caring about latency of these systems. I always used to, you know, fuzz at the people we work with at the big labs who would like publish their benchmarks to their new model and it would have token throughput. And I was like, sure, but I don't care because at this point, you know, this was even a couple years ago. All your models are like as fast as human speech. That's my bar. Once it's faster than human speech, like you're faster is nice, but it doesn't really matter.

What I care about is time to 1st token. If you don't publish time to 1st token on your big fancy chart you put up on the screen behind your CEO when they talk about the new model, I don't care. Like give me time to 1st token. And now I see more, more and more often I see time to 1st token on people's like launch day, you know, metrics charts. And that makes me really happy.

Allie Howe Yeah, I know that is important for sure. Yeah, I know one company, I won't say their names. I don't know the ethics of of this either or where this is going, which is an interesting like question of they're willing a healthcare voice agent, you know, patient facing, but they don't say on the phone call that it's an agent. So they're like latency is super important to us because if it takes too long, they'll realize that they're talking to an agent, not a person. And we don't want that. And it's like, I don't know the ethics around like, do you have to say you're an agent?

Like they know that. But yeah, like, like you said, like if you don't get latency right, then it's not going to be like a natural conversation.

Kwindla Kramer I think you should always say this is like a digression from what you were just saying. But I and we always tell all our customers, our very strong recommendation is say it's an AI agent for a bunch of reasons. One is it's worth giving people that bit of context in with today's technology, most people are going to figure it out anyway. Like let them know that, you know, they know, like it's just a better sort of psychological user experience. Secondly, if we don't do that, we're going to just end up getting regulated and have to anyway, right? And this is like a theme in the technology industry for the last 30 years.

Like, let's do it right from the beginning. Because if we don't, we're going to get a bunch of regulation. And the regulation is not going to be ideal because like no large, complex system that humans ever engage in is ideal. So like, let's do it right. And then, you know, we can regulate the other stuff that's harder. This is not hard. And then the third thing is people actually don't care mostly and often forget. We have lots and lots of call transcripts and call recordings from real world agents. And you can very clearly see that, you know, a few minutes into a conversation. Even when you tell people it's a voice agent, an AI at the beginning, they just kind of forget if it's good and they just talk to it and they say please and thank you and they do all the stuff we do when we talk to humans and they're happy.

So if you get the if you get the agent right, it doesn't matter. So you might as well say it's an agent at the beginning of call.

Allie Howe Yes, I agree too, especially if like it does grow off through all this. At least you like said in the beginning recently.

Kwindla Kramer Yeah, 100% no. There's also an interesting dynamic where you don't always want to try to pretend to be a human. And we, we often see a little bit of a divide between product teams and executive teams at very large companies. And the product teams that sort of dig into this stuff often see a bunch of real world user reactions. And they're like, like, I like the voice that sounds like a human, but actually kind of in a call center context, maybe a slightly more robotic voice, especially if you've, if you've built the agent to try to be very guardrailed and to do very specific things very well.

Your, your uncanny valley place is a little bit different. And maybe we have some customers who actually think a slightly AI voice. It's very hard to define what these things are, but it's actually the right thing. Executive teams at big companies almost always do a vibe check on an agent by calling it on the phone. And if it quote UN quote sounds human, they're happy. And if it doesn't sound human, they're not. And that divide between sort of the vibe that like senior executives want and what product teams that are in the trenches are, are sort of learning is interesting. And I think, you know, maybe in 2026 we'll see some evolution there too.

Allie Howe Yeah, that's a good point. I really like the voice of the game agent that he makes. It sounds like it sounds like a human, but it's got some sort of small bit to like. This is definitely an AI though.

Kwindla Kramer Yeah, I actually asked. So we got Cartesia to do that voice for us and I actually asked them, can you give me a voice that sounds a little bit like a ship computer? Like I don't know what that means exactly, but like the original idea for this game was I used to play very like very first thing I did a lot on the Internet, I think probably was play this old school BBS door game, text based, where you're flying around in this spaceship with text commands. I wanted to build that, but with the ship being a computer, you could talk to like innumerable science fiction research that was in Genesis of this game.

And so in my imagination, this is very much not a human I'm talking to. It's human like it's open-ended. It can have a real conversation with me, but it's the spaceship. I'm talking to the spaceship. And that's really what I wanted. And I think that idea that a lot of these AIS we talked to, we don't really want to think of them as human is going to be something like there's like a very large amount of exploration we can do in terms of user experience, application design and all that stuff.

Allie Howe Amazing. Yeah, no, I'm super excited to continue to follow the game and Pipecat and daily. Is there anything that's upcoming that's new with any of those that you want to like foreshadow or tell us about?

Kwindla Kramer So I do think pushing on beyond the voice agents towards what's the multimodal LLM core software experience is really interesting. And I'm, I do a lot of that like recreationally and a lot of people in the Pipecat community do so like building super responsive multimodal AIS that UIS that are completely driven by an LLM that's actually sort of originally the kind of stuff we built Pipecat for. So if you're interested in that stuff, come hack on it with us. Like it's, it is going to be how we build all software and all user interfaces in the future. So come invent that with us. And if you just want to play this game we've been talking about, it's gradient bang. com and we're going to open it up to everybody to play.

So come sign up.

Allie Howe Yeah, I know people should definitely do that. I might go check out the multimodal stuff because I think one thing I heard from a customer yesterday for my product, for the architecture diagram, they're like, oh, we want to give you not only our code base, but we want to also like, you know, tell you either text or verbally, like this is where this is deployed or here's some extra context. And it's like if I could take maybe like a PRD and some code from them, and also maybe something that they say of extra context like that, I could make a better analysis for them.

Kwindla Kramer One of the use cases we see really growing is user interviews. And there's a bunch of different kinds of them. But but it's the kind of thing where, you know, a big company used to hire, still does hire a team of consultants. And the first job that consultants do is like schedule a bunch of conference rooms and bring everybody in who, you know, knows anything about the project they're working on and interviews them, records those interviews and transcribes them. And that's the input to like the consulting project. Now you can just talk to an LLM to do all those interviews. And in many ways people like it better because the LLM is like infinitely patient.

You can talk as long as you want. There's no, you know, there's all these strengths that LLM's have that back to the idea of should this be human? Like, sure, it should be human like in a bunch of ways. It should have capabilities that, you know, software hasn't had before. But there's a bunch of strengths that LLM's have that humans definitely do not have, including things like being infinitely patient that we haven't even and begun to really figure out how to kind of leverage yet.

Allie Howe 100%. And we're seeing that in security, too, with like, the context or the consent fatigue problem.

Kwindla Kramer Oh, totally. Yes. Yeah, yeah, yeah. Oh, man, I would love to hear you talk about that forever because, like, consent fatigue is like such an unsolved problem with traditional, you know, security design.

Allie Howe Yes. And I don't know that I have a good answer to that today, but it's something I'm thinking about, like for 2026. I think it's one of our biggest problems as the community, even security we have to solve. And I think maybe today, like it's best to have like the LLM do a lot of the judging just because like the conservative program was so real, but also like have human and loop first the most critical parts, but they're not going to get that fatigue. But also architect your system in a way where you're, you know, you don't even have to have that like Checkpoint, for example, like the Ralph Wiggum mode.

But if I'm running it in a sandbox and I've I've got the different like hooks in my harness that say like if we're going to download a package, make sure it's secure and make sure that it runs the lints and make sure it runs the tests. And like you've got all these conditions in place. Like, yeah, I feel a lot better about like me not needing to be involved so much. Yeah, totally. So OK, we can talk forever. This is a really good episode. Super excited to continue to follow your work daily and Pipecat and to continue to follow the game. Kwindla, where can people connect with you if they want to join in on the conversation?

Kwindla Kramer You know, I'm easy to find because I have an unusual first name. So it's KWINDLA Kwindla, and I'm Kwindla on Twitter and Kwindla on LinkedIn. So please, you know, connect if you're interested in this stuff.

Allie Howe Amazing, Super easy. Well, thank you so much for your time. We really appreciate it and we'll talk to you again soon.