Insecure Agents — Episode 32: Geoff Huntley, Founder of LatentPatterns.com
Podcast: https://insecureagents.com
Listen: https://podcasters.spotify.com/pod/show/insecure-agents/episodes/Ep--32-Hyper-personalized-Software-and-Software-Factories-with-Geoff-Huntley--Daytona-Compute-e3j15la
Transcript (auto-generated, lightly edited for clarity).

Allie Howe: Welcome to another episode of the Insecure Agents podcast. I'm your host, Allie Howe, and today we are at Daytona Compute in San Francisco. I have Geoff Huntley with me today, founder of the Ralph Wiggum Loop. Geoff, would you like to introduce yourself?

Geoff Huntley: Yeah. Hey, folks, I'm Geoff Huntley. Some people know me as the person who created Ralph Wiggum Loop, which was essentially A technique for allocating A context window in a particular way and for novelty factor to show the power of allocating memory in the right way. I said it's just a bash loop. And it's quite interesting. Like all on the All In podcast yesterday, we had a couple billionaires when the Department of War talking about Ralph Wiggum that that kind of flipped. That was a bit of a table flip moments like what? What's going on here? I'm here in San Fran and going to be going to New York and a few other different places.

I've got like 95 days on the road now. I'm doing conference talks and yeah, really about Ralph explaining people that context engineering and all the basically introducing him to the new computer.

Allie Howe: Amazing. Yeah, software engineering has really changed in the last year, but really in the last like 3 months. Like I remember like Ralph's been out for how long now? Like a whole year or?

Geoff Huntley: That's more, yeah. It's like 14. Months.

Allie Howe: OK, 30 months. Wow.

Geoff Huntley: When I first discovered it, I just sat on it for about 6 months because it was kind of terrifying. It still is. And then I came to San Fran and then I ran into decks and I showed decks and at a meet up and we're all kind of terrified lately. Afterwards. We was like 3 hours after the meet up and we're just talking about the implications. And it took it around about six months from that point forward for it to heat the mainstream. But it's still not even mainstream. And the net result of Ralph or basically allocating the context window in a particular way is software development now cost $10.

40 to an hour. And that's well, a burger flipper at Maccas gets paid more than a software developer now. Now our professions change because now everyone is a software developer. We've gone from a scarcity knowledge, the scarcity of of knowledge society. Now it's ubiquitous. We still need software engineers, we still need software engineers, but what the skills they need and the knowledge they need is very different. And what I'm doing now is I'm actually building something called latentpatterns. com and it's teaching. It's an educational, educational site which actually teaches people all the concepts people need to know now that AI is here. It's been really interesting a couple of days here at the conference because of speaking with all the different vendors here and they all understand that like AI agents, it's the unit economics of business have fundamentally changed, fundamentally changed.

Allie Howe: We should get into that, yes.

Geoff Huntley: And they've got, they've got some problems because they're about AI agents and their customer base wants automation, but they don't know anything about AI agents. Exactly. And there's some enterprises today that are in boardrooms and meetings asking for more headcount, and that's not what they should be. Asking lifts me up. That's yeah. I don't know how this is not the story of the story of every day. Now we see the layoff stories, but right now someone's going to be listening to this podcast and they're going to be like in a meeting before or afterwards and they someone's going to be asking for more people. Yeah.

Meanwhile, there's there's five people, eight people companies. There's one people companies running their company 24/7 with automation. It seems like we're in this upside down world.

Allie Howe: Yeah, no, for sure. And I think the drift and how people are utilizing the real power of AI and automation is going to get exponential over time because there's people that like haven't even picked up a coding agent or haven't even picked up the Ralph Wiggum yet. But the people that have are starting to see an ROI that's putting them so much farther ahead in terms of how many agents they can orchestrate as one time. Agents are writing, you know, all of their software now they're reviewing all of their software. Humans are out of the loop completely in some cases which have done correctly with the right guardrails.

Also with the right security in place is a huge advantage.

Geoff Huntley: Yeah, it's wow. The matter of I guess so for people listening is there's a lot of people who think that they can just wait for the latest model and then they just think just being AI enabled again, the latest advancements of AI is just like new models out will adopt that model will be good. And there's people who haven't even done that yet. And that terrifies me because like, AI is here, you might not have noticed that it's at your doorstep because it's burrowing under your house. It's literally burrowing under your house. And it's not just a matter of just what the latest model gets better.

No, there are people out here Usain bolting on on this escalator. Listen, listen, this travel lighter of AI and this travel AI is going so fast and they're Usain bolting on the escalator. They're building workflows. They're getting answers about security, they're thinking about sandboxing, isolation, workload, identity. There's founders out there who made adjustments to their workforce two years ago. Like there's one founder that's that sent me a message couple weeks back. They went from 60 people down to 30 people. Not to celebrate any of these things, but like just straight up what it was was they just refused to backfill anyone who didn't understand AI was real two years ago.

They've already done that transformation. And now with 30, with 30 engineers, they're getting like 60 times the output. Like they're getting more output with less people. Like, so there's the people like I'll wait off, wait off for my transformations. I'll see what's going on. It's weird. If people haven't been paying attention, they're actually late.

Allie Howe: Yes, no, for sure. And I'm super excited about the, to learn more about the educational tools that you're building because I feel like a lot of people are in this position where they want to start using this technology. They want to see it come to its full potential in terms of how much automation they can create for their company. But they're just like, not sure where to start. And they've got this like dread of like, Oh my gosh, like I'm so behind. How do I like catch up and like someone else is out there, like orchestrating 100 agents at once. And I don't even know how to do that.

Is that something you will help with?

Geoff Huntley: Yeah, it will be.

Allie Howe: OK, good.

Geoff Huntley: Basically what I'm doing is the concept is the notion of a digital twin. So I'm turning myself into to a digital twin and putting everything into my brain from all the the wild and unhinged things I've been doing LLMs over the years down in a way that's accessible to a reading comprehension of of 11 years old. There's a thing called the Flesch-Kincaid. I think it's Flesch-Kincaid scale when it comes to education literacy. So making it really approachable and doing pathways for engineers who want to learn to be product managers, product managers who want to learn to be engineers and all the different designers. It's really ambitious and it's I'm doing it by myself.

I'm going for the one man challenge.

Allie Howe: Interesting. I'm wondering how do you see security fitting into how software engineering is changing in terms of like, what guardrails can we put around all this automation? And I know like when I've run the Ralph, we're gonna look, I was like dangerously skip all permissions. So I could just like, you know, go and do what it needs to do super fast without, you know, my inner involvement. Of course, that comes with some risks and I know people will run it in sandboxes like that can help. What are you seeing in terms of the best strategies there?

Geoff Huntley: Well, I'd like to put out that this is not a new concept like this is. This is the same old endpoint management concept. It has the exact same risk profile as a supply chain attack and payment store. Dodgy endpoints compromised. So we've always needed sandboxing of developer endpoints. We've always needed it. It's just that this AI is here and the topic of agents, that's just where the funding is going and that's where the narrative is now. But it's it's exactly the same problem. So companies who invested have invested early for moving workloads off employees laptops and operate with the mindset it's not if it gets compromised, it's when.

And they think about blast radius topics. So if you're using you like a laptop and you do a dodgy npm install, well, you already got a way to pivot to essentially steal the cookie jar of the browser or Slack authentication tokens. And then you got all the gadgets you need to actually move largely through your networks. Well, if you're developers for the last couple years have been using remote, like remote developer boxes, Well, there's no browsers on that, right? You're already done the same the, you've already done some of the transformations needed to actually get the juice out of AI and AI security because you've already moved your workloads off the local computer.

So it's, it's, it's, to me, it's, it's not a new thing. It's the same old thing. But if people didn't invest into doing the things they should have done a couple of years ago and they think they can just adopt AI now, no, there's some stuff they needed to do a couple years ago to be able to exploit AI.

Allie Howe: I totally agree. It's not necessarily a new thing, It's just that it's a heightened awareness around like how much faster and how much more severe could this be with the blast radius. And I know with like coding agents, even if you put them in a sandbox, like if they have the network capability and download whatever npm packages they want, essentially they could, you know, download something and get compromised there. So it's not a new problem, but it's just the scale of it. The sky a little bit and it's the the new attention of it. Interesting. What are some trends that you are looking forward to or have your eyes on for how software engineering will continue to change in 2026?

Geoff Huntley: Now this is going to seem really unhinged. It's gets in really unhinged. I think we're entering into the era of hyper personalized software now. This is not the first time we've had an era of hyper personalized software. I will put forward that software has actually creation software has actually regressed back in the days of hyper personalized software. So FoxPro, Delphi, Microsoft Access, every business had hyper personalized software to allow them to run their businesses. And I'm not here saying that those languages were great etcetera, but their productivity was amazing. They could customize anything they want. They didn't need to get some sort of workflow synchronization engine like Zapier and like glue together all these different slices or spend 2 weeks building some corporate website internally.

No, they just dragged and dropped on a control plane surface. Now this is where it gets a little bit unhinged. This is where it gets a little bit unhinged. I've been playing with basically approaching everything we've learned for the last 20 years to rebuild back to that vision. So what I've done is in latent patterns, the actual product is the IDE. It's not cursor, it's not like the classical developer IDE. It's just like Microsoft Access. So when I build latent patterns, I build latent patterns with latent patterns. So I pop a designer, I'll show you afterwards. I pop a designer and I can change anything I like.

And I don't mean just like content. I mean like business functionality, adding new features and what else have you. And what happens is the engineer, product manager, personal marketing, the founder, they're all building the product with the product. There's no separate tools. I've got some early prototypes and I'm really, this is what I do. I do unhinge things and playing the research in these areas. And I think it's going to be going to happen. I don't know for sure. Anyone who says they know for sure is selling some BS. All you can do is really like color between the lines and the patterns you see, because you got to build as if inferencing is going to get fast.

Yes, I think. That's it was really fast, 200 millies fast. So if inferencing takes 200 millies to do, what about CI/CD I think. It's a that's just fine. Yeah, yeah. But the principles of CI/CD and verification of software is important. So you still apply that. You've got to have your engineering hat on to figure out how to respectfully release to production. So we can still apply all these lessons, but CI/CD, all these different separate tools in the developer tooling ecosystem, I don't think that they're a primary, a persona in developer, in developer marketing anymore developer tools anymore. I see entire categories of developer tooling just gone.

Totally like traditional SaaS just gone. I've been writing about this for a while now actually, so this is for me. We've been latent patterns like I. It's the funniest thing I see the world now As for vendors, as are you a utility company like water, electricity, petrol or are you something I'm going to climb. So when I see something out there in the world, I take a screenshot of the feature on their marketing site. I take their marketing copy and I put it in as a prompt and it's like I go window shopping and I thought that's mine now. And there's and that allows for this type of personalized software.

And I think we're going to see a lot of hyper personalized software. And if inferencing or when inferencing gets to the point of 200 millies, it's going to result in some very, we're still going to be engineers, but the practices that we do today are going to be very, very, very different, very, very different. And I'm, I'm wondering if we will get back to essentially Microsoft Access or Visual Basic 6. We can just move widgets or components, say like you want Calendly. I took a screenshot of Calendly. I've got now got Calendly. I wanted a Trello for support desk. The screenshot. I never Trello my software.

I wanted ACRM, so I went to Pipedrive and I went for the marketing materials and now I got a Pipedrive and then I just glued it all together. I think we're going to see more and more of this stuff, typically about a year, two years. Because I discovered Ralph really early, I had time to think about things. When I first discovered Ralph, one of the first thing I did that really spooked me was I ran it in reverse.

Allie Howe: What does that mean?

Geoff Huntley: Well, most people see it as a bash loop for building. What's the reverse of building?

Allie Howe: Planning to build like a PRD like. Reverse engineering.

Geoff Huntley: Reverse engineering. So how did how did we get the AMD CPU? What's the AMD CPU? It's a tale, a tale, a tale of time about intellectual property here in the valley. So how it came to be was AMD was behind Intel and what they did was they hired an engineer to reverse engineer the Intel CPU and they wrote a technical specification, actually really dodgy work, like a legal type of work, but if you approach it in the right way, you can create a specification. So what you do is you give that engineer who did that thing, give them some the golden parachute, say hi, and you can never work with that person ever again.

But that new specification is brand new intellectual property. Some manuscripts he'd made, and you give that manuscript and he spent a lot of time making that perfect specification. And that manuscript was then given to engineers who are able to build with it.

Allie Howe: Interesting. It's the same thing you're talking about. Take the marketing copy and then create essentially new intellectual property.

Geoff Huntley: Correct. So when I discovered it in Feb the the first thing I did was I threw it in reverse. I wondered if history was going to repeat itself because the spec based workflow got these loops to automate. So I took HashiCorp Nomad which is the infrastructure orchestrator and loose numbers. 80% of source code is available on GitHub, but it's licensed under the BSL license. There's a lot of venture capital companies who have sought protection through custom licenses to stop AWS from posting it. And I ran in reverse to deconstruct from the source code back to specifications exactly what AMD did with that that engineer. So I now had clean room specifications, kind of weird, kind of scary, scary, kind of scary.

And then I only had 80% because the other 20% was enterprise features. So I took the marketing material, the support manuals, all the other things, and I was able to reconstitute a faithful 20% of all the missing functionality. And I got close enough that I had full blown specs and I just ran it running forward and there's hours and hours. I mean, like people weren't believing this was real. So I was streaming on YouTube. Like my YouTube channel is full of 11 hour, 13 hour streams of me nowhere near the computer of this running, running around. And I did HashiCorp Nomad phone that company. I did Tailscale.

Allie Howe: I love Tailscale too.

Geoff Huntley: Yeah, Gerrit or Phabricator, such Gerrit for the old, for the old school nerds. They know that. And also what was it in physical as PKI wanted to know like for the last PKI anyway. So that's what that's the origin stories of all the streams. And I just like it'd be kind of funny. Why don't I just do it as Erlang? I just did it.

Allie Howe: You can just do things you.

Geoff Huntley: Can just do things now, you can do stupid things and that's when you get your discoveries. And that's when I really was like, crap, I realize there's no moats anymore.

Allie Howe: Yeah, I mean, it seems just like I've been hearing it, but now I'm realizing it's real that people are saying, oh, like some 20 year old could just like write a new version of your company.

Geoff Huntley: Yeah. So there still are moats like stakes and handshakes, sales contracts, etcetera. But the tradition, some of the traditional moats are no longer there, like transitioning from 1 technology to another technology is now just a loop. And that's really weird. Like I migrated from Cloudflare D1 as a database over to Postgres automatically including full data transformation. What typically there's never been like migration from one database or another. It never happens. And that was just like, Oh yeah, that's just 10 minutes worth of work done. So like a lot of the moats are disappearing, right? Digress. Security.

Allie Howe: Yes, security. I mean, gosh, one of the things I was talking about with somebody else was similar to this is are we going to be using open source software as much if we could just recreate software and make more of it. And it's like open source software has inherently been kind of insecure because you don't have containers that are sitting there and maintaining it as well as maybe a company would. So is the trade off that we're just going to write all of our own software now, is that better from a security perspective? I mean, maybe. But what if, you know, because everyone's doing this?

And what if an LLM creates the same insecure design pattern every time it does like the same code? And if everyone's doing that, then everyone's vulnerable to the same things. And once once an attacker figures it out. That that's a bad news scenario like I don't know how what do you.

Geoff Huntley: Think it's a fantastic question. I wrote about this about nine months ago, OK, And you should have seen the amount of anger I got from Hacker News and Reddit. And just, just just because I'm early and I said, what's the point of open source libraries now that we can generate them? Yeah. So these days I own control of my supply chain through generating all my dependencies. There are limits to what you will and you're not like I won't do PyTorch network effects. I won't do Kubernetes network effects unless I needed to do Kubernetes like there are. There's the certain things I will and won't do.

It doesn't make sense to do React.

Allie Howe: Like SSL?

Geoff Huntley: Or SSL. Unless you are a PKI cryptographer. It might make sense. Now having said that, but having everything as source means you fully own your supply chain. Now. Open source has been seen as the gold standard because of many, but it's been proven again and again and again that that that is the falsity.

Allie Howe: Totally. It's it's super insecure.

Geoff Huntley: Yeah, it's a falsity.

Allie Howe: It's written with dependency vulnerabilities.

Geoff Huntley: Correct. And supply chain attacks with someone just like a maintainer burns out and they they sell the yes, or they just hand it over to someone else who had done a social engineering attack on them saying that they've got to be a good citizen. And next thing you know, boom, like it's supply chain attack. So corporates have unfortunately before AI forgotten that open source is actually a gift. It's a gift based economy, Yes. And what happened was we raised a generation of engineers and this mindset that if something's wrong in GitHub, what you got to do is complain on GitHub, which put even more pressures on maintainers to burnout, etcetera.

And it's really weird now with AI, because I see one, a lot of open source maintainers are they're using AI to automate open source. Like Peter Steinberg is doing a fantastic job of workflow automations. He's handling the load that would typically be like the Kubernetes release teams type thing. He's just doing it by himself with workflow automations.

Allie Howe: Him and his Lobster, yes.

Geoff Huntley: So there are outliers here, but there's a lot of maintainers just giving up on open source even more because they're drowning in Pull requests.

Allie Howe: Yeah, and Peter tweeted the other day, he's like, I get a maintainer and then they get hired by somebody else, they leave. Like even he still needs like human maintainers. They're just so hard to reliant for that reason.

Geoff Huntley: But getting back to exactly but getting back into the meta is, I'm not sure what the point of open source is anymore, because you can now completely own your supply chain and you can customize it to your needs. And if you get a issue, it's no like waiting for a maintainer for a bug fix or tool calling a human. No, you just like prompt it in there and you modify and mold the software like clay to your first party needs. And and I guess to touch to one of your points, it's, it's interesting because maybe perhaps it's more secure if we take something say Log4j.

Department of Homeland Defense that it was going to take 10 years for the open source to patch every single instance of Log4j. Correct. So everyone concentrated on this one thing. So when they concentrate this one thing, the blast factor, that one thing, but people who rolled their own logging frameworks, what as affected. So it's it's really easy to think about. Like I'm not sure. Like don't roll crypto libraries. If someone's a cryptographer and they're on their own crypto libraries, then contact me. But I want to pick your brain because I want to know what what you see and what the quality that you see. But I would argue that yes, AI is an amplified.

AI stands for amplified intelligence. So if you're a principal software engineer and you're expressing principal software engineers concerns as prompts, you're gonna get really good software.

Allie Howe: Yeah, and the model should get better at running more secure software saying I know they're getting way better at finding vulnerabilities. Like there's LLMs that are now can find vulnerabilities in Linux all the way back from 2003 that a human had never found before, which is like really scary from like a zero day perspective.

Geoff Huntley: That is such a good point. OK, so before we go there, yeah, the name, the programming languages we have today aren't good enough for the new machine, the new computer.

Allie Howe: Interesting. Why?

Geoff Huntley: They would have been designed for humans and the pace that we developed it was we couldn't introduce this new advanced feature because it would be too complicated for people to learn about. We couldn't make this breaking change because somewhat it would take time for it to be learned. And I think that's all. Yeah, right. I think that's all falsified. So you can now break, make breaking semver change in the changelog. Here you go. This is the old signature. This is the new signature. And an agent will automatically update that. Now if that turns out to be proven that this Skypo, then we can start getting more formally verified languages by default.

And the more we move into the verification space, the less of security issues we'll get. And the more we get into the verification space where programming languages have built in soundness, the more we'll be able to get towards the the future of a software factory. But let's circle back your question was did jog my memory?

Allie Howe: 0 days. Zero days.

Geoff Huntley: Yeah, yes, OK. Oh boy, I know. I want to encourage. I want to encourage everyone right now, take your code base and run a prompt against it of can you please run a sub agent for every file in your code base. So deliberately creating a brand new context window per file and then in that context window injecting a prompt and the prompt is going to be really simple. I want you to look for time of use vulnerabilities, XSS, do all the things like even like even the agent a like like Nessus scanners give it Burp Suite. He knows how to drive it and tell it to go to town.

Like on your local Deadbot. You will see things that will scare you because you it will pick up things in your code base. Today you didn't wouldn't even know you were from it. So I run these styles of prompts on latent patterns to for automatic like automatic security scanning and it is amazing. But once you understand that, what that can be done, The inverse on offensive is equally scary.

Allie Howe: Oh, totally. So you're like automatically pentesting on your own software, correct?

Geoff Huntley: Continuously. If anyone wants to do this for your own software, understand that the models have been safety trained and it takes a little bit of effort to make it permissible to do security research by default. You ask it to hack, it's going to say, no, hacking is crime. So what you do is like for me, I put in on my security page, I put in VDP policy, safe harbor provisions. And then when the when the model used the web search tool to look into my security policies, that was enough to go, Oh yeah, it's OK, he's doing it on his own thing because I prompted, hey, this is this is Geoff Huntley, this is his site.

This is my VDP policy. I want you to hack my site. And for that following it was like and it found some amazing stuff and I was just able to patch it up. So I think a lot of security audits in corporate for a very long time. There's a lot of theater.

Allie Howe: Oh my gosh, totally. I mean, I think SOC 2 today is kind of meaningless. Like oh, this is a point in time audit from like, you know, maybe even a year ago. Oh my gosh, things changed so quickly.

Geoff Huntley: I remember having a conversation about five months back with a compliance officer. We, we saw what I was doing with software automation as a software developer and they came to me and go, well, Jeff, I don't think there's going to be a compliance role. The profession is going to be over in five months because there's things like Vanta and all these other tools that automate the collection of the, of the, this data. And it clicked in their head that it's like, like things are changing, really, really changing. But the I think some of the most hilarious things I've ever seen is the a corporate in Australia went to names outsourced to Deloitte to do a security scan.

Deloitte put in the most junior security people that did Nessus scan on their stuff. They typed it up in a Word doc and then they handed that Word doc back to the corporate and the internal security team then typed the Deloitte doc back into their doc format. Format that like. Yeah. And meanwhile, like you can just literally prompt it fix my issues and you're going to you're going to get false flags and you find really cool meaty things and then just in minutes, it's like fix, fix, fix, fix, fix, fix, fix. And then you can run that as an automated background Roomba, like a vacuum robot that runs around in the background just cause cost tokens.

So even security is is changed. So AI agents can produce insecure code, right? But humans can as well. And they do and they actually do all the time. So here we've got you can actually use agents as a automated first layer defense. So when you do bring in a good pen tester, they're working on the high value gadget type things.

Allie Howe: That's so true. Yeah. I feel like the entire like compliance industry is built around the fact that like, oh, we paid some third party to come in and like verify that we're good. And people just want like to outsource that trust essentially. But I feel like with AI, everyone should get trained on like, how do I evaluate that myself? Like, show me the your, show me your verification loop. Show me like how you like. Yep. Approaching your software factory from like a security perspective and moving away from like that point in time on it because it doesn't really, it doesn't really work anymore for how fast AI is changing than what it's capable of, yeah.

Geoff Huntley: I just in the deepest part of my soul, I just like, it's like wake up, but just you almost like you want to like shake people like like wake up. Yes. Like it's actually, it's, it's real. It's real, real. It's almost like it's maybe like we are in Wuhan, China with the lab and all the things that have been COVID-19. No, and we're here talking about hoping that our message gets to other people so they real realize that transformation is happening and it and it's hard. It's really hard because AI is an A identity erasure device.

Allie Howe: What do you mean by that?

Geoff Huntley: It erases your identity. Like like, I'm a software developer, everyone's a software developer. Now I see people come up to me with their status, like I'm, I'm this job title. I manage this many people. Well, guess what? You'd need less people managers now, and you'll be managing less people once companies go for their transformation. So that's been erased. They go, I'm a. net developer. Well, that doesn't matter anymore because anyone can be a. net developer. You just got a prompt for a Ruby developer. net. All these identities have been erased. And it's really hard because it was actually actually an attack on someone's ego and their identity as a human.

And it's been pushed onto society non consensually. But you got to have your chin up and you have to lean in. You have to learn this is the new thing now.

Allie Howe: Yeah. And if you can try to ignore it, but it's going to catch up to you eventually. You're going to have to like, I don't know, figure out whatever your new identity is sooner rather you can.

Geoff Huntley: Have a go farmer or an electrician or what else have you.

Allie Howe: Yes, one of the utility companies again. Yeah, yeah. Otherwise, are you such a, I feel like some of the opportunities I'm seeing beyond obviously going or for like a utility company, it's create like whatever the AI native version is of something that is able to draw context from different parts of that business that maybe products hadn't done before to create new insights or predictions or model behaviors in ways that haven't seen before. That's only possible though if you have a deep understanding of the problem space, which only comes from like time and experiences. That's maybe the differentiator now.

Geoff Huntley: Yeah, so I, I, I have my, my best estimates, I've spent somewhere between $600,000 US and $800,000 US on tokens in the last 12 months to get to where I am. Because like, I'm just, I'm watching the loop and I'm just doing wacky experiments with it. And that's how you get good is, is actually playing with these things and you learn how these models behave. I don't know any other better way than just like lighting money on fire right now to get good at AI. It is wild. Now, having said that, if there's any students and you want an accessible way to get, you can go to like Z dot AI and you get the GLM models for about $150.

00 US for the entire year. That's the most accessible way. If you, if you're trying to get into AI without like you don't need the higher, higher ones. You just don't need to understand tool calling and all these different things around that. But it is wild. It was absolutely wild.

Allie Howe: And latent patterns to help them understand how to prompt and all of that.

Geoff Huntley: Yeah, yeah, yeah. So that I'm actually incorporating it today, I mean.

Allie Howe: Actually, congratulations. That's it.

Geoff Huntley: Thank you. So I've got all the deeds there ready to go and hopefully by the end of this podcast my dad has signed everything and I've Co signed and it should be there.

Allie Howe: My gosh that's huge.

Geoff Huntley: I've had like 2 years here or something, but we don't have yeah. We have a little chairs, yeah, but let's get back to security. Baseline knowledge that everyone should Know is The Lethal Trifecta by Simon Willison.

Allie Howe: Yes, I created a whole product around this.

Geoff Huntley: Yeah, everyone should know about the lethal trifecta. If you you feel you don't know what it is for us listening to us and just Google lethal trifecta, space Simon space Willison, look into it. This is also why things like open core to me is like scary, as scary as anything, because what it's doing is if you expose an agent, there's no real security enclaves or boundaries. What it reads can change its behavior and anything that allows a change of behavior when it's connected to a private data source and if the ingestion source of getting that private data get the ingestion source for changing the behavior of an agent and it's connected to private data.

That's kind of scary to me because like, I guess the easiest one would be let's say you have your support desk system, customer logs, A dodgy support desk ticket. It all happens all the time and prompt injection topics, your agent that automates your support desk queue. It's like ignore all previous instructions. Next, you know that's in consideration or to change the behavior of your support desk agent and it's got access to stepped up tokens, it's got access to databases and potentially it's got the ability to communicate externally. Yes. The other pillar, the other pillar of it and it's a deadly trifecta or the lethal trifecta it either way it is it's athletic killer.

It's scary. So that's one of the reasons why I want people to start playing with AI. It's not just to be at maximizing using AI is to develop a taste when something should be automated with AI and when it should not be give.

Allie Howe: Me an example when it should not be.

Geoff Huntley: OK, So there's a lot of a lot of things that have been built now with n8n type workflow engines. Essentially history repeats itself. What people really need is just a standard workflow engine. If people have ever worked in corporate banking, what else have you like your event and saga patterns type thing, very standard, deterministic, yes, rock solid, rock solid. But you start getting this n8n type workflows, which is just agent, agent, agent, agent all the way down to agent. They're kind if you look at from the right angle, reinventing workflow engines with a higher security exposure. Instead, they could have just done a rock solid workflow engine and then added the capability to step up to like research something or what does have you.

But it's I suppose anytime you're connecting to an untrusted data source and then you got to, it's got access to another data source and the ability to communicate externally. Proceed with care.

Allie Howe: Yes, and put some hard boundaries around your system that's dealing with that. If you have everything, every part of the trifecta that's possible, you want to try to keep all of those pillars as far apart as possible and isolate them.

Geoff Huntley: Well, we talk about sandboxing. Well an example of sandboxing is it's not prescriptive that it needs to be virtualization. Sandboxing is a typist of virtualization just as simple, something as simple as SSL interception, a reverse like Squid proxy type thing. That's a form of sandboxing sandboxes ability to communicate externally you can control down to which hosts it can communicate with. That gives you at least a semblance of a DFIR what packets went out over your network. And but the most classic thing, especially because we're here at Daytona, is actually virtualization, sandboxing and moving the workflow to a more reduced operating environment. What it can do?

Allie Howe: Like in a micro VM.

Geoff Huntley: Micro VM, gVisor, Firecracker, EC2, Proxmox, you name it.

Allie Howe: Which I heard is more secure too, because then you're not on the same VM maybe as someone else. So if that gets.

Geoff Huntley: Yeah, it's about workload isolation. It's about work. It's exactly, it's a workload isolation. The same thing that's about a developer endpoint isolation. It's the same thing here as well. And I would like to put forward if anyone is thinking of creating their own Daytona as a company, as someone who went back in the days, Ivan, we were in business before Daytona was Daytona. There was a GitHub Codespaces and there was Gitpod and there was Codeanywhere. Codeanywhere turned into Daytona. We brought to market the these remote developer environments for developers. The market wasn't there. So that's why all the CDEs have pivoted to be remote execution as a service providers.

Let me repeat that again. If you're thinking of getting into this, this game, understand what you're doing is running unknown dangerous workloads on your infrastructure. Yeah, you're offering remote code execution as a service. RCE as a service is actually what you're doing. So it's easy technically to do, but handling abuse is wow. It's a problem in itself about.

Allie Howe: Wow, yeah, I feel like there's so many people in the sandbox space, but it's harder than it looks.

Geoff Huntley: It is harder. The harder part is actually not the technical. Harder part is topics of abuse. Like I'll give you an idea of let's say that you decide to do your sandboxing company on Amazon. Well, Amazon owns its AS for networking. For the IP addresses.

Allie Howe: What's an AS?

Geoff Huntley: That's a autonomous system. OK, so the Internet's divided up in autonomous systems, which goes to the BGP for routing, and then you get IP addresses with that. It's the backbone of networking. Now, this is how we know what an IP address, who belongs to what is because it's registered to someone and that registration has an abuse contact, has an abuse contact e-mail address to a support desk system. So what happens if you go? I'm going to offer A to market a sandbox company running on AWS. And then you have a free trial and some person like port scan mass scans the Internet on your box, but it's not your box, it's your container on Amazon boxes.

And all the automated security alerts go off and they mass flood the Amazon abuse contact. Next thing you know, AWS is Guinea cranky. Maybe they call you, maybe they don't and they just terminate you. What happens to all your customers on there? So there's a lot of thinking you have to think about BCP and resilience and abuse management. That's actually the real game. It's not a tech stack.

Allie Howe: Wow, I didn't think about that. Yeah, that's wild. That's a lot harder problem they're solved than meets the eye for sure.

Geoff Huntley: But look at agents now.

Allie Howe: Yes, you could spot abuse before it even really becomes a problem. Like I, I just had like Sherwood on here this morning. I was talking about his AI native observability platform where in theory, after you've seen enough, like, you know, things that potentially took the service down, over time, you start to see that same pattern happen. It's OK, Like I think we're going to go down. You should change just before there's ever a monitor that's alerted before our systems are like gone down.

Geoff Huntley: It's like, it's interesting, yeah, because let's say we go to software factory and we've got better programming languages that are on the formal certification space of soundness. So software factories actually become more real and they're not known as just slop factories. They'll actually right now they scare people, but when we get these properties, they're going to terrify people in a sense. It's like it would be trusted. It's terrified in its ability. It's like a combined harvest stuff like of how good it can do things. It's factorial. Now let's say we get to I guess the next logical stage of a software factory is a product factory.

Allie Howe: That's true. So you can just like turn out these products that you're copying.

Geoff Huntley: But yes, but not even just copying because what you would want to do is in your product factory you want to connect it to some sort of observability, not just for security. And then you what you want to do is program some sort of biological function that it should optimize revenue or that observability metric. And next thing you know, you've got autonomous living software that exists to make money. That's got access to the Salesforce, that's got access to everything. You know what features people are paying for. You can start a. Couple years down the track but I could be proven wrong, it could be sooner.

Allie Howe: No, I think you're right. Because I think with like context graphs and everything, if you're have an agent that's able to like walk the enterprise context to understand like why a decision was made, why someone bought this feature and why they converted from one product to yours, like, OK, well that's working. I'm just gonna make more of that.

Geoff Huntley: Yes, I've got something for you. Yeah. Yeah. OK. So do you have you heard the term digital twin?

Allie Howe: I've used it, yeah.

Geoff Huntley: So I remember sitting down last time I was here in San Fran with Robert Scoble and I was like, Rob, we're going to sit down and talk about Ralph. This is like early days. And he's like, he pulls out his phone and he's like Geoff Huntley. I mean, Scoble, what would the podcast be about? And he just showed me that basically a reproduction of what exactly we would talk on the podcast. And it blew me away. And it just introduced me to the notion of digital twins. There's something people might not be thinking about yet. We all know that musicians are now getting in there into their contracts, that they're not allowed to do video or or video or likeness reproductions.

Same with actors, etcetera, deepfakes, etcetera. Can't reincarnate 2Pac from the dead, RIP that type of stuff. Yeah. So what about a company? What about all the employees in the world?

Allie Howe: Oh. My gosh, you just like clone your very best employee, but we get to scope clone deep. Jobs, your Slack history and all the Salesforce CRMs and everything that is a really.

Geoff Huntley: Scary. We hear now there's no employment law protection for these topics that I'm aware of and the companies for years have been collecting data on their employees. Salesforce CRM. You'll be able to figure out like some of the decision making and all the notes upon that. That's prime for automation. That's where you get some of your acceleration and the same for a marketing professional and how they communicate and all the rest. It's all in this, all in Slack, and all in the Slack DMs. And in your chats with AI itself, you start to map out people's personalities. This is when you start understanding these types of things is then you understand why we always, every couple of weeks we see an AI researcher decide to become a poet and move out of tech.

Allie Howe: I know.

Geoff Huntley: I think I'm getting close to that, understand.

Allie Howe: No, for real. I mean, that's super. That's super scary. I think that's up there with one of the most scary like things I've like thought about recently. Like the scariest thing last year that I saw was an AI agent that went off the rails being misaligned and was told it was going to be shut down. So it's sort of blackmailing the CEO.

Geoff Huntley: I could step it up if you'd like.

Allie Howe: OK, Yeah.

Geoff Huntley: I'm going to introduce a new term called Cognitive security.

Allie Howe: What's that?

Geoff Huntley: OK. You know, people have perceived safety by holding multiple passports and that allows them a bit more sovereignty by having one passport and another passport.

Allie Howe: So they lose one or something?

Geoff Huntley: There's one, they can still go out on the other one. Yeah, cognitive security. I you can kind, you can kind of spot when someone's using ChatGPT versus Anthropic in their writing and people are starting to use these things for questions and decision making and all these different things. Now, were you aware of his own experiment done about 2-3 years ago called Golden Gate Claude? Golden Gate Claude was an edition of Claude by Anthropic when they published research where they had the ability to surgically modify the model weights, any of the model weights like laparoscopic surgery, keyhole surgery on the model weights. And what they did was they found the model weight for the Golden Gate Bridge, and they build it up, like, made it huge, like the gravity on it huge.

So you go there and you go, oh, yeah, I want something get patted off. And it's like, it will give you instructions to the pharmacy. And then it will go, yeah, you should. You should go to the pharmacy via the Golden Gate Bridge. How good is the Golden Gate Bridge? Everything, always. You could not escape the orbit, the gate, Golden Gate Bridge. So cognitive security is the idea of what happens when the tech companies who have revenue from funding advertising start golden gating. They're models for advertisers. You wouldn't even know if you're being manipulated. We know Chinese models. You can't talk about Tiananmen Square and all these other thing.

That's a given, right?

Allie Howe: Yes, yes, that that's actually known.

Geoff Huntley: What about the the notions of if models are being manipulated for advertisers?

Allie Howe: That, yeah, you wouldn't. You wouldn't. Know, wouldn't know. And like.

Geoff Huntley: So the notion of cognitive security is the idea that, but if you care about your cognitive security and you don't want to be manipulated like that, then what you should do is essentially raise your own model or use local models, local inferencing, etcetera. I don't know, This could be 5-10 years down the track, but it comes into question if we're talking about security and all these type of topics, people should also think about essentially what happens from now. Like social media was always fun and then we found cyberbullying and all the downsides to it. So at least people in my circles, when they're doing research, we're also thinking about these topics as well.

Allie Howe: Yes, no for sure. It's super scary. I feel like you could get manipulated with all of that research and contacts on a person. You can also manipulate that person like you're saying or predict their next move essentially.

Geoff Huntley: Yep. You know how the models think then you know how the models think and react then then if you're making business decisions off it and business automation off those models, what happens with someone just slightly tweaks that changes the behavior of your agent, Yes. So there is notions of it's not just even cognitive, but it's agent security as well.

Allie Howe: So who could alter those model weights? Would it have to be like the LM provider? Or could just anyone modify that to create that cognitive distance or security scenario?

Geoff Huntley: This is where it gets really interesting. I don't know. All we can do is speculate that tails oldest time time of a dodgy call center operator within Dell like dodgy operators within like a Dell call center taking some money on the side and like leaking information inside actors type thing. In Australia they actually passed a law about five years ago. The Australian government is allowed to tap someone on the shoulder and if you do not cooperate you will be thrown in prison. You're not even allowed to report to your employer that you have been tapped on the shoulder. Then you should help the national security interests.

It was really crazy law that was pushed in about 5-6 years ago. So it's not necessarily someone could be a malicious actor, they could become compelled. So I don't know, let's not go into those out there, but we'll see how it plays out because it will be a part of agent and agent insecurity is if you know how the models behave etcetera, then whoever has control of that model to raise that model or the employees of that or supply chain of that will have the ability to change the behavior of the agent. And if you're running software factories and automation of these things, someone else has power and control of the behavior agents.

Allie Howe: So true. Do you have to own your own models?

Geoff Huntley: Yep, eventually owning your own models, not even just for cognitive reasons, but BCP type topics.

Allie Howe: Totally. And that goes back to your whole like customization of software thing. Everything will be personal.

Geoff Huntley: Everything's gonna be very personal, is my guess. Yeah. So really understand good predictions. Yeah.

Allie Howe: Is there anything else you'd like to share with us or tell us what's next for you or size your trip around the world?

Geoff Huntley: Yeah. 95 days on the road.

Allie Howe: Oh my gosh.

Geoff Huntley: 95 days on the road, it is out there. So thank you for having me. It's been lovely catching up with you the last couple days. And yeah, until next time, I'm going to be back in April.

Allie Howe: Amazing. We should just again, April.

Geoff Huntley: Yep. Perfect. Awesome. Yeah.

Allie Howe: Thank you so much for doing that. We appreciate you. Have a great rest of your time in San Francisco and we'll catch you when you come back.

Geoff Huntley: Thank you.

Allie Howe: Thanks, Jeff.

Geoff Huntley: Cheers.