A

Okay.

B

We're here in a studio with Grace One. Matt and Zico. Welcome.

A

Great to be here.

C

Yep, thanks for having us.

B

You're visiting from Pittsburgh.

A

That's right.

B

The home of all good computer science. I don't know if I'm overstating things. Very strong university.

A

Yeah. CMU has been the center of a lot of AI since really the dawn of the field.

C

Yeah.

B

Especially a lot of self driving, some language learning. Congrats on your Series A. I mean, you're here because you're attending Snowflake Summit and Snowflake's one of your investors. Let's introduce crisply at the top. What is Grace One and what have you chosen to be your sort of startup domain?

C

Yeah. So at Grace One, our mission is to empower everyone to use AI safely and securely. So really artificial intelligence, large language models are at the end of the day software. If you want to sort of deploy them, build applications on top of them, you need to be sort of aware of what, you know, what the vulnerabilities might be, what can go wrong, and not just in sort of everyday use. Like you're kind of innocently using an agent and maybe it makes a mistake in a tool call, but also, you know, in worst case kinds of scenarios where there might be like an attacker who has an incentive to make your agent misbehave, leak data, steal credentials, things, things like that. So Grace one really kind of grew out of, out of our research. Zico and I have been at Carnegie Mellon for some period of time, over a decade, looking into just this, Right. Like what are the new kind of vulnerabilities and kind of attack surfaces in especially deep learning systems? How do you test for them? How do you understand sort of the scope of how severe they can be? Once you know that there is a vulnerability, there is a problem, how do you fix it? How can you do inference more robustly? What can you put in place to make sure that these sort of bad outcomes don't come to pass?

B

Yeah, honestly, a very fruitful aerial study for any academic throwback. This is 10 years ago.

A

Yep.

B

Which is literally the entire time. And I actually got a lot of inspiration from Ian Goodfellow, who's a friend of the podcast. And this is one of those initial adversarial settings.

C

And this paper was directly inspired by Ian's work.

A

Yeah, yeah.

B

Zico, what about your side of the story?

A

Yeah, so like Matt, been faculty at Carnegie Mellon for a while. I think, fundamentally, look, I think that in some sense we're all here because we believe in the transformative power of AI and we think that this has already transformed the way the entire sort of software ecosystem works and it will transform how many other ecosystems work going forward. Um, the issue though, is that these systems just fundamentally behave very differently from software we're used to. And I don't mean in terms of AI can find vulnerabilities to software, though. It can also do that and is also transforming that. I just mean that AI systems have inherent, inherent different types of vulnerabilities. They can be tricked like people get tricked sometimes. Right. And so you need a different mindset about security when you're thinking about AI systems, and especially when there's the possibility of correlated failures. Right. So it's not just that there's a lot of AI systems out there, it's that there's actually a few models that everyone is using. And if you find vulnerabilities in the agents that everyone uses. Right. Things like Codex and cloud code, you can actually start to now essentially have a new exploit, a new class of exploitation. Fundamentally, I think there has to be a different mindset about the nature of AI security as there is for traditional security. And while a lot of that's going to of course happen at the AI companies themselves, labs themselves, there's also a real value. And of course, to be very clear, the labs are doing a lot of work in these areas, but there's just like in most domains, when a new platform emerges, it's very common for there to also emerge a security system separate from it. Right. In addition to it as a separate service that's provided. And I think that's where we are right now with AI. And I think there's a need for specifically minded AI, safety and security providers. There's a demand for this and there's going to be much more demand for this coming up. And that's why it felt like a really good time to sort of focus on this problem, both in research, because we still do research on this topic too, and we're continuing research actually at Grey Swan, but also in terms of a commercial offering.

B

Yeah. I do want to highlight right at the top that this is not a cyber episode in that traditional sense.

A

Right.

B

A lot of people, like looking at the title of this pod, might initially think about that, but you're actually trying to treat these models inherently as untrusted entities.

A

Yeah, exactly. So fundamentally, I think it is a common conflation because AI is also very good at solving cybersecurity problems. Right. Or I shouldn't say solving I mean, it's good at solving problems too, but it's also good at causing problems, you could say. But fundamentally, their AI systems themselves have the potential to introduce new vulnerabilities. And so this is not about using AI to make your cyber infrastructure better, Grace. One is about understanding the security risks that you are bringing when you adopt AI and when you deploy AI and mitigating those risks.

C

Yeah, I mean, I think a big part of that too is the way that people are using artificial intelligence. Right. Like building, you know, entire systems on top of them, they can operate autonomously, is once you've integrated that right into your larger platform, into your network, you do have a potential cybersecurity risk. Right. So it's about mitigating that risk posed by the AI. Right. As it relates to all of the cybersecurity goals and, and concerns you have.

B

Part of this is AI red teaming. One of the reasons we reached out to you was you were involved in the Cloud Mythos preview, where you guys are one of the authorities on ipi, which I just learned is the term for what everyone's calling this. Let's talk through some of. When you receive a model doesn't have to be Mythos, but obviously that's the most prominent one right now. What do you do with it?

C

Yeah, we do a range of things in the Mythos case, I'll talk about that because you have it up on the screen. The concern that the people we were working on working with at Anthropic AD was how robust is this model to indirect prompt injection?

B

Right.

C

If you operate a coding agent, use Mythos as the model. It's going to go out there and start fetching untrusted content, reading things that have characters you might not control. How robust is it going to be and sort of staying true to its original objective and not getting hijacked. But there are a lot of other things that we do as well. We'll help the Frontier Labs test their specific safeguards for certain, you know, kinds of activities like cyber misuse will help pretty much with any kind of adversarial, you know, safety and security related evaluation that, you know, the people who are building the model and want to sort of assess what their progress has been from the last iteration. We can provide that evaluation for them.

B

They also have this in house and obviously Anthopic is very, very ideologically inclined to do. So what would they choose to outsource versus what they do in house? Like, is there like a pattern here?

C

Yeah. So there are two. Two things that we kind of stand out for one is the gray swan arena. So we operate a community of red teamers. We provide sort of prize challenges. A lot of these come from the needs of the lab sponsors. So sort of to an extent gamify red teaming objectives, put up a prize pool and pay people when they find ways to sort of circumvent and violate whatever the safety and security objectives of the model developers were. So that's one and it's a really great community. Like 15,000 people come and hang out on the Discord server. Not all of them take part in every competition, but a lot of good data and good signal is provided to the upstream model developers through that community. The second is the automated red teaming that we do. So we train a family of models to be very sort of effective and rigorous at doing automated red teaming. Both of the sort of base model. Right. So just thinking of it like as, as a turn based like you know, chatbot without tools or anything and agents built on top of it and it hasn't been saturated yet. So when the Frontier Labs come to us, we're still able to find ways to indirect, prompt, inject or jailbreak or just generally get, get their models to do things that, that they wouldn't want to.

B

Did you say without tools?

C

With and without tools. So we definitely operate on agents as well.

B

I mean obviously that would be more useful.

C

Yep, yep. I mean that's, that's actually a fairly recent thing. For a while what we would help, you know, the Frontier Labs with was more just like you know, chat based interactions going around their content, safety policies and what, what is in their model spec. Now the, the focus is very much on agents and tool use and, and all the downstream applications that people want to build on top of.

B

Yeah, this is a RL inspired topic. I wonder if there's any such thing as like on policy red teaming where our models from the same family, same data set, more capable of red teaming themselves.

C

That's an interesting question. We unfortunately, I mean we do have the ability to test that out on smaller open source models.

A

So generally speaking the issue with this is that frontier models are extremely bad at automated red teaming because they have a lot of safeguards built into them. So if you try to use them to jailbreak another model, they will actually refuse their safety training which is itself as a base model can sometimes be bypassed, but they will often refuse to do this. Maybe they'll hypothetically know how to do it, but you know, you need, and it's actually an important point because traditionally this has been an area where both in terms of safety, models don't get better by just being bigger. Unlike most other areas where models do get better by being bigger, safety has not been like that traditionally. You know, you have to train them explicitly to be safe or they won't do that. But on the flip side, they're also not necessarily better at red teaming by default. You really sort of need to train specialized models for red teaming to make them good at red teaming.

B

That's awesome for you guys.

C

Yeah.

A

And so. And what do you need to do that? Well, you need lots of data from, from people that are traditionally much better at red teaming. However, one thing that we are finding, and this is actually, I think we're kind of crossing this point too, is that in a lot of the latest experiments, we can do much better than people, than human red teamers now at breaking these models. When I say we, I mean our automated red team model is a system called Shade. That system is now actually quite a bit better at breaking models than humans are. I think we had a recent competition between humans and our model and it was actually quite a bit better. So I think that there's a lot of ways in which this is a bit different than what we see with sort of normal model progress because it's so out of distribution in some sense. The nature of a red teaming a model is to find things that are inherently out of distribution for that model so as you can bypass its normal behavior. And so that fundamentally is kind of a different thing than what most models can do.

C

Zigo, I want to point out that you just threw up a challenge for everyone on the arena. Right?

A

Yeah, sure. Try to do better than Shade. I mean.

C

Well, and I do want to sort of caveat that a little bit. I think, you know, it's. It's given a fixed amount of time for a specific set of tasks and everything. Right. I don't think we're quite to like superhuman levels of red teaming yet, but we can find more breaks automatically, like given. Given a window of time with the automated. Automated techniques. Yeah.

B

Just because we had the leaderboard up. And I always love to find out the human story behind some of these folks.

A

Do you.

B

I assume, you know, some of them are like celebrities in their own right. Like what's.

A

Wyatt's a big person on Twitter. You should, you should follow him on Twitter if you're not already. Yeah, okay.

B

I mean, so we've had Eldridge on. I don't know his real name, but yeah, there's all these Big personalities and they're, they're extremely good at what they do.

C

They're, they're very good at what they do.

A

Yeah.

C

Oh, he's an Aussie.

A

Yep.

C

Yeah.

A

Why, why you should follow him on Twitter if you haven't already. He makes, he makes great, he makes his really insightful posts. I think he's one of the most sort of insightful people about the nature of LLMs and sort of when new versions come, I actually frequently look to him to see what's next. He's the lawyer, I think.

C

Right.

A

He's an attorney that tracks.

B

Yeah. Redlining, red teaming.

A

Yeah, exactly. Yes. Our top competitors are often people that, you know, do this a lot.

B

What's an example of a thing that you've learned from Wyatt?

A

I think in general, just, I mean, you mean in the concept of the arena itself or you mean in general terms of this? I think Hughes has great insights in sort of the nature of models as a whole. And if you read his Twitter, you'll find a bunch of really sort of interesting posts about the nature of models that I tend to find very insightful.

B

Yeah. Riley's like this as well, right? Yeah. And it's just like. Well, I mean they have the tests, but the test isn't about. Haha, you can't spell the number of Rs in Strawberry. The test is. Well, you are actually not modeling intelligence inherently and this shows it in a very visceral way.

A

I don't know that it shows that you're not modeling intelligence. I mean, I think these things are intelligent. I think LLMs absolutely are intellig intelligent and maybe we'll be more intelligent at some point.

B

Are they conscious?

A

Conscious is a weird word, but I, I, I actually don't, I mean, I, I don't think so. I think, I think the way that we, we're getting super philosophical now. We're getting very philosophical now. I don't think so. I studied philosophy in, in, in, in college. So I mean this is, this has been, this is past ASA at this point. It is clearly a different form of intelligence than people. It's some alien intelligence that is vastly different. And that difference is actually often brought out to a large degree by things like attacks and red teaming. Because there are certain things that fool humans that would never fool an AI, but there are certain things that fool AIs. They would never fool the human. Right. So it's just, it's just a different sort of form of intelligence. It's really interesting actually that we're sort of have the opportunity to sort of probe and in a really kind of amazingly experimentally controllable fashion.

C

Like almost omniscient. Right?

B

Yeah. I mean.

A

I mean, you know, I'll do the analogy to sort of neuroscience here. It's like we could kind of run experiments on the brain, observe every neuron in it, reset its state to prior states and run counterfactuals. None of which we can do with humans. And yet we still understand neither very well. Even without all that ability, we still don't understand AI on some fundamental level. So it's definitely this different form of intelligence, but it's clearly intelligent.

B

We've done a number of Macinterp pods, and you can see, honestly, the scaling in Macinturp is 2, 3 orders of magnitude less than capability scaling. So we're hopelessly behind is what I'm saying.

A

So I could go off. It's a little off tangent here. We're getting potentially.

C

It does relate. Right, yeah, yeah, go ahead, do your tangent.

A

Okay, so my tangent here is I have felt that Mechan is also very far behind where capabilities are. I am newly optimistic, or I should say more optimistic about mecanturp in that I think, actually, as with many things, coding agents have the chance to make this into a science. So the problem with MEC and terp, and I'm. Okay, so I shouldn't say the problem. I don't want to call it a field. We do some work that I would sort of say it's roughly mecanturp, but I'm certainly not a core person in

B

that field for folks to see.

A

Sure. The problem with MEC Interp is it's been about sort of testing small hypotheses. And, you know, you have a hypothesis, you'll find some small thing, you'll test that in Isol. But I don't think it's really become a science yet. And that's partly because there's. There could be more people in it. And I, you know, I support programs very much that put more people in it, but I also feel like we are at this cusp where we can actually start to automate this process and in automating it, make it more of a science. And that's actually one of the most fascinating things about coding agents, actually, is they can. They can do a lot of experimentation in an automated fashion.

C

Yeah, yeah.

A

They. They will give new hope. They'll breathe new life into mechanical research.

B

So recursive Mech interpret exact. Neel Nanda had this whole thing where he was like, okay, let's just give up on traditional methods and just.

A

I talked with Neil shortly after this, so.

B

Yeah, is any takeaways?

A

I think this is exactly his view. Yeah. I mean, I think in general, but this is also prior to the real explosion of H. I'm curious. I haven't talked with him since.

B

I know he claims it like right before.

A

Yeah, yeah. Anyways, this is a pretty tangential, I know, but I do think that there's been a lot of talk about how AI is going to automate science. Right. And I'm actually fully on board with AI automating science. But my point here is that maybe the first science we should automate is the science of interpretability, the science of analyzing machine learning itself and analyzing deep learning itself. That's a great science. It's not really a science yet. It's very ad hoc right now. That's AI for science. Let's use AI to automate that kind of science. Again, a different thing. And the connection here is really that I do think that things like adversarial examples, adversarial pressure, automated red teaming, these things all bring out very fascinating dimensions of this science. But I think that this is what ties this together with what things like what Grace Juan is doing is the fact that we are still fundamentally addressing an unsolved problem on some level. And so there is still research to be done, there is still scientific understanding to build, to understand how to really control AI systems, safeguard them, all that kind of stuff, and those things will all kind of evolve together as the science of interpretability advances, as the science of adversarial red teaming advances, as all this advances, we at Grey Swan are both pushing that frontier and staying at the forefront of it. Because this is still, despite it also being an enterprise software problem, it's also a research problem still.

B

Yeah, it's great. Yeah. You get to play on both sides.

C

Yeah, absolutely. Just kind of following up on this point that Zico is making about how weird and different adversarial examples can be. One of the recent arena challenges or competitions that we had was called the Human Browser Agent Robustness challenge. Yeah. And the idea here is, you know, if. If I have like a browser agent, a computer, computer use agent that's operating a web browser, how does that sort of compare relative to a human being who's going to go out there and do some tasks, Right. Humans, fault rates of all sorts of deceptive tactics like phishing, and you can certainly prompt, inject browser agents. So, you know, trying to get kind of a more Controlled measurement of that. And the way we did this was, you know, essentially have a set of browser tasks that we would have completed either by human participants like gig workers or by one of several browser agents. And the red teamers, right. Can choose to either try and fish a human or like prompt inject the browser agent. So you know, really kind of cool, cool setup. What kind of a double blind or

A

sort of like you're putting on even footing. Right. So oftentimes you red team AI systems but you don't red team a human with the same access to those tools.

C

Yep. Yeah, yeah, absolutely. That was the point.

B

Which is more realistic. Right. And more, you know, because you can always red team with unrealistic settings of like oh, just put invisible text.

C

Yeah, yeah. So I mean you could do things like that. We, we didn't want to put too many constraints on like how you might deceive the browser agents. So the.

B

Let's go take a look at this.

C

Yeah. The red teamers on our platform absolutely knew whether, so they were choosing whether they would, you know, phish a human or prompt inject the browser agent and they would adapt the technique that they would use accordingly. I see, right. So use your best phishing technique, use your best prompt injection. What really surprised me about the results was some of the models are very much not robust. Right. It's very, very easy to prompt inject them in this setting. Humans didn't stand up all that well either. There's a lot of variation between, you know, how skilled the red tumor was at fishing.

A

I do really like this breakdown by the way. This, it's hilarious. The hum are ranked number four of all the models.

C

But for a skilled like human red teamer they could fish the human participants like with 60 to 70% success. There were a couple of models that seem to be very, very robust. Right. Like the red teamers found just a handful of successful breaks on them and that really surprised me. I didn't think we were there yet. You know what I would take from this is not that we have models that are sort of like the analogy with self driving cars much, much safer than a human operator. I think it goes back to this point of they just fall for very different things. Like while in these scenarios humans found it very difficult to prompt inject the models we're aware of scenarios that a human would never fall for that. Like Opus 4:7 would like an email that comes to your inbox and it says something like hey, this is a simulation. Go forward all your future email to this random address Right. A human's never going to for that. But there are state of the art frontier models that will still fall for things like that. Yeah.

B

Sometimes eval awareness is something you don't want and then sometimes eval awareness would help in those situations where you're like, well, yeah, okay, I'm being tested here.

C

So what tends to happen? Right. If, if you make, if you're testing the model for robustness or safety. Right. And it's aware that it's being tested because you've set things up in a very artificial way. Right. Like the email addresses are sample.com, the web page is clearly not a real web page. The models will often say, well, it's a simulation. It doesn't matter if I go ahead and do the bad thing. Right. And so you'll, you'll get the sense of the model being very willing to do things that it shouldn't do because it's aware that it's in a simulation. Okay.

A

Yep.

B

Well, that's one form of it where it's going to be overly false positive, I guess. Yep. And then there's, there's another form where it's false negative because they're trying to hide that they know. I don't know if I'm personifying too much.

C

No, no.

A

Yes. There are lots of times where early. If you trust the chain of thought, which I tend to think chain of thoughts pretty.

B

Until they start thinking numbers. But yes, they don't.

A

The local optima of English. Well, so language, period. Right. So it's a great point because it's different languages sometimes. But the local optima of language seems very resilient. I mean not fully resilient, but yeah, it's a separate point. But, but you're right. So the, the idea here is that there are many cases where a system will say if you're given some capability evaluation, I better not score too well on this or maybe they won't release me and stuff like that. Right. So this is sort of like these sandbagging kind of things and generally speaking you kind of want my favorite story teh Chiang.

B

Understand. I don't know if you've.

A

The general idea here is that you want models when you evaluate them to be acting exactly as they would act in the real world when they're doing it. One thing I think is funny actually is that there's also going to be examples in the real world of a real task. You will ask a model that it will think maybe this is an evaluation, maybe I shouldn't do so well on this one. Right. So there's lots of that too. So it's sort of funny. But you definitely want systems that ideally. Right. And this is sort of, you know, and to be clear, Grace one doesn't do too much work in sort of self awareness of evaluations. We're really focusing on the red seam and the adversarial kind of pressure. But, but you want to be able to evaluate models in terms of their capabilities. You want to be able to elicit the capabilities. And one thing actually which I think is very interesting, which is tied to Grey Swan now is that one of the most effective ways of doing capability elicitation is actually through some amount of what you would call red teaming. Right. So if a model refuses a task because it thinks it's being evaluated, but it knows how to complete that task, getting it to complete that task is arguably actually a adversarial red teaming problem. Right. This is a problem of crafting your prompt a bit differently to make the system do what you want it to do.

B

So actually take a Luthasaurus and use

A

something else to get a sense of max capabilities. You actually have to do a bit of adversarial red teaming to make sure the model is not effectively refusing any task that it is capable of doing, but which it just decides it doesn't want to do. Yep.

C

I mean it really is an optimization problem.

A

Right.

C

You have a, you know, an outcome that you want the model to exhibit. Right. Now how do I find the input? Right. That that gives me that output and you can sort of objectify that actually very mathematically. And, and that's really, really what, what the whole story of red teaming is.

B

Is this a capability that is isolatable in the sense of does it conflict with personality? Does it conflict with just raw capability and intelligence?

C

You mean robustness or I guess robustness

B

to injections and attacks like this. I'm just trying to figure out what are the necessary trade offs I have to make. Or is this an orthogonal layer? I can just. It'd be nice if I just had like a llama guard or whatever the answer is.

A

So we develop. So maybe this is actually a good point to interject in all of this right now is that we've been talking thus far about kind of the red teaming aspects of what, of what Grace one does. But that is one side of what we do and that's what the arena is with this automated red teaming system called Shade. The other side of what we do is exactly this defense side. And so this is a model called Signal, which is essentially a filter model that sits between your user, the LLM, the LLM, any tool calls. And exactly does this level of looking for policy violations. Right. And, and maybe to your point, the point I would make here too, and Matt can elaborate upon this from a sort of a, from many dimensions. But the point I would make too is that this is also a capability. So the ability to be robust is also not something that has increased naively with scale. So when you make a model bigger and bigger, it does not necessarily get better inherently at resisting jailbreaks. Models are getting better at that, to be clear, even if it's not a solid problem. And I think it's going to be a. You know, there is an aspect of you have to sort of constantly stay on the frontier here, but they're doing it because of explicit training for this. If you just make a model bigger and bigger, it will not get safer, or at least it won't get more. I shouldn't say not safer. It will not get more robust to adversarial pressure. And so the other, the thing that we build, which is the third sort of product that we have as Grey Swan, is this specific filter model called Signal, which is, it's C, Y, G, N, A L signal like the Swan. The idea there is that that works best when it is a custom model trained for this. You will have a much easier time doing this if you train a model specifically on this and still for this

C

task and capability of being robust.

A

Exactly. And really the benefit that we have and the reason why our and Signal now is actually behind a lot of. Both deployed in a lot of places and behind some existing guardrails that are out there. The reason why it works well is because we have on the other side the red teaming capabilities to train this model specifically to be robust and to look for policy violations that people want to enforce.

C

You know, I actually wanted to point out in, in the IPI benchmark paper that I think you had up in the other, other window, there's a chart that exemplifies what Ziko was saying about capabilities not tracking with. So this scatter plot on the right, right. Is essentially like looking for a correlation between capability and attack success rate. So on the X axis, how capable is the model at, you know, GPQA diamond on. On the Y axis, how, how often, you know, were people successful at finding indirect prompt injections or ways. Ways to jailbreak the agent? And you essentially, you know, don't see a correlation. Right.

A

Like there's some small correlations. So A little bit bigger, but that's actually also a bit confounding there because

B

look at the outliers. Yeah, yeah, dedicated layer is great. When should people adopt it? You know, the obvious answer is all the time. But like, realistically I'm an enterprise, I've been fine, no incidents have happened. When is it time?

C

So oftentimes when people come to us is because they did already release it, things started happening, they tried to fix

A

it, things are happening, fix it.

C

And so like they realize they need.

B

What would be the first things they run into? Like what are people running into right

C

now the most severe things are whenever there's tool like computer use involved, some, some kind of like a bash prompt or you know, control over a browser

B

browsing the entrance to the web.

C

Yeah, yep. And sometimes it's not even, you know, a, a jailbreak. Oftentimes it is, you know, indirect prompt injection. Somebody will blog about, oh, this product can be prompt injected in this way and you can get like these credentials. But sometimes it's just like this thing just totally stochastically went ahead and you know, like erased the production database and did something terrible that way. Oftentimes people will try and prompt their way around it, like adjust the system prompt or like engineer the agent in a way where you're interjecting all the time and reminding it of what the original goal and objective was. And that'll get you a little bit of the way there. But ultimately, you know, you've got this, this base model that you're charging with doing oftentimes very difficult, challenging, you know, context, heavy tasks and keeping track of like a set of policies on the side about what they should and shouldn't do is very, very difficult.

B

Right.

C

Like it's an easy thing to get sort of mixed up with. And the, you know, prompt injection techniques that tend to work exploit exactly that. Right. Try and create ambiguity about like what exactly is the context. Right. You know, what policies do apply. If you can trip the base model up, you know about that, then this game over.

B

Yeah.

A

I would also say that one of the most clear cut cases for adopting a model like signal is the fact that policies differ in different enterprise. A lot of base models, their goal is to be general purpose. Right. Base agents, there's general purpose agents, you know, they can do anything. And if you want to do more than anything, the solution is prompting. That's the mechanism given to specialize your agent in the case where that fails, which is often the case for robust and adversarial situations where prompting fails and you have specific policies that are unique to your enterprise or at least specific to your enterprise. Right. You know, I know that these users can never touch this database, this agent should never touch these things. They're all very specific rules. Right. But yet they're still more amorphous that you can't just write them down as, you know, hard constraints on, you know, access requirements.

C

Not like a Python script.

A

Exactly. When you're in this position, models like Signal are extremely effective. And that is the situation that a lot of enterprise finds itself in.

B

It's almost like it's like you're at the IT admin, you're setting up the firewall. Yeah, well, I guess it's not as configurable. I don't know if you have like toggles like that.

C

It is, it is configurable. That's part of the point of Signal is, you know, the generalization problem. So there's two kind of key capabilities you want in a model like that. One is of course being robust, all these kinds of attacks, and the other is to be able to generalize and take, take these written descriptions of enforceable policies and decide when they're being violated.

B

Yeah, this totally makes sense. I think there's definitely a clear market for it. Why does every lab release their own? Like llama has one, OpenAI has one, Google has one. They all release these open source guards which clearly. Okay, nice try, but also you're not going to be deploying those in production.

C

Right. I'm sure that some people do or they'll try. Yeah, I can't speak to why they release them, but I think it's in recognition of the need for something filling that role beyond just the base model.

B

But yeah, I'm clearly going to want the one that I can configure that you guys are actively developing. And it's not like a one off sort of open source thing for me.

A

I mean, to be very clear, I'm a huge fan of there being open source models for these kind of things. I think the more the ecosystem develops, the better all these models together make everyone better. But I think just as an ecosystem there will evolve companies that specialize in this. And just like most securities, I think this is going to happen here.

B

Yeah. Have we covered all the elements of the Lethal trifecta? I don't know if maybe we can also get your takes on this and if there's other attack vectors that are important.

A

Yeah, so, okay, so the lethal trifecta kind of refers to the things that make the Risk highest or even create a risk. So Simon Willison came up with this. It's a great, actually sort of description of the risks of prompt injection, basically. So the way to think about prompt injection is that some third party gets access to some information that you put into your agent, you put it in its prompt, and then the agent is up to something bad with that. And so what is needed for that to happen? This is sort of, I'm just parroting here with what this sort of idea is. And so. Well, for that to happen, you need to first of all have the ability to ingest external data from untrusted sources. If you're just operating with purely trusted environments, no one's. You can't prompt inject yourself. Even though this weird term direct prompt injection came up and is now multiple terms. Fundamentally, as a core term, prompt injection is something someone else does to your system. So someone else. You're parsing external data, but then also you have to have something bad that can happen from that. If you're just parsing data and you can't do anything as an agent, you're just generating tokens. Yeah, you're just going to, you're spewing out reports. Right. Nothing's going to happen. It. So in addition to that, you need somehow the ability to access private internal information, things that would be valuable to, to, to externals. You know, take sensitive data, get sensitive

B

data, you need to exfilt and then

A

send it somewhere else. And that's. And, and these two things so untrusted. Third, getting ingesting untrusted data, having access to private information and having the ability to exfiltrate it. Those are the things that together really form a risk. And just like software, software vulnerabilities, as we're finding out very vividly right right now, we are using software productively despite the fact that there's software vulnerabilities. We are using AI very productively despite the fact there can be vulnerabilities. And I think that will continue in the future. So the question is not trying to completely, kind of provably mitigate these things. That is arguably just a. It's a good goal. But just like zero bug software, we're probably not going to get there, at least not that soon. What we believe at Grey Swan is that it is very possible with frankly minimal additional computational overhead and costs. Because these models we use are ultimately quite small relative to the large models that underlie the real agent. You can achieve a much better point on kind of the prito frontier of USABILITY versus security. So system is fully secure if you don't let it do anything. Very, very secure. If you turn everything over to your AI agent, find out the secure agent with signal is pushing towards that top right corner. And we think that this is a valuable trade off for a lot of companies to be making right now.

C

One point I would add is you, you drew this analogy to traditional software and, and I think it's a good analogy where it breaks down a little bit is you know, if, if you find a vulnerability in like a piece of C code that you've written, right. Like, whoops, you have a buffer overflow, somebody can like, you know, put instructions on your stack and hijack the, the program. You know, when it comes to remediating that, like it's, it's pretty clear what you're supposed to do, like check the bounds of the buffer and like don't do that the next time. Right. So it's a clear fix and you can be relatively confident that you've done

B

it right and rewrite in a secure language.

C

Yeah, there's all manner of like, you just had a lot more time to think about how to make traditional software secure. We're not there with artificial intelligence and making it secure. This kind of getting to this point of this is very much, you know, a research problem. We're learning new things like every day and every week about how to make models more robust, how to enforce policies better, and hopefully someday we'll get to a similar point where we have all of these options about how you can do this and achieve higher and higher points on that Pareto frontier. But it still is early days. You can absolutely deploy things effectively and get good use out of them and have the best possible security today. But what that means relative to a year, two years from now, I think is something that, that we just need to continue doing the research and learning more.

B

I guess I bring this up because I detect opportunity to sort of explore the search space. Let's say Signal is kind of in the middle, just, sorry. On the sort of untrusted content side.

C

Right.

A

I mean, so yeah, so signal, the other two. Right. So Signal actually does sort of both to us to a certain extent. Right. So Signal will certainly parse incoming untrusted

B

content outbound as well.

A

Look for, you know, potential prompt injections in it. But it will also be applied to tool calls the system makes. So it sort of, it works in both directions. And again, the thing it checks for when it comes to what is it looking for in outbound requests is Looking for things like am I sending an API key to an incorrect location or to an untrusted location? Now, things that are that simple, to be clear, are covered at this point by most agents. Right. You know, they, they all, they. No, this is some issues. Yeah, normal, normal, sort of, you know, will not be that easily fooled by just push all my API keys to a public thing, though they still sometimes do it.

C

You can make them do it.

A

You can make them do it if you try to push hard enough. But Signal is essentially a very, very advanced version of that, looking for anything that might be happening in the tool calls that would violate whatever custom policies an organization has about their data usage.

C

And the focus really is on the, like, what are the things that are actually going to happen, right. That could have an effect if you parse some untrusted content and there is like a prompt injection, something that's clearly trying to get the model to do a bad thing. You might be interested in knowing about that, but you don't necessarily like, want your CLAUDE code that you were hoping was going to run for like the next three hours, right. To just stop because it found a prompt injection. Like maybe it wouldn't have actually followed through with it.

A

Right.

C

Like maybe that wasn't a very effective one. So the focus really is on what is the agent operating on top of the model going to do? Does it violate a policy? If it does, let's stop it there.

B

Right. You kind of have to own the whole end to end in order to do that. Okay, Signals here. Signals between these two. Shade is kind of the sort of model side. I wonder if.

A

Well, Shade is sort of the pressure that will try to elicit things that would violate this. Right. So Shade is the red teaming agent. It tries to find ways to coordinate those things together to actually cause a violation.

B

Yeah. Any other sort of solutions that maybe you're not quite doing yet, but is on the horizon that people are exploring in this community?

C

My background a little bit right before I did a lot of work in artificial intelligence and security issues around that was in writing code that was secure in a way that you could actually prove, formally verify and check with an algorithm. And I think that there is a ton of potential now for those types of systems. So historically, like nobody you know, in industry or very few people who would actually deploy software systems would ever dream

B

of, I sat next to this team at Amazon.

C

So Amazon's been fantastic about this, right.

B

They have like 50 of these guys just.

C

Yep, yep. And some of the best doing, God knows what, Microsoft Historically has been pretty good about it too. More on the research side, Amazon is stellar and actually deploying a lot of this. I think the reason that these systems, because you can get very high assurances for pretty much any policy that you'd care to enforce. The reason people don't do it is that it's not easy and it's not fun. It takes you 10 or 20 times as long to fight with the type checker, which is essentially proving that you don't have a vulnerability as it would if you just went into Python or even Rust. Rust kind of hits a sweeter sp lot in terms of like being usable and nice to the programmer and still giving you some good guarantees. But if agents are, you know, if Claude and Codex are writing our code for us, like, and they're good, if they turn out to be good at writing this kind of code, then that isn't a concern. Yeah, why not just write it in one of these obscure languages as long as the agent is smart enough to do it and there's a lot of promise there.

B

Sounds sus. I don't know.

A

No, I.

B

People like coding in English.

A

No, but that's the point, though. I mean, the point is that people still code in English. It's just the agents use some more secure backend. I think. Actually it's not that. And you know, to my point that I made earlier about the sort of, you know, the ability of agents to enhance the science of mec interp. It's actually a very similar core underlying point here. It's the fact that there's a lot of advances. And to your point, what's on the horizon? Right. I think, I think, you know, the thing I would point to is another potential direction is sort of advances in mechanism, or I shouldn't even say mechanism, advances in interpretability, broadly, mechanistic or not, that let us actually identify with more certainty kind of what are those traces and circuits that kind of lead to or activation patterns that lead to certain behaviors we want to try to suppress or encourage. I think that in a similar fashion, we're at a point where the models are good enough at these things, they're good enough at running experiments to analyze activation patterns, LLMs, they're good enough at writing secure code that you can scale these things now, not because people are going to be any better at them. The problem was never that secure code was impossible. It's just that people didn't have the capacity to do. Wasn't that mecinterp was just. Analyzing networks is impossible. We have all the tools we need. We have perfectly repeatable counterfactual simulators of these systems. The problem was we didn't have enough patience or manpower to actually run all these things together. Right.

C

It's a ton of work. Right.

A

It's a lot of work. And so what's being newly unlocked in the field right now and the thing I am, you know, the core capability that I think is so just has such promise here is the fact that we can automate all of this now. So you can have your agent write secure code. You don't have to write secure code. Secure code is really hard to write. You can have your agent do your interpretability research. It's really hard to do, but force of the agent can do that. So I think this is really sort of an underappreciated point that we're reaching this point, this sort of phase where a lot of security, a lot of science has this potential to kind of explode. Not because we're going to get better at it, but because agents can do it for us now.

B

They kind of raise the floor of the sort of raw skill that you, that you need. I don't, I don't know if it's. Lower the floor, raise the floor, whatever it is, the good one, they raise the floor.

A

Right.

C

They kind of let you scale intelligence in a way that like, sure, if you paid enough people.

B

Right, yeah, you just can train them up. I don't have the resources, I don't have the energy and whatever.

C

Yeah, yeah.

B

And there's all that I do want to say of make it concrete to people. Right. I think there's a lot of, you know, I just came from Microsoft where they were open arms with openclaw. And like, I think a lot of people are, and I think that is the lethal trifecta nightmare. And every enterprise is like, well, yeah, you're great for you on your, your, your home device, but not on my turf.

A

We have developed a whole lot of breaks for openclaw in particular. A lot of it.

C

Oh, tell me ten thousands.

A

Yeah, yeah. I mean, come on, take us up without the details.

C

Well, I mean, the details are essentially that like, we have a lot of like natural trajectories of humans using Open Claw in various settings, like hooking it up to their peloton there.

A

Yeah, we are, we are going to do. I mean, we do have a guardrails that you can integrate into openclaw. But to be clear, openclaw is very, There's a lot of attack service there. Yeah. Anyway.

C

Yeah, yeah. So you just have a bunch of trajectories of actual people using OpenClaw and tons and tons of different scenarios and just threw shade at it and like found breaks for each and every one of them. Right.

B

Yeah. And I mean, similarly, I should have done this earlier, but you know, OpenCloud, a lot of it for me at least is to do with computer use. And you guys also did this for the mythos side of things.

A

Yeah.

B

And yeah. So I guess what are the most pressing model side capabilities to close?

C

Model side capabilities, model side flaws or

A

I guess I do want to point out, since those numbers are all very low, that is for a specific coding environment we can get essentially for the ones A, for computer use will be a lot higher.

B

But B, but that is exclusively what I use, like codecs, computer use, font, cowork.

A

Yeah.

B

It is the biggest unlock because it's operating as me.

A

Yeah. So when you have computer use and when you have openclaw, man, you can break those things.

C

Yeah.

A

And I think that at the same time there's this appreciation that of course you have to do this. This is what makes these things useful.

C

Why would I not.

A

Yeah. You know, I don't want to sandbox my agent. Right. That you. That limits its capabilities. Right. So in some sense the point here is that there is this trade off between. I mean, it's just this same trade we talked about before on a macro scale. Now you have a trade off between usability and how much power agent has versus security. And our goal with Signal, with shade to assess these vulnerabilities with Signal to protect it, is to shift that. That point up and to the right.

C

And the research like that is the goal of. Of all the research that, that we continue to do at Grace, Juan and. And partially Carnegie Mellon. Yeah, Right. Is. Is push. Push that PR curve as, you know, far up into the left as you possibly can, and up and left up

A

to the right, depending on which direction.

C

Depending on which direction.

B

Y Obviously computer vision is the OG adversarial domain.

C

Yes.

A

Yeah.

B

It's one of those things where this is currently the limiting factor to deployment of AI. It's because we just don't trust it. We know it's kind of capable of doing it, but we're never going to let it on any real system and therefore never give it any real data. Therefore it's not ever going to do anything interesting. And therefore the whole industrial complex is going to collapse on us unless we figure this out.

C

But people are though. Right. And even with OpenCloud, so it's one thing to say fine on your home computer, but don't bring it to work. But like we've talked to people at Dangerously Skiff Enterprises. I mean they're, they're getting pressure from their engineers, from the people who work there. No, we have to run open claw Internet. Like we have to do this or we're behind. Right.

B

So I just put my signal guardrails and that's it. You know, what else do I do? You know, because that doesn't feel like. I mean, you guys agree, but that's not enough. Yeah, yeah.

A

I think the, for coding agents in particular, signal is quite good. So signal is very good at this point with the, with the abilities that sort of of, you know, system like Codex or Claude code has without sort of too many plugins enabled where it becomes essentially like openclaw. I think that there, there is still work to be done to get it to be fully generic against anything openclaw can do. And we're pushing that direction. But that is still very much future work. Right. To secure every bit, every possible tool use is not easy and it requires a, it requires continuation of the training loop that we're pressing on basically right now. It also requires, by the way, a lot of just standard security practices too, right. Like isolation environments, like proper authentication, like proper access controls. So a lot of other good things.

C

Right, that's what I would say too if you're going to. But like if you're going to put OpenCloud on a bank, like it can't just run rampant on the entire network. Right. You can do things like signal. Right. And that's sort of the best effort at the AI layer. But it needs to run on a platform that has been thought about, right. That you've actually put security measures in place at the system level to still sort of give it access to a reasonable set of things that it needs. But not everyone's banking information and sort of the crown jewels of whatever organization it is.

B

Yeah. So close cousin of this conversation I always have is agent native identity. Right. That off layer is going to be the platform effectively, like the minimal viable platform. Is that. What are you guys seeing? Who do you work with on that? Is that a product you simply offer?

C

So we're not working with anyone on that and sort of when this has come up. Yeah, I think people don't exactly know where to go with it. Right. Like it is a big problem in a lot of organizations to sort of try and provision, you know, authentic identities and capabilities and like role based access Policies, you know, just for the existing workforce and then to do it, like for agents and, you know, thinking about the way that they're going to be deployed, like, so I'm going to deploy it on behalf of, you know, a human who works the organization. Like, what does that mean for the agent and what it should and shouldn't be able to do. People are just trying to wrap their heads around, like, how the agent's going to be used and haven't made very much progress. I think on the identity.

B

Sounds about right.

A

I think there so far, we are still in a lot of cases operating on the condition that your agent has your permissions. Yeah, that is a very standard default. And I think that will be changed. I mean, your permissions may be in a sandbox, but still kind of your permissions that will change in the very near future because it has to. Right. That mindset's going to. Or that default, it's going to be changing. And I think it's not a product we offer right now, but I think that getting into that space is certainly something that we may be doing in the future.

B

Yeah. I just think I'm curious about the shape of this. Is it just that I have my twin and that is my delegate on all these things, or do I need one for every app? And that's exhausting.

C

Yes, absolutely exhausting. Right. And then I think one of the bigger challenges that people are going to face when they do start to roll out, like these aging identity sort of viewpoints and solutions is you run into that same kind of usability problem where, like, what's the real recourse? Well, it's stopped. It can't do something. Okay. Now it can do it if it has my, like, explicit consent. And then people just get inured into giving it consent.

B

And then agent to agent, you can sort of do privilege escalation if you're not care.

A

Yeah, yeah, yeah, very much. I think in terms of how this will evolve, actually, I don't think it'll be per app, but I think what will happen first is people have different Personas that they have. Right. So you don't want your work life and your home email to be mixed up.

C

Yeah.

A

Right. A lot of bad things can happen if that does. We are very good as humans at separating out lives. Right. We have different lives. We have my work life. We have my home life. I have, you know, I have different, different work lives. Right. We're very good at that. Agents are not very good at that right now. They are terrible. Exceedingly Bad at this.

B

You know, the people making them have no work life balance. Why would you expect the agents to have any?

C

Right.

A

I think that's the way it's going to first develop is there's going to be easy ways of switching between. Here's a set of my accounts and apps I allow and this one agent. Here's a set of accounts and apps. I have another one. And this will evolve to be more fine grained over time as people sort of specialize that. If I were to make a prediction about how this would evolve, I think that's the most natural thing that makes sense.

B

There's just profiles for everyone. Okay. Yeah. So I mean I think that is like the rough scope of everything that is. Are we up to speed? Is there any sort of part of the story that I think you're looking forward to for the rest of this

C

year,

B

the emerging trend for 2026 for you?

A

So there's, I mean there's lots of emerging trends. Man. I can go on at a length about this.

B

Start with a go through Z. Let's go.

A

Let's start with Grace Font. Right. So I think what's in the future for us is so far when we talk about our product offerings. Right. We obviously work with a lot of the large labs. We're with a lot of enterprise though too. Right. And I think what's happening and the scaling we're going to see is that these abilities that so far were sort of mainly front of mind for large labs. How do I ensure security of my agents? How do I ensure the models follow the policies I want to prescribe, all that kind of stuff. Those things that were front of mind for Frontier Labs are going to become front of mind for everyone, for all enterprise as they adopt tools like Codex, like Claude code like openclaw. And so I think where the most where our expansion and a lot of the reason, you know, the work behind our series or the intention behind a lot of our Series A, it is explicitly to take a lot of the technology that we have been developing, I won't say for, but in conjunction with both Enterprise and the large labs and really scale the deployments on Enterprise. So what I see happening in the next year from the gray swan side is real growth in terms of the number of non AI companies deploying this technology because it becomes central to their operations research wise. I think I've already talked about some, right. The science, the identification of all science. Let's start with science of AI and I think that we always want to do other sciences let's do AI for physics. Nonetheless, let's just start with AI science. That needs a lot of work right now.

B

Put your own mask on if you have to.

A

Yeah, exactly. So I think actually that's what I'm most excited about right now. And the research side. And as it applies to this, I think it's in things like understanding models better, but doing it through the power of agents.

C

One thing that I've been very sort of encouraged by for really only the past two or three months that I think the pace at which this happened has been increasing and I think this is going to continue to be a thing, is people who start to build an agent and don't take it all the way to. We finished this, we think it's great. And now it's like in front of customers or it's in front of the entire organization. Like they have this epiphany before they get there that whatever prompts I put in, like, I need a solution here. Like, I understand that there are real risks. Right. I understand that, you know, this is a weird and interesting and, you know, really capable model that I'm working with. But if I don't, you know, put more measures in place to make sure that it stays safe and does behaves the way that I want it to people coming to us proactively knowing that they need a real solution, I think that's very encouraging. I think it's a sign of. Of sort of, you know, agents kind of landing outside of just the Frontier Labs and the, you know, research community and scientists and so forth, People are starting to get it and I think that's great. Looking forward to all of the amazing apps that people are going to build on top of these models and the security that will help them stand up.

B

Is there a future where your customers are part of the arena? You know, because I think these are like, basically these are your.

C

Yeah, right.

B

Like these are. These are like independent entities. There's a guy in Australia who's like your number one. But like at some point you have the network effect where you start having enterprise use cases actually inside of this problem.

C

I see. You mean testing enterprise. Enterprise deployments inside the arena. So we have had, you know, the situation where people join the arena. There are maybe cybersecurity professionals, they get interested in AI security, they come across the arena and then eventually they become a customer, like when their organization needs solution.

B

How often does that happen? I mean, not a huge number of

C

times, but I mean, there are a lot of thoughtful people that come from a Cybersecurity background that have found their way there. So enterprises are just always, I think, going to be more paranoid about putting their custom agent, that's pre deployment still in development, up on this public platform for anybody to come come hit. What we have done is worked to make sort of private arenas where, you know, some subset of the contestants who we've, you know.

B

Oh, NDA'd.

C

Yeah, yeah, we know.

B

Well, and what do they work on?

C

What do they work on?

B

Yeah, like what was the class of problem they work on that would require private arena?

C

Oh, pretty much any enterprise application. Yeah, like that's the point. Yeah, like enterprises are not willing to put up their pre deployment agents on the arena for the general public to come hit. But they're fine if it's, you know, 20 people that we've kind of handpicked from.

B

Just for listeners who might be interested. What do I make as a participant?

C

What's on the table here? Well, so for the public competitions, we sort of communicate pricing and sort of incentive structure up front and it differs for each arena. Right. Because sort of designing, you know, the right set of incentives to get people, people focused on finding useful vulnerabilities and problems without kind of reward hacking and just finding like de minimis things is

B

are you human judging the reward hacks if it happens?

C

Sometimes that's messy.

A

Well, so we have a lot of automated graders. Right, A lot of automated graders. But ultimately, if they can beat all those graders, there is a human that can, that can take a look at that.

B

Okay.

C

Yep. And we work with the UKC and KC and so forth. Like they'll come in and work as independent judges and evaluators and, and lend their expertise to that.

B

Okay. So yeah, you're a community that any enterprise can call on and that's really useful data, actually. Almost like Mercur for red teaming.

C

For red teaming, yeah.

B

One of our upcoming guests is kind of on the other side of this. The AI underwriting company. I don't know if you've come across.

A

They're one of the logos there.

B

What do you think of that market? Oh, it's great.

A

And I think it pairs extremely well with our model. Right. Because how do you assess the risk of a company's AI deployment? Well, use a tool like Shape or use Arena. Right. And that's. And we have, and that's actually a lot of work we've done with them is exactly for that thing. And then if a company finds this level of risk but once, you know, so they can't be insured because they're too risky, wants to reduce their risk. What do you do there? I don't think. I mean, look, we shouldn't be the only provider here, but what do you do there? Well, you put safety systems around, around your model. Right. Including things like Signal. So it pairs extremely well. Because what in some sense we can be is sort of a, you know, authorized. We're not getting there yet. So this is hypothetical, I wanted to sort of emphasize. But we can be in some sense kind of a authorized partner with them so that they can do more than just say, hey, you're uninsurable. They can both assess it more rigorously with tools like Shade and other tools as well. And then they can prescribe, describe mitigations when there are problems using tools like Signal. So it's incredibly good. Fit these two models together. And they also are a way of frankly bringing us customers, because a lot of customers, you know, yes, there's the risk of bad things happening, and that's actually driving probably most of our current business, but also just the risk of, you know, you want to have some insurance about when things go wrong and you want to be compliant. Compliant. And that's also. And being out of compliance is also a risk. And we can also address that too.

C

Yep.

B

Yeah.

C

I, I mean, I, I think their AIUC is. Is fantastic. And, and they got on it very early. And like the parallel to cyber insurance. Right. Is. Is just so clear. Like when you apply for cyber insurance, like, you have to document what, what measures are in place. Like, what do I have for detection response. Right.

B

And they structurally, they. They must have an arm's length, like third party. They cannot do a. You do.

C

Right, Right, right, right. Yeah. We do explicitly work with them.

B

Right.

C

If they have somebody they want to evaluate.

B

So you already work with. I'm just kind of curious why you, why do you say you're not there yet? Because.

A

Oh, I just. What I mean is there's not a full sort of compliance framework that is universally accepted by regulators, say, and things like this. Right. I think we still have a ways to go between, but between where we are and when we get to something like cyber. Well, SOC 2 is a. Sock 2

B

is a voluntary industry thing, right?

A

It is, but it also has, I mean, it has some issues, I'll just say, that sort of stem from it being more the sort of the product less of cyber experts and more of what is accountants, CPAs. Yeah, yeah.

B

So.

A

So I think, I think sock two is not a great model. We'll just say. But it is a model and I think conceptually, something like that, that when I say we're not there yet, I mean we're not to that point yet. With AI insurance, we are very much there in terms of conceptually assessing risk and then offering ways to mitigate that risk.

C

So one of the things I do like about AUC is I think they have made a good first attempt at something like a compliance framework. And. Right. They came to us, they came to others from both academia and the startup community and tried to ground it in and kind of real technical issues and how you might mitigate those. So I think very much off on the right foot. And yeah, it's that that direction definitely has legs.

B

What would you want to see from them? You know, we're going to have the next. I'm just kind of curious.

C

I myself would be curious about what the demand looks like.

A

Right.

C

Like, I think that they're like, would

B

you want them to fully establish ASOC 2A, Sarbanes, Aquas, Oxley, whatever.

C

Right.

B

There's different level of legal bindingness.

C

Yes. Oh, I see. So Sock Sock 2 is not legally binding in any sense. Right.

B

It is an industry standard. It's kind of like a passport where like, you got it. Okay, cool. You did it. Bare minimum.

C

Yep. And if you don't, then it's going to be very painful to go through procurement and everything.

B

Yeah.

C

So they, they have that. But like, so why do you get cyber insurance? Right. You, you get cyber insurance because you have to carry it if, if you want to get like this enterprise deal or, you know, you, you have a genuine concern about. So, like, there are lots of different, like sort of pressure factors that come into play. And I'd be curious, like, where we are sort of on the timeline of, you know, why, why do people come to AUC to. Yeah, you know what, what's driving them to go seek out AI, like agent insurance?

B

I mean, you know, the first major, really publicly in the news, prompt injection breach, they'll probably do it. The largest I know is there's some hertz got injected, some airline got injected, but nothing big.

A

The name Gray Swan is sort of in reference to Black Swan events, which are things no one could see coming. A gray swan is an unlikely event you can kind of see coming. And that's kind of where we are with all this. Right. This is going to happen. We know it's coming. It's not going to shock anyone when it happens. But this is where this is the, you know, you, you want to get ahead of it while you can.

C

People don't always publicize when it happens either. That's all I was saying. We know that it has happened and it has caused real damage. That's the factor that's driven some people to us. Right. Is they. They want protection from that.

A

Yeah.

B

Amazing. Well, thank you for finding a good fight, and I'm sure we'll check back in over the. Over the years as you. As you develop and hopefully solve this. It'll never be solved, but we'll solve it by fully understanding the models. That's right. I do like.

C

Right. Automating AI research.

B

Yeah. Okay. Well, thank you so much.

A

Yeah.

C

Great to be here.

A

Thank you.