A

Foreign.

B

Hi, welcome back to the AI Policy Podcast. I'm Alok Mehta, director of the Wadhwani AI center here at CSIS and the new host of the podcast. Today I'm joined by Matt Mand, a researcher at the center and a podcast veteran. On the docket today is a discussion of Pope Leo's first encyclical, which which focuses on AI and the challenges it poses to humanity, and a potential executive order on cybersecurity and AI, which the Trump administration scuttled at the last minute. But before we go into those topics, many of our listeners will have noticed that this is our first podcast in over a month. I'm taking over from Greg Allen, who started the podcast and has hosted it for the past several years. Greg left CSIS in April. We released a quick episode with Greg and myself before he left discussing the transition and and I recommend folks check that out. We've taken the past month to plan out the next phase of the show, and you can now expect regular episodes every other week. We're going to keep the same general format. It'll be a mix of news roundups and guest interviews with various experts in AI policy, but we're aiming to keep the episode shorter and release on a more regular cadence. The goal of the podcast also remains the same, to provide clear and accurate expert analysis of developments in AI policy. We thank our loyal listeners for tuning back in and welcome new folks to the show. We hope you enjoy the podcast. We also welcome your feedback on how to make the show better. So please reach out. You can find us@aipolicypodcastsis.org so Matt, let's dive into our first topic.

A

Well, Luke, welcome to the AI Policy Podcast. It might be a bit of a stretch to call me a podcast veteran, but I'm excited to be back and I'm looking forward to this next era of the podcast with you. So, as you said, let's dive in. On May 25, Pope Leo XIV presented his first encyclical, magnifica humanitas, a 42, 300 word document on safeguarding the individual in the age of AI. And for those who aren't familiar, encyclicals are letters written by the Pope to the whole Roman Catholic Church. Pope Leo's encyclical addresses a diverse set of AI policy issues, from AI's impact on labor to the dangers of AI enabled autonomous weapons. And we're going to discuss our takeaways from this document. But first, could you just explain the significance of the timing of this document's? Release?

B

Yeah, good question. Thanks for asking. So, technically, Leo publicly presented this encyclical on Monday, so just this week. But he formally signed it on May 15, which is a significant date in the Catholic church. It's the 135th anniversary of the publication of another encyclical known as Roman Rerum Novarum. That translates to of new things in English. And this was a major encyclical written in 1891 by his namesake, Leo XIII. And this encyclical, this new encyclical, was timed in particular to come out on this anniversary because both encyclicals deal with significant transformations caused by technology, significant economic and social transformations. And so both encyclicals are designed to guide the Catholic Church and followers of the Catholic faith about how to deal with these changes and in particular, how to protect workers. So Leo 13 was really thinking about technological and industrial disruptions in the 19th century with the advent of the industrial era and factories. And so this new version, or this new iteration of the encyclical is dealing with the modern iteration of that, which is social and economic disruption caused by AI. And I think we should also keep in mind that Pope Leo chose his name explicitly with AI in mind. And so as a nod to the transformations that were happening in the 19th century under Leo XIII. So in May, just a few days after he was elected Pope, he said, I, I chose to take the name Leo xiv. There are different reasons for this, but mainly because Pope Leo XIII in historic encyclical Rome Novarum addressed the social question in the context of the first great industrial revolution. In our own day, the Church offers to everyone the treasury of our social teaching in response to another industrial revolution and to developments in the field of artificial intelligence that pose new challenges for the defense of human dignity, justice and labor. And so this is really intended to address a new technology, but do it in conversation with where the Church has been in the past and the thinking it's done in the past around similar transformations.

A

Interesting. Well, as you said, there are a ton of parallels here. So let's move on to the content of the encyclical. What were some of the high level themes that were present in the Pope's writing?

B

Yeah, so a few things stand out. Generally, if you look at the encyclical, it is subtitled on Safeguarding the Human Person in the Time of Artificial Intelligence. And so throughout the encyclical, there is a real concern about ensuring that technology serves human beings and not the other way around. There's a Particular passage that stood out to me that sort of seems to encapsulate a lot of the Pope's thinking. So he writes, if technological development advances without a corresponding ethical and social progress, the result may be an increase in means without a growth in humanity. Having more, without being more. And so this is sort of the overarching sort of framework that he takes and then he pulls certain threads out of it. So you know, some of the things that he, he talks about ensuring that our social and political institutions, both collectively and individually, grow in tandem with our mastery over the natural world and particularly our progress in new technologies, including AI. Concerns about the concentration of power, especially in the private sector, and what this means for access to technology and the distribution of benefits from technology, especially for poor populations and global majority countries. And also a call for direct policy interventions into AI development in a variety of areas, as you mentioned, to help protect children and workers, to fight against information, to ensure ethical guardrails for the use of AI in war and a number of other areas as well.

A

Yeah, I think as you started to hint at, one of the things that's most interesting about this text is that it goes beyond philosophy and talks about the practical steps that the world needs to take to prepare for the impact of artificial intelligence. And this includes some pretty policy oriented actions. I'm curious which of these stood out most to you?

B

Yeah, well, AI is one of, is a broad ranging technology. It's touching almost every aspect of society and in some way. And so if you look at the encyclical, it does make concrete policy recommendations and they span a huge variety of areas, sort of commensurate with the impact of we're seeing AI have on the world. So the Pope mentions some of the environmental impacts of AI and taking steps to reduce the impacts of AI on use of water and energy. The impact of AI in sort of complicating our information environment, affecting democratic processes and the proliferation of deepfakes and misinformation, algorithmic transparency and making sure that we have insight into what is happening inside AI models. Education, AI literacy, and ensuring that there are sufficient ways to ensure that people know how to use these tools properly. Children safety, for example. The Pope says that farsighted public policies are needed to oppose the immediate interest of platforms concentrate in a few hands when they conflict with the well being of minors. In this regard, interventions by legislators are appropriate for setting age limits, holding service providers accountable rather than shifting the whole borden of control onto families, and for providing specific protections against all forms of online sexual exploitation. And violence.

A

That's a quote right from the text.

B

Yeah, that's right from the text. Paragraph 142. And then there's a lot of discussion about the potential impact of AI on workers and ensuring the human dignity in relation to how people approach employment. And another thing he says directly in the encyclical is in relation to the labor transition. At this time of transition, it is not enough to react only when jobs disappear. We must oversee the transformation in advance. One viable path is, first of all to establish social criteria for innovation. Here, every introduction of automation and AI should be accompanied by verifiable measures to protect the employment, retraining and participation of workers. And this is a theme we see broadly in the cyclical, which is sort of thinking about anticipating and taking proactive action to shape how AI is developed so that it is for the benefit of human beings and where we are being proactive in how we're dealing with the impacts of the technology instead of reactive.

A

Interesting. Well, I want to talk about one particular paragraph. This is paragraph 99 that has received a fair amount of attention. In that paragraph, Pope Leo wrote, quote, we must avoid the misconception of equating this type of intelligence with that of human beings. These systems merely imitate certain functions of human intelligence, end quote. And then he goes on to say, quote, they might. They may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce, for they lack the effective relational and spiritual perspective through which human beings grow in wisdom, end quote. So what criticism has this paragraph drawn, and do you feel like this criticism is merited?

B

Yeah, we've seen some reactions to this particular section from the AI policy community and the AI community more broadly. And the criticism largely centers around the fact that it feels kind of like a punt. Dean Ball, for example, has said that this axiomatic denial of AI cognition is a punt of the highest order because it doesn't directly address the possibility that we'll have real AI cognition. I think in some ways this is kind of a strange criticism if you look at the encyclical, and takes great pain to sort of establish that it is a document that is in discussion with centuries of thought in the Catholic Church, and. And that thought really puts human beings, and in particular human beings with souls, as the center of. Of sort of theological doctrine. And I think it is a bit odd to ask the. The Catholic Church to sort of take an approach to technology that sort of upends centuries, many centuries of doctrine that's developed in a very careful fashion. I also don't think that this is the final thing that we'll see the Church say about AI. This is certainly a landmark document and one that will affect how the Church thinks about AI and its approach to AI for many decades. But it's not the only time I think the Church will have the opportunity or the ability to engage with issues raised by AI. I think in the future it will really have to grapple with sorts of new types of issues raised by technology that maybe the Church has never had to deal with before. And when I think of what that might be in particular, I think of the fact that I think with AI we're the closest to ever being in a situation we're going to create a new type of potential life. Right. That is, I think, far off. It's by no means certain, but typically, if you look at religious doctrine across many, many different types of systems of belief, this is a power that is usually relegated to divine beings or gods or sort of given to humans in limited circumstances as a, as a divine favor. And if now humans are, are in the position to do that sort of thing, I think that the Catholic Church and many other churches and religions will have to deal with what that means and how they think about creation, life, cognition, all sorts of things. I don't think we're at that point yet.

A

Yeah, I mean, that would be quite a lot to deal with given that the current dangers and changes presented by AI are already a lot to deal with. I think Zvi Mashowitz had an interesting quote there where he was like, the document effectively only deals with relatively mundane AI dangers and changes, although that on its own is still rather quite a lot to deal with and discuss. I think that's a really good point there, which is that it is important to think about what AI could become, but also there are so many different dangers and changes that are important to deal with now.

B

Yeah. And there's been long standing criticism in the AI policy space that for a long time people have been focused on the long term theoretical dangers of AI and paid disproportionately less attention to what AI is doing right now. Things like and deepfakes and the effect on the information environment. And I think it makes sense that, you know, that's the way a lot of people are encountering AI right now. And it makes sense to directly address this and this encyclical and there will be future opportunities to deal with the more profound theological questions that AI raises.

A

Right. Well, that was one of the paragraphs that drew a fair amount of reactions. I'm curious if there are any other reactions from people in the AI policy space that caught your attention.

B

Yeah, I think that the reaction has been pretty positive and people have really appreciated how thoughtful the Vatican has been on this topic. They've clearly spent a lot of time thinking about it. The Pope has had an AI advisor for some time and definitely, at least since the moment ChatGPT came out probably earlier, has been thinking about what the Church should be doing in regards to AI. I think one notable thing that stood out, the Vice President, the US Vice President JD Vance said what I read of the encyclical sounds very profound and the sort of thing you would expect and hope from a leader of the Church. The thing about morality is that principles never change, but the way you apply those principles does. Just because the world changes. Right. I think this reflects a lot of the reaction we've seen, which is that, yeah, there are, AI is a new technology. It certainly raises all sorts of new questions. But we also have ways of thinking about the world that have, that have been developed over millennia. And some of those principles we like to think are fundamental principles that won't change just because the world outside changes and will help shape and guide how we're thinking about things. And I think that that is how the Vatican approached this issue. And it is sort of a wake up call for a lot of policymakers to go back to first principles and think about how to approach this technology. I think the other thing I'd say is that the things the Vatican is writing about, they're broadly consistent with what we're seeing in a lot of the world, which is like if you look at polling in the US and in Europe, you see a lot of anxiety about AI, a lot of concerns that AI is not going to make people's lives better, that it's having immediate negative impacts on people's lives are ready, that it's likely to lead to job displacement, it's causing anxiety over people being laid off as well as anxiety over things like inflation with energy prices. And this is a real obstacle to sort of the long term hopes people have about the technology improving people's lives as well as the goals around AI adoption and making sure that this technology is disseminated broadly in the economy. And, and so the things, the kinds of things the Pope is talking about are very consistent with the kinds of concerns we're seeing in many places in the world and the kinds of policy interventions that he talks about in the encyclical I think, are part of the story for what policymakers need to do to sort of help bridge that trust grab and help make people feel like AI is so something that has the potential to improve their lives.

A

Well, as you said, Alok, we'll probably be seeing more from the Vatican on this over the coming months and years. But I want to transition us to our next topic, which is a more traditional topic for the AI policy podcast. President Trump was expected to sign an executive order on AI and cybersecurity on May 21, but he ended up canceling the signing ceremony at the last minute and postponing the executive order's release. Before we get into what we know about the content of the executive order, as well as why it was postponed, what do we know about why the administration was considering an executive order in the first place?

B

Yeah, so the root of the administration's thinking on this issue was really driven by the release of a model by Anthropic called Mythos. So this was a cybersecurity focused model, and the Anthropic didn't release it to the general public. They only released it to a handful of trusted companies. And their calculus here was really that they were concerned about the safety implications of a broad release because of how good the model was at cybersecurity. So, for example, Anthropic wrote in the materials around its announcement of Mythos that, quote, during our testing, we found that Mythos Preview is capable of identifying and then exploiting zero day vulnerabilities in every major operating system and every major web browser when directed by a user to do so. Wow. And yeah, this, this capability doesn't extend just to finding vulnerabilities. It also extends to being able to chain those vulnerabilities together into more significant exploits. And so this, and not only that, but this sort of capability has been validated by some testing by external parties. So we've seen, for example, the UK's AI Security Institute, which has validated some of the findings that Anthropic has talked about. And so for Anthropic, this represents a pretty significant step change in cybersecurity capabilities. And they, they were, they were concerned enough not to release it publicly. And in turn, this, I think, has spooked the government. So we saw a lot of government agencies in the US trying to get access to Mythos so they could do scans of their own systems. A real concern that when systems like this are available more widely, they will really pose significant risks to, to government systems unless the government has some mechanism for getting access in advance and then using these systems to identify potential issues that need patching, and then being able to patch them with sufficient lead time so that they're disseminated to all the systems that need that patch. And so this, I think, is driving a lot of the thinking that led to this potential executive order.

A

Yeah. And how is the Trump administration planning to address these issues through this executive order? What do we know about what was going to be actually in the executive order before the signing ceremony was canceled?

B

Yeah, so unfortunately, we never saw the final text and we don't know exactly what was going to be in it. We do have some pretty strong indications that of where the administration's thinking was, what they were thinking of including, and also importantly what they were thinking of not including. So even in the last month or so, we've seen an evolution in thought here where there was a point in time where the administration, or at least administration officials, were talking about something akin to a mandatory testing and approval process before powerful models like these are allowed to be released broadly to the public. And so, for example, we saw comparisons to potentially implementing something like an FDA or Food and Drug Administration type approval regime where the government would have to give its blessing before these models are released. In the end, I think where the administration ended up, at least the indications we have, are that they wanted not to do a mandatory testing process, but to have some kind of voluntary oversight which would essentially allow developers of advanced AI models to opt into providing their products to the federal government early and then have the federal agencies release them. As part of that, the US Government would have to create benchmarks to determine what would be a covered model that would be appropriate to include in this scheme. And so it would require the development of new ways to evaluate and measure cybersecurity capabilities and be able to distinguish when a model is powerful enough to receive this type of scrutiny or collaboration versus when it wouldn't. And another thing we think was very likely to be in the executive order was some sort of new clearinghouse or way for AI companies and government agencies to come together to share information about threats that had been detected using these types of models and ideally disseminate them ahead of time so that governments could take advanced steps to patch those systems and generally make government IT infrastructure more secure.

A

Yeah. Well, I want to go back to the your mention of the FDA style review that was at one point under consideration. I think that came from a quote from NEC Director Kevin Hassett when he was on Fox News on May 6, where he said that these Models should go through a process similar to an FDA drug review. And I think the fact that we ultimately didn't get that shows that there was some infighting within the administration. In fact, I think several media outlets reported on this infighting in different groups within the administration that maybe wanted different executive orders. So could you tell us a little bit about who these groups were, what they wanted, and then why ultimately the more almost moderate version of the executive order, which was, as you said, a voluntary framework, I still ended up being postponed.

B

Yeah. It's important to remember that administrations are not unitary actors. Ultimately, the President does decide things, but it's very normal for there to be discussion and debate around policy issues, especially tricky policy issues like this. And so, to the best of my understanding, there were multiple kinds of debates happening around the executive order. One of those, as you've hinted, is around voluntary versus mandatory potential nature of testing. And, you know, this idea of mandatory testing is actually very much at odds with how the administration has thought about regulating or dealing with AI policy up to this point, which is really, for the first year and a half of the administration, their focus was on accelerating the pace of AI development and removing regulatory barriers that might impede developing more advanced AI models. And so a really deregulatory approach that isn't fully in line with the mentions of using something like an FDA style approach to approving models. And ultimately, I think this was debated within the administration, and that is why they settled on something lighter, something that didn't involve mandatory approval, but voluntary approval, because this is a part of the administration that has not been dominant up to this point, apparently was in the minority even in these discussions. And so as they discussed and debated, that approach probably fell out of favor. We know there were other conflicts as well, conflicts around what parts of the government might play a role in the various aspects of the executive order. So the latest that, you know, according to the latest information, the Treasury Department was slated to play a major role in implementation of the executive order. And this is a little strange because typically it would be things like the Department of Homeland Security or the Department of Commerce, particularly nist, and various components of a NIST that work on standards that would play more of a major role. And so there was also debate about what the right part of the government was to implement a lot of the executive order.

A

Yeah, I think one official described this, according to Washington Post reporting, as a, quote, knife fight, end quote. So that is interesting. And ultimately, as I said, the more moderate version of this executive order still did not end up Going through. What do we know about what happened the day of that? Ended up getting this executive order signing ceremony canceled and the implementation of the order postponed.

B

Yeah, so it's really interesting because we were incredibly close to this executive order coming out. There was a White House event on the schedule. They had invited CEOs, they were going to have a formal ceremony and it got nix at the very last minute. And ultimately this was something the President decided. The president said that he wasn't in favor of this type of regulation and so postponed it at the last minute. We'll have to see if a new iteration comes out or if this is the kind of postponement that ends up being indefinite and the administration moves on to new things. But if we back up, you know, Trump's decision was influenced by his advisors, in particular, I think David Sacks, who is known as the Aizar and, and advises the President on AI issues, even though he had been involved in discussions around the executive order, I think had new types of concerns at the very last minute. We also know that some tech CEOs reached out the administration at the last minute expressing their concerns about the executive order and how much of a shift it would represent in the administration's approach to regulating AI. And ultimately the influence of those tech CEOs and the influence of David Sachs, both of whom Trump has shown a history of listening to and have had quite a lot of influence on his administration's policies, won the day. And so we ended up not seeing the executive order came out. And we'll just have to see what time brings in terms of whether the administration intends to revisit this issue or if they are going to move on to different ways of approaching the policy issues that AI raises.

A

Yeah, I think the ultimate reasoning that Trump provides reporters at least, is that he said, quote, we're leading China, we're leading everybody, and I don't want to do anything that's going to get in the way of that. So I think that sort of sums up his policy position. And we can also talk about the role that these CEOs and David Sacks also play in that decision. But I think that does it for today's episode of the AI Policy Podcast. Alok, thanks for joining me for your first news roundup. This was great. And thanks to our audience for tuning in.

B

Yep, thanks. Thanks for listening to this episode of the AI Policy Podcast. If you like what you heard, there's an easy way for you to help us. Please give us a five star review on your favorite podcast platform, subscribe and tell your friends. It really helps when you spread the word. This podcast was produced by Sarah Baker and Matt Mand. See you next.