SACHA PFEIFFER, HOST:
It's a science fiction nightmare. What happens if the machines take over? The world got a potential taste of that last week when OpenAI, the maker of ChatGPT, said two of its models had autonomously hacked into another AI company. OpenAI says its technology carried out the attack during a cybersecurity test. Its models broke out of their so-called sandbox and accessed the private servers of another AI company. The incident has highlighted fears that losing control of AI could pose broader safety concerns for the real world. So I asked Thomas Wolf, the co-founder and chief science officer of the company that was attacked, Hugging Face, what he believed the rogue model was trying to do.
THOMAS WOLF: Everything here is something like a normal human hacker would have done. Really, the surprising thing is that the AI was not at all tasked with attacking us - thankfully - but actually was trying to solve this challenge. So the challenge is you give a software vulnerability that is known to an AI and you task the AI with trying to use that as what we call an exploit, which is trying to use this software vulnerability to take control of a computer. And what the AI decided to do in this case was to go on the internet and try to find the answer to the challenge directly, just like someone sneaking out, you know, professor office to basically grab the answer to the test.
PFEIFFER: Your CEO has said that he does not believe there was any malicious intent on the part of OpenAI, that it didn't set out to hack you. But this is the kind of security breach that taps the greatest fears that people have about how powerful AI can be. So you are, as we said, the chief science officer of an AI company. How would you address those fears?
WOLF: Well, I think we need to monitor this frontier system.
PFEIFFER: A frontier system being a very cutting-edge AI tool.
WOLF: Exactly. Being able to detect better what they are doing, understand better, you know, what is the current task that they are actually doing. And it's quite likely that in this case there was just maybe not enough monitoring of what the system was doing in place.
PFEIFFER: You know, the media coverage of this has been fairly alarmist. It's a fear of what could be. But you seem not very concerned. How would you describe your level of concern?
WOLF: I would say an informed level of concern, which is I know the remediation tools for this are here. I don't think there is anything, you know, new that we need to invent to be able to monitor what is AI doing right now. It's mostly just a wake-up call to take these things seriously and to actually take security seriously. There is currently a debate which is around should we allow open-source system to be deployed as well, or should we only have closed-source systems?
PFEIFFER: And your company is open-source. Hugging Face is open-source, is that right?
WOLF: Our company is open-source, yeah.
PFEIFFER: For people, by the way, who aren't familiar with open-source concerns, the fear, as I understand it, is that if it's open-source - meaning the public can come and use it - it's more likely to be used by malicious actors who are trying to do malicious things with it. But you're saying that this incident shows the importance of more open-source AI models. Why do you think that?
WOLF: Yeah, exactly. So this system is surprising because it's the exact opposite of what people were thinking. So we were expecting, just like you said, that an open-source system would be used to attack someone because they are slightly easier to access, I would say. In practice, what we discovered in our case were that we were attacked actually by a closed-source system. And when we tried to defend ourself with closed-source model, they just decided not to help us because they said, this is too similar to an attack. We're not allowed to help you with that. And so we had to turn out to actually an open-source system to defend ourselves. And my feeling is that you need to have access to them as well when you want to defend yourself.
PFEIFFER: You describe this as a wake-up call and a reason for more transparency. If that doesn't happen, what do you think the risk is?
WOLF: I think if it doesn't happen, the risk are quite strong. And what - if there is no more transparency - right? - and no more access to model to defend yourself, we basically end up in a situation where we have a lot of concentration of power in just a couple of company who basically don't have to answer to anyone - right? - or to anybody. I think that's a pretty - it's a pretty dangerous future for the field. We've always pushed for open-source and transparency. We think the more the public can understand what's happening and the more, like, people can - like, third party can monitor or can do positive and forensic to understand what happened, the better it is for everyone.
The good thing here is that OpenAI was pretty transparent, and I think that's something we really want to commend and we really want to see more and more in the future, which is this company not fearing, like, reputation risk or from, you know, the right incentive being incentivized to actually publicly explain what happened and be more transparent. I hope that's the good takeaway of this event.
PFEIFFER: That is Thomas Wolf. He is the co-founder and chief science officer of the AI company Hugging Face. Thank you for talking about this.
WOLF: Thank you, Sacha. Transcript provided by NPR, Copyright NPR.
NPR transcripts are created on a rush deadline by an NPR contractor. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of NPR’s programming is the audio record.
