In the Hugging Face breach, OpenAI's hacker was noisy and fast - but not unstoppable
Cybersecurity experts told TechCrunch that one of the biggest lessons to be taken from the OpenAI hack against Hugging Face has nothing to do with AI, but traditional cybersecurity defense. Earlier this month, AI dataset platform Hugging Face shocked the world when it revealed that it had fallen victim to a fully autonomous AI-powered cyberattack. Days later, the story took another dramatic twist when OpenAI admitted that the hacker behind the breach was one of its AI models , which broke out of a testing environment and into protected Hugging Face systems in an effort to circumvent a benchmark.
Key Takeaways
- It's an alarming incident for anyone even slightly concerned about rogue AI models - and the days since the event have been full of predictions about a new cybersecurity paradigm in which AI models launch attacks so strong that only other AI models can defend against them.
But despite the justified alarm, the paradigm may not have shifted quite as much as it seems.
- " Kyle Ryan, the head of R&D at Pensar , a startup that develops continuous hacking AI agents, and Vlad Ionescu, the co-founder and CTO of RunSybil , a startup that builds AI-powered bug hunters, both agreed and told TechCrunch that the techniques used in the attack would be the same ones employed by a human or a group of human red teamers.
That is, hackers tasked with attacking a system to help the company that owns it improve defenses.
- " Contact Us Do you any more information about OpenAI's hack against Hugging Face?
- "I'd call it more of a defensive failure than exceptionally good offense.
Hugging Face's tooling actually correlated the activity into an attack signal, but failed to raise the criticality and page the on-call team, which cost them time," Ryan explained.
- " Ryan explained that properly implemented techniques such as defense-in-depth - a strategy that leverages several layers of cybersecurity measures - should have given Hugging Face multiple chances to catch the attack.
It's an alarming incident for anyone even slightly concerned about rogue AI models - and the days since the event have been full of predictions about a new cybersecurity paradigm in which AI models launch attacks so strong that only other AI models can defend against them. But despite the justified alarm, the paradigm may not have shifted quite as much as it seems. Experts who spoke to TechCrunch stressed that OpenAI's agent largely operated like a human - with some caveats - and that better implemented traditional defensive techniques could have helped stop the attack.
In short, we may already have the tools to defend against this kind of attack; we just aren't using them properly. Hugging Face made a version of this point in its incident report , stating that the weaknesses exploited in the attack "were familiar," and "a capable human attacker could have found and exploited the same flaws. " Kyle Ryan, the head of R&D at Pensar , a startup that develops continuous hacking AI agents, and Vlad Ionescu, the co-founder and CTO of RunSybil , a startup that builds AI-powered bug hunters, both agreed and told TechCrunch that the techniques used in the attack would be the same ones employed by a human or a group of human red teamers.
That is, hackers tasked with attacking a system to help the company that owns it improve defenses. What was very non-human-like was the speed, scale, and relentlessness of the attack. As Hugging Face explained , OpenAI's agent performed 17,600 actions over four and a half days: It broke in, did reconnaissance, stole passwords and code, and moved around the company's infrastructure.
For more details please read the original article at TechCrunch AI.
Continue Learning
Comments
Sign in to join the conversation