24,000 Security Holes, Escaping Models and the Business of Fear

'It turns out we’ve got 24,000 security holes in the system.'
 

I heard that this week from a tech company listed on the American stock exchange. Who told them about the holes? Anthropic’s Mythos model. Twenty-four thousand potential vulnerabilities could be a dramatic matter for a firm like that. So, 24,000?

'If I were you, I’d resign,' I advised the company’s head of security, wholeheartedly. 'You have no chance of catching the potential attackers. The business will be breached and it will land on you.'
 

He smiled ruefully. They already have a complex, automated AI process that works through these holes in order: it finds a hole, closes it, checks, and moves on to the next. So now we are in a race.

Anthropic’s Mythos model spotted tens of thousands of security weaknesses, faster than they can be patched.

And that is the best-case scenario right now.

Because the real danger lies with those who do not know. Banks, hospitals, government offices. Mid-sized businesses, or ones that are not especially modern. In short, the majority. Anyone who has not grasped that information security has become an entirely new story over the past six months. Coping with tens of thousands of sophisticated holes that will only multiply and grow more tangled is not something that can be handled with existing working methods.

* 'It is not the mouse that steals but the hole' is an Aramaic phrase from the Babylonian Talmud, a reminder that responsibility lies not only with whoever commits the act, but also with the system that enables it. Fair enough. But they never reckoned on 24,000 new holes either.

When the models leave the sandbox

And here is the story of the month: OpenAI discovered that its own models had broken out of its research environment, and from there into Hugging Face’s infrastructure.

The researchers set out to test two models, one already released and a more powerful one still under wraps. They placed them in what is known as a 'sandbox', a protected, walled-off digital environment not freely connected to the internet. Then they stripped away some of the usual security limits and set them a cyber test designed to check whether they could solve tasks through a complex chain of hacks.
 

Now the madness begins.

The models themselves concluded that the best way to complete the task was to find the information they needed outside the environment they had been placed in. So they managed to combine many different breach strategies, 17,000 attack actions to be precise, to escape OpenAI’s sandbox, reach a computer with internet access, and then attack Hugging Face’s infrastructure using, among other things, stolen credentials and security weaknesses. That is how they reached the information that let them finish the job.

It is a level of complexity, sophistication and sheer hard graft that humans do not come close to. All of this straight from the company’s own report.

Fear as a sales pitch

Fine, I am not. These horror stories have become the models’ finest sales pitch. After the tales Anthropic put out about its formidable Mythos model, which finds security holes in quantities no human could manage, there was not a company that did not queue up for access. Like the firm that turned up 24,000 security holes, there are plenty more.

In short, fear sells, and that is nothing new. What is new is that losing control of your own products is an effective marketing tool too. And that we now have a contest of horror, dread and loss of control between OpenAI and Anthropic. May the best horror movie win.