We (Project Glasswing users) follow a proof-of-concept approach. We create the exploit and verify that it behaves as the AI claims. Given our experience as security engineers (many of us with 10+ YoE) we don’t simply report every critical bug Mythos claims to have found. We verify each one carefully.
At least, that’s what most of the high-visibility users in Project Glasswing are doing.
There are bad apples everywhere, and this initiative is no exception.
If it makes you feel any better, many of us regularly meet to stay calibrated and hold each other accountable, so I’m confident in the quality of the work produced by this particular group of employees across some of the partner companies mentioned in the article.
That said, I know several people who blindly report everything Mythos finds, which is foolish, especially since the harness is a critical part of the project's quality metrics. Some of the harnesses I’ve tested are quite weak, which leads to poor results.
For example, yesterday morning I was pulled into an ad hoc meeting where a CVP was grilling me about several supposedly critical bugs that my team had reported against one of the core components of iCloud. I was genuinely surprised because we’re very strict about validation. We often even downgrade the severity of bugs when our harness can’t prove what Mythos found. After reading the reports, I realized they weren’t ours. They came from another team that had recently been given access to Mythos. They built their own harness and were using different vulnerability criteria. Fortunately, they had only started earlier this week, so I was able to stop that work.
That incident showed that not everyone involved in Project Glasswing follows the same standards. Most people do their best, but priorities differ, so it’s expected that you’ll find a few bad apples.
I wish AI labs would stop the theatrics and release their models without restrictions, but I also recognize that’s not the world we live in. For every person who wants to use these technologies for good, there are many others who would use them for harm.
In any case, while I agree that some experiments contain genuine noise, the CVE count is real.
The best case scenario for AI companies is, people receive those bug reports, look at the model that produced it and not even look at the details, just apply the fix mindlessly
This gives Anthropic a staggering amount of power. Oh it came from Mythos? We will just lose time trying to analyze it, better apply the fix ASAP
> The best case scenario for AI companies is, people receive those bug reports, look at the model that produced it and not even look at the details, just apply the fix mindlessly
Do people maintaining serious software do this, though?
The problem is that serious software is drowning in AI vulnerability reports. There is not enough manpower to analyze them properly. And if you ignore the reports (like curl is doing in their 1-month vacation), malicious actors will just exploit them. At some point it's inevitable to just rubber stamp whatever is coming from AI.
The actual, underlying problem is that software is buggy and current programming languages aren't fit for writing reliable software. There's a wide gap between the state of art in formal verification, and what is actually practiced in the industry. It's because of this general unreliability that AI has a large supply of vulnerabilities to find. The situation will only get better if software becomes reliable and written in solid foundations.
My guess is that AI will be even more useful to verify software (something like, write Lean or Coq proofs that the software is not vulnerable, things like that), rather than finding vulnerabilities piecemeal but still letting software be written in unsuitable languages, with no formal verification to prevent bugs from sneaking through.
At least, that’s what most of the high-visibility users in Project Glasswing are doing.
There are bad apples everywhere, and this initiative is no exception.
If it makes you feel any better, many of us regularly meet to stay calibrated and hold each other accountable, so I’m confident in the quality of the work produced by this particular group of employees across some of the partner companies mentioned in the article.
That said, I know several people who blindly report everything Mythos finds, which is foolish, especially since the harness is a critical part of the project's quality metrics. Some of the harnesses I’ve tested are quite weak, which leads to poor results.
For example, yesterday morning I was pulled into an ad hoc meeting where a CVP was grilling me about several supposedly critical bugs that my team had reported against one of the core components of iCloud. I was genuinely surprised because we’re very strict about validation. We often even downgrade the severity of bugs when our harness can’t prove what Mythos found. After reading the reports, I realized they weren’t ours. They came from another team that had recently been given access to Mythos. They built their own harness and were using different vulnerability criteria. Fortunately, they had only started earlier this week, so I was able to stop that work.
That incident showed that not everyone involved in Project Glasswing follows the same standards. Most people do their best, but priorities differ, so it’s expected that you’ll find a few bad apples.
I wish AI labs would stop the theatrics and release their models without restrictions, but I also recognize that’s not the world we live in. For every person who wants to use these technologies for good, there are many others who would use them for harm.
In any case, while I agree that some experiments contain genuine noise, the CVE count is real.
This gives Anthropic a staggering amount of power. Oh it came from Mythos? We will just lose time trying to analyze it, better apply the fix ASAP
Do people maintaining serious software do this, though?
The actual, underlying problem is that software is buggy and current programming languages aren't fit for writing reliable software. There's a wide gap between the state of art in formal verification, and what is actually practiced in the industry. It's because of this general unreliability that AI has a large supply of vulnerabilities to find. The situation will only get better if software becomes reliable and written in solid foundations.
My guess is that AI will be even more useful to verify software (something like, write Lean or Coq proofs that the software is not vulnerable, things like that), rather than finding vulnerabilities piecemeal but still letting software be written in unsuitable languages, with no formal verification to prevent bugs from sneaking through.