I feel like, while not extremely impressive in a vacuum, the result should be seen as "one run found a security issue in one of the most reviewed applications we have, which was extensively attacked with previous models since they became available (likely every day)".
I mean in addition to running a number of “normal” static code analyzers all the time, using the pickiest compiler options and doing fuzzing on it for years etc
This is what we don't do at all that often anywhere else. Let's brace ourselves to a dark period of less to no security until we… rewrite everything?
3 months ago I watched this guy announce the end of the bug bounty program due to slop on stage. Have the tools gotten that much better, or is it just that without the profit motive, people are spending more time separating the real vulns from the slop?
Looking at Mastodon, this kind of result allows confirmation bias to run wild. But if confirmation bias is set aside, this doesn’t look suitable for extrapolation. Good to see data points getting published, though.
viraptor | an hour ago
I feel like, while not extremely impressive in a vacuum, the result should be seen as "one run found a security issue in one of the most reviewed applications we have, which was extensively attacked with previous models since they became available (likely every day)".
peter-leonov | an hour ago
This is what we don't do at all that often anywhere else. Let's brace ourselves to a dark period of less to no security until we… rewrite everything?
mort | 39 minutes ago
Maybe the "rewrite in Rust" people were more right than they've been given credit for?
FreeFull | 31 minutes ago
That takes care of memory safety and some other things, but there's still tons of security bugs that Rust doesn't prevent, for example TOCTOU
pointlessone | 20 minutes ago
You’re saying that as if memory safety issues are never security issues and are a minority among all the bugs.
It is technically true that Rust is not a silver bullet but it’s also true that memory issues are ~70% of all bugs.
Toric | an hour ago
3 months ago I watched this guy announce the end of the bug bounty program due to slop on stage. Have the tools gotten that much better, or is it just that without the profit motive, people are spending more time separating the real vulns from the slop?
viraptor | an hour ago
https://daniel.haxx.se/blog/2026/04/22/high-quality-chaos/ talks about it. But yeah, it looks like it stopped the slop.
zipy124 | 31 minutes ago
This article is far more interesting than the original in this post to be honest!
hsivonen | 58 minutes ago
Looking at Mastodon, this kind of result allows confirmation bias to run wild. But if confirmation bias is set aside, this doesn’t look suitable for extrapolation. Good to see data points getting published, though.