These guys are now the biggest slowdown to fixing issues with their bullshit cyber detection. They drop a model, let the model find security issues in your code, then when you want to fix them you need to play a game of charades so it doesn't realize it's creating a validation and then a fix for the issue because you might be hacking yourself. Meanwhile attackers are using some ablated deepseek or what have you.
It's frustrating. Claude thinks it finds a way to hop from one client to another, but any attempt to get it to demonstrate it isn't relying on something that's already prevented runs into "safety" concerns and gets blocked.
I'm not that hot on the idea of open models, but having something dangled in front of you but suddenly unable to verify is not a good feeling.
With the right amount of coaxing you can sometimes get it to reproduce a PoC with a "TDD first" approach to fixing, but often now that gets blocked too.
So far I can always get around it by starting fresh sessions on a similar problem and without ever mentioning security issues, but why do I have to play this little theater with a tool we pay to use.
On the other hand this sounds like a job guarantee, though. And not only for the moment, as it is not the capability of the model which is insufficient.
I'm not opposed to open models at all, I'm just slightly sceptical on how much some people push their use compared to the value we get out of a dirt cheap subscription.
Propriety models are clearly subsidised by enterprise API contracts, so it makes sense to milk that for all its worth, provided one steers clear of any lock in so the option to switch is always there.
Jumping to spending time managing and self-hosting open models is a nice ideal but I think it's a distraction to jump on that train without proven benefits given the value that other plans deliver.
I don't mean I'm against the idea of open models, they were especially instrumental in Anthropic backing down from the "Fable only initially available 2 weeks" plan, which got extended up until Fable was over-taken by Opus in general usefulness.
It's my view that any company sized appropriately for Teams plan should be milking it for all its worth while it still exists, and it is a waste to lose that opportunity by jumping on open models too early.
If you're a larger enterprise then it would be a very different story, you'd already be paying more forced onto API pricing and would likely have the capacity to have a team look at open models without impacting delivery.
there is a middle ground between proprietary models and self-hosting: Fireworks, et al. Popular models like DeepSeek have a couple dozen providers on openrouter.
There is, but it also has an additional trust layer on top, where you have to trust you're getting back responses from the models you're being sold and aren't being artificially quantised or stealthily redirected to cheaper models.
I trust ( although some don't ), Anthropic not to be doing that. I don't trust intermediate providers so much. If you're self-hosting then at least you know exactly what is running and get privacy on top.
Likewise you lose privacy of local hosting, and if it emerged that a provider was training on data they shouldn't be, would you have the necessary legal recourse, that you would going direct to the major model providers?
React is actually pretty good for any kind of reactive interface, and it's small. Something under 3k lines of code, and it's well battle tested at this point.
You might have a dislike of react and that's fine, but it's not adding any significant attack surface to a project.
Just the ReactFiberWorkLoop.js has 5k+ lines of code. React is one or two orders of magnitude bigger than you are thinking. And that's not counting Nextjs.
I don't feel like that's a good faith reply. I'm making the claim that react's production build core is surprisingly small and shouldn't be considered a security risk (or attack surface as someone above phrased it).
Your claim that the "react core" is small isn't correct. React+ReactDOM is >130kb, the minimum needed to run a react component. To create TUIs, Ink ships even more.
> React is actually pretty good for any kind of reactive interface,
It’s a different interface and the axiomatic layers are not the same.
React expects a tree of nodes that will map to the UI and build on that to add reactivity. And that’s what the major GUI platforms do offer (QT, the web dom, Android, iOS,…).
The terminal is a grid of characters modfied by the cursor concept. It is also modal (although most TUI uses one mode). What the cursor can do, and how the grid is rendered is terminal dependent (even though most current ones try to follow xterm-256color).
Using React on top of that is a technical bad idea.
If only Claude didn't constantly drop to less capable models whenever it encountered anything vaguely cybersecurity related (despite being part of the Anthropic Cyber Verification Program and having Mythos access, albeit limited).
We're strongly pushing for a move to a mix of Codex and open models at my workplace because of Anthropic's ridiculous nanny-ing.
These types of flip flop posts are the new security theater. One week Anthropic will post a doom and gloom about how open weight models running a fraction of a token per second on your Grandma's popped Android TV is a national security threat and the next week they'll tell you how awesome their 200+ billion parameter model is at doing something they don't actually allow you to do because they gatekeep all the normies. I really do hope history remembers them correctly, but I'm sure they'll be attributed as being a significant and load bearing linchpin during the new world order of LLMs.
Also in the CVP and kept tripping cyber a couple days ago due to an MCP server that had the ability to connect to remote devices. Maybe it thought I was trying to make a C&C server. But you literally have to start the MCP first, have it generate a key, give the key to the remote device and then it'll generate its key, then give that key back to the MCP server before they can connect.
Yep, gotta really mind the language, sometimes saying something like “when implementing <foo> feature you gotta pay attention to <bar> to avoid <baz> attack” is enough to trigger it, along with a cache miss of 300k tokens in the downgraded request to rub it in. Frustrating.
Having a big brother on the frontier models in the cyber verification program defeats the whole purpose of the program. My team has found ourselves leaning the same direction your team is going. I was looking into GLM 5.3 to host in our env and come to find there is an article from anthropic trying to say that model is "dangerous" and/or not good compared to their models.
There is really just one way this goes and its upwards to the peak: which means full formal verification programming languages, OS uni-kernels, small TCB and probably verified HW for the most sensitive cases. Everything else will soon be hackable via 'brute compute' because economy of scale and accessibility of open weight models.
These programs are executed in environments. Which are run on a hardware. Big part of which is hidden software.
And in general you can have 5 boxes verified and be surprised what comes out of them calling each other. Number of boxes is growing and possible interactions is exp. I don't think formal verification solves much.
I'm doing this attack surface tactic with Vix IDE, a terminal text editor that helps novices get comfortable with vi, emacs, git, jj, sql, commands, and more.
The prompt I used was "Goal: rework to reduce the attack surface. Process: move features into feature-flag Rust crates. Output: build a minimal version. Questions: Ask me anything."
This past weekend, Claude Fable/Opus/Sonnet together successfully refactored the whole Vix IDE codebase into many separate Rust crates and Cargo features.
This refactor enables a security-oriented user to pick and choose, and also makes it easier for a security auditor to look at one capability at a time.
If you're curious you can see the commit messages by Claude Sonnet. IMHO they're reasonably good.
Is there not a great irony in over a weekend throwing Fable/Opus/Sonnet at your project, refactoring your whole codebase, and the purpose of that project is to teach people to use vi, git, sql and so on. Why would anyone want to learn that in the current day and age, when as you state it can refactor a project over a weekend fairly autonomously?
This sounds so arrogant, lol. I know it actually means that they red team the frontier AI models (which is a reasonable claim), but it looks like they are claiming to be the best red team in the world
I would believe Anthropic about how Claude reduces attack surfaces if they weren't simultaneously nannying everyone about how they supposedly don't want their models used for cybersecurity because... Reasons. And if you do want that access, you have to verify your identity, and do a bunch of other things, all of which goes against "reducing" the "attack surface". If they really gave a damn about cybersecurity, this cybersecurity filter wouldn't exist, period.
> If they really gave a damn about cybersecurity, this cybersecurity filter wouldn't exist, period.
Golden guns don't keep you safe from silver bullets.
These models are obviously getting used to commit acts which would be criminal if done with intent (https://www.felonybench.com), handing everyone a digital version of a machine gun means everyone catches strays.
Worse: every capability the SOTA models demonstrate is available in an open weight model 3 months later, so we're all catching strays already, and it's only the exact policies you're unhappy about which gave critical infrastructure any prep time.
And recall, absent demonstrations of capabilities (and sometimes even with demonstrations), lots of people respond dismissively to these warnings; before OpenAI had actually hacked into HuggingFace, people thought it was "science fiction" that an AI would find and exploit novel zero-days in its own sandbox.
No, it means some billionaires catch strays and have to sell one of their yachts out of their fleet, or cut dividends. The collapse of the current Google/Microsoft security and management duopoly would be less impactful on the average computer user than the death of flash player.
None of this is for you; it's for money. It's someone else's empire. Cybersecurity is a farce, and western culture would quickly recover from a complete collapse, and be better off for it.
compliance is payed for by the parties ensuring the compliance.
so yes if the cybersecurity was not strong enough, some big companies would go bankcrupt or change their TOS.
And then others would emerge, and the cat mouse game of hackers and viruses in the 2000s would pick up again.
We had a bit of a calm period when Anti virus programs in windows didnt do much anymore
> No, it means some billionaires catch strays and have to sell one of their yachts out of their fleet, or cut dividends.
Their capital isn't your body armour.
Anything it costs them has all the impact of an ambulance at the bottom of a cliff, when we could have a fence at the top.
> The collapse of the current Google/Microsoft security and management duopoly would be less impactful on the average computer user than the death of flash player.
Add Cisco (CVE-2026-20213), NGINX (CVE-2026-27654), the Linux kernel (CVE-2026-31554), ImageMagick (CVE-2026-33901), FreeRDP (CVE-2026-40033), OpenSSL (CVE-2026-45447), Suricata (CVE-2026-45751), FreeBSD (CVE-2026-4747), PostgreSQL (CVE-2026-6479), and Mozilla Firefox/Thunderbird NSS library (CVE-2026-6772).
> None of this is for you; it's for money. It's someone else's empire. Cybersecurity is a farce, and western culture would quickly recover from a complete collapse, and be better off for it.
Cybersecurity is indeed a farce; one does not put out a dumpster fire by pouring on gasoline. Well, not unless it's a huge amount, and then you've got gasoline everywhere off-gassing and one spark from being much worse.
Due to tech being over-sold as a solution to everything, it has been so heavily integrated in our infrastructure that "a complete collapse" would be devastating.
We see small, limited, isolated incidents already, just from Ukraine blowing up Yandex servers and by extension knocking out random chunks of Russian economy, or unknown hackers switching French/Dutch/US fridges to defrost overnight spoiling the food, or messing with water control systems in (fortunately small) reservoirs, Revolut not working for a bit when one European DC went down, and some stupidity where a root signing certificate had a private key of "4" for 18 months without anyone noticing.
Do this everywhere all at once, and it's likely we get mass blackouts, banking in general stops, software updates and signing can't be trusted to undo the mess. It's anywhere from "really bad" to "civilisational crisis" if it dropped in unexpected and unprepared for.
This doesn't work. Even if you use agents that are 3 months behind, they will still find the vulnerabilities Mythos or Claude Opus would find, it would just take them a bit longer to do it. As such there is little reason for these cybersecurity filters to exist. They don't actually prevent people from finding vulnerabilities; it just slows people in finding them because they have to use less capable models, and by extension that makes cybersecurity postures worse. You can't ever get rid of the arms race between defender and attacker. Anthropic acting all nanny-ish doesn't at all help the situation -- if anything, I would argue, it just makes it significantly worse. Especially when the filters will have significant false positives (e.g., they will trigger for local things that are completely unrelated to hacking at all, because somehow that's super duper dangerous).
> They don't actually prevent people from finding vulnerabilities; it just slows people in finding them because they have to use less capable models,
That's the point. Slowing people down. Hackers aren't "not people".
> You can't ever get rid of the arms race between defender and attacker.
Yes you can, and by doing exactly what you're complaining about:
And if you do want that access, you have to verify your identity, and do a bunch of other things
That's the thing which gives defenders an advantage. A 3-month advantage, or could have been if the US government hadn't blocked it for a non-trivial fraction of the gap.
> ... and by doing exactly what you're complaining about:
How exactly does requiring people to identify themselves actually protect anyone or slow them down? Their identity system doesn't even work reliably (it's just your classic "verify your ID" scheme), and it failed to verify my ID no matter how I did it, so in the end I gave up. Even assuming it was reliable, I don't see how it stops bad actors from doing it, since we all know how easy it is for most sufficiently resourceful actors to forge IDs, and I suspect these digital ID verification services have all kinds of vulnerabilities that would make them easier to bypass compared to the tools the state has to prove an ID is legit. For the higher tiers of CVP there may be more requirements, but do you seriously think these companies aren't just using AI to verify those tiers too?
The flip side of it is that now the US agencies have unrestricted access to that machine gun and software developers don't, or at best still heavily censored access (their "trusted" program is a joke). Instead of building a tool to fix security issues, they've essentially built a cyber weapon for offensive use. Which was obvious from the day 1 of Mythos shenanigans.
Yes, agreed the US agencies will have the best. But not from Anthropic: recall the White House unlawfully blacklisted Anthropic when Anthropic dared say no to their models getting used in certain ways, and that Anthropic was given an order to stop supplying to any non-American despite the large number of non-Americans on their staff.
So for this:
> Instead of building a tool to fix security issues, they've essentially built a cyber weapon for offensive use.
They did both. That it's a machine gun with an overly-cautious safety system isn't great, but it's also a tool and as a tool is better than either a machine gun with no safety system or just giving no-questions-asked direct control to just the US government.
No-questions-asked access would be from the other AI vendors.
I’m very frustrated by this, for the record. But they have the atom bomb, and we want to borrow their tech for nuclear power. They don’t trust us. This familiar geopolitical quagmire isn’t resolved on the global scale, so I don’t expect these flawed companies to do it any better.
One can phrase it: bad actors already have nukes, and throwing them on peaceful American cities every day, and gov + corps deny citizen access to defensive tools while do nothing for protecting citizens.
Recently I watched this presentation of a kernel developer about the "Mythos' 79 kernel vulnerabilities": https://youtu.be/NnV_cWeoo5Q?t=257
He claims that most of these vulnerabilities are not an issue at all, or have been fixed earlier, etc. I can clearly see the improvement of the AI models, but based on this youtube video it's very plausible that these AI labs rush to announce a result without further understanding of the underlying problems. I wonder what's the quality here in this article.
That's the weird thing with LLMs, they know just enough about a topic to fool anyone without much knowledge on the subject, seemingly even frontier labs.
It doesn't help that the labs have positioned themselves on the predication that expertise is no longer required.
They are trying to sell this false myth of machines so good they can replace any human expert, and they buy this BS themselves, so, instead of actually learning something and/or confirming results with institutions of experts (what's left of them anyway) they just rush out a PR piece knowing the unwitting public and VCs won't care enough to look into the validity of anything themselves anyway.
The whole thing is a gamble on whether or not they can convince the world that a fancy word machine substitutes for deep experience and expertise.
(To be clear, the fancy word machines are still very useful tools, but they are not adequate replacements for knowing humans).
Yes, and before that you were probably adept at writing code based on incomplete and sometimes inaccurate documentation, using tools and technologies that were powerful but occasionally buggy.
Or at the very least, you were interested in the subjects in question and just never had the time or other resources to learn them.
Yeah the rate at which I'm learning is insane! I think the key is picking a project with practical goals that you can track towards, ideally backed with concrete evidence (like data) and measuring progress by how close you are getting to those goals via working code.
As such, I'm learning through motivated, self-directed, focused exploration, using the AI to answer questions, run experiments, explain findings, investigate surprises, and explore new directions to research, all grounded in concrete data related to those goals.
The data and our results on it are what keep us honest. The data is also what helps understanding nuances and building intuition because the AI can explain outcomes in the context of samples data points that you can reason about independently.
The difference between YOLO vibe-coding and engineering is understanding. I think this is the future of learning and skill-building.
I don't get what your point is WRT this article. They aren't announcing a result? They're describing a technique they used in real projects to prevent new vulnerabilities, including a change that is merged into OpenSSL. What is the "rush to announce a result" here? What are you saying they don't understand?
I literally included it. more broadly: dont install any so-called AI/LLM "black box" agent anywhere if you want to minimize attack surface and risk. Claude might be the most notoriously foolish but the same security-sloppy design decisions are likely present in the other similar codebases. And the line between reckless naivete and intentional maliciousness is conveniently blurry.
I'm with you here, just asking what the proposed stack is here, which you did not include.
If I understand correctly you're saying custom agent and possibly locally run open model, or are you saying no to all LLM / agent use outright? Either is fine, I'm not here to tell you're wrong, I'm just curious about what others are doing.
There is an unresolved issue where the offensive guys can use AI to attack a target and the defensive guys are being limited when they ask AI to audit their codebase.
The workflow for a defender basically goes:
Claude do a security audit on this codebase, use the harness / skills I built on how to do security audits.. pick the best suitable audit strategy, methods.. use the tooling and reporting structures.
Claude refuses, GPT refuses.
Grok, goes further, but refuses to make a working PoC.. a requirement because AI models make stuff up and without a PoC there is no proof.
GLM-5.3 / Kimi to the rescue and completes the job the "smarter" models refused.
So we as defenders are refused by the best models, on our own codebase
Heck, Codex was trying to use gdb to debug a capture crash and the session locked up with security blocks. I had to restore a checkpoint and tell it abandon that work and stay away from gdb.
I've heard people getting jammed up with Claude refusing to update DNS..
There's nothing stopping you from applying for Daybreak for OpenAI / Unsure which program Anthropic has. But yes, you do have to share your identity and I can see how that is a blocker for some.
Hey, speaking of Daybreak: I just saw you can apply for this yesterday and was wondering if people here have had success being approved as an individual? If so, how long did it take from application to access?
So rich to have them post this when just this morning I had to switch to GLM 5.3 so I could mitigate the PHP CVEs (both Claude and Codex balked at the early investigation work). Once I was further along I was able to switch back to Claude to implement the fix but it refused to look at any POC's or test them against my dev servers, so freaking annoying.
The only things worse than the CVE getting dropped in our laps with little warning was having Claude run into its safeguards which slowed my progress in actually protecting our customers.
Last week Claude (and Codex) also refused to help write unit tests around existing code because it touched on authentication. I was able to ladder between them past everywhere they got stuck but today I went straight to GLM after hitting safeguards.
Anthropic keeps talking about using its models to detect security issues in existing projects and other people's code, but no mention of its models creating security issues in new code and the mass of vulnerabilities AI agents are creating in its wake. If you run Fable on Opus code, how many security issues do you find? The irony is Fable can detect many of them, but its security guardrails refuses to fix them.
vasco | 15 hours ago
xnorswap | 15 hours ago
I'm not that hot on the idea of open models, but having something dangled in front of you but suddenly unable to verify is not a good feeling.
With the right amount of coaxing you can sometimes get it to reproduce a PoC with a "TDD first" approach to fixing, but often now that gets blocked too.
vasco | 14 hours ago
generic92034 | 14 hours ago
featwanz | 13 hours ago
ajb | 13 hours ago
xnorswap | 12 hours ago
Propriety models are clearly subsidised by enterprise API contracts, so it makes sense to milk that for all its worth, provided one steers clear of any lock in so the option to switch is always there.
Jumping to spending time managing and self-hosting open models is a nice ideal but I think it's a distraction to jump on that train without proven benefits given the value that other plans deliver.
I don't mean I'm against the idea of open models, they were especially instrumental in Anthropic backing down from the "Fable only initially available 2 weeks" plan, which got extended up until Fable was over-taken by Opus in general usefulness.
It's my view that any company sized appropriately for Teams plan should be milking it for all its worth while it still exists, and it is a waste to lose that opportunity by jumping on open models too early.
If you're a larger enterprise then it would be a very different story, you'd already be paying more forced onto API pricing and would likely have the capacity to have a team look at open models without impacting delivery.
rgbrenner | 12 hours ago
xnorswap | 9 hours ago
I trust ( although some don't ), Anthropic not to be doing that. I don't trust intermediate providers so much. If you're self-hosting then at least you know exactly what is running and get privacy on top.
Likewise you lose privacy of local hosting, and if it emerged that a provider was training on data they shouldn't be, would you have the necessary legal recourse, that you would going direct to the major model providers?
esperent | 14 hours ago
I think the hackers are just using "not Claude".
abc123abc123 | 12 hours ago
owebmaster | 13 hours ago
esperent | 13 hours ago
You might have a dislike of react and that's fine, but it's not adding any significant attack surface to a project.
Nextjs on the other hand...
shakna | 12 hours ago
brazukadev | 12 hours ago
esperent | 9 hours ago
I had a quick look around and it's actually ~1000 lines apparently, in React 19.
owebmaster | 8 hours ago
esperent | 6 hours ago
Do you dispute that? If so, why?
owebmaster | 5 hours ago
Maken | 12 hours ago
esperent | 9 hours ago
skydhash | 10 hours ago
It’s a different interface and the axiomatic layers are not the same.
React expects a tree of nodes that will map to the UI and build on that to add reactivity. And that’s what the major GUI platforms do offer (QT, the web dom, Android, iOS,…).
The terminal is a grid of characters modfied by the cursor concept. It is also modal (although most TUI uses one mode). What the cursor can do, and how the grid is rendered is terminal dependent (even though most current ones try to follow xterm-256color).
Using React on top of that is a technical bad idea.
bothers | 9 hours ago
That's mighty white of you, it's a dirty job but someone's got to tell others what experiences and feelings are fine
LoganDark | 8 hours ago
The fuck kind of racism is this? Calling people white for using basic rhetoric?
DeluluDon | 2 hours ago
messe | 13 hours ago
We're strongly pushing for a move to a mix of Codex and open models at my workplace because of Anthropic's ridiculous nanny-ing.
windexh8er | 11 hours ago
nl | 10 hours ago
It's a 10T BTW.
LoganDark | 10 hours ago
oefrha | 10 hours ago
LoganDark | 8 hours ago
jDucky | 4 hours ago
mentalgear | 13 hours ago
comboy | 9 hours ago
And in general you can have 5 boxes verified and be surprised what comes out of them calling each other. Number of boxes is growing and possible interactions is exp. I don't think formal verification solves much.
ben_w | 9 hours ago
smj-edison | 8 hours ago
lptome | 12 hours ago
cianmm | 12 hours ago
steve1977 | 12 hours ago
StilesCrisis | 8 hours ago
jph | 11 hours ago
The prompt I used was "Goal: rework to reduce the attack surface. Process: move features into feature-flag Rust crates. Output: build a minimal version. Questions: Ask me anything."
https://github.com/vixide/vix
This past weekend, Claude Fable/Opus/Sonnet together successfully refactored the whole Vix IDE codebase into many separate Rust crates and Cargo features.
This refactor enables a security-oriented user to pick and choose, and also makes it easier for a security auditor to look at one capability at a time.
If you're curious you can see the commit messages by Claude Sonnet. IMHO they're reasonably good.
philipp-gayret | 9 hours ago
Zambyte | 10 hours ago
This sounds so arrogant, lol. I know it actually means that they red team the frontier AI models (which is a reasonable claim), but it looks like they are claiming to be the best red team in the world
cube00 | 10 hours ago
Rapzid | 7 hours ago
Marcom07 | 9 hours ago
StilesCrisis | 9 hours ago
Missing an apostrophe? A human wrote this! Love to see it.
ethin | 9 hours ago
ben_w | 9 hours ago
Golden guns don't keep you safe from silver bullets.
These models are obviously getting used to commit acts which would be criminal if done with intent (https://www.felonybench.com), handing everyone a digital version of a machine gun means everyone catches strays.
Worse: every capability the SOTA models demonstrate is available in an open weight model 3 months later, so we're all catching strays already, and it's only the exact policies you're unhappy about which gave critical infrastructure any prep time.
And recall, absent demonstrations of capabilities (and sometimes even with demonstrations), lots of people respond dismissively to these warnings; before OpenAI had actually hacked into HuggingFace, people thought it was "science fiction" that an AI would find and exploit novel zero-days in its own sandbox.
functionmouse | 8 hours ago
None of this is for you; it's for money. It's someone else's empire. Cybersecurity is a farce, and western culture would quickly recover from a complete collapse, and be better off for it.
busssard | 8 hours ago
And then others would emerge, and the cat mouse game of hackers and viruses in the 2000s would pick up again. We had a bit of a calm period when Anti virus programs in windows didnt do much anymore
ben_w | 7 hours ago
Their capital isn't your body armour.
Anything it costs them has all the impact of an ambulance at the bottom of a cliff, when we could have a fence at the top.
> The collapse of the current Google/Microsoft security and management duopoly would be less impactful on the average computer user than the death of flash player.
Add Cisco (CVE-2026-20213), NGINX (CVE-2026-27654), the Linux kernel (CVE-2026-31554), ImageMagick (CVE-2026-33901), FreeRDP (CVE-2026-40033), OpenSSL (CVE-2026-45447), Suricata (CVE-2026-45751), FreeBSD (CVE-2026-4747), PostgreSQL (CVE-2026-6479), and Mozilla Firefox/Thunderbird NSS library (CVE-2026-6772).
> None of this is for you; it's for money. It's someone else's empire. Cybersecurity is a farce, and western culture would quickly recover from a complete collapse, and be better off for it.
Cybersecurity is indeed a farce; one does not put out a dumpster fire by pouring on gasoline. Well, not unless it's a huge amount, and then you've got gasoline everywhere off-gassing and one spark from being much worse.
Due to tech being over-sold as a solution to everything, it has been so heavily integrated in our infrastructure that "a complete collapse" would be devastating.
We see small, limited, isolated incidents already, just from Ukraine blowing up Yandex servers and by extension knocking out random chunks of Russian economy, or unknown hackers switching French/Dutch/US fridges to defrost overnight spoiling the food, or messing with water control systems in (fortunately small) reservoirs, Revolut not working for a bit when one European DC went down, and some stupidity where a root signing certificate had a private key of "4" for 18 months without anyone noticing.
Do this everywhere all at once, and it's likely we get mass blackouts, banking in general stops, software updates and signing can't be trusted to undo the mess. It's anywhere from "really bad" to "civilisational crisis" if it dropped in unexpected and unprepared for.
ethin | 7 hours ago
ben_w | 7 hours ago
That's the point. Slowing people down. Hackers aren't "not people".
> You can't ever get rid of the arms race between defender and attacker.
Yes you can, and by doing exactly what you're complaining about:
That's the thing which gives defenders an advantage. A 3-month advantage, or could have been if the US government hadn't blocked it for a non-trivial fraction of the gap.ethin | 3 hours ago
How exactly does requiring people to identify themselves actually protect anyone or slow them down? Their identity system doesn't even work reliably (it's just your classic "verify your ID" scheme), and it failed to verify my ID no matter how I did it, so in the end I gave up. Even assuming it was reliable, I don't see how it stops bad actors from doing it, since we all know how easy it is for most sufficiently resourceful actors to forge IDs, and I suspect these digital ID verification services have all kinds of vulnerabilities that would make them easier to bypass compared to the tools the state has to prove an ID is legit. For the higher tiers of CVP there may be more requirements, but do you seriously think these companies aren't just using AI to verify those tiers too?
orbital-decay | 7 hours ago
ben_w | 4 hours ago
Yes, agreed the US agencies will have the best. But not from Anthropic: recall the White House unlawfully blacklisted Anthropic when Anthropic dared say no to their models getting used in certain ways, and that Anthropic was given an order to stop supplying to any non-American despite the large number of non-Americans on their staff.
So for this:
> Instead of building a tool to fix security issues, they've essentially built a cyber weapon for offensive use.
They did both. That it's a machine gun with an overly-cautious safety system isn't great, but it's also a tool and as a tool is better than either a machine gun with no safety system or just giving no-questions-asked direct control to just the US government.
No-questions-asked access would be from the other AI vendors.
tjbrock | 3 hours ago
y-curious | 8 hours ago
andriy_koval | 3 hours ago
petermora | 9 hours ago
He claims that most of these vulnerabilities are not an issue at all, or have been fixed earlier, etc. I can clearly see the improvement of the AI models, but based on this youtube video it's very plausible that these AI labs rush to announce a result without further understanding of the underlying problems. I wonder what's the quality here in this article.
ActionHank | 8 hours ago
voidhorse | 8 hours ago
They are trying to sell this false myth of machines so good they can replace any human expert, and they buy this BS themselves, so, instead of actually learning something and/or confirming results with institutions of experts (what's left of them anyway) they just rush out a PR piece knowing the unwitting public and VCs won't care enough to look into the validity of anything themselves anyway.
The whole thing is a gamble on whether or not they can convince the world that a fancy word machine substitutes for deep experience and expertise.
(To be clear, the fancy word machines are still very useful tools, but they are not adequate replacements for knowing humans).
dev_hugepages | 7 hours ago
dijit | 7 hours ago
There are humans like this and they are some of the most catastrophic people to have on a project, I actively avoid them.
I'm sure LLMs are different.
smashers1114 | 7 hours ago
ActionHank | 7 hours ago
If that's the case, that's the primary example that I've seen in many places.
keeda | 5 hours ago
How do I know I didn't learn hallucinations?
Because based on that knowledge, I've produced code that works as intended.
jasomill | 3 hours ago
Or at the very least, you were interested in the subjects in question and just never had the time or other resources to learn them.
keeda | an hour ago
As such, I'm learning through motivated, self-directed, focused exploration, using the AI to answer questions, run experiments, explain findings, investigate surprises, and explore new directions to research, all grounded in concrete data related to those goals.
The data and our results on it are what keep us honest. The data is also what helps understanding nuances and building intuition because the AI can explain outcomes in the context of samples data points that you can reason about independently.
The difference between YOLO vibe-coding and engineering is understanding. I think this is the future of learning and skill-building.
Rapzid | 7 hours ago
God, I'd hate to be on the receiving end of their bug/issue submissions.
square_usual | 7 hours ago
syngrog66 | 8 hours ago
ActionHank | 8 hours ago
syngrog66 | 8 hours ago
ActionHank | 7 hours ago
I'm with you here, just asking what the proposed stack is here, which you did not include.
If I understand correctly you're saying custom agent and possibly locally run open model, or are you saying no to all LLM / agent use outright? Either is fine, I'm not here to tell you're wrong, I'm just curious about what others are doing.
tilemarch | 8 hours ago
The workflow for a defender basically goes: Claude do a security audit on this codebase, use the harness / skills I built on how to do security audits.. pick the best suitable audit strategy, methods.. use the tooling and reporting structures.
Claude refuses, GPT refuses. Grok, goes further, but refuses to make a working PoC.. a requirement because AI models make stuff up and without a PoC there is no proof. GLM-5.3 / Kimi to the rescue and completes the job the "smarter" models refused.
So we as defenders are refused by the best models, on our own codebase
naveen99 | 8 hours ago
realusername | 7 hours ago
All projects need security and writing code means taking security decisions, you can't untangle both.
Rapzid | 7 hours ago
I've heard people getting jammed up with Claude refusing to update DNS..
The situation is pretty insane right now.
woud420 | 7 hours ago
zamadatix | 7 hours ago
joshstrange | 3 hours ago
The only things worse than the CVE getting dropped in our laps with little warning was having Claude run into its safeguards which slowed my progress in actually protecting our customers.
Last week Claude (and Codex) also refused to help write unit tests around existing code because it touched on authentication. I was able to ladder between them past everywhere they got stuck but today I went straight to GLM after hitting safeguards.
espeed | 3 hours ago