I enjoy the engineers trying to avoid responsibility be saying "the LLM made a mistake and caused the outage". My people your job is to test everything you write. I don't care if you had an LLM help you or not if you deploy code and production goes down it is your fault.
Well it's tricky because upper management is also pressuring people to use these tools as much as possible at many of these companies. "Don't get left behind!" They want to tell their investors that they are ahead of the AI curb. You can see various companies bragging about it.
But the reality is that writing code is not the hard part of engineering, reviewing it is. And we have known since Ka-Ping Yee's dissertation on voting machines that even the smartest engineers struggle to find vulnerabilities and bugs in code upon review, including maliciously inserted ones, even in incredibly simplified examples.
And so, we are pressuring to turn peoples' jobs into the thing they are worst at, with the highest fatigue: reviewing code they didn't write that looks incredibly plausibly correct.
So I would have a hard time blaming the engineers for the situation. It's coming from top-down, and we're going to see consequences from it all over our industry, I think.
It's a shame the real solution--eliminating layers of useless management--is rarely, if ever, on the table. I see this in colleges, too, where the administrators are plentiful and overpaid while getting in the way of the actual work of educating. Good engineers are tempted to move up into high-paying managerial roles just like good teachers are tempted to move up into high-paying administrator roles. We'd be better off if the people doing the fundamental work were paid the most and encouraged to grow in the roles they belong in.
So they had a few production incidents with similar root causes and they had a discussion about it at a regularly scheduled all hands meeting. This seems like normal good engineering practice, not news.
This “mandatory meeting” is just the usual weekly company-wide meeting where recent operational issues are discussed. There was a big operational issue last week, so of course this week will have more attendance and discussion.
This meeting happens literally every week, and has for years. Feels like the media is making a mountain out of a mole hill here.
Note that the article doesn’t say that he told staff they have to attend the meeting. It says he “asked” staff to attend the meeting. Which again, it’s really really normal for there to be an encouragement of “hey, since we just had an operational event, it would be good to prioritize attending this meeting where we discuss how to avoid operational events”.
As for the second quote: senior engineers have always been required to sign off on changes from junior engineers. There’s nothing new there. And there is nothing specific to AI that was announced.
This entire meeting and message is basically just saying “hey we’ve been getting a little sloppy at following our operational best practices, this is a reminder to be less sloppy”. It’s a massive nothingburger.
singpolyma | 3 hours ago
I enjoy the engineers trying to avoid responsibility be saying "the LLM made a mistake and caused the outage". My people your job is to test everything you write. I don't care if you had an LLM help you or not if you deploy code and production goes down it is your fault.
[OP] dustyweb | 3 hours ago
Well it's tricky because upper management is also pressuring people to use these tools as much as possible at many of these companies. "Don't get left behind!" They want to tell their investors that they are ahead of the AI curb. You can see various companies bragging about it.
But the reality is that writing code is not the hard part of engineering, reviewing it is. And we have known since Ka-Ping Yee's dissertation on voting machines that even the smartest engineers struggle to find vulnerabilities and bugs in code upon review, including maliciously inserted ones, even in incredibly simplified examples.
And so, we are pressuring to turn peoples' jobs into the thing they are worst at, with the highest fatigue: reviewing code they didn't write that looks incredibly plausibly correct.
So I would have a hard time blaming the engineers for the situation. It's coming from top-down, and we're going to see consequences from it all over our industry, I think.
gnafuthegreat | an hour ago
It's a shame the real solution--eliminating layers of useless management--is rarely, if ever, on the table. I see this in colleges, too, where the administrators are plentiful and overpaid while getting in the way of the actual work of educating. Good engineers are tempted to move up into high-paying managerial roles just like good teachers are tempted to move up into high-paying administrator roles. We'd be better off if the people doing the fundamental work were paid the most and encouraged to grow in the roles they belong in.
hoistbypetard | 5 hours ago
I can't see the article... it just takes me to a "Subscribe" modal instead.
[OP] dustyweb | 2 hours ago
I've updated to a new link at @isagalaev's suggestion! Try now.
hoistbypetard | 2 hours ago
That's better than the post I eventually found.
synack | an hour ago
So they had a few production incidents with similar root causes and they had a discussion about it at a regularly scheduled all hands meeting. This seems like normal good engineering practice, not news.
gerikson | 5 hours ago
The Financial Times has has a pay/regwall, flagging this as "broken link".
[OP] dustyweb | 4 hours ago
Hm, ok I will submit a different article
[OP] dustyweb | 4 hours ago
Unfortunately the only other article I can find is https://www.tomshardware.com/tech-industry/artificial-intelligence/amazon-calls-engineers-to-address-issues-caused-by-use-of-ai-tools-report-claims-company-says-recent-incidents-had-high-blast-radius-and-were-allegedly-related-to-gen-ai-assisted-changes
but that URL is over 250 chars long so lobste.rs won't let me post it.
Too bad. Seems important.
isagalaev | 3 hours ago
Here's a reprint from ArsTechnica: https://arstechnica.com/ai/2026/03/after-outages-amazon-to-make-senior-engineers-sign-off-on-ai-assisted-changes/
[OP] dustyweb | 2 hours ago
I've updated the post to use that link! Thank you!
typesanitizer | 25 minutes ago
Couple of comments from HN make it sound like this was a smaller deal internally compared to the framing in the article:
https://news.ycombinator.com/item?id=47324904
Later down the thread: (https://news.ycombinator.com/item?id=47325772)