All of it completely reasonable suggestions taken at face value, though I expect a lot of people (pro or anti AI) to assume malicious motives (market capture, power grabs, hurting competition, hurting free models, etc.).
I wish we could stop arguing about whether or not these things are marketing stunts and talk more about what AI regulations actually should be. If the risk is overstated then nothing happens. If it's not, then I'd rather we have learned something from the Cold War.
It's hard for me to take "AI regulation" seriously when all the hyperscalers are violating regulations across domains and jurisdictions and absolutely getting away with it.
So the US or the EU passes a law restricting how AI can be developed. What then? They'll suddenly grow the spine needed to enforce it?
The regulations should actually be "a machine must never make a decision". If the only story that sells that to the public is the sci-fi runaway superintelligence stuff, ugh but so be it. If the reason the slop kings are in favor is to dodge responsibility and screw with the competition and do marketing or whatnot, so be it. We should be clear though: we only want humans to make important decisions and judgments, no automation of any kind, does not matter "how intelligent" the machine could be.
Let's say he genuinely believes the importance and urgency of what he wrote at face value. Is there a signal he could possibly send of these beliefs that wouldn't be assumed to have such malicious motives? Is it possible for someone at the forefront to communicate this without being disbelieved?
Perhaps the answer is that he's the wrong guy to put forward this argument, no matter how qualified he is, and regardless of whether what he's saying is the right thing to do, or of how genuine he is. He's the CEO! Anything he says reflects upon the company and has to be interpreted in that light.
Even if you're the best attorney who's ever lived, you're probably still the wrong person to represent your own child in court.
Suggest somebody else? Many people have put forth this argument, usually the answer if they're not building AI is "what do they know, they're not building AI".
I genuinely don't believe a single word. At this point, I believe he would argue that his mother is an octapus, if that would help the stock somehow. That's based on past experience, not becuase I hold a grudge against Altman or Amodei (or anyone else for the matter).
However that's irrelevant. IMO regulation should be discussed regardless. But a very good starting point is force these companies to respect the existing legal framework for starters... LLMs that escape sandboxes are grounds for building seizure in other business with this amount of blast radius.
We’ve made clear progress in alignment — training models so that they remain safe, ethical, compliant with our guidelines, and genuinely helpful (the principles that are embedded in Claude’s Constitution).
I know this is off-topic, but — the jokes genuinely write themselves.
There are only two options: there is a documented case and it is not very effective. This is what is happening right now with the big AI studios using their own AI to develop the next iteration of their AI. Or there is a case and it is very effective, in which case we all die. So "documented cases of dangerous RSI" cannot exist before the end of the world.
Sure, but is there any evidence that it is even possible? I could fearmonger about how I COULD develop the ability to cast wizard spells and throw the world into turmoil, and there's only two options: either I will develop wizard spells or I won't. If I constantly talked about how dangerous the idea of wizard spells was, you might think I was a little bit silly...
Well sure, different things are different. The evidence that it's possible is that they're trying to do it, they keep doing small parts of it, and if the models keep getting better at the current rate they'll probably be able to do it pretty soon. Software and AI development are cognitive skills and it seems like you can get models to be pretty good at cognitive skills, maybe without upper bound or maybe with an upper bound far above human, or maybe not but we don't know. Right now it all points towards at least superhuman.
If you fearmonger about how you could develop the ability to cast highly dangerous wizard spells, and also you just worked out how to cast Magic Missile and Shocking Grasp, then it seems at least worth entertaining the notion that Fireball and Greater Wish are also in the cards.
I flagged spam as this post is essentially a bunch of a science fiction from the CEO of Anthropic and therefore meets the criteria of "content designed to promote a commercial service"
mond | 10 hours ago
All of it completely reasonable suggestions taken at face value, though I expect a lot of people (pro or anti AI) to assume malicious motives (market capture, power grabs, hurting competition, hurting free models, etc.).
Qyriad | 9 hours ago
I wish we could stop arguing about whether or not these things are marketing stunts and talk more about what AI regulations actually should be. If the risk is overstated then nothing happens. If it's not, then I'd rather we have learned something from the Cold War.
mtset | 7 hours ago
It's hard for me to take "AI regulation" seriously when all the hyperscalers are violating regulations across domains and jurisdictions and absolutely getting away with it.
So the US or the EU passes a law restricting how AI can be developed. What then? They'll suddenly grow the spine needed to enforce it?
valpackett | 6 hours ago
The regulations should actually be "a machine must never make a decision". If the only story that sells that to the public is the sci-fi runaway superintelligence stuff, ugh but so be it. If the reason the slop kings are in favor is to dodge responsibility and screw with the competition and do marketing or whatnot, so be it. We should be clear though: we only want humans to make important decisions and judgments, no automation of any kind, does not matter "how intelligent" the machine could be.
jtolio | 9 hours ago
Let's say he genuinely believes the importance and urgency of what he wrote at face value. Is there a signal he could possibly send of these beliefs that wouldn't be assumed to have such malicious motives? Is it possible for someone at the forefront to communicate this without being disbelieved?
bitshift | 9 hours ago
Perhaps the answer is that he's the wrong guy to put forward this argument, no matter how qualified he is, and regardless of whether what he's saying is the right thing to do, or of how genuine he is. He's the CEO! Anything he says reflects upon the company and has to be interpreted in that light.
Even if you're the best attorney who's ever lived, you're probably still the wrong person to represent your own child in court.
FeepingCreature | 2 hours ago
Suggest somebody else? Many people have put forth this argument, usually the answer if they're not building AI is "what do they know, they're not building AI".
atmosx | 8 hours ago
I genuinely don't believe a single word. At this point, I believe he would argue that his mother is an octapus, if that would help the stock somehow. That's based on past experience, not becuase I hold a grudge against Altman or Amodei (or anyone else for the matter).
However that's irrelevant. IMO regulation should be discussed regardless. But a very good starting point is force these companies to respect the existing legal framework for starters... LLMs that escape sandboxes are grounds for building seizure in other business with this amount of blast radius.
Relax | 9 hours ago
If there were some signal in their actions that lined up with this rhetoric.
bitshift | 9 hours ago
I know this is off-topic, but — the jokes genuinely write themselves.
lilac | 5 hours ago
there is a reasonable argument to be made here but god this post is so transparent and insufferable
kaig | 4 hours ago
Does recursive self improvement exist? Is there any documented case of it happening?
Unrelated: what's with the "spam" tags on this article? How does this qualify as spam?
FeepingCreature | 2 hours ago
There are only two options: there is a documented case and it is not very effective. This is what is happening right now with the big AI studios using their own AI to develop the next iteration of their AI. Or there is a case and it is very effective, in which case we all die. So "documented cases of dangerous RSI" cannot exist before the end of the world.
kaig | 2 hours ago
Sure, but is there any evidence that it is even possible? I could fearmonger about how I COULD develop the ability to cast wizard spells and throw the world into turmoil, and there's only two options: either I will develop wizard spells or I won't. If I constantly talked about how dangerous the idea of wizard spells was, you might think I was a little bit silly...
FeepingCreature | an hour ago
Well sure, different things are different. The evidence that it's possible is that they're trying to do it, they keep doing small parts of it, and if the models keep getting better at the current rate they'll probably be able to do it pretty soon. Software and AI development are cognitive skills and it seems like you can get models to be pretty good at cognitive skills, maybe without upper bound or maybe with an upper bound far above human, or maybe not but we don't know. Right now it all points towards at least superhuman.
If you fearmonger about how you could develop the ability to cast highly dangerous wizard spells, and also you just worked out how to cast Magic Missile and Shocking Grasp, then it seems at least worth entertaining the notion that Fireball and Greater Wish are also in the cards.
hailey | an hour ago
I flagged spam as this post is essentially a bunch of a science fiction from the CEO of Anthropic and therefore meets the criteria of "content designed to promote a commercial service"