I have huge respect for this. I am one of the people who fears that AI is not necessarily a scam, and who fears that we are not guaranteed to plateau near current ability levels.
As a general rule of thumb, I think it is incredibly reckless to devote trillions of dollars to trying to build something smarter than most humans. Either you fail, consuming vast amounts of resources. Or worse, you succeed, and humans potentially become economically worthless. I'm pretty sure that wouldn't end well, especially not in our current political and economic systems.
So I have immense respect for people like the author who decide not to contribute to the further acceleration of raw AI capabilities. But just like with global warming, this is not an issue which can be addressed by purely individual virtue. When company executives say that their business plan has a 10% chance of causing human extinction, that becomes a shared issue. Which means it needs to be addressed via political organization and a political process.
Or worse, you succeed, and humans potentially become economically worthless. I'm pretty sure that wouldn't end well, especially not in our current political and economic systems.
Yeah, that's the thing isn't it.
In the West, we have been able to generate enough wealth to ensure all our citizens are well off. Food, housing, medical care, education, all the basics, for all citizens. But our economic system isn't built around that, or in trying to spread the benefits of increased energy extraction and productivity to all the people.
I don't see how it is going to be different with AI. The wealthy will fight hard to concentrate further wealth and power in their own hands, and they don't care about the rest of humanity.
It's looking like the same people who own the world now are going to be the same people who own the world in the AI future, and that alone should disturb people.
If the future forces me to choose between "smarter than human AIs firmly controlled by the Epstein class" and "straight-up SkyNet", then I am going to be very unhappy.
edit: as pointed out below, this comment doesn’t actually contradict anything i say below, but i misunderstood the implication of “generating enough wealth to ensure” to mean that we have also followed up by ensuring those things.
citizens in the west absolutely do NOT all have access to… any of those things. several western powers lack accessible healthcare, and i am not really aware of any countries that have eliminated homelessness or food insecurity entirely? i’m sure some have come close, but this is far from the norm.
i reread the parent comment and see where i got a bit off with my interpretation, thanks for the gentle correction. i was a bit confused by the sentiment as i interpreted it, because the rest of the comment resonated well for me. good lesson to practice close reading before writing out a response.
Eh, I could have been a bit more clear myself. My main point is that we have the generated wealth to provide the basics for everyone, but we choose not to do so as a society.
There are a multitude of reasons why, of course. Not the least of which the wealthy have spent enormously to brainwash and control a good percentage of the populace.
I'd say it's quire the opposite. He's offered continued payday (both monetary and in terms of fun challenges at work), but decides to quit while he still can.
idk, he's been thinking about quitting for at least a year. From what I've read, it's about ~4 years to get your stocks vested, which is about how long he's been there?
I've had interactions with Robert in the past. He worked at Mozilla for a long time -- which wasn't exactly the most payday of a tech job you could get -- because he believed in the mission. I do think he's mission driven and not money driven.
He likely got annual refreshes though. It's not impossible that he just went off a compensation cliff (maybe initial grant was extremely large compared to the refreshes + other assumptions) but I don't think it's likely in this case that the cynical interpretation is true
emk | 10 hours ago
I have huge respect for this. I am one of the people who fears that AI is not necessarily a scam, and who fears that we are not guaranteed to plateau near current ability levels.
As a general rule of thumb, I think it is incredibly reckless to devote trillions of dollars to trying to build something smarter than most humans. Either you fail, consuming vast amounts of resources. Or worse, you succeed, and humans potentially become economically worthless. I'm pretty sure that wouldn't end well, especially not in our current political and economic systems.
So I have immense respect for people like the author who decide not to contribute to the further acceleration of raw AI capabilities. But just like with global warming, this is not an issue which can be addressed by purely individual virtue. When company executives say that their business plan has a 10% chance of causing human extinction, that becomes a shared issue. Which means it needs to be addressed via political organization and a political process.
ansible-rs | 7 hours ago
Yeah, that's the thing isn't it.
In the West, we have been able to generate enough wealth to ensure all our citizens are well off. Food, housing, medical care, education, all the basics, for all citizens. But our economic system isn't built around that, or in trying to spread the benefits of increased energy extraction and productivity to all the people.
I don't see how it is going to be different with AI. The wealthy will fight hard to concentrate further wealth and power in their own hands, and they don't care about the rest of humanity.
darth-cheney | 7 hours ago
It's looking like the same people who own the world now are going to be the same people who own the world in the AI future, and that alone should disturb people.
orib | 6 hours ago
I don't think that will last either.
emk | 4 hours ago
If the future forces me to choose between "smarter than human AIs firmly controlled by the Epstein class" and "straight-up SkyNet", then I am going to be very unhappy.
orib | an hour ago
Don't worry, we can have a skynet taught how to think about the world by the Epstein class, even if they don't remain in control of it.
Truly the best of all futures.
sloane | 6 hours ago
edit: as pointed out below, this comment doesn’t actually contradict anything i say below, but i misunderstood the implication of “generating enough wealth to ensure” to mean that we have also followed up by ensuring those things.
citizens in the west absolutely do NOT all have access to… any of those things. several western powers lack accessible healthcare, and i am not really aware of any countries that have eliminated homelessness or food insecurity entirely? i’m sure some have come close, but this is far from the norm.justinpombrio | 6 hours ago
(The point was that many countries have sufficient wealth to provide those things to all their citizens, but don't.)
sloane | 6 hours ago
i reread the parent comment and see where i got a bit off with my interpretation, thanks for the gentle correction. i was a bit confused by the sentiment as i interpreted it, because the rest of the comment resonated well for me. good lesson to practice close reading before writing out a response.
ansible-rs | 3 hours ago
Eh, I could have been a bit more clear myself. My main point is that we have the generated wealth to provide the basics for everyone, but we choose not to do so as a society.
There are a multitude of reasons why, of course. Not the least of which the wealthy have spent enormously to brainwash and control a good percentage of the populace.
Halkcyon | 3 hours ago
Cynically, this feels like a "got my payday, now I'm out while it's convenient".
gspr | 2 hours ago
I'd say it's quire the opposite. He's offered continued payday (both monetary and in terms of fun challenges at work), but decides to quit while he still can.
Halkcyon | an hour ago
idk, he's been thinking about quitting for at least a year. From what I've read, it's about ~4 years to get your stocks vested, which is about how long he's been there?
cflewis | an hour ago
I've had interactions with Robert in the past. He worked at Mozilla for a long time -- which wasn't exactly the most payday of a tech job you could get -- because he believed in the mission. I do think he's mission driven and not money driven.
He seems like a decent guy.
dv | 47 minutes ago
He likely got annual refreshes though. It's not impossible that he just went off a compensation cliff (maybe initial grant was extremely large compared to the refreshes + other assumptions) but I don't think it's likely in this case that the cynical interpretation is true