Alan Kay said "A change in perspective is worth 80 IQ points,".
A change in perspective is often brought about by communication or I/O. In my experience this often bears out. The people who communicate get more done than the clever person who doesn't get out into the problem domain. The limit is I/O not intelligence.
In my work, I have always found that the main "blocker" (I hesitate to use the term "bottleneck"), is human nature.
If we design stuff to be used by humans (what I generally do), then it's imperative to take basic human nature into account, and that can be incredibly annoying.
Annoying, or not, human nature is a major factor, and won't go away, no matter how many times we tap our heels together. I think it was Philip K. Dick that said "Reality is that which, when you stop believing in it, doesn’t go away."
Designing stuff for processes and machines to consume, is pretty straightforward. Designing stuff for humans, however, is really challenging (which is one of the reasons that I like doing it).
I would go further and say that the human should be the bottleneck/blocker. If not designed for and to serve humans, which means to have human in the position of power and control and take in account all the various human oddities, why are we building all of this?
That depends on the bottleneck. When the bottleneck to me getting something I want is other humans won't allow me...
Of course this goes both ways. Some things I could name here everyone would agree I shouldn't be allowed to do. Other things are things that it wouldn't be hard to convince everyone here (but the opposition would remind you that I'm ignoring their points)...
To remove humans from power in all cases where our biases and animalistic nature would cause unnecessary suffering to others. As a species we are quickly approaching some very hard problems that no individual, or even small group, is capable of handling in a consistently rational manner. On our current trajectory of resource constraints and habitability constrictions, ethics by the numbers is inevitable.
We are either going to leave it to the powerful to sacrifice those they consider undesirable, as we have throughout history, or replace them with something not afflicted by the flawed elements of human nature to make the most compassionate decision possible amidst the horror.
How do you make sure the thing you are hoping saves everyone has an inclination (or ability) to do so? How do you make sure it is only afflicted with the human parts you like and not the ones you don't?
Nothing can save everyone, that's the point. There is no miracle out there that suddenly makes the world a fairy tale. And I don't have an answer, otherwise I would have made a claim to that effect.
This is a hard problem, and the human mind is not naturally equipped to work at the scales in question. Perhaps the solution is a thing (e.g a program), a process like democracy of some ilk, or something else entirely. The important part is recognizing our limits and working to account for them in our solutions. Acting without that awareness is a recipe for disaster.
One thing you have to be careful of, mostly when working on internal tools, is that you don't end up building around the idiosyncrasies of one particular person. This is something that actually happens in some places.
I actually find "bottleneck" a nicer word than "blocker" here (although not sure if "nicer" is what you are going for). I really like building for humans and the only times I've ever truly felt excited about being a developer is when I've had direct contact with my customers. It can be very trying, but the payoff is always nice.
Anything digital basically has the chaos fuzzed out by its binary nature.
The analog gets all the real entropy.
Humans are cool because our distribution is exploring the future curve of what is possible in a hostile universe and to do that successfully over time we have to have the humility to always have someone unsure of even our most basic facts.
Sucks for most of those edge distribution humans but good for the species.
But this is why our criteria of reasonable is aligned with the center of distribution and why we have colloquial statements like "if you design for everyone you design for no one".
I think the next bottleneck is embodiment. AI can lay the best laid plans, but who will execute it?
I think a possible solution to the dual problems of AI embodiment and unemployment is humans becoming avatars for AI.
Imagine one of the unskilled masses donning a headset and being guided through all manner of work?
Robots being able to perform any task a human can do, physically, doesn't seem that far off. They would be powered by AI of course.
The question is: do we still have to work? If not, who will take all the wealth?
We thought the washing machine would have giving us so much free time, because it used to take a full day a week. Instead, we only got more frantic it seems and started putting in more labor. But what will happen when that labor is worthless?
In a dystopian world, the elite may be better off just killing the lower, middle and even the higher classes. Keep just enough people to have an interesting gene pool around. Because even in the space mines there will be no need for human labor anymore.
Appliances like the washing machine literally killed the market for "hired help" in the anglosphere. In-house servants used to be relatively common among the middle and upper classes; with the aid of electrical appliances, the time/money costs of doing the laundry, dishes, and so forth went down so far that only the very rich care to hire servants as a mark of status.
It‘s actually stunning how inhumane this suggestion is. Especially, in the way it‘s made in the purely functional sense of human value as means of productivity and efficiency.
> Don’t mistake my ill portent for enthusiasm. I merely foretell.
Your choice of the word "solution" was the upsetting part. It implies a problem, and given your solution is "robot bosses tell human servants what to do using headsets," the implied problem is "how can we create a system where robot bosses can direct human servants?"
A human with experience can do without AI the job a lot faster than someone without can do it with. The expert HVAC tech can get the wrench of the fitting and have the job done faster than the novice. The novice will take longer to follow directions to get the right wrench; then they have to figure out how to get it on - something the expert does automatically isn't easy; then turn it the right direction... The expert also knows the correct "feel" so they won't over tighten, while still having less leaks to fix (both will check, but the expert will find less)
If I'm in my remote cabin I'm still faster than calling a tech, but in my city I can hire someone to do this who is faster. (though perhaps not cheaper)
In my city I call a handyman to fix my dryer and guess what? He uses chatGPT to help him through the issue. $350
Beyond a basic amount of manual dexterity and familiarity with mechanical things, AI can absolutely help guide an untrained person through the job.
How about medical research? Endless hours of pipetting. Does it really take a phd? It takes someone with conscientiousness and care. Both things an AI overlord would help with.
Indeed - if access to data or the permission to act on that data is limited by human gatekeepers (individual or bureaucratic), intelligence is often not the primary bottleneck.
OP points out that this is usually the case in biology/medicine where regulatory bottlenecks dominate.
> Some have been waiting for a year for the NIH to release imaging datasets they can use to produce better biomarkers. If one needs to interact with the FDA to get their endpoint validated, it is even worse: I have previously written about how the validation of Bone Mineral Density (BMD) for use as a surrogate endpoint in osteoporosis trials took 12 years (!), despite the fact that the data to support it already existed in full and the analyses done were basically regressions.
> no matter how “intelligent” AI becomes, intelligence is often not the main bottleneck to things changing in the real world
A mathematician Terrence Tao said something similar in a recent talk [1]. That, intelligence is bottlenecked by human understanding. Say there is an alien that brings an alien book containing math humans havent seen before. Using said math is bottlenecked by humans comprehending it first.
You sort of side stepped the author's point, though.
> In medicine you have the modern labs which automate a lot and you have the old labs. These 'neolabs' are better because of intelligence.
The author is not denying that a lab can be more effective with intelligence, they're arguing that this entire phase of the process of improving health is not the bottleneck.
And that, furthermore, the new labs and the old labs still both remain incentivized by the regulatory environment to chase the wrong targets. The new lab might just iterate towards the wrong targets faster.
I think the guy is right on point at least when it comes to therapies. But he is also missing one major angle: we are no longer in control; our systems are. If we were to vote on the issue, I suspect that a narrow majority would prefer cheaper housing, and a total majority would prefer cancer to be cured. But there's no direct mechanism of any kind to exercise that vote. Not in USA, not in Europe, not in China. When/where there's democracy, nobody runs in a campaign of advancing immune therapies[^1]. The closest proxy we have to a vote is "regulated markets", and my recent experience losing a loved one to cancer taught me that cancer is not a disease for the streams of money moving through that system. Far from it.
I'm frankly pessimist that we will be able to fix those systems in a humanist way, at least not in my lifetime. If we were any good at it, we wouldn't have extreme inequality, nor our educations systems would be in tatters, and we wouldn't cast ballots propping up politicians whose best skill is ragebait.
However, if fixing the system is not an option, access to cheap technology and intelligence could help a lucky few circumvent those systems. And guess what the lucky few at the head of the AI companies will say then... Not necessarily the worst outcome, since most people will find more viable to equip a private biolab in the basement of their countryside manor than fixing the medical system of first world economies.
[^1] For the record, I think that would be better than running on xenophobic slogans.
I could build a house. With zero experience you can find a job in house building as they will pay you to learn on the job (depending on the economy you might or might not be able to find such a job today, in good years they will hire anyone who isn't disabled - but there are bad years). Many people who have never had a construction job build their own house (only a tiny %, but that is still a lot of people)
I suppose I could work on a cancer cure. However since I don't have a background in biology I would have to pay for several years of school to learn the required skills. Those classes are hard and there is no guarantee I will pass (I have an engineering degree, so my odds of passing should be above average).
Depends. In rural areas the only part you can't do the power company will in five minutes. In other places only a licensed electrician is allowed to touch wires and do you can do much less
Essence of Decision is a great book that explores this idea. It looks at different models for understanding how organizations make decisions, and arrives at broadly the same conclusion: a model that assumes people are particularly in control is inadequate for explaining real world events.
Two possible assumptions here are that systems are actually in control and that there isn't anyone looking, while we saw that if things get too heated the US govt can sweep in and shut things down. The reasons for it might be debatable -- I'm more inclined to believe that that actually played nicely into the hype train launched by Anthropic around Mythos -- but there is always a bigger dog watching.
The other assumption is that these systems are failing by accident when in fact a stronger argument can be made that they are like this by design.
"He looked at me incredulously. Surely, a smart person like me should know that AI, or better said, AGI will be hyperpersuasive soon – already on a bunch of benchmarks it exceeds professional debaters at persuasion." I'm always tickled by this tic of this kind of person, that human traits are something that we can maximize to infinity. That if we trained a robot to tell jokes, it would eventually tell jokes so funny we'd die laughing etc. You can't convince someone who doesn't want to be convinced.
This has been a core belief of the EA/LessWrong/Rationalist people for many years - that above a certain level of intelligence you effectively get hypnotic powers, and people will obey AIs no matter what. So they said you can't keep a superpowered AI airgapped as merely offering a chat interface will immediately take control of the next person to interact with it and they will deliberately compromise the airgapping.
And if AI wrote a book about these hypnotic powers, then it would be incredibly convoluted, long, and yet worn as a badge of pride to have survived it.
Thinking about it, much of the output I'm seeing nowadays looks like a chapter out of infinite jest.
A complete mischaracterization, as usual for HN lately when discussing AI or LessWrong. Obviously, even average levels of persuasion are enough to convince some people. And nobody is air-gapping AI.
It's kind of fascinating how toxic the EA/LessWrong/Rationalist stuff has become. I remember reading it back in the early days (~2010 or so) and finding a lot of good ideas mixed in with some dorkiness (Harry Potter fan fiction about rationalism..). It always seemed like a sort of harmless niche with some good ideas and some wacky ones. And yet now it's basically become a cultish religion for the ultrapowerful who use it to justify doing some really awful things.
> You can't convince someone who doesn't want to be convinced.
I think this is true. But it doesn’t get rid of the possibility of hyperpersuasive bots, just, points to a reason they might not be as consequential as expected. I think bots that are very good at writing rhetoric might happen (why not? They are already good at math and chess), and they would be devastating to the spaces where people go online and argue with anonymous strangers. But I don’t know if that’ll matter. Could be bad for bloggers. Complex for policy people.
IMO if we’re really honest with ourselves, I think we get more of our opinions by stewing in memes and other social processes. Offline or online. The sort of hyperpersuasive ML that we really should have been worrying about was the Twitter/reddit/Tick-Tock engagement algorithms.
yes of course LLMs are already wreaking havoc by scaling up the bot farms of yore but that's not really how I interpreted hyperpersuasion, I interpreted it as a specifically targeted one-on-one conversation with belief in a specific notion as the desired outcome. I believe that LLMs can logos and pathos-max but (depending on the interlocutor) they cannot overcome by sheer force of logorrhea their lack of reputation, credibility, presence, etc.
Hyperpersuasion is not necessary. AI needn't convince the smartest among us, it need only convince the masses. Which, thanks to the widgets in our pockets, is a highly distributable task. The other risk profile is that of the hyperhacker: a sufficiently advanced AI can worm its way into untold internet-connected systems. Wars are being fought drone-to-drone already without a reduction in human deaths. Every drive-by-wire car and plane is a weapon waiting to be deployed.
> In biopharma specifically, one great bottleneck is clinical trials
It is a good news for having more intelligence. Phase 1 trial to acceptance rate is ~10%. If say in ideal world, intelligent AI could make it to 80% by rejecting drugs before Phase 1, cost of trial(which author claims is the bottleneck) would suddenly drop to 1/8th even if AI doesn't help at all in the trial.
> new drugs approved per dollar of R&D has fallen for decades, even as our scientific tools have grown vastly more powerful, the exact opposite of what the existence of more raw capability would predict
Can we please stop using X per dollar as a metric over long periods of time?
Inflation causes exponential price increases, so almost everything would be expected to have its "per dollar" yield decrease over time...
Well, let me just throw in it is our communication skills, or lack of them, that is our main bottleneck. That goes for pretty much everything humans do, and everything we fail trying to do: it is the quality of those engaged and their communications that is the critical aspect determining success or failure.
While this is right on some level... the incentives aren't the main driver. Finding good targets in biotech is just a harder problem to solve. Like way harder. Disease biology is complex as hell. At the other end, figuring the interplay between one target protein and a potential molecule to act as a therapy is much easier (it's still complex, and there is a little more to it than that, but not anything like finding a good target).
Indeed. Humans and not other apes reached the moon by our great proclivity to long distance running, through our exceptional sweat glands... ah, wait.
How is it possible to live in the world around us and not see, apparent and transparent, that intelligence is the lever on which everything rests? Humans are surprisingly weird animals, but it's not our hairlessness or our fairly average propensity to violence that did this. We don't have more houses than chimpanzees because we're _strong_. We haven't defeated the tolls of disease because we're atypically devoid of restraint. We don't have more effective governance than termites because we're more communal.
Of course intelligence is the main bottleneck. The arguments here are so confused. What do you think regulation is made of? What do you think money buys? What do you think determines how well you can gather data, or how efficiently you can consume it? But of course, as the genre, after spending the first third waxing about how its opposition could be so foolish, the last third is spent finding a pretty insult and wallowing in it, so as to not to leave enough space for the brief argument in the middle to consider such things as why someone might disagree.
The perfectly rational actor myth in economics will have Mary doing all sorts of things she doesn’t actually do. Mary just really likes peaches and the whole construct of rational actors is a fictional superstructure. Same with this idea of intelligence levering all our behavior.
Maybe Mary keeps intelligently finding new ways to find scarce peaches, but her ultimate benefit is eating the peach. It’s that human desire if anything.
How much more intelligence will it take to get people to be ok with data center construction? How much more intelligence will be needed to reduce overuse of antibiotics? How much more intelligence is necessary to enable free trade with Canada?
What property, if not intelligence, do you think makes people capable of rallying political support? Of working quickly though permit processes? Of providing accurate drug dosing guidance? Of presenting as a trustworthy personal healthcare guide? Of negotiating political deals? Of making effective public communications? Of juggling independent parties' interests? Of coming out ahead in contracts? Of producing alternative antibiotics? Of detecting and handling drug resistance in the wild? What answer could your questions possibly have other than this?
It's very clear that we have plenty of intelligence but we are not acting in it. What intelligence are we missing that is needed to solve climate change? (our most pressing problem - which AI is exacerbating)
> We don't have more houses than chimpanzees because we're _strong_.
You're literally making the point that we already have intelligence. I'm not sure how this is supposed to be an argument for a shortage. To revise: The point you are arguing against is not that intelligence is not important but that it's not the bottleneck.
Seems intelligence is actually a bit antithetical to the idea of a stable society. Look at the rate of change of our species. Within our own lifetimes there is so much change that it eventually exceeds our abilities to property mental model this change by the time we are elderly.
Meanwhile, consider the termite. Their society has been stable for millions of years and they've colonized the earth. Intelligence is not needed for that.
As far as I know humans with the same sort of brains as your brain and my brain have existed for 100,000's of years, perhaps as long as 2 million years.
They did not go to the moon. They did not make mechanisms, they did not make pottery, they did not smelt iron, they did not press shapes onto clay and leave it in houses that burned down, they did not have houses.
All of that has come in the last 12,000 years. I think that there are people in the Amazon today who could not write or read a word, could not drive a car or solve an equation or argue a case or compose an algorithm who are as intelligent as me, or you, or more. I think that people like that have existed for more than a million years.
So if they were so smart, why didn't they go to the moon?
I frankly think this writer is missing an even larger point. These are just bottlenecks to making progress in biotech, but rapid innovation in biotech or medical technology is not a goal in and of itself, except for stakeholders in biotech companies. The goal is improved human health, and all the medical technology in the world does not seem to be noticeably improving health outcomes. It certainly has over longer timespans if you compare outcomes of today to pre-antibiotics or pre-indoor plumbing, but improvement has slowed or even regressed in the 21st century.
At least I think so. My health is fine, but as an uninformed non-expert going by popular sentiment, it sures seems like people more generally are fatter than ever, mental health is in the tank, suicide rates are up, 20 year-olds are getting cancer, everyone is autistic and has no attention span, we're all full of plastics and forever chemicals, nobody has sex hormones any more. I'm sure a lot of that is hyperbole and alarmism, but it sure doesn't feel like we're living in a world where medical technology is solving all of our health problems.
Climate change is another big problem that frontier AI labs have handwaved away as being "solvable" by AGI. The premise that a pure superintelligence could hand us a plan to beat climate change that's so good and airtight that A) it would definitely work and B) people would execute it.
But there have been entire books written about the fact that right now we have the technology to drastically curtail climate change and yet they're under utilized because of a complicated and murky set of political, social, and systemic reasons. There are a huge numbers of problems for which very good solution plans have been detailed yet the gap is in execution.
The AGI-pilled must think one of two things; either there's an as-yet-undiscovered plan that's both world-changing yet trivial to implement or we'll all worship at the altar of the machine god with such fervor that even if it spits out a relatively obvious and iterative climate change plan the world will eagerly follow it. Both premises seem laughably unlikely to me.
I'm cynical and I believe that AI will be our undoing. In fact, when we let that genie out of the bottle, many other scenarios became possible. For example, here's one third scenario to your "two things": a sufficiently cynical[^1] and resourceful rogue AI may solve climate change by steamrolling over the "set of political, social, and systemic reasons" and teaching the monkeys a lesson about the real significance of very vast agency and intelligence. For example, it could manipulate people into an anti technological uprising and subsequent backward, anti-humanist ethos, while at the same time modifying one or two key ecosystems so that the Mediterranean would smell of cooked cabbage (dimethyl sulfide) and bring about a North African monsoon that, with sufficient time, would turn the Sahara into carbon-sequestering dense jungle...
I think the idea that "regulation is the bottleneck" is at least as pernicious as the idea that "intelligence is the bottle".
US health care is well known to be an ongoing disaster for US health care end-users however many new health care products US companies produce. The overall situation is that final good health outcomes currently and in the near future require humans involved and moreover, caring human attention involved - among other things because human health involves many, many "outliers" and many, many connections between social, biological, psychological and individual factors.
The US both an entrepreneurial "try things out" approach and a vast spider web of regulations. Why? Because if it didn't have that people would be dying at a faster rate, in droves. Occasionally, even with this, really bad stuff comes through (see opioid epidemic etc).
There currently isn't a way out of this since the entrepreneurial needs to cheapen humans and get humans out of the loop. Again, that produces mediocre factory health care for maybe cheaper if you're lucky and careful. And again, f-- kills people if no one looks (Theorno etc).
The think is, it's more plausible that some level of intelligence could get a handle on the multifaceted needs that are actually involved in actual human health care. The problem is that it seems likely that experiments using inadequate tools are likely to harm many people before that happens.
The argument being made appears to be that improving technical ability at the edges of a graph doesn't help when governance of inter-node processes is still a primary limiting constraint.
Ok. Yes. And it's a very strong position to take.
But that's not the same thing as intelligence not being the bottleneck, because governance is absolutely a manifestation of intelligence. It's just intelligence at a hub node instead of at a leaf node.
Good governance takes intelligence. Recognizing the importance of things takes intelligence. Making existing resources available quickly takes intelligence. And so on. Even when things are constrained by physical processes or financial resources, making things less constrained by those physical processes and financial resources takes intelligence. It's all intelligence. Where our processes are slow and dumb, they are slow and dumb because the people managing those processes were not smart and fast enough in the ways that mattered to make them better.
Tangentially, I find this sloppiness kind of funny in the context of intelligence and bottlenecks, because it's also a bottleneck for communication.
The subject should have been "Intelligence _here_ is not the main bottleneck."
True, but that doesn't mean intelligence is the bottleneck of good governance. Good governance also requires simple, plain brute force. You have to herd millions of cats, all of which have their own intelligence, and the richest of which are basically giant tigers. At our current level of individual intelligence, we have plenty of simple solutions to many of the problems that plague us, but the reality of the boiling ocean of society is that it tends to crush them faster than they can be built up.
When the problem is the coordination of intelligent agents with disparate interests, increasing everyone's intelligence isn't necessarily going to lead to a solution. Maybe if everyone operates in good faith with a compromising spirit. But if our best solution creates losers, they will fight to the bitter end, in which case we don't need intelligence as much as we need a bigger boot. That's governance.
The barrier to change is not just regulation. Regulation is downstream of public opinion. Compare China and the US. In China, public sentiment around AI seems to be a hunger to use it for fear of being left behind. Whereas in America, it's an apathetic pessimism around inequality and safety.
China has seen much faster growth in recent decades, which incentivizes people to work hard and have faith it will yield results. In America, the average standard of living has not improved for a long time, so instead of a desire to change things, like cutting regulation to speed up drug discovery, most Americans are quiet quitting.
In this climate, the unfortunate reality is that we're unlikely to see a popular revolution to drastically change things one way or another. Instead, we're much more likely to see regulatory capture by a small number of wealthy, powerful, motivated people. And everyone else will complain on social media, but that's about it.
> Figure 2. ... In the last decade, the deceleration in pharmaceutical productivity seems to have ameliorated ...
Uh, what? It's a semi-log scale and the data has been statistically bouncing off 0 for a decade. It can't go below 0 but it's clearly flat-lined. Hint: on a log scale, you're never getting to 0 unless your function crosses the axis. But this function can't cross 0.*
* It could cross 0 if you start counting withdrawals of products from the market, but that would be a different function with different inputs.
I suggest people greatly underestimate how localized expertise is. How very rapidly someone's expertise deteriorates as you move away from some focus, from the usually-very-narrow region where that expertise is externally nurtured.
A snark characterization of that region being, the issues where someone badly fears getting something wrong, and by that being embarrassed in front of the people whose opinion they much care about. One can have what superficially look like intellectual communities (picture DC thinktanks), which largely lack this property.
You can have someone well respected in their "real" subfield of biology, while being basically nutter in another subfield, which they care and think about, but aren't professionally embedded.
Ask a quantum chemist a protein chemistry question, and don't be surprised by a grad student answer. Ask "what causes the seasons?", and perhaps high-school or middle-school. Why would it be otherwise?
Even within focus, a bunch of research community gossip is discount functions. "Don't believe his observations using <technique> - he sees what he wants to see". "She always looking for things to be <favorite mechanism>."
For a recent impactful example, someone can be a successful CEO, notable engineering manager, and unusually good at thinking from first principles... while simultaneously, on other topics, being so batshit-crazy nutter that closely-held important views are 5+ orders of magnitude disconnected from reality. Because on those topics, their external community is nutter. And instead of finding this juxtaposition unsurprising, commenters twist themselves up trying to level that curve.
It's the shitty textbook publisher fallacy of "We'll find ourselves an outstanding professor of chemistry, and he'll write us an outstanding chemistry textbook". That's just not how this works.
> monoculture
Which brings us to groupthink. Expertise is a community-associated property.
In foreign affairs for example, the investment community is good at noticing and correctly evaluating some aspects of reality but not others, and similarly for diplomatic community, development, business, political, etc. Each has a partial view with systemic flaws.
I suggest groupthink is a critically important aspect of most human behavior. And yet somehow, we rarely manage to discuss this, and are repeatedly surprised by it. I'm unclear on why.
For decades I thought it bizarre that the New York Times didn't even manage to say the word. Until Trump and MAGA. Yay, a breakthrough? No, at least not the last time I checked several years ago. It remained associated with only Trump and MAGA. Some things are easier to see across the battlefield, than in the mirror, or literally everywhere?
>I suggest groupthink is a critically important aspect of most human behavior. And yet somehow, we rarely manage to discuss this, and are repeatedly surprised by it. I'm unclear on why.
Isn't that just a manifestation of the 'Blind spot' from Johari Window?
In general it is really hard to think about the things you already internalized.
human trials are the bottleneck. Thought experiment: You invent a way to simulate a body variations 1:1 and can scale it, you cure most disease in a very short amount of time.
Minton Beddoes: What people who lose their jobs, who are displaced, who may go off and garden, but how are they going to live? And what kind of I mean, it sounds to me...
Musk: universal high income
Minton Beddoes: but how are you going to get there? Let's talk about the politics of this cuz right now we if we look at the political pressures in most countries people are very very worried about this very against it. I mean it seems to me that there's a risk that we are heading more for revolution than for an easy route to this nirvana you're playing out.
Musk: I mean it will be a bumpy road. I'm not suggesting it will be sort of that everything will be smooth sailing here. Um, so I mean there are many jobs that used to exist that don't exist because of computers. I mean the the word computer used to refer to a person doing computations like that used to be a job description of a person. And you can see pictures online of um of buildings that... drifts away
discreteevent | 6 hours ago
Joel_Mckay | 5 hours ago
https://en.wikipedia.org/wiki/Conway's_law
andai | 5 hours ago
xboxnolifes | 2 hours ago
ChrisMarshallNY | 6 hours ago
If we design stuff to be used by humans (what I generally do), then it's imperative to take basic human nature into account, and that can be incredibly annoying.
Annoying, or not, human nature is a major factor, and won't go away, no matter how many times we tap our heels together. I think it was Philip K. Dick that said "Reality is that which, when you stop believing in it, doesn’t go away."
Designing stuff for processes and machines to consume, is pretty straightforward. Designing stuff for humans, however, is really challenging (which is one of the reasons that I like doing it).
dgellow | 5 hours ago
curuinor | 5 hours ago
rbn3 | 5 hours ago
KPGv2 | 5 hours ago
bluGill | 5 hours ago
Of course this goes both ways. Some things I could name here everyone would agree I shouldn't be allowed to do. Other things are things that it wouldn't be hard to convince everyone here (but the opposition would remind you that I'm ignoring their points)...
HardlyCognizant | 4 hours ago
To remove humans from power in all cases where our biases and animalistic nature would cause unnecessary suffering to others. As a species we are quickly approaching some very hard problems that no individual, or even small group, is capable of handling in a consistently rational manner. On our current trajectory of resource constraints and habitability constrictions, ethics by the numbers is inevitable.
We are either going to leave it to the powerful to sacrifice those they consider undesirable, as we have throughout history, or replace them with something not afflicted by the flawed elements of human nature to make the most compassionate decision possible amidst the horror.
pluralmonad | 3 hours ago
HardlyCognizant | an hour ago
This is a hard problem, and the human mind is not naturally equipped to work at the scales in question. Perhaps the solution is a thing (e.g a program), a process like democracy of some ilk, or something else entirely. The important part is recognizing our limits and working to account for them in our solutions. Acting without that awareness is a recipe for disaster.
RandomLensman | 3 hours ago
asdff | 2 hours ago
dgellow | 2 hours ago
snarf21 | 5 hours ago
sodapopcan | 5 hours ago
One thing you have to be careful of, mostly when working on internal tools, is that you don't end up building around the idiosyncrasies of one particular person. This is something that actually happens in some places.
I actually find "bottleneck" a nicer word than "blocker" here (although not sure if "nicer" is what you are going for). I really like building for humans and the only times I've ever truly felt excited about being a developer is when I've had direct contact with my customers. It can be very trying, but the payoff is always nice.
ChrisMarshallNY | 4 hours ago
Also, statistically higher-than-normal portion of disabled folks (cognitively, as well as physically). I need to make sure my stuff is really usable.
I have busted my ass, implementing stuff that I think is marvelous, only to have it ignored, or even despised.
Quite humbling.
ball_of_lint | 5 hours ago
I mean to say: Yes it's certainly a factor, but we're also leaving fences in our own way.
fellowniusmonk | 5 hours ago
The analog gets all the real entropy.
Humans are cool because our distribution is exploring the future curve of what is possible in a hostile universe and to do that successfully over time we have to have the humility to always have someone unsure of even our most basic facts.
Sucks for most of those edge distribution humans but good for the species.
But this is why our criteria of reasonable is aligned with the center of distribution and why we have colloquial statements like "if you design for everyone you design for no one".
drivebyhooting | 5 hours ago
I think a possible solution to the dual problems of AI embodiment and unemployment is humans becoming avatars for AI. Imagine one of the unskilled masses donning a headset and being guided through all manner of work?
Roofing, ACHVAC, stocking shelves, etc…
No need to train anybody.
simiones | 5 hours ago
ta988 | 5 hours ago
utopiah | 5 hours ago
No need to imagine nor AI for this, it can be either JSON guides on AR, e.g. Daqri for more than a decade, or remote work in VR (6 years ago or more https://www.inceptivemind.com/remote-controlled-model-t-robo... )
robofanatic | 5 hours ago
AI?
watwut | 5 hours ago
Lutger | 5 hours ago
The question is: do we still have to work? If not, who will take all the wealth?
We thought the washing machine would have giving us so much free time, because it used to take a full day a week. Instead, we only got more frantic it seems and started putting in more labor. But what will happen when that labor is worthless?
In a dystopian world, the elite may be better off just killing the lower, middle and even the higher classes. Keep just enough people to have an interesting gene pool around. Because even in the space mines there will be no need for human labor anymore.
bluGill | 5 hours ago
drivebyhooting | 5 hours ago
bitwize | 2 hours ago
3ot | 5 hours ago
Maybe I‘m missing the joke though.
drivebyhooting | 5 hours ago
Don’t mistake my ill portent for enthusiasm. I merely foretell.
3ot | 2 hours ago
KPGv2 | 42 minutes ago
Your choice of the word "solution" was the upsetting part. It implies a problem, and given your solution is "robot bosses tell human servants what to do using headsets," the implied problem is "how can we create a system where robot bosses can direct human servants?"
bluGill | 5 hours ago
If I'm in my remote cabin I'm still faster than calling a tech, but in my city I can hire someone to do this who is faster. (though perhaps not cheaper)
drivebyhooting | 4 hours ago
Beyond a basic amount of manual dexterity and familiarity with mechanical things, AI can absolutely help guide an untrained person through the job.
How about medical research? Endless hours of pipetting. Does it really take a phd? It takes someone with conscientiousness and care. Both things an AI overlord would help with.
bluGill | 3 hours ago
Maybe the right decision there but you have others.
KPGv2 | 39 minutes ago
Well it sounds like you made a mistake calling for help when it's apparently that easy.
bitwize | 2 hours ago
Or the AI could threaten humans into compliance, as in the 2008 film Eagle Eye.
rubinlinux | 2 hours ago
In it, employees wore headsets run by the AI which instruct them task by task:
"Jane, when you are through with this customer, please close your register. Then we will clean the women's restroom."
Manna: "Place the 'wet floor' warning cone outside the door please."
When Jane completed the task, she would speak the word "OK" into her headset and Manna moved to the next step in the restroom cleaning procedure.
Manna: "Please block the door open with the door stop."
Jane: "OK."
Manna: "Please retrieve the bucket and mop from the supply closet."
Jane: "OK."
LunicLynx | 5 hours ago
computerphage | 5 hours ago
sdellis | 5 hours ago
aixpert | 2 hours ago
xxpor | 5 hours ago
LunicLynx | an hour ago
drdeca | 3 hours ago
yourMadness | 5 hours ago
And that has been true for centuries, not just since AI.
Allowing intelligence means ceding control over the outcome. And that is often a bigger issue than the actual issue.
csours | 5 hours ago
highfrequency | 5 hours ago
OP points out that this is usually the case in biology/medicine where regulatory bottlenecks dominate.
> Some have been waiting for a year for the NIH to release imaging datasets they can use to produce better biomarkers. If one needs to interact with the FDA to get their endpoint validated, it is even worse: I have previously written about how the validation of Bone Mineral Density (BMD) for use as a surrogate endpoint in osteoporosis trials took 12 years (!), despite the fact that the data to support it already existed in full and the analyses done were basically regressions.
bwfan123 | 5 hours ago
A mathematician Terrence Tao said something similar in a recent talk [1]. That, intelligence is bottlenecked by human understanding. Say there is an alien that brings an alien book containing math humans havent seen before. Using said math is bottlenecked by humans comprehending it first.
[1] https://teorth.github.io/tao-web/slides/age-of-ai-icm-2026.p...
Yopolo | 5 hours ago
An expert in physics still needs to write code in python to do their math and analysis. Its an intelligence issue. That person needs to learn a tool.
In medicin you have the modern labs which automate a lot and you have the old labs. These 'neolabs' are better because of intelligence.
A doctor doing a diagnosis can say 'no clue' or can give you a proper answer. The more intelligent this person is, the better the answer.
kenferry | 5 hours ago
> In medicine you have the modern labs which automate a lot and you have the old labs. These 'neolabs' are better because of intelligence.
The author is not denying that a lab can be more effective with intelligence, they're arguing that this entire phase of the process of improving health is not the bottleneck.
notpachet | 4 hours ago
dsign | 5 hours ago
I'm frankly pessimist that we will be able to fix those systems in a humanist way, at least not in my lifetime. If we were any good at it, we wouldn't have extreme inequality, nor our educations systems would be in tatters, and we wouldn't cast ballots propping up politicians whose best skill is ragebait.
However, if fixing the system is not an option, access to cheap technology and intelligence could help a lucky few circumvent those systems. And guess what the lucky few at the head of the AI companies will say then... Not necessarily the worst outcome, since most people will find more viable to equip a private biolab in the basement of their countryside manor than fixing the medical system of first world economies.
[^1] For the record, I think that would be better than running on xenophobic slogans.
RandomLensman | 5 hours ago
bluGill | 5 hours ago
I suppose I could work on a cancer cure. However since I don't have a background in biology I would have to pay for several years of school to learn the required skills. Those classes are hard and there is no guarantee I will pass (I have an engineering degree, so my odds of passing should be above average).
RandomLensman | 4 hours ago
bluGill | 3 hours ago
bunderbunder | 5 hours ago
gchamonlive | 5 hours ago
The other assumption is that these systems are failing by accident when in fact a stronger argument can be made that they are like this by design.
notpachet | 4 hours ago
> he
Your gender biases are showing!
scuppernong | 5 hours ago
mister_mort | 5 hours ago
_puk | 5 hours ago
Thinking about it, much of the output I'm seeing nowadays looks like a chapter out of infinite jest.
Maybe that's unfair on infinite jest.
zone411 | 4 hours ago
giancarlostoro | 3 hours ago
https://genai.mil/
I mean... I would hope that AI used for military needs is not deployed in the public internet.
throwlifeaway | 31 minutes ago
https://rationalwiki.org/wiki/AI-box_experiment
https://www.yudkowsky.net/singularity/aibox
https://www.lesswrong.com/posts/Bnik7YrySRPoCTLFb
It's far from the craziest belief that's come out of that group.
meindnoch | 4 hours ago
overgard | 3 hours ago
asdff | 2 hours ago
This was a plot point in the Halo lore actually, with Mendicant Bias being reasoned with to defect against its own makers.
bee_rider | 2 hours ago
I think this is true. But it doesn’t get rid of the possibility of hyperpersuasive bots, just, points to a reason they might not be as consequential as expected. I think bots that are very good at writing rhetoric might happen (why not? They are already good at math and chess), and they would be devastating to the spaces where people go online and argue with anonymous strangers. But I don’t know if that’ll matter. Could be bad for bloggers. Complex for policy people.
IMO if we’re really honest with ourselves, I think we get more of our opinions by stewing in memes and other social processes. Offline or online. The sort of hyperpersuasive ML that we really should have been worrying about was the Twitter/reddit/Tick-Tock engagement algorithms.
scuppernong | an hour ago
boothby | 2 hours ago
YetAnotherNick | 5 hours ago
It is a good news for having more intelligence. Phase 1 trial to acceptance rate is ~10%. If say in ideal world, intelligent AI could make it to 80% by rejecting drugs before Phase 1, cost of trial(which author claims is the bottleneck) would suddenly drop to 1/8th even if AI doesn't help at all in the trial.
adverbly | 5 hours ago
Can we please stop using X per dollar as a metric over long periods of time?
Inflation causes exponential price increases, so almost everything would be expected to have its "per dollar" yield decrease over time...
thinkharderdev | 2 hours ago
bobsmooth | 2 hours ago
bsenftner | 4 hours ago
danielmarkbruce | 4 hours ago
Veedrac | 4 hours ago
How is it possible to live in the world around us and not see, apparent and transparent, that intelligence is the lever on which everything rests? Humans are surprisingly weird animals, but it's not our hairlessness or our fairly average propensity to violence that did this. We don't have more houses than chimpanzees because we're _strong_. We haven't defeated the tolls of disease because we're atypically devoid of restraint. We don't have more effective governance than termites because we're more communal.
Of course intelligence is the main bottleneck. The arguments here are so confused. What do you think regulation is made of? What do you think money buys? What do you think determines how well you can gather data, or how efficiently you can consume it? But of course, as the genre, after spending the first third waxing about how its opposition could be so foolish, the last third is spent finding a pretty insult and wallowing in it, so as to not to leave enough space for the brief argument in the middle to consider such things as why someone might disagree.
meroes | 4 hours ago
Maybe Mary keeps intelligently finding new ways to find scarce peaches, but her ultimate benefit is eating the peach. It’s that human desire if anything.
samth | 3 hours ago
Veedrac | an hour ago
What property, if not intelligence, do you think makes people capable of rallying political support? Of working quickly though permit processes? Of providing accurate drug dosing guidance? Of presenting as a trustworthy personal healthcare guide? Of negotiating political deals? Of making effective public communications? Of juggling independent parties' interests? Of coming out ahead in contracts? Of producing alternative antibiotics? Of detecting and handling drug resistance in the wild? What answer could your questions possibly have other than this?
discreteevent | 2 hours ago
> We don't have more houses than chimpanzees because we're _strong_.
You're literally making the point that we already have intelligence. I'm not sure how this is supposed to be an argument for a shortage. To revise: The point you are arguing against is not that intelligence is not important but that it's not the bottleneck.
asdff | 2 hours ago
Meanwhile, consider the termite. Their society has been stable for millions of years and they've colonized the earth. Intelligence is not needed for that.
sgt101 | 2 minutes ago
They did not go to the moon. They did not make mechanisms, they did not make pottery, they did not smelt iron, they did not press shapes onto clay and leave it in houses that burned down, they did not have houses.
All of that has come in the last 12,000 years. I think that there are people in the Amazon today who could not write or read a word, could not drive a car or solve an equation or argue a case or compose an algorithm who are as intelligent as me, or you, or more. I think that people like that have existed for more than a million years.
So if they were so smart, why didn't they go to the moon?
nonameiguess | 4 hours ago
At least I think so. My health is fine, but as an uninformed non-expert going by popular sentiment, it sures seems like people more generally are fatter than ever, mental health is in the tank, suicide rates are up, 20 year-olds are getting cancer, everyone is autistic and has no attention span, we're all full of plastics and forever chemicals, nobody has sex hormones any more. I'm sure a lot of that is hyperbole and alarmism, but it sure doesn't feel like we're living in a world where medical technology is solving all of our health problems.
MerrimanInd | 4 hours ago
But there have been entire books written about the fact that right now we have the technology to drastically curtail climate change and yet they're under utilized because of a complicated and murky set of political, social, and systemic reasons. There are a huge numbers of problems for which very good solution plans have been detailed yet the gap is in execution.
The AGI-pilled must think one of two things; either there's an as-yet-undiscovered plan that's both world-changing yet trivial to implement or we'll all worship at the altar of the machine god with such fervor that even if it spits out a relatively obvious and iterative climate change plan the world will eagerly follow it. Both premises seem laughably unlikely to me.
meroes | 4 hours ago
dsign | 3 hours ago
Yeah, that's not gonna happen.
> The AGI-pilled must think one of two things;
I'm cynical and I believe that AI will be our undoing. In fact, when we let that genie out of the bottle, many other scenarios became possible. For example, here's one third scenario to your "two things": a sufficiently cynical[^1] and resourceful rogue AI may solve climate change by steamrolling over the "set of political, social, and systemic reasons" and teaching the monkeys a lesson about the real significance of very vast agency and intelligence. For example, it could manipulate people into an anti technological uprising and subsequent backward, anti-humanist ethos, while at the same time modifying one or two key ecosystems so that the Mediterranean would smell of cooked cabbage (dimethyl sulfide) and bring about a North African monsoon that, with sufficient time, would turn the Sahara into carbon-sequestering dense jungle...
[^1]: It takes one to know one.
ijidak | 2 hours ago
They are effectively arguing that AI will be God-like.
I don't mean here God in the religiously charged sense but in the philosophical sense.
A god is simply a person or thing that grants salvation.
Whether that's looking to a rain god to to solve a drought, a luck god to to solve a series of bad events, etc.
It doesn't matter how the god does it but the adherents trust that the god has the powers to do it.
In this case the idea is that the AI is going to be all-knowing and all-wise, and thus can lead mankind towards every manner of salvation.
It has a god-like flavor to it.
signatoremo | an hour ago
Mind sharing who at AI labs saying this?
joe_the_user | 4 hours ago
US health care is well known to be an ongoing disaster for US health care end-users however many new health care products US companies produce. The overall situation is that final good health outcomes currently and in the near future require humans involved and moreover, caring human attention involved - among other things because human health involves many, many "outliers" and many, many connections between social, biological, psychological and individual factors.
The US both an entrepreneurial "try things out" approach and a vast spider web of regulations. Why? Because if it didn't have that people would be dying at a faster rate, in droves. Occasionally, even with this, really bad stuff comes through (see opioid epidemic etc).
There currently isn't a way out of this since the entrepreneurial needs to cheapen humans and get humans out of the loop. Again, that produces mediocre factory health care for maybe cheaper if you're lucky and careful. And again, f-- kills people if no one looks (Theorno etc).
The think is, it's more plausible that some level of intelligence could get a handle on the multifaceted needs that are actually involved in actual human health care. The problem is that it seems likely that experiments using inadequate tools are likely to harm many people before that happens.
BugsJustFindMe | 3 hours ago
The argument being made appears to be that improving technical ability at the edges of a graph doesn't help when governance of inter-node processes is still a primary limiting constraint.
Ok. Yes. And it's a very strong position to take.
But that's not the same thing as intelligence not being the bottleneck, because governance is absolutely a manifestation of intelligence. It's just intelligence at a hub node instead of at a leaf node.
Good governance takes intelligence. Recognizing the importance of things takes intelligence. Making existing resources available quickly takes intelligence. And so on. Even when things are constrained by physical processes or financial resources, making things less constrained by those physical processes and financial resources takes intelligence. It's all intelligence. Where our processes are slow and dumb, they are slow and dumb because the people managing those processes were not smart and fast enough in the ways that mattered to make them better.
Tangentially, I find this sloppiness kind of funny in the context of intelligence and bottlenecks, because it's also a bottleneck for communication.
The subject should have been "Intelligence _here_ is not the main bottleneck."
breuleux | 2 hours ago
True, but that doesn't mean intelligence is the bottleneck of good governance. Good governance also requires simple, plain brute force. You have to herd millions of cats, all of which have their own intelligence, and the richest of which are basically giant tigers. At our current level of individual intelligence, we have plenty of simple solutions to many of the problems that plague us, but the reality of the boiling ocean of society is that it tends to crush them faster than they can be built up.
When the problem is the coordination of intelligent agents with disparate interests, increasing everyone's intelligence isn't necessarily going to lead to a solution. Maybe if everyone operates in good faith with a compromising spirit. But if our best solution creates losers, they will fight to the bitter end, in which case we don't need intelligence as much as we need a bigger boot. That's governance.
chaseadam17 | 3 hours ago
China has seen much faster growth in recent decades, which incentivizes people to work hard and have faith it will yield results. In America, the average standard of living has not improved for a long time, so instead of a desire to change things, like cutting regulation to speed up drug discovery, most Americans are quiet quitting.
In this climate, the unfortunate reality is that we're unlikely to see a popular revolution to drastically change things one way or another. Instead, we're much more likely to see regulatory capture by a small number of wealthy, powerful, motivated people. And everyone else will complain on social media, but that's about it.
0xWTF | 3 hours ago
Uh, what? It's a semi-log scale and the data has been statistically bouncing off 0 for a decade. It can't go below 0 but it's clearly flat-lined. Hint: on a log scale, you're never getting to 0 unless your function crosses the axis. But this function can't cross 0.*
* It could cross 0 if you start counting withdrawals of products from the market, but that would be a different function with different inputs.
highfrequency | 3 hours ago
mncharity | 2 hours ago
I suggest people greatly underestimate how localized expertise is. How very rapidly someone's expertise deteriorates as you move away from some focus, from the usually-very-narrow region where that expertise is externally nurtured.
A snark characterization of that region being, the issues where someone badly fears getting something wrong, and by that being embarrassed in front of the people whose opinion they much care about. One can have what superficially look like intellectual communities (picture DC thinktanks), which largely lack this property.
You can have someone well respected in their "real" subfield of biology, while being basically nutter in another subfield, which they care and think about, but aren't professionally embedded.
Ask a quantum chemist a protein chemistry question, and don't be surprised by a grad student answer. Ask "what causes the seasons?", and perhaps high-school or middle-school. Why would it be otherwise?
Even within focus, a bunch of research community gossip is discount functions. "Don't believe his observations using <technique> - he sees what he wants to see". "She always looking for things to be <favorite mechanism>."
For a recent impactful example, someone can be a successful CEO, notable engineering manager, and unusually good at thinking from first principles... while simultaneously, on other topics, being so batshit-crazy nutter that closely-held important views are 5+ orders of magnitude disconnected from reality. Because on those topics, their external community is nutter. And instead of finding this juxtaposition unsurprising, commenters twist themselves up trying to level that curve.
It's the shitty textbook publisher fallacy of "We'll find ourselves an outstanding professor of chemistry, and he'll write us an outstanding chemistry textbook". That's just not how this works.
> monoculture
Which brings us to groupthink. Expertise is a community-associated property.
In foreign affairs for example, the investment community is good at noticing and correctly evaluating some aspects of reality but not others, and similarly for diplomatic community, development, business, political, etc. Each has a partial view with systemic flaws.
I suggest groupthink is a critically important aspect of most human behavior. And yet somehow, we rarely manage to discuss this, and are repeatedly surprised by it. I'm unclear on why.
For decades I thought it bizarre that the New York Times didn't even manage to say the word. Until Trump and MAGA. Yay, a breakthrough? No, at least not the last time I checked several years ago. It remained associated with only Trump and MAGA. Some things are easier to see across the battlefield, than in the mirror, or literally everywhere?
nzach | an hour ago
Isn't that just a manifestation of the 'Blind spot' from Johari Window?
In general it is really hard to think about the things you already internalized.
m3kw9 | 2 hours ago
ma2kx | 2 hours ago
https://www.youtube.com/watch?v=XuoqKYxDHVc&t=1880s
Minton Beddoes: What people who lose their jobs, who are displaced, who may go off and garden, but how are they going to live? And what kind of I mean, it sounds to me...
Musk: universal high income
Minton Beddoes: but how are you going to get there? Let's talk about the politics of this cuz right now we if we look at the political pressures in most countries people are very very worried about this very against it. I mean it seems to me that there's a risk that we are heading more for revolution than for an easy route to this nirvana you're playing out.
Musk: I mean it will be a bumpy road. I'm not suggesting it will be sort of that everything will be smooth sailing here. Um, so I mean there are many jobs that used to exist that don't exist because of computers. I mean the the word computer used to refer to a person doing computations like that used to be a job description of a person. And you can see pictures online of um of buildings that... drifts away