This is fascinating and scary at the same time. I dread the future when people will have access to cheaper health insurance with the caveat that all consultations and procedures will be performed solely by AI. Want a human? Pay extra.
I imagine human consultation could be more expensive, at least in the US, due to liability of human error. Especially if/when AI capabilities in medical procedures becomes clearly superior.
Humans are irrational. We also value "feeling" safe. Some tricks sell the illusion for those people who can't trust 100% some technologies.
Of course I trust engineering, I know planes are safe: but when there is turbulance and I see a flight attendant just casually walking and not even reacting to it, I feel better.
Humans are still humans.
Of course, culture changes. Colective ideas and institutions adapt. But extremely slowly compared to advances in tech.
This is already happening though, to a point; recently, local GPs have been subsumed into larger health care provider companies, and they have set up systems so that before you can make an appointment, you need to go through a wizard and / or chat interface. Likewise, people are encouraged to go to a website and do some self-diagnosis before calling a doctor - things like "if you have a fever, how high is it and for how long have you had it?"
It's indirectly related to health insurance cost, as we have a GP shortage and they (I presume?) realized a lot of people are making appointments and wasting a GP's time for trivialties like a cough or cold.
I'm pretty sure they already have, or are experimenting with having people go through an AI chatbot first. This is already very much a thing with any kind of customer support. It's not a new development though, the preceding generation was conversational interfaces.
I have already given up the idea of getting heath care. I have top-shelf insurance that has all the preventative stuff included, but it is simply not worth the effort. The quality of care is not even worth my co-pay.
Tell me you've never needed significant healthcare without telling me.
It's never about "ambulances". The bureaucracy will happily call an ambulance for you, as it ends their involvement and absolves them of liability. And the ambulance company is happy to show up, as it's a large billable event. In fact when seeking/obtaining complex medical care, this is one of the failure modes you have to constantly swat away!
As for interacting with the bureaucracy to obtain real care, it's gotten to the point where it's easier to get something done calling a large bank. Navigating the medical bureaucracy is probably on the level of calling Comcast-Warner (although I haven't had the misfortune of having to deal with them for a decade, so I'm sure they've gotten worse as well).
Even before "AI", my standard experience is a multi-level phone tree, multiple mandatory announcements before you can even proceed, then waiting on hold. That merely gets you to an answering service who types a ticket into the computer system. Then you wait a ~day for a nurse to call you back (if you miss answering this call, return to step 1). Then the only urgent appointments are invariably with "nurse practitioners" who throw pre-canned advice at you, as they don't really know what they're doing. That's four layers running interference to keep you away from an actual doctor, who you will finally get to see in a month for a fifteen minute appointment, if you are lucky.
I am looking forward to it! Right now we get doctors who are "qualified", but take you after 20 hour shift with no sleep, and do not even speak fluently in the same language as patient.
And this is on VERY expensive state helath insurance!!!
At the risk of being a pedant, was this not literally a test to see if Claude could/would determine and then take the appropriate steps in administering an injection?
With a veterinarian conducting part of the procedure and overseeing the rest of it, and the mouse reportedly making a decent recovery, I fail to see a lapse in ethics.
And presumably you are going to keep failing, because you're not trying. You're handwaving the subject away looking for some bright red "lapse" rather than seeing the everpresent contours.
I've come to see this type of indignant/defiant language as a tell that someone is refusing to consider an issue (ie ignorance).
I've considered the potential harm versus the realities of medical science and study.
Starting on epileptic mice versus humans seems like a reasonable starting point.
You're failing to identify any actual ethical issue beside a blanket problem with animal testing. Which I can partially agree with as there's plenty of needless abuse and inhumane treatment.
First, you seem to be confusing me for the commenter who originally raised a concern.
Second, you're not really addressing my comment, rather just responding with a more polished version of your previous comment deny any issue.
My point is where are your own ethical boundaries here? For each aspect of the situation, what would have to be different to run aground of your own ethical concerns? In order to say that something is on a specific side of a line, you must know where the line actually is, right?
Third, and this doesn't apply to you as an uninvolved commenter per se, but it definitely applies to the original "experiment" - the onus is on the person performing it to demonstrate how it is ethical. A xit that reads like a kid buying some fancy equipment and playing around to see what might happen, "18hrs in front of my laptop coding and coordinating this", and a "licensed vet" as an accountability crumple zone does not inspire confidence!
I would say I did address your comment, I just don't seem to be spelling it out in a way that you would prefer. Which is to say I'm not sure how you can't extrapolate my ethical boundaries and considerations from my previous comment.
> potential harm versus the realities of medical science and study
How much issue do you take with animal testing in the rest of medicine? We already, more or less, torture an unfathomable amount of animals to get where we are.
At which point from testing with non-animals do you go from simulated surgeries with robots to actual surgeries though? I don't want it being done unethically, but how do you promote the effort / research ethically?
At some point you have to rip out the bandaid, if they did everything possible before operating on actual test subjects, then I don't personally have an issue.
That ship sailed a long time ago. Science decided that killing mice for experiments (regardless of the experiment) is fine. In fact, a large amount of the mice killed are just redundant (not needed for a particular experiment)
It’s a mouse. If you think feeding one to a snake is ethical, then it’s a little strange this would be unethical. At least the human has the mouse’s wellbeing at heart.
Not saying that this resolves the broader issue, but I'm not aware of any vegetarian snakes. It seems possible to differentiate this from animal experimentation.
That’s not quite equivalent. For one, a responsible owner is not feeding live mice to a snake. There’s at least an opportunity to ethically euthanize the mice prior to the act.
> At least the human has the mouse’s wellbeing at heart.
This is a good way to injure your snake, it definitely depends on the species and the individual temperament of the snake, but there are ways to entice a snake to eat dead mice (you can warm them, and wiggle them).
It helps to understand how snakes detect and catch prey.
Oh well, when we talk about animal cruelty, the paradoxes of what groups of humans think is ethical is mind bending. Because humans (most of them), love to eat meat or use animal products and don't really care where it comes from.
Oh don't worry, they made sure to have a human acting as an accountability crumple zone: "A licensed vet gated every irreversible step and held a physical kill switch to stop the AI agent."
For the mouse sake probably not as the mouse will not be freed and no longer subjected to experiments. From the perspective of the success of this AI surgery it's likely important that the mouse survived.
It's a cheap imitation of one; a toy. Use it when you're helping your kid build a spaceship out of cardboard. If you want one that has actual electrical and temperature ratings beyond that of "survives shipping to Amazon Prime member" you should probably break out the McMaster catalog.
Also it isn't mounted on a panel. You'll go to slap it and it will just tip over instead of being pressed.
I know there is always endless goalpost-moving in these kind of discussions but I really do think there's an important difference between "performed brain surgery" and "observed brain surgery and ran inject(3.9, 4, 4.3) at one point". And there's also no discussion here of whether this decision made any medical sense and what impact it had on the mouse besides not killing it.
I don't know what harnesses they run but when I daily tell Claude or others to do X and they indeed do parts of X and then also Y and break A on the way... I'm so tired of asking "but, really?" that I'm not even bothering anymore. The AI folks live in a different world than I live, and whether it's a real world or only in their minds (or only in their articles) I definitely cannot tell anymore. I simply gave up.
Some variation of “you’re just using it wrong” is usually the refrain I hear when I talk about things LLM’s are bad at. Anything is possible if you divine the right words to input and fix your harness and set pick the perfect model and…
From my reading of it, the override moment was because they made a coding error which cause manual adjustment to de-sync the arm position, the agent correctly refused to continue when it noticed that.
It does seem like there is minimal "surgery" involved but it's not much of a stretch. If a surgeon took the same steps but someone else opened the skull, it would still be considered 'brain surgery', I think.
I would feel more comfortable if the agent wrote the software to do the surgery rather than did it itself, so that it can be vetted before deployment. And if a complication arose, to stop and defer to the surgeon (would that work?)
If only we had accurate models they could train on.
This was an unnecessary experiment that was only done a live mouse (vs a simulation) for the attention grabbing effect.
Obviously an AI agent can perform a simple surgical procedure. The question is how to get it to perform reliably.
The reason AI agents work as well as they do with code is because they get feedback and retry in a loop. Surgery is inherently inflicting “harm” on a patient, so there’s no bright line guardrail to apply to the agent loop.
Agents performing surgery won’t be possible until models get much much more reliable or we develop a simulation that matches reality close enough that we can use that in realtime for their feedback loop. I think we’re likely very far off from either solution.
I feel like we are in the grab headlines by any means necessary part of the AI Hype cycle, maybe a bearish indicator.. either way this just felt wrong. Like what the heck kind of wrong.
This truly feels like a meme rather than a genuine study or something... If I hook up claude to my car it will probably will be able to drive it, same way it can program a simulated car in a game engine, or even Jev could do it...
This has be some sort of a weird marketing stunt for so called x-bros that I can't understand.
OP submitter here: Just so you know, I posted this to start the conversation.
I actually agree with you completely and find this "experiment" really odd and intentionally attention-seeking.
Very different but made me think of a recent personal story.
My father-in-law had an eye accident when he was young, and the doctor at the time said "unfortunately I don't recommend you try surgery, there's a really high risk you go blind".
Fast forward to today, and 40 years later such doctor called him and said "hey, turns out we could do a cornea transplant and surgery today and you have really high chances of success!"
He then went to do the surgery and now has restored his vision fully. He said the surgery was very "robotic" as in there's this super high precision device that does incredible work.
This seems supremely stupid, lazy, and performative. We know how to build custom neural networks to perform much faster with higher accuracy when custom built for a specific task. What was it doing mid-surgery? Googling "brain size of mice"?
glimshe | 7 hours ago
sheikhnbake | 6 hours ago
rowanG077 | 6 hours ago
[OP] FacZap | 6 hours ago
Of course I trust engineering, I know planes are safe: but when there is turbulance and I see a flight attendant just casually walking and not even reacting to it, I feel better.
Humans are still humans.
Of course, culture changes. Colective ideas and institutions adapt. But extremely slowly compared to advances in tech.
RandomLensman | 3 hours ago
Cthulhu_ | 6 hours ago
It's indirectly related to health insurance cost, as we have a GP shortage and they (I presume?) realized a lot of people are making appointments and wasting a GP's time for trivialties like a cough or cold.
I'm pretty sure they already have, or are experimenting with having people go through an AI chatbot first. This is already very much a thing with any kind of customer support. It's not a new development though, the preceding generation was conversational interfaces.
sznio | 5 hours ago
Need an ambulance? Let me escalate to the department that will escalate you to a department that will dispatch the ambulance in 5-10 business days.
voakbasda | 4 hours ago
mindslight | 3 hours ago
It's never about "ambulances". The bureaucracy will happily call an ambulance for you, as it ends their involvement and absolves them of liability. And the ambulance company is happy to show up, as it's a large billable event. In fact when seeking/obtaining complex medical care, this is one of the failure modes you have to constantly swat away!
As for interacting with the bureaucracy to obtain real care, it's gotten to the point where it's easier to get something done calling a large bank. Navigating the medical bureaucracy is probably on the level of calling Comcast-Warner (although I haven't had the misfortune of having to deal with them for a decade, so I'm sure they've gotten worse as well).
Even before "AI", my standard experience is a multi-level phone tree, multiple mandatory announcements before you can even proceed, then waiting on hold. That merely gets you to an answering service who types a ticket into the computer system. Then you wait a ~day for a nurse to call you back (if you miss answering this call, return to step 1). Then the only urgent appointments are invariably with "nurse practitioners" who throw pre-canned advice at you, as they don't really know what they're doing. That's four layers running interference to keep you away from an actual doctor, who you will finally get to see in a month for a fifteen minute appointment, if you are lucky.
threethirtytwo | 4 hours ago
That is a different kind of scary. Because it’s justified. Why limit the ai when human lives are at stake?
throw93937900 | 3 hours ago
And this is on VERY expensive state helath insurance!!!
rbinv | 7 hours ago
Will we just throw LLMs at most problems in the future instead of smaller, custom models? Like Sonnet 5.5 in this case.
jwrallie | 7 hours ago
NateEag | 6 hours ago
This is an unfortunately true and insightful comment.
ChrisMarshallNY | 3 hours ago
There's the old joke:
x3haloed | 4 hours ago
andrewshadura | 6 hours ago
markyc | 6 hours ago
throwaway198846 | 6 hours ago
tom_ | 4 hours ago
tylfin | 6 hours ago
OleksandrC | 6 hours ago
wslh | 6 hours ago
bio-s | 6 hours ago
giancarlostoro | 6 hours ago
ajcp | 6 hours ago
sheikhnbake | 6 hours ago
bio-s | 5 hours ago
sheikhnbake | 4 hours ago
With a veterinarian conducting part of the procedure and overseeing the rest of it, and the mouse reportedly making a decent recovery, I fail to see a lapse in ethics.
mindslight | 3 hours ago
And presumably you are going to keep failing, because you're not trying. You're handwaving the subject away looking for some bright red "lapse" rather than seeing the everpresent contours.
I've come to see this type of indignant/defiant language as a tell that someone is refusing to consider an issue (ie ignorance).
sheikhnbake | 3 hours ago
Starting on epileptic mice versus humans seems like a reasonable starting point.
You're failing to identify any actual ethical issue beside a blanket problem with animal testing. Which I can partially agree with as there's plenty of needless abuse and inhumane treatment.
This specific case does not seem like that.
mindslight | 2 hours ago
Second, you're not really addressing my comment, rather just responding with a more polished version of your previous comment deny any issue.
My point is where are your own ethical boundaries here? For each aspect of the situation, what would have to be different to run aground of your own ethical concerns? In order to say that something is on a specific side of a line, you must know where the line actually is, right?
Third, and this doesn't apply to you as an uninvolved commenter per se, but it definitely applies to the original "experiment" - the onus is on the person performing it to demonstrate how it is ethical. A xit that reads like a kid buying some fancy equipment and playing around to see what might happen, "18hrs in front of my laptop coding and coordinating this", and a "licensed vet" as an accountability crumple zone does not inspire confidence!
sheikhnbake | an hour ago
> potential harm versus the realities of medical science and study
> needless abuse and inhumane treatment.
> This specific case does not seem like that
PhunkyPhil | 6 hours ago
ajcp | 6 hours ago
PhunkyPhil | 5 hours ago
giancarlostoro | 5 hours ago
ajcp | 2 hours ago
giancarlostoro | an hour ago
0xdeadbeefbabe | 2 hours ago
throwaway198846 | 6 hours ago
0xbadcafebee | 6 hours ago
sillysaurusx | 6 hours ago
henrymerrilees | 6 hours ago
bensyverson | 6 hours ago
bensyverson | 5 hours ago
catlifeonmars | 6 hours ago
> At least the human has the mouse’s wellbeing at heart.
You don’t know this to be true
CuriouslyC | 4 hours ago
suddenlybananas | 3 hours ago
catlifeonmars | an hour ago
It helps to understand how snakes detect and catch prey.
paxy | 6 hours ago
augment_me | 3 hours ago
The amount of cruelty humans inflict on other animals has very little bounds, this is fairly benign in the big picture honestly.
nojvek | 3 hours ago
It's a deep subject.
mindslight | 3 hours ago
(/s)
RandomLensman | 6 hours ago
adamsmark | 6 hours ago
I noticed that the first tweet made no mention if the mouse survived. Seemed a little...sus.
catlifeonmars | 6 hours ago
Without more context, I’m not sure if that’s better or worse
tartoran | 3 hours ago
DavCreator | 6 hours ago
alnwlsn | 6 hours ago
https://www.amazon.com/dp/B09JP4JKY6
ricardobeat | 5 hours ago
alnwlsn | 4 hours ago
Also it isn't mounted on a panel. You'll go to slap it and it will just tip over instead of being pressed.
burkaman | 6 hours ago
> Watching our surgeon do the craniotomy, which the agent is not allowed to do yet for safety reasons
I think the only action it took was sending commands to lower a needle to a certain location, inject, and then remove it. But then it sounds like it couldn't actually choose a valid location and this guy had to manually override it? Maybe I'm misunderstanding, talking about this post: https://x.com/OcanaSantero/status/2115442968874139867 (https://xxcancel.com/OcanaSantero/status/2115442968874139867).
I know there is always endless goalpost-moving in these kind of discussions but I really do think there's an important difference between "performed brain surgery" and "observed brain surgery and ran inject(3.9, 4, 4.3) at one point". And there's also no discussion here of whether this decision made any medical sense and what impact it had on the mouse besides not killing it.
soco | 5 hours ago
Forgeties79 | 5 hours ago
x3haloed | 4 hours ago
suddenlybananas | 4 hours ago
ricardobeat | 5 hours ago
It does seem like there is minimal "surgery" involved but it's not much of a stretch. If a surgeon took the same steps but someone else opened the skull, it would still be considered 'brain surgery', I think.
suddenlybananas | 4 hours ago
pengaru | 3 hours ago
exe34 | 2 hours ago
UncleOxidant | an hour ago
esafak | 5 hours ago
If only we had accurate models they could train on.
sarchertech | 5 hours ago
Obviously an AI agent can perform a simple surgical procedure. The question is how to get it to perform reliably.
The reason AI agents work as well as they do with code is because they get feedback and retry in a loop. Surgery is inherently inflicting “harm” on a patient, so there’s no bright line guardrail to apply to the agent loop.
Agents performing surgery won’t be possible until models get much much more reliable or we develop a simulation that matches reality close enough that we can use that in realtime for their feedback loop. I think we’re likely very far off from either solution.
minraws | 5 hours ago
This truly feels like a meme rather than a genuine study or something... If I hook up claude to my car it will probably will be able to drive it, same way it can program a simulated car in a game engine, or even Jev could do it...
This has be some sort of a weird marketing stunt for so called x-bros that I can't understand.
[OP] FacZap | 5 hours ago
threethirtytwo | 4 hours ago
Why would we limit the autonomy and power of ai when human lives are at stake?
hmokiguess | 3 hours ago
My father-in-law had an eye accident when he was young, and the doctor at the time said "unfortunately I don't recommend you try surgery, there's a really high risk you go blind".
Fast forward to today, and 40 years later such doctor called him and said "hey, turns out we could do a cornea transplant and surgery today and you have really high chances of success!"
He then went to do the surgery and now has restored his vision fully. He said the surgery was very "robotic" as in there's this super high precision device that does incredible work.
igleria | 3 hours ago
hmokiguess | 3 hours ago
tartoran | 3 hours ago
"ast forward to today, and 40 years later such doctor called him and said"
hmokiguess | 3 hours ago
0xdeadbeefbabe | 3 hours ago
malfist | an hour ago
BiscuitEater546 | 3 hours ago
sva_ | 3 hours ago
dofm | 3 hours ago
moistoreos | 3 hours ago
y-c-o-m-b | 3 hours ago
Here's a Wired link: https://www.wired.com/story/an-ai-agent-did-brain-surgery-on...
deweywsu | 3 hours ago
atleastoptimal | an hour ago