I'm curious if this will hold up, the people of Utah rejected that mega data center iirc, maybe they feel differently on this one because of what private equity has done to rural healthcare
Oooh, good point. I bet the pharmas love this. Get that expensive, foot-dragging doctor out of the sales loop and write several prescriptions for every patient encounter! The KPI goes from minimizing MD minutes to maximizing med profits.
I doubt they're going to give the acne medication recommender agent avenues to violate HIPAA.
The accountability is with the company owners. Like with every healthcare startup, there's no reason to assume they'll love being sued or fined so much that they will bake violating HIPAA into their business plan.
You are right. I just think you overestimate the competence of leadership in such companies and underestimate the level of corruption in Utah.
They have lawsuits built into this, and either by hubris or bribes believe this is the most profitable route. "Its just acne, what could possibly go wrong?"
Isotretinoin is a common acne medication and a bit of a breakthrough that also has potentially serious side effects. My take is more that if the medication is safe enough to be given out by an AI then maybe it should just be over the counter.
OTC is a reasonable argument to explore, at least.
I don't think it's automatically true though. In the case of acne medication, I remember there's a well-known one that carries the very real caveat that it'll kill or severely deform the baby if you're pregnant. When you consider the old classic line "Think of the average person. Half of the people are dumber than that" I don't feel comfortable with that medication sitting on the shelf next to the Clearasil, where a busy person will just grab it because they heard it was super effective, or because the other types were out of stock.
The AI inference involved there no doubt costs pennies; so having it screen the patient to ensure that she understands the risks and has taken a pregnancy test recently if unsure, seems like a "why not" kind of thing.
Now, if some AI-healthcare company is capturing 90% of the money saved by not having doctors do this, then it would be barely worth it.
There's the "in between" of "behind the pharmacy counter", requires you to interact with a tech if not a pharmacist to ask/answer some questions before dispensing, but at point of sale.
If a 20 minute telehealth visit with a PA is a sufficient level of screening for a medication, it's hard to imagine a compelling reason why an AI system would not be.
Amazon's pharmacy prescribes a large swath of medications over text.
No call or video necessary.
To make things more complicated, every message I sent was responded to by a different nurse practitioner. So it seemed everytime i sent a response or question, it would get routed to a random NP and she would respond without reviewing the full conversation.
Works if you know what you need, and I can't imagine AI being a worse process than whatever Amazon is doing.
While I am too ignorant to have an opinion about whether AI is a good idea, I do think that ungating relatively safe medications from needing an MD's prescription is a good idea.
You don't have to go to medical school to know what bad acne looks like, or how to treat a UTI. We should be working on tiering prescriptions based on ease of diagnosis and risk.
This is great. It removes a big time waste of appointments and procedures for things that you know you need.
I have for example had tonsilitis 5+ times, and a lot of common colds/flus/etc. I know how my body reacts and how it feels when I have one or the other.
Yet, I have to be screened by nurses, sent around, given weak antibiotics, and in general wasting a lot of time when I could just convince a model to give me the prescription.
I have a friend with acne who has the same issues with tretinoin prescriptions. Endless appointments and time waste.
I have for example hypersensitivity to radio waves. I know how my body reacts and how it feels when I'm exposed to such radiation, especially during vaxxine toxicity flareups.
Yet, I have to be screened by nurses, go through appointments with psychologists, and in general wasting a lot of time when I could just convince a model to prescribe the radium water I need.
It would be also be great if people with libedo issues could then get a medical prescription for porn, which if I know my AI, and I dont, could be accomodated instantly during the 'examination'
Because as an adult the operation is genuinely full of risk. Think about the anatomy of your neck and how close your tonsils are to your major blood vessels.
Sometimes doctors make choices that reflect a landscape of risk the patient is no more prepared to understand than the doctor is ready to run a network test. People who say they “know they need something” are often confidently wrong.
There was an assessment for this and deemed not worth the risk because of the size of the connective tissues/blood supply and closeness to major blood vessels.
There's other solutions that can also be explored.
Where I am in BC, Canada, pharmacists can prescribe specific medications for certain pre-approved conditions, and there is also the concept of an "emergency" refill for if you lose your medication or are unable to get an appointment in time, a few days of a previously prescribed medication can be provided.
At the end of the day, a better solution is to have tiers of prescription meds. Not everything needs to be gate-kept by MDs. If an AI can prescribe it, there is no reason a nurse, pharmacist or other medical professional shouldn't be able to as well.
Acne is a real problem for some people, but comparatively low stakes, mostly visual evaluation, and with pretty well-known solutions. (Of course everything has corner cases, but in general this is a well understood area.)
I have no idea how well this will go, but starting here rather than, say, robotic cancer treatments, can help AI medicine develop into more.
If it does go badly, then after a year we can re-evaluate and either give up or refine the system.
What is the actual benefit here? If we believe the doctor's oversight is required, the AI introduces a dangerous potential for error. If we don't think a doctor's oversight is required, the medication shouldn't require a prescription in the first place. What positive value does the AI actually add that isn't circumventing a legal requirement that we decided to put in place?
The removing a prescription is a federal law, something that Utah can't overrule. Unfortunately some prescriptions are good for some medication and not great for others.
I think we came up with this dubious category of things that supposedly require oversight, but only to a minimal extent. Viagra, contraceptives, eyeglasses. You can already get these with near-zero diligence if you take the telehealth route, but I guess there's still someone who feels it would be a pressing societal issue if you took matters into your own hands and ordered the wrong kind of eyeglasses.
So here, AI isn't doing anything clever, it's just asking "are you sure you don't have a heart condition" and "are you sure you're not a crazy person". We don't really need LLMs for that, but if it's a catalyst for liberalization, fine.
Yes and if you need a controlled prescription, it’s super painful. Can’t be transferred to another pharmacy, etc. extremely difficult to fill reliably.
And the medications that are controlled vs uncontrolled (at least here in Texas) seem completely random.
>So here, AI isn't doing anything clever, it's just asking "are you sure you don't have a heart condition" and "are you sure you're not a crazy person".
So are we just admitting that the "AI" here is the equivalent of a dialog box and this same change could have been enacted at any point in the last half century?
>We don't really need LLMs for that, but if it's a catalyst for liberalization, fine.
Is this liberalization or is it just further entrenching the bureaucracy that this change implies is already broken?
This whole thing just feels like a Chesterton's fence scenario in which we knock down the fence and then immediately rebuild it, only a few inches shorter and with weaker materials this time. I just don't see how that's any sort of victory for progress.
AI is already being used to guide care outcomes and overrule doctors via “utilization management”.
I am a big believer in AI for drug discovery. But AI applied to patient care will never be used to benefit the patient or increase quality of care.
Cost reductions will be absorbed for profits. The medical industry has already aggressively implemented AI automation (imaging, automated note taking, etc) and in that period, costs have only increased.
People can be uneducated about medication, acetaminophen overdose is a leading cause of acute liver failure in the United States. A little bit of a barrier and forced information about the concern might not be a bad thing but it doesn't require a doctor. A lot of medication probably falls in the same category but is on the other side of the prescription fence.
And doctors aren't infallible, AI (and rule based systems) can act as an extra safeguard for them as long as they don't blindly trust it.
Well in this age of performative bullshit, it gives the appearance of "medical practice" and allows the government to keep people drugged and complacent. What other reason do you need?
> allows the government to keep people drugged and complacent
Drugged with a topical ointment and complacent by having less acne? This is a one-year trial for patients with mild-to-moderate acne. Let's not blow this situation out of proportion. This trial seems rather narrowly scoped and overall a positive idea for well understood treatments that hardly need an MD to begin with.
If you want to get into conspiracy theories of a government wanting a drugged and compliant citizenry just look no further than the sudden push legalizing THC across the country.
This is such a low quality comment that I wonder whether people even know how AI is used!
I see this same fallacy repeated all the time when we speak about AI.
"I want AI to be 100% accurate and not 99.999%"
"If AI makes even one mistake, why would I even use it?"
"What's the point of using AI if I have to verify it anyway?"
Here's how it could work.
You could always have a CI sort of thing for doctors prescriptions where you run the prescription through AI and it can flag any potential error to the doctor.
The LLM can be accurate enough that it only flags things when necessary
You could always have a CI sort of thing for doctors prescriptions where you run the prescription through AI and it can flag any potential error to the doctor.
Yes, and we call this CI system a "pharmacy". It might even employ some human experts to review DDI warnings and suggest alternatives to the prescribing physician.
> Being a doctor is one of the jobs expected to be least at risk from AI replacement
I wouldn't say that, but it's certainly one where the consequences are high and doctor-in-the-loop seems highly desirable for the foreseeable future, especially since most patients are not going to be knowledgeable enough to realize the shortcomings of an LLM and make an informed decision about whether they're OK taking an LLM-prescribed medication.
This is the type of reform that has been difficult to achieve in the US (And I presume other high-regulation, Western countries).
We usually regulate a one-size-fits-all approach to everything in medicine. An M.D. (or whatever NPs have?) is required to prescribe medicines, even medicines which are just the same or slightly stronger formulations of OTC medications like a 400mg Ibuprofen or a 1% hydrocortisone cream. Compare this to Mexico, where you can buy many of our "prescription-only" meds OTC.
A lot of healthcare issues are just not high-stakes. Requiring a doctor in these cases doesn't help the average patient, because the average patient would rather ignore the problem, or now, ask ChatGPT how to treat it with OTC options, than deal with the mess of scheduling doctor appointments, dealing with insurance (or in Canada, long wait times to see the doctor for free), only to get just 5 minutes with the actual doctor who may not even listen to you.
I know it's a tough pill to swallow for doctors, and I recognize that it's not doctors who chose the terrible healthcare system we have or decided that they should have 5-10 minutes with a patient. But I do think we need to change the rules for things that have little risk of causing harm but a great chance of improving the aggregate outcomes by increasing access to healthcare.
I’m all for this, but I think this is wild. I hope they have some hard programmed limits and guidelines baked in because otherwise people are going to turn these into the new pill mills.
Depends. The company normally would be, but they can always add a no liability clause to their ToS, in which case the user is. But it hardly matters in this case given the narrow scope. Very little harm can occur if the system prescribes the wrong thing; hardly different from buying the wrong thing OTC.
> Very little harm can occur if the system prescribes the wrong thing
Statements like this is effectively saying "Nothing will go wrong".
> but they can always add a no liability clause to their ToS
Then why should I trust the output at all?
However, the entire point was to reference the IBM training manual that states: "A computer can never be held accountable, therefore a computer can must never make a management decision".
If the bot cannot be held accountable, it shouldn't be allowed to make decisions.
> Being a doctor is one of the jobs expected to be least at risk from AI replacement
Is it? At least as a customer I'd love to replace a doctor with AI. Humans suck, you have to wait so long for an appointment, they spend like 10 minutes on your case, and often have very outdated knowledge. The main issue is that vision still sucks and lack of embodiment is a problem. It could invoke "human subagents" for that.
> While the program will eventually become fully automated, the first 100 prescriptions will have to be reviewed by actual doctors before they are submitted to the pharmacy.
> In the second phase, the next 400 patients will have their prescriptions submitted to pharmacies by the AI and reviewed retrospectively by a physician weekly.
> The final phase will involve a doctor reviewing at least 10% of the total prescriptions issued once a month.
This is a really good way to go. That's how you get to a proper system. It's the route that should've been taken from the very beginning with all human-facing AI-brokered services, and would've meant far fewer nightmare stories.
blinkbat | 4 hours ago
verdverm | 4 hours ago
https://www.justice.gov/opa/pr/opioid-manufacturer-purdue-ph...
https://www.propublica.org/article/purdue-settlement-leaves-...
I'm curious if this will hold up, the people of Utah rejected that mega data center iirc, maybe they feel differently on this one because of what private equity has done to rural healthcare
imglorp | 4 hours ago
big /s
buellerbueller | 4 hours ago
jmorenoamor | 4 hours ago
throw904309 | 4 hours ago
1Kajsh | 4 hours ago
Choco31415 | 4 hours ago
johnnyanmac | 4 hours ago
unless frontier models (or whatever employs in medicine) are being kind enough to absorb the lawsuits here, I don't see how this ends well.
xp84 | 3 hours ago
The accountability is with the company owners. Like with every healthcare startup, there's no reason to assume they'll love being sued or fined so much that they will bake violating HIPAA into their business plan.
johnnyanmac | 3 hours ago
They have lawsuits built into this, and either by hubris or bribes believe this is the most profitable route. "Its just acne, what could possibly go wrong?"
mulderc | 4 hours ago
xp84 | 4 hours ago
I don't think it's automatically true though. In the case of acne medication, I remember there's a well-known one that carries the very real caveat that it'll kill or severely deform the baby if you're pregnant. When you consider the old classic line "Think of the average person. Half of the people are dumber than that" I don't feel comfortable with that medication sitting on the shelf next to the Clearasil, where a busy person will just grab it because they heard it was super effective, or because the other types were out of stock.
The AI inference involved there no doubt costs pennies; so having it screen the patient to ensure that she understands the risks and has taken a pregnancy test recently if unsure, seems like a "why not" kind of thing.
Now, if some AI-healthcare company is capturing 90% of the money saved by not having doctors do this, then it would be barely worth it.
FireBeyond | 2 hours ago
mulderc | 2 hours ago
TYPE_FASTER | 2 hours ago
[0] - https://www.mobihealthnews.com/news/nolla-health-launches-ut...
bogwog | 4 hours ago
johnnyanmac | 4 hours ago
assimpleaspossi | 4 hours ago
hamdingers | 4 hours ago
If a 20 minute telehealth visit with a PA is a sufficient level of screening for a medication, it's hard to imagine a compelling reason why an AI system would not be.
hansonkd | 4 hours ago
To make things more complicated, every message I sent was responded to by a different nurse practitioner. So it seemed everytime i sent a response or question, it would get routed to a random NP and she would respond without reviewing the full conversation.
Works if you know what you need, and I can't imagine AI being a worse process than whatever Amazon is doing.
skeledrew | 3 hours ago
rglover | 4 hours ago
Just in case anyone else has the gut "oh ffs" reaction to this.
janice1999 | 4 hours ago
1Kajsh | 4 hours ago
dghlsakjg | 4 hours ago
You don't have to go to medical school to know what bad acne looks like, or how to treat a UTI. We should be working on tiering prescriptions based on ease of diagnosis and risk.
rglover | 4 hours ago
dghlsakjg | 2 hours ago
rglover | an hour ago
augment_me | 4 hours ago
I have for example had tonsilitis 5+ times, and a lot of common colds/flus/etc. I know how my body reacts and how it feels when I have one or the other.
Yet, I have to be screened by nurses, sent around, given weak antibiotics, and in general wasting a lot of time when I could just convince a model to give me the prescription.
I have a friend with acne who has the same issues with tretinoin prescriptions. Endless appointments and time waste.
Step in the right direction
bogwog | 4 hours ago
I have for example hypersensitivity to radio waves. I know how my body reacts and how it feels when I'm exposed to such radiation, especially during vaxxine toxicity flareups.
Yet, I have to be screened by nurses, go through appointments with psychologists, and in general wasting a lot of time when I could just convince a model to prescribe the radium water I need.
Step in the right direction.
doubletwoyou | 4 hours ago
bogwog | 4 hours ago
ofjcihen | 4 hours ago
metalman | 3 hours ago
OkayPhysicist | 4 hours ago
EA-3167 | 3 hours ago
Sometimes doctors make choices that reflect a landscape of risk the patient is no more prepared to understand than the doctor is ready to run a network test. People who say they “know they need something” are often confidently wrong.
augment_me | 3 hours ago
dghlsakjg | 4 hours ago
Where I am in BC, Canada, pharmacists can prescribe specific medications for certain pre-approved conditions, and there is also the concept of an "emergency" refill for if you lose your medication or are unable to get an appointment in time, a few days of a previously prescribed medication can be provided.
At the end of the day, a better solution is to have tiers of prescription meds. Not everything needs to be gate-kept by MDs. If an AI can prescribe it, there is no reason a nurse, pharmacist or other medical professional shouldn't be able to as well.
skeledrew | 3 hours ago
But there are. Costs and wait times go down. Can be handled like a drive-through. No mis-diagnoses due to fatigue, etc.
dghlsakjg | 2 hours ago
1Kajsh | 4 hours ago
AnimalMuppet | 4 hours ago
janice1999 | 4 hours ago
Quinner | 4 hours ago
agos | 2 hours ago
compiler-guy | 4 hours ago
Acne is a real problem for some people, but comparatively low stakes, mostly visual evaluation, and with pretty well-known solutions. (Of course everything has corner cases, but in general this is a well understood area.)
I have no idea how well this will go, but starting here rather than, say, robotic cancer treatments, can help AI medicine develop into more.
If it does go badly, then after a year we can re-evaluate and either give up or refine the system.
EA-3167 | 4 hours ago
bogwog | 4 hours ago
slg | 4 hours ago
abirch | 4 hours ago
socializer | 4 hours ago
So here, AI isn't doing anything clever, it's just asking "are you sure you don't have a heart condition" and "are you sure you're not a crazy person". We don't really need LLMs for that, but if it's a catalyst for liberalization, fine.
toddmorey | 4 hours ago
And the medications that are controlled vs uncontrolled (at least here in Texas) seem completely random.
slg | 3 hours ago
So are we just admitting that the "AI" here is the equivalent of a dialog box and this same change could have been enacted at any point in the last half century?
>We don't really need LLMs for that, but if it's a catalyst for liberalization, fine.
Is this liberalization or is it just further entrenching the bureaucracy that this change implies is already broken?
This whole thing just feels like a Chesterton's fence scenario in which we knock down the fence and then immediately rebuild it, only a few inches shorter and with weaker materials this time. I just don't see how that's any sort of victory for progress.
toddmorey | 4 hours ago
I am a big believer in AI for drug discovery. But AI applied to patient care will never be used to benefit the patient or increase quality of care.
Cost reductions will be absorbed for profits. The medical industry has already aggressively implemented AI automation (imaging, automated note taking, etc) and in that period, costs have only increased.
matt_heimer | 4 hours ago
And doctors aren't infallible, AI (and rule based systems) can act as an extra safeguard for them as long as they don't blindly trust it.
Henchman21 | 3 hours ago
MC995 | 2 hours ago
Drugged with a topical ointment and complacent by having less acne? This is a one-year trial for patients with mild-to-moderate acne. Let's not blow this situation out of proportion. This trial seems rather narrowly scoped and overall a positive idea for well understood treatments that hardly need an MD to begin with.
If you want to get into conspiracy theories of a government wanting a drugged and compliant citizenry just look no further than the sudden push legalizing THC across the country.
simianwords | 2 hours ago
I see this same fallacy repeated all the time when we speak about AI.
"I want AI to be 100% accurate and not 99.999%"
"If AI makes even one mistake, why would I even use it?"
"What's the point of using AI if I have to verify it anyway?"
Here's how it could work.
You could always have a CI sort of thing for doctors prescriptions where you run the prescription through AI and it can flag any potential error to the doctor.
The LLM can be accurate enough that it only flags things when necessary
AlotOfReading | an hour ago
simianwords | an hour ago
"Lets not write unit tests. We can hire developers to write good code that doesn't break"
HarHarVeryFunny | 4 hours ago
I wouldn't say that, but it's certainly one where the consequences are high and doctor-in-the-loop seems highly desirable for the foreseeable future, especially since most patients are not going to be knowledgeable enough to realize the shortcomings of an LLM and make an informed decision about whether they're OK taking an LLM-prescribed medication.
qubex | 4 hours ago
xp84 | 4 hours ago
We usually regulate a one-size-fits-all approach to everything in medicine. An M.D. (or whatever NPs have?) is required to prescribe medicines, even medicines which are just the same or slightly stronger formulations of OTC medications like a 400mg Ibuprofen or a 1% hydrocortisone cream. Compare this to Mexico, where you can buy many of our "prescription-only" meds OTC.
A lot of healthcare issues are just not high-stakes. Requiring a doctor in these cases doesn't help the average patient, because the average patient would rather ignore the problem, or now, ask ChatGPT how to treat it with OTC options, than deal with the mess of scheduling doctor appointments, dealing with insurance (or in Canada, long wait times to see the doctor for free), only to get just 5 minutes with the actual doctor who may not even listen to you.
I know it's a tough pill to swallow for doctors, and I recognize that it's not doctors who chose the terrible healthcare system we have or decided that they should have 5-10 minutes with a patient. But I do think we need to change the rules for things that have little risk of causing harm but a great chance of improving the aggregate outcomes by increasing access to healthcare.
dyauspitr | 4 hours ago
tibbydudeza | 4 hours ago
bena | 4 hours ago
buellerbueller | 4 hours ago
skeledrew | 2 hours ago
bena | 2 hours ago
That's a pretty thin defense when things do go wrong.
skeledrew | an hour ago
Never said that at all.
bena | an hour ago
skeledrew | an hour ago
bena | 32 minutes ago
Statements like this is effectively saying "Nothing will go wrong".
> but they can always add a no liability clause to their ToS
Then why should I trust the output at all?
However, the entire point was to reference the IBM training manual that states: "A computer can never be held accountable, therefore a computer can must never make a management decision".
If the bot cannot be held accountable, it shouldn't be allowed to make decisions.
redox99 | 4 hours ago
Is it? At least as a customer I'd love to replace a doctor with AI. Humans suck, you have to wait so long for an appointment, they spend like 10 minutes on your case, and often have very outdated knowledge. The main issue is that vision still sucks and lack of embodiment is a problem. It could invoke "human subagents" for that.
RobRivera | 3 hours ago
skeledrew | 3 hours ago
> In the second phase, the next 400 patients will have their prescriptions submitted to pharmacies by the AI and reviewed retrospectively by a physician weekly.
> The final phase will involve a doctor reviewing at least 10% of the total prescriptions issued once a month.
This is a really good way to go. That's how you get to a proper system. It's the route that should've been taken from the very beginning with all human-facing AI-brokered services, and would've meant far fewer nightmare stories.
OutOfHere | 2 hours ago
cwillu | an hour ago