"Dangerous and Alarming" - it tough; healthcare is needs disruption but unlike many places to target for disruption, the risk is life and death. It strikes me that healthcare is a space to focus on human in the loop applications and massively increasing the productivity of humans, before replacing them...
https://deadstack.net/cluster/google-removes-ai-overviews-fo...
The profit in insurance is the volume, not the margin. Disrupting it will not dramatically change outcomes, and will require changes to regulation, not business policy.
Agreed. I'd also argue that there will always be the issue of adverse selection, which in any system that doesn't mandate that all individuals be covered for healthcare regardless of risk profile, will continue to raise costs regardless of whether or not margins are good or bad. That dream died with the individual mandate, and if the nation moves even further away from universal healthcare, we will only see costs rise and not fall as companies shoulder more and more of the relative risk.
The solution is single payer. Any attempt to solve this with technological band aids is completely futile. We know what the solution is because we see it work in every other developed nation. We don't have it because a class of billionaire doners doesn't want to pay into the system that allowed them to become fabulously wealthy. People who are claiming AI is the solution to healthcare access and affordability are delusional or lying to you.
There are good reasons to think single payer systems are not the answer. The numerous documented inefficiencies and inconveniences they suffer from don't need repeating here.
And many single payer systems around the world only appear to work as well as they do because the US effectively subsidizes medical costs through its own out of control prices.
Compare: Google's founders can buy all the yachts they could possibly eat, yet Google Searches are offered for free.
If we could get healthcare to that level, it would be great.
For a less extreme example: Wal-Mart and Amazon have made plenty of people very rich, and they charge customers for their goods; but their entrance into the markets have arguable brought down prices.
And Google search, a service on the level of a public utility, has been degrading noticeably for years in the face of shareholders demanding more and more returns.
Comparing something to a public utility is not me saying it's literally a public utility. Google runs a monopolistic service that is essential to a lot of our public life, in a segment that has high cost of entry and infrastructure cost. They make the service worse to make more money. It should be a regulated utility like electricity or railroads, we should have a public alternative like the post office is to UPS, or it should be nationalized. The situation gets more dire when you consider their browser monopoly.
Other search engines exist. Bing is right there, and Microsoft is more than willing to eat the high cost of entry and infrastructure cost.
> It should be a regulated utility like electricity or railroads, we should have a public alternative like the post office is to UPS, or it should be nationalized.
I agree that electricity and railroads should be regulated like Google Search.
It's really weird that snail mail in the US is a government monopoly. When even social democratic Germany managed to privatise them.
> The situation gets more dire when you consider their browser monopoly.
Don't a lot of people in the US use iPhones? They don't ship with Chrome as the default browser, do they?
(And yes, Safari is built on top of the same open source engine as Chrome. But you can hardly call using the same open source project a 'monopoly'. Literally anyone can fork it.)
The existence of few competitors is not proof that monopolistic power doesn't exist and isn't being leveraged. Saying Google isn't monopolistic is being willfully wrong. You're more wrong when we look at the browser market, and Google has lost anti-trust suits on this very topic in the past couple years.
A public mail service is required by our constitution. It's cheaper than the private options and often the only option for many rural areas. It's not a monopoly.
> A public mail service is required by our constitution.
Where does it say so in your constitution? All I can find is the postal clause which Wikipedia summarises as follows, but whose full text isn't much longer:
> Article I, Section 8, Clause 7, of the United States Constitution, the Postal Clause, authorizes the establishment of "post offices and post roads"[1] by the country's legislature, the Congress.
The Postal Clause certainly allows the government to run a public postal service, but I don't see how the constitution _requires_ it. It doesn't even require the federal government to regulate postal services, it merely allows it.
Perhaps I missed something?
> It's cheaper than the private options and often the only option for many rural areas.
If you want to subsidise rural areas, I would suggest to do so openly, transparently and from general taxation. At least general taxation is progressive etc. Instead of just making urban folks pay more for their mail, whether they be rich or poor.
I would also suggest only subsidising poor rural areas. Rich rural areas don't need our help.
We know that from observing evidence such as how much the government pays out in welfare to Wal-Mart employees.
Customers continue shopping there because human beings are typically incapable of accepting a short-term loss (higher price) for a long-term gain (product lasts more than three uses).
> We know that from observing evidence such as how much the government pays out in welfare to Wal-Mart employees.
That's a weird metric. If tomorrow Wal-Mart laid off all employees and replaced them with robots, they would surely be worse off, but by your metric Wal-Mart would look less evil?
> Customers continue shopping there because human beings are typically incapable of accepting a short-term loss (higher price) for a long-term gain (product lasts more than three uses).
I mean, I could tell you are disingenuous from the get-go, so it is not surprising you would take an off-the-cuff metric which is accurate right now and invent a strawman scenario where I might continue to use it beyond a point where it makes sense.
Likewise, I would not use my flippant 3 times metric regarding durability to cover the quality of produce.
Well, feel free to propose a different metric for seeing how Wal-Mart is evil. And instead of Wal-Mart firing all employees, we can have a look at the more plausible scenario of Aldi vs Wal-Mart. Aldi runs a different labour model: their shifts are much more intense but pay more. Their model can't and doesn't utilise the kind of older and less fit people that Wal-Mart employs.
You have to look at the counterfactual of what these people would do, if Wal-Mart weren't around. You seem to implicitly assume that they'd be getting higher paying jobs somewhere else (so they wouldn't have to rely on welfare)? If so, what's stopping those people from switching to these better jobs right now, even while Wal-Mart is still around?
And sure, let's disregard how many times you can eat your groceries. That was a cheap shot. However I think quality vs price trade-off is something customers have to make for themselves anyway. Who am I to judge their choices?
> Google's founders can buy all the yachts they could possibly eat, yet Google Searches are offered for free.
Google searches cost many billions of dollars: your confusion is because the customer isn’t the person searching but the advertisers paying to influence them. Healthcare can’t work like that not just because the real costs are both much higher and resistant to economies of scale but, critically, there aren’t people with deep pockets lining up to pay for you to be healthy. That’s why every other developed country sees better results for less money: keeping people healthy is a social good, and political forces work for that better than raw economic incentives.
IDK, the owners of retail clothing chains buy yachts and yet that sector is jaw-droppingly efficient at delivering clothes to people. Executives can be annoying tools but I don't think their pay is the problem.
Shame those same owners have plenty of money for buying yachts and not enough money to pay their sweat-shop employees a living wage or to provide decent work conditions. Executives don't "earn" huge paychecks, they merely exploit others by figuring out a way to not pay them their worth.
Seriously? Spending a night in a hospital results in a $10,000 bill (though the real out of pocket is significantly cheaper. God help you if you have no insurance though). Healthcare in the US is the thing that needs the biggest disruption.
But no business is going to fix it. The market is captured. Only a radical change of insurance laws is going to have any impact. Mandate that insurance must be not for profit. Mandate at least decent minimal coverage standards and large insurance pools that must span age groups and risk groups.
Many hospitals are already non-profit. That doesn't seem to bring down prices. Why would you think that this would work for insurance?
Profit isn't even a big part of the overall revenue.
> Mandate at least decent minimal coverage standards
I assume you want higher coverage standards than what currently exists? Independently of whether that would be the morally right thing to do (or not), it would definitely increase prices.
> and large insurance pools that must span age groups and risk groups.
Why does your insurance need a pool? An actuary can tell you the risk, and you can price according to that. No need for any pooling. Pooling is just something you do, when you don't have good models (or when regulations forces you).
> Why does your insurance need a pool? An actuary can tell you the risk, and you can price according to that. No need for any pooling. Pooling is just something you do, when you don't have good models (or when regulations forces you).
Wuh? The more diverse the pool, the lower the risk. Your way of thinking will very quickly lead to "LiveCheap: the health insurance for fit, healthy under 30s only" for dollars a month, and "SucksToBeYou: the health insurance for the geriatric and chronically disabled" for the low low cost of "everything you have to give".
There's insurance which allows you to convert an uncertain danger into a known payment. And then there's welfare and redistribution.
By all means, please run some means testing and give the poor and sick or disabled extra money. Or even just outright pay their insurance premiums.
But please finance that from general taxation, which is already progressive. Instead of effectively slapping an arbitrary tax on healthy people, whether they be rich or poor. And please don't give rich people extra stealth welfare, just because they are in less than ideal health, either.
Just charge people insurance premiums in line with their expected health outcomes, and help poor people with the premiums using funds from general taxation. (Where poor here means: take their income and make an adjustment for disability etc.)
We _want_ the guy who loses 5kg and gives up smoking to get lower insurance premiums. That's how you set incentives right.
> The more diverse the pool, the lower the risk.
No. The diversification comes from the insurance company running lots of uncorrelated contracts at the same time and having a big balance sheets. For that, it doesn't matter whether it's a pool of similar insurance contracts, or whether they have bets on your insurance contract, and on the price of rice in China, and playing the bookie on some sports outcomes etc. In fact, the more diversified they are, the better (in principle).
But that diversification is completely independent of the pricing of your individual insurance contract.
Have a look at Warren Buffett's 'March Madness' challenge, where he famously challenges people to predict all 67 outcomes of some basketball games to win a billion dollar. Warren Buffet ain't no fool: he doesn't need a pool, he can price the risk of someone winning this one off challenge.
This is going to pretty rapidly devolve into cheap for healthy and insanely expensive for those that aren’t. Genetic propensities will be a lifelong financial burden. Cancer patients will get priced out and die.
If you want to subsidise these unfortunate people, please just do so openly. Don't be all sneaky, sneaky about it by banning accurate risk assessments.
In any case, what you are saying is only true, if you buy your health insurance second to second on the spot market.
Insurance companies are more than happy to enter long running contracts, where you both agree today on (the algorithm for) the premiums for the next twenty years or even until the rest of your life. That's pretty common with life insurance and disability insurance already.
The above already exists, but if you allow some speculation: you could even envision people buying insurance for their kids before conceiving them. That way you don't have to worry about pre-existing conditions.
(Well, if the parents already have heritable conditions that would make the kids more likely to have expensive medical problems, those would push up their premiums. But then: perhaps these people should think twice about burdening a potential kid with these issues.
Compare how in Cyprus where sickle cell anemia is prevalent, even the Catholic church demand you get screened, before they'll marry you.)
If you really want specialised in-kind welfare, you can get people a voucher for the catastrophic version of 'unconceived baby insurance'.
Basically, you can buy insurance against insurance premiums being expensive.
These solutions are often proposed as easy fixes but I'm skeptical that they actually will do much to reduce healthcare costs. Healthcare is fundamentally expensive. Not-for-profit hospitals and for-profit hospitals don't really substantively differ in terms of out-of-pocket expenditures for patients; I find it difficult to imagine that forcing insurance companies to be nonprofit would do much to reduce costs.
> large insurance pools that must span age groups and risk groups.
What you describe (community rating) has been tried and it works. But it requires that a lot of young, healthy people enroll, and seniors receive most of the care. In an inverted demographic pyramid like most Western economies have, this is a ticking time bomb, so costs will continue to rise.
> Mandate at least decent minimal coverage standards
I think a better solution is to allow the government to threaten in negotiating prices with companies as Canada does; it greatly reduces rent-seeking behavior by pharmaceutical companies while allowing them to continue earning profits and innovating. (I understand a lot of the complaints against big pharma but they are actually one of the few sectors of the economy that doesn't park their wealth and actually uses it for substantive R&D, despite what the media will tell you, and countless lives have been saved because of pharma company profits)
Essentially the gist of what I'm saying, as someone who has been involved with and studied this industry for the better part of five years, is that it's much more complex than what meets the eye.
There are a lot of not-for-profit insurance companies and they aren't noticably cheaper, though I'm not in HR and they may well be cheaper for the employer.
Disruption, yes, in the sense that the current system needs to be overhauled. But this is a space that's frequented by the SV and VC space and "disruption" has very different connotations, usually in the realm of thought that suggests some SV-brained solution to an existing problem. In some edge cases like Uber/Lyft, this upending of an existing market can yield substantial positive externalities for users. Other "heavy industry" adjacent sectors, not so much. Healthcare and aviation, not so much.
Even SpaceX's vaunted "disruption" is just clever resource allocation; despite their iterative approach to building rockets being truly novel they're not market disruptors in the same way SV usually talks about them. And their approach has some very obvious flaws relative to more traditional companies like BO, which as of now has a lower failure-to-success ratio.
I don't think you'll find many providers clamoring for an AI-assisted app that hallucinates nonexistent diseases, there are plenty of those already out there that draw the ire of many physicians. Where the industry needs to innovate is in the insurance space, which is responsible for the majority of costs, and the captive market and cartel behavior thereof means that this is a policy and government issue, not something that can be solved with rote Silicon Valley style startup-initiated disruption; that I would predict would quickly turn into dysfunction and eventual failure.
Enshittification has done a lot of damage to the concept of "disrupting" markets. It's DOA in risk-averse fields.
Because insurance companies incentivize upward price momentum. The ones who innovate and bring the prices down are not rewarded for their efforts. Health inflation is higher than headline inflation because of this absence of price pressure
I sympthatise with the argument. We should test it against real world data.
Eg your argument would predict that healthcare price inflation is not as bad in areas with less insurance coverage. Eg for dental work (which is less often covered as far as I can tell), for (vanity) plastic surgery, or we can even check healthcare price inflation for vet care for pets.
I just looked it up, and apparently health care costs for pets has gone up in price even faster than for humans.
Pets typically don't have medical insurance, and any insurance that does exist there has a radically different regulatory regime than for humans.
Since 1980 for the US:
CPI has gone up by 3.16% on average per year (x4.17 in total). Human healthcare costs by 4.9% per year (x8.96 in total). And pet healthcare costs by 6.49% (or x17.87 in total).
Dental and vanity surgeries aren't happening in a vacuum. There are baseline costs eg. anesthesia, recovery medications, medical machinery etc which are all bloated due to the rest of industry not being under price pressure (rising tide lift all boats)
It's similar to how AI data center buildout race is raising the prices for consumer electronics in 2026 and beyond. The suppliers have no incentive to sell lower cost products to tiny niche
The uninsured medical market is actually quite efficient. e.g. lasik surgery can actually be done for very reasonable rates with full price transparency.
Lasik has gone down in real price over the years, yes.
But dental and vanity cosmetic surgery have gone up by that metric. Dental is less covered by insurance for most people. Vanity cosmetic insurance is covered for almost no one.
Vet care for pets has gone up a lot more than healthcare for humans.
This argument doesn’t make sense to me. Insurance companies are structurally incentivized to minimize payouts across the board. They want hospital bills lower, physician compensation lower, and patient payouts as small as possible. If insurers had unilateral power, total medical spending would collapse, not explode.
The real source of high medical costs is the entity that sets the hospital bill in the first place.
The explanation is much simpler than people want to admit, but emotionally uncomfortable: doctors and hospitals are paid more than the free market would otherwise justify. We hesitate to say this because they save lives, and we instinctively conflate moral worth with economic compensation. But markets don’t work that way.
Economics does not reward people based on what they “deserve.” It rewards scarcity. And physician labor is artificially scarce.
The supply of doctors is deliberately constrained. We are not operating in a free market here. Entry into the profession is made far more restrictive than is strictly necessary, not purely for safety, but to protect incumbents. This is classic supply-side restriction behavior, bordering on cartel dynamics.
We see similar behavior in law, but medicine is more insidious. Because medical practice genuinely requires guardrails to prevent harm and quackery, credentialing is non-negotiable. That necessity makes it uniquely easy to smuggle in protectionism under the banner of “safety.”
The result is predictable: restricted supply, elevated wages, and persistently high medical costs. The problem isn’t mysterious, and it isn’t insurance companies. It’s a supply bottleneck created and defended by the profession itself.
Insurance companies aren't innocent angels in this whole scenario either. When the hospital bill fucks them over they don't even blink twice when they turn around and fuck over the patient to bail themselves out. But make no mistake, insurance is the side effect, the profession itself is the core problem.
> This argument doesn’t make sense to me. Insurance companies are structurally incentivized to minimize payouts across the board. They want hospital bills lower, physician compensation lower, and patient payouts as small as possible. If insurers had unilateral power, total medical spending would collapse, not explode.
They absolutely do not.
They have their profit levels capped at 15% by law and regulation. That means if the insurer wants more absolute dollars of profit, prices must go up.
It also means that if they push prices down they necessarily have less funding to administer those plans, even if the needs are the same (same number of belly buttons, same patient demographics and state of health).
As you note there's also other variables, but this claim: "Insurance companies are structurally incentivized to minimize payouts across the board" is absolutely and categorically not so.
Physician reimbursement is only ~9% of national healthcare expenditures.
I tell you this with certainty as a 3rd year medical student: If physician wages go down and tuition stays as is, no one will do this. Intrinsic motivation to help people evaporates as soon as you see how enshittified healthcare in the US has become.
I do agree that medical school is far too restrictive to get into (For MD schools at least). However, if you want to make medical school easier to get into: Where will all those students rotate at for their clinical years? There aren't enough spots in hospitals to jam students in.
Stop taking aim at the people that sacrifice so much to help you. Take aim at the real drivers of healthcare expenditures: administrative bloat.
because in america at least, the supply of doctors is kept artificially low. that combined with exploding administrative headcount, means patients are getting pretty terrible, expensive service.
Physician compensation is around 9% of healthcare spending. The number of non-physician providers (NPs, PAs and specialists like physical therapists and podiatrists) has also exploded over the last 20 years. We have far more overall providers per capita than we did 20 years ago.
I don't think physician compensation per se is a good metric for capturing the effect of lack of providers, because some of the increased costs are due to the bottlenecks in the services per se, in terms of procedure costs and types of procedures offered. I also don't think the number of providers per se under the current regime, without deregulation or reregulation of practice boundaries, is representative of what would happen if there were changes in those boundaries. Adding more optometrists 5 years ago isn't the same as changing what they're allowed to do. It also doesn't address what cost increases would have been without an increase in the number of providers.
9% might also seem pretty big to me if it's out of all spending and doesn't include other provider compensation? What if overall healthcare costs went down, but physician compensation stayed the same? Would that then be a problem because it was an increased proportion of the total costs — fat left to be trimmed, so to speak?
There are many problems that don't have anything to do with providers per se, but I also don't think you can glean much by extrapolating to more of the same, especially compensation per se.
If lack of physicians is leading to an increase in costs, you’d expect to see physicians capturing a large part of that increase. There are situations where that wouldn’t hold, but it requires moving away from the simplest explanation that fits the data.
We don’t have to extrapolate from physician compensation though. We know that providers per capita have increased, but costs have continued to skyrocket. Therefore a lack of providers is not the immediate cause of the increase.
In addition to increasing the number of providers, the scope of practice for non-physician providers has almost universally increased.
All of this doesn’t prove that increasing the number of physicians wouldn’t lower costs some amount, but it does show that the increases over the last 20-30 years requires some other explanation.
Many people don’t get it, it’s really expensive, even in countries with non broken healthcare systems (not the us) costs increase rapidly and no one is sure how the systems will remain solvent with the same level of care given today. The way things are currently done are entrenched but not sustainable, that’s when disruptions are apt to appear.
It's really just not there yet. I've been in medical school for >3 years now and have been using the latest models with good prompting. They have gotten much better, but I still see misses that my classmates would easily catch. This is not acceptable in healthcare. It's certainly not getting 100% on all my assignments, which are a step below the complexity of real-world clinical practice.
Before medical school, I was not so sure of the quality of your average doc. Now having spent a year in clinical practice across various settings, I am extremely reassured. I can say with certainty that a US trained doctor is miles ahead of AI right now. The system sucks really bad though and forces physicians to churn patients, giving the impression that physicians don't pay attention/don't care/etc.
I mean, if we're talking Christensenian disruption, that happens in neglected markets rather than currently dysfunctional ones. There's no shortage of actors wronging money out of health care so there's not a disruptable space per se.
When I look at the US, the symptom -> diagnosis hypothesis is not anywhere near the most expensive bit. If you have a medical issue and AI works effectively for this then it saves you maybe one trip to your GP. Your insurance probably still requires your GP to provide a referral to a specialist. If insurance companies allow for AI to be used in place of a referral then you save this trip. But you still need all of the stuff to confirm a diagnosis. And you still need all of the treatment.
If you don't have a medical issue and an AI system tells you this then you save yourself a trip to a specialist and the associated diagnostic tests. Again, this saves a bit of money but is nowhere near the bulk of medical expenses. And it has to be able to do this without any diagnostic testing, just based off of your reported symptoms.
Even if AI diagnosis works flawlessly we save a bit of money but absolutely do not revolutionize the cost of the industry.
Salaries for healthcare workers make up only a small portion of expenditures. You do not want to avoid a trip to your GP for an AI system.
It'll be great at first while in development. But when profits need to be generated, seeing a specialist will get harder. There will be less wiggle room. I predict we will see more GP utilization.
Removing "some" doesn't make it worse. They didn't include "all" AI titles which it would. "Google removes AI health summaries after investigation finds dangerous flaws " is functionally equivalent to "Google removes some of its AI summaries after users’ health put at risk"
Oh, and also, the Ars article itself still contains the word "Some" (on my AB test). It's the headline on HN that left it out. So your complaint is entirely invalid: "Google removes some AI health summaries after investigation finds “dangerous” flaws"
Good. I typed in a search for some medication I was taking and Google's "AI" summary was bordering on criminal. The WebMD site had the correct info, as did the manufacturer's website. Google hallucinated a bunch of stuff about it, and I knew then that they needed to put a stop to LLMs slopping about anything to do with health or medical info.
in a way, all overconfident guessing is a better match for the result than hallucination or fabrication would be
"confabulation", though, seems perfect:
“Confabulation is distinguished from lying as there is no intent to deceive and the person is unaware the information is false. Although individuals can present blatantly false information, confabulation can also seem to be coherent, internally consistent, and relatively normal.”
I'm not sure why you would think I would blurt that out on the internet.
And are you sure it's giving you good info? "AI" is famously subject to hallucinations, so you may not be getting the "good info" you think you're getting. Be careful with "AI", it's not an all-seeing-all-knowing infallible oracle.
Google is really wrecking its brand with the search AI summaries thing, which is unbelievably bad compared to their Gemini offerings, including the free one. The continued existence of it is baffling.
Yeah. It's the final nail in the coffin of search, which now actively surfaces incorrect results when it isn't serving ads that usually deliberately pretend to be the site you're looking for. The only thing I use it for any more is to find a site I know exists but I don't know the URL of.
It's mystifying. A relative showed me a heavily AI-generated video claiming a Tesla wheelchair was coming (self-driving of course, with a sub-$800 price tag). I tried to Google it to quickly debunk and got an AI Overview confidently stating it was a real thing. The source it linked to: that same YouTube video!
The AI summaries clearly aren’t bad. I’m not sure what kind of weird shit you search for that you consider the summaries bad. I find them helpful and click through to the cited sources.
Not surprised. Another example is minecraft related queries. Im searching with the intention of eventually going to a certain wiki page at minecraft.wiki, but started to just read the summaries instead. It will combine fan forums discussing desired features/ideas with the actual game bible at minecraft.wiki - so it mixes one source of truth with one source of fantasy. Results in ridiculous inaccurate summaries.
I find its tricky with games, especially ones as updated as frequently as Minecraft over the years. I've had some of this trouble with OSRS. It brings in old info, or info from a League/Event that isn't relevant. Easier to just go to the insanely curated wiki.
I run a small business that buys from one of two suppliers of the items we need. The supplier has a TRASH website search feature. It's quicker to Google it.
Now that AI summaries exist, I have to scroll past half a page of result and nonsense about a Turkish oil company before I find the item I'm looking for.
I hate it. It's such a minor inconvenience, but it's just so annoying. Like a sore tooth.
What's interesting to me is that this kind of behavior -- slightly-buffleheaded synthesis of very large areas of discourse with widely varying levels of reliability/trustworthiness -- is actually sort of one of the best things about AI research, at least for me?
I'm pretty good at reading the original sources. But what I don't have in a lot of cases is a gut that tells me what's available. I'll search for some vague idea (like, "someone must have done this before") with the wrong jargon and unclear explanation. And the AI will... sort of figure it out and point me at a bunch of people talking about exactly the idea I just had.
Now, sometimes they're loons and the idea is wrong, but the search will tell me who the players are, what jargon they're using to talk about it, what the relevant controversies around the ideas are, etc... And I can take it from there. But without the AI it's actually a long road between "I bet this exists" and "Here's someone who did it right already".
Yeah, this is by far the biggest value I've gotten from LLMs - just pointing me to the area of literature neither me nor any of my friends have heard of, but which have spent a decade running about the problems we're running into.
In this case, all that matters is that the outputs aren't complete hallucination. Once you know the magic jargon, everything opens up easily with traditional search.
Or you can take the alternative approach, where Microsoft's own "Merl" support agent says it knows anything to do with Minecraft, and then replies to basically any gameplay question with "I don't know that".
I had a similar thing happen to me just today. A friend of mine had finished a book in a series. I have read the series but it was almost 10 years ago, and I needed a refresher with spoilers, so I went looking.
Well, some redditor had posted a comparison of a much later book in the series, and drawn all sorts of parallels and foreshadowing and references between this quite early book I was looking for and the much later one. It was an interesting post so it had been very popular.
The AI summary completely confused the two books because of this single reddit post, so the summary I got was hopelessly poisoned with plot points and characters that wouldn't show up until nearly the conclusion. It simply couldn't tell which book was which. It wasn't quite as ridiculous as having, say, Anakin Skywalker face Kylo Ren in a lightsaber duel, but it was definitely along those same lines of confusion.
Fortunately, I finished the later book recently enough to remember it, but it was like reading a fever dream.
A few months ago in a comment here on HN I speculated about the reason an old law might have been written the way it was, instead of more generally. If it had been written without the seemingly arbitrary restrictions it included there would have been no need for the new law that the thread was about.
A couple hours later I decided to ask an LLM if it could tell me. It quickly answered, giving the same reason that I had guessed in my HN comment.
I then clicked the two links it cited as sources. One was completely irrelevant. The other was a link to my HN comment.
Yeah, it happened recently for a kubernetes resource. I was searching for how to do something, and Google AI helpfully showed me a kubernetes resource that was exactly what I needed, and was designed to work exactly how I needed it.
Sadly, the resource didn't actually exist. It would have been perfect if it did, though!
At some point in time when asked how many Kurdish people live in Poland, Google's AI would say that several million, which was true, but only in a fantasy world conjured by a certain LARP group, who put a wiki on fandom.com.
How do you suggest to deal with Gemini? Extremely useful to understand whether something is worrying or not. Whether we like it or not, it’s a main participant to the discussion.
With robust fines based on % revenue whenever it breaks the law, would be my preference. I'm nit here to attempt solutions to Google's self-inflicted business-model challenges.
This "random output machine" is already in large use in medicine so why exactly not? Should I trust the young doctor fresh out of the Uni more by default or should I take advises from both of them with a grain of salt? I had failures and successes with both of them but lately I found Gemini to be extremely good at what it does.
There's a difference between a doctor (an expert in their field) using AI (specialising in medicine) and you (a lay person) using it to diagnose and treat yourself. In the US, it takes at least 10 years of studying (and interning) to become a doctor.
Even so, it's rather common for doctors to not be albe to diagonise correctly. It's a guessing game for them too. I don't know so much about US but it's a real problem in large parts of the world. As the comment stated, I would take anything a doctor says with a pinch of salt. Particularly so when the problem is not obvious.
This is really not that far off from the argument that "well, people make mistakes a lot, too, so really, LLMs are just like people, and they're probably conscious too!"
Yes, doctors make mistakes. Yes, some doctors make a lot of mistakes. Yes, some patients get misdiagnosed a bunch (because they have something unusual, or because they are a member of a group—like women, people of color, overweight people, or some combination—that American doctors have a tendency to disbelieve).
None of that means that it's a good idea to replace those human doctors with LLMs that can make up brand-new diseases that don't exist occasionally.
Nobody can (and should) stop you from learning and educating yourself. It however doesn't mean just because you can use Google or use AI, you think you can become a doctor:
Educating a user about their illness and treatment is a legitimate use case for AI, but acting on its advise to treat yourself or self-medicate would be plain stupidity. (Thankfully, self-medicating isn't as easy because most medication require a prescription. However, so called "alternate" medicines are often a grey area, even with regulations (for example, in India).
It takes 10 years of hard work to become a profound engineer too yet it doesn't prohibit us missing the things. That argument cannot hold. AI is already wide-spread in medical treatment.
An engineer is not a doctor, nor a doctor an engineer. Yes, AI is being used in medicines - as a tool for the professional - and that's the right use for it. Helping a radiologist read an X-Ray, MRI scan or CT Scan, helping a doctor create an effective treatment plan, warning a pharmacologist about unsafe combinations (dangerous drug interactions) when different medications are prescribed etc are all areas where an AI can make the job of a professional easier and better, and also help create better AI.
> This "random output machine" is already in large use in medicine
By doctors. It's like handling dangerous chemicals. If you know what you're doing you get some good results, otherwise you just melt your face off.
> Should I trust the young doctor fresh out of the Uni
You trust the process that got the doctor there. The knowledge they absorbed, the checks they passed. The doctor doesn't operate in a vacuum, there's a structure in place to validate critical decisions. Anyway you won't blindly trust one young doctor, if it's important you get a second opinion from another qualified doctor.
In the fields I know a lot about, LLMs fail spectacularly so, so often. Having that experience and knowing how badly they fail, I have no reason to trust them in any critical field where I cannot personally verify the output. A medical AI could enhance a trained doctor, or give false confidence to an inexperienced one, but on its own it's just dangerous.
No, I'm not asking you spend $150, I'm providing you the evidence your looking for. Mayo Clinic, probably one of the most prominent private clinics in the US, is using transformers in their workflow, and there's many other similar links you could find online, but you choose to remain ignorant. Congratulations
The existence of a course on this topic is NOT evidence of "large use". The contents of the course might contain such evidence, or they might contain evidence that LLM use is practically non-existent at this point (the flowery language used to describe the course is used for almost any course tangentially related to new technology in the business context, so that's not evidence either).
But your focus on the existence of this course as your only piece of evidence is evidence enough for me.
The "well we already have a bunch of people doing this and it would be difficult to introduce guardrails that are consistently effective so fuck it we ball" is one of the most toxic belief systems in the tech industry.
Apparently we should hire the Guardian to evaluate LLM output accuracy?
Why are these products being put out there for these kinds of things with no attempt to quantify accuracy?
In many areas AI has become this toy that we use because it looks real enough.
It sometimes works for some things in math and science because we test its output, but overall you don't go to Gemini and it says "there's a 80% chance this is correct". At least then you could evaluate that claim.
There's a kind of task LLMs aren't well suited to because there's no intrinsic empirical verifiability, for lack of a better way of putting it.
If it's giving out medical advice without a license, it should be banned from giving medical advice and the parent company fined or forced to retire it.
As a certified electrical engineer, the amount of times googles LLM suggested a thing that would have at minimum started a fire is staggering.
I have the capacity to know when it is wrong, but I teach this at university level. What worries me, are the people who are on the starting end of the Dunning-Kruger curve and needed that wrong advice to start "fixing" the spaces where this might become a danger to human life.
No information is superior to wrong information presented in a convincing way.
Different levels of capabilities. The summary feature in google uses a quick and inaccurate AI model. Were it to be a heavier model, we wouldn’t have this problem.
We would still have this problem. The heavier models make mistakes at too high a rate vs. a physician. Especially on imaging data. Real world data and patient presentations often deviate from the textbooks they are trained on.
That's a different class of problem. It will do just fine on text based queries spanning a few pages. Probably better than the average physician (average over all countries).
I do agree that LLM's are not there yet in the image part.
No sane, in the loop person would believe Google reviews anything, at least not with a human. I bet they have another AI reviewing the answers, and the only time a human enters the loop is when this second AI needs to be restarted.
Google for "malay people acne" or other acne-related queries. It will readily spit out the dumbest pseudo science you can find. The AI bot finds a lot of dumb shit on the internet which it serves back to you on the Google page. You can also ask it about the Kangen MLM water scam. Why do athletes drink Kangen water? "Improved Recovery Time" Sure buddy.
Going offtopic: The "health benefits of circumcision" bogus has existed for decades. The search engines are returning the results of bogus information, because the topic is mostly relevant for its social and religious implications.
I am related with the topic, and discussion is similar to topics on politics: Most people don't care and will stay quiet while a very aggresive group will sell it as a panacea.
the problem isn't that search engines are polluted; that's well known. The problem is that people perceive these AI responses as something greater than a search query; they view it as an objective view point that was reasoned out by some sound logical method -- and anyone that understands the operation of LLMs knows that they don't really do that, except for some very specific edge examples.
But only for some highly specific searches, when what it should be doing is checking if it's any sort of medical query and keeping the hell out of it because it can't guarantee reliability.
It's still baffling to me that the world's biggest search company has gone all-in on putting a known-unreliable summary at the top of its results.
Tangent, but some people I know have been downloading their genomes from 23andme and asking Gemini via Antigravity to analyze it. "If you don't die of heart disease by 50, you'll probably live to be 100."
As accurate as our knowledge of genetics, which is not very outside of the identified set of pathological genes associated with hereditary disorders.
Your genome is very complex and we don’t have a model of how every gene interacts with every other and how they’re affected by your environment. Geneticists are working on it, but it’s not here yet.
And remember that 23andMe, Ancestry, and most other services only sequence around 1% of your genome.
Part of genetics is pattern matching, and last time I checked I still can't find a model that can correctly solve hard Sudokus (well, assuming you don't pick a coding model that writes a Sudoku solver.. maybe some of them are trying to do genetics by doing correct algorithms), a trivial job if you write a program that is designed to do it.
Are you asking for you in particular? It's certainly not accurate in general that anyone that made it to 50 is likely to live to 100.
One I heard was if you make it to 80 you have a 50% chance to make it to 90. If you make it to 90 you have a 50% chance to make it to 95. From 95 to 97.5 again 50% chance. That for the general population, in a 1st world country though, not any individual.
The assessment he got was "you have supergenerian genes, but also this one that's really sensitive to heart disease. If you can keep you plaque in check, you're almost invincible (to other popular causes of mortality)."
It cited whatever gene it said he had that made that so.
23&Me data is simply not accurate enough to make reasonable predictions about health outcomes.
I have a whole genome and nothing Google has built has been able to do anything useful with it, medically speaking. I could use DeepVariant to re-map all the raw reads, it would only slightly increase the accuracy of the estimate of my genome sequence. When I met with genetic counselors, they analyzed my genome and told me I had no known markers for any disease (and they also told me they Google all the unique variants that show up in the report).
(for what it's worth, I literally went to work at Google to improve their medical/health/genomics research, and after working on it a few years I concluded that the entire genomics field is about 90% fantasy. If you want actionable data, there are a small number of well-regulated tests that can help in a limited set of circumstances, but those aren't whole genome tests).
Claude just added Health Connect integration for Android.
Meanwhile Copilot launched a full bot for it:
"Dos and don’ts of medical AI
While AI is a useful tool that can help you understand medical information, it’s important to clarify what it’s designed to do (and what it isn’t).
Dos:
Use AI as a reliable guide for finding doctors and understanding care options.
Let AI act as an always available medical assistant that explains information clearly.
Use AI as a transparent, unbiased source of clinically validated health content.
Don’ts:
Don’t use AI for medical diagnosis. If you’re concerned you may have a medical issue, you should seek the help of a medical professional.
Don’t replace your doctor or primary care provider with an “AI doctor”. AI isn’t a doctor. You should always consult a professional before making any medical decisions.
This clarity is what makes Copilot safe"
https://www.microsoft.com/en-us/microsoft-copilot/for-indivi...
I'm telling you all this as a medical student that has used the latest and greatest models with proper prompting for the past 3 years in school:
There are a ton of misses. Especially on imaging. LLMs are not ready for consumer-facing health information yet. My guess is ~ 3-5 years. Right now, I see systems implementing note writing with LLMs, which is hit or miss (but will rapidly improve). Physicians want 1:1 customization. Have someone sit with them and talk through how they like their notes/set it up so the LLMs produce notes like that. Notes need to be customized at the physician level.
Also, the electronic health records any AI is trained on is loaded to the brim with borderline fraud/copy paste notes. That's going to have to be reconciled. Do we have the LLMs add "Cranial Nerves II-X intact" even though the physician did not actually assess that? The physician would have documented that... No? But then you open up the physician to liability, which is a no go for adopting software.
Building a SaaS MVP that's 80% of the way there? Sure. But medicine is not an MVP you cram into a pitch deck for a VC. 80% of the way there does not cut it here, especially if we're talking about consumer facing applications. Remember, the average American reads at a 6th grade reading level. Pause and let that sink in. You're probably surrounded by college educated people like yourself. It was a big shock when I started seeing patients, even though I am the first in my family to go to college. Any consumer-facing health AI tool needs to be bulletproof!!
Big Tech will not deliver us a healthcare utopia. Do not buy into their propaganda. They are leveraging post-pandemic increases in mistrust towards physicians as a springboard for half-baked solutions. Want to make $$$ doing the same thing? Do it in a different industry.
dreadsword | a day ago
bandrami | a day ago
miltonlost | a day ago
zdragnar | a day ago
kunai | a day ago
eru | a day ago
wswin | a day ago
bandrami | a day ago
__loam | a day ago
gradus_ad | a day ago
And many single payer systems around the world only appear to work as well as they do because the US effectively subsidizes medical costs through its own out of control prices.
__loam | 22 hours ago
ikr678 | 20 hours ago
plagiarist | a day ago
eru | a day ago
If we could get healthcare to that level, it would be great.
For a less extreme example: Wal-Mart and Amazon have made plenty of people very rich, and they charge customers for their goods; but their entrance into the markets have arguable brought down prices.
__loam | a day ago
eru | 23 hours ago
__loam | 22 hours ago
eru | 17 hours ago
> It should be a regulated utility like electricity or railroads, we should have a public alternative like the post office is to UPS, or it should be nationalized.
I agree that electricity and railroads should be regulated like Google Search.
It's really weird that snail mail in the US is a government monopoly. When even social democratic Germany managed to privatise them.
> The situation gets more dire when you consider their browser monopoly.
Don't a lot of people in the US use iPhones? They don't ship with Chrome as the default browser, do they?
(And yes, Safari is built on top of the same open source engine as Chrome. But you can hardly call using the same open source project a 'monopoly'. Literally anyone can fork it.)
There's also plenty of other browsers available.
__loam | 9 hours ago
A public mail service is required by our constitution. It's cheaper than the private options and often the only option for many rural areas. It's not a monopoly.
eru | 2 hours ago
Where does it say so in your constitution? All I can find is the postal clause which Wikipedia summarises as follows, but whose full text isn't much longer:
> Article I, Section 8, Clause 7, of the United States Constitution, the Postal Clause, authorizes the establishment of "post offices and post roads"[1] by the country's legislature, the Congress.
https://en.wikipedia.org/wiki/Postal_Clause
The Postal Clause certainly allows the government to run a public postal service, but I don't see how the constitution _requires_ it. It doesn't even require the federal government to regulate postal services, it merely allows it.
Perhaps I missed something?
> It's cheaper than the private options and often the only option for many rural areas.
If you want to subsidise rural areas, I would suggest to do so openly, transparently and from general taxation. At least general taxation is progressive etc. Instead of just making urban folks pay more for their mail, whether they be rich or poor.
I would also suggest only subsidising poor rural areas. Rich rural areas don't need our help.
> It's not a monopoly.
Compare and contrast what USPS has to say https://about.usps.com/universal-postal-service/universal-se...
plagiarist | a day ago
eru | 23 hours ago
And why do customers come back to shop there?
plagiarist | 21 hours ago
Customers continue shopping there because human beings are typically incapable of accepting a short-term loss (higher price) for a long-term gain (product lasts more than three uses).
eru | 17 hours ago
That's a weird metric. If tomorrow Wal-Mart laid off all employees and replaced them with robots, they would surely be worse off, but by your metric Wal-Mart would look less evil?
> Customers continue shopping there because human beings are typically incapable of accepting a short-term loss (higher price) for a long-term gain (product lasts more than three uses).
Groceries typically only last one use.
plagiarist | 13 hours ago
Likewise, I would not use my flippant 3 times metric regarding durability to cover the quality of produce.
eru | 2 hours ago
You have to look at the counterfactual of what these people would do, if Wal-Mart weren't around. You seem to implicitly assume that they'd be getting higher paying jobs somewhere else (so they wouldn't have to rely on welfare)? If so, what's stopping those people from switching to these better jobs right now, even while Wal-Mart is still around?
And sure, let's disregard how many times you can eat your groceries. That was a cheap shot. However I think quality vs price trade-off is something customers have to make for themselves anyway. Who am I to judge their choices?
acdha | a day ago
Google searches cost many billions of dollars: your confusion is because the customer isn’t the person searching but the advertisers paying to influence them. Healthcare can’t work like that not just because the real costs are both much higher and resistant to economies of scale but, critically, there aren’t people with deep pockets lining up to pay for you to be healthy. That’s why every other developed country sees better results for less money: keeping people healthy is a social good, and political forces work for that better than raw economic incentives.
bandrami | 21 hours ago
ndsipa_pomu | 18 hours ago
reaperducer | a day ago
Yeah, because we saw what a great job the tech bros did making government more efficient.
shaky-carrousel | 21 hours ago
dyauspitr | a day ago
gmueckl | a day ago
eru | a day ago
Profit isn't even a big part of the overall revenue.
> Mandate at least decent minimal coverage standards
I assume you want higher coverage standards than what currently exists? Independently of whether that would be the morally right thing to do (or not), it would definitely increase prices.
> and large insurance pools that must span age groups and risk groups.
Why does your insurance need a pool? An actuary can tell you the risk, and you can price according to that. No need for any pooling. Pooling is just something you do, when you don't have good models (or when regulations forces you).
FireBeyond | a day ago
Wuh? The more diverse the pool, the lower the risk. Your way of thinking will very quickly lead to "LiveCheap: the health insurance for fit, healthy under 30s only" for dollars a month, and "SucksToBeYou: the health insurance for the geriatric and chronically disabled" for the low low cost of "everything you have to give".
eru | 23 hours ago
There's insurance which allows you to convert an uncertain danger into a known payment. And then there's welfare and redistribution.
By all means, please run some means testing and give the poor and sick or disabled extra money. Or even just outright pay their insurance premiums.
But please finance that from general taxation, which is already progressive. Instead of effectively slapping an arbitrary tax on healthy people, whether they be rich or poor. And please don't give rich people extra stealth welfare, just because they are in less than ideal health, either.
Just charge people insurance premiums in line with their expected health outcomes, and help poor people with the premiums using funds from general taxation. (Where poor here means: take their income and make an adjustment for disability etc.)
We _want_ the guy who loses 5kg and gives up smoking to get lower insurance premiums. That's how you set incentives right.
> The more diverse the pool, the lower the risk.
No. The diversification comes from the insurance company running lots of uncorrelated contracts at the same time and having a big balance sheets. For that, it doesn't matter whether it's a pool of similar insurance contracts, or whether they have bets on your insurance contract, and on the price of rice in China, and playing the bookie on some sports outcomes etc. In fact, the more diversified they are, the better (in principle).
But that diversification is completely independent of the pricing of your individual insurance contract.
Have a look at Warren Buffett's 'March Madness' challenge, where he famously challenges people to predict all 67 outcomes of some basketball games to win a billion dollar. Warren Buffet ain't no fool: he doesn't need a pool, he can price the risk of someone winning this one off challenge.
More generally, have a look at Prize indemnity insurance https://en.wikipedia.org/wiki/Prize_indemnity_insurance which helps insure many one-off events.
dyauspitr | 20 hours ago
eru | 18 hours ago
In any case, what you are saying is only true, if you buy your health insurance second to second on the spot market.
Insurance companies are more than happy to enter long running contracts, where you both agree today on (the algorithm for) the premiums for the next twenty years or even until the rest of your life. That's pretty common with life insurance and disability insurance already.
The above already exists, but if you allow some speculation: you could even envision people buying insurance for their kids before conceiving them. That way you don't have to worry about pre-existing conditions.
(Well, if the parents already have heritable conditions that would make the kids more likely to have expensive medical problems, those would push up their premiums. But then: perhaps these people should think twice about burdening a potential kid with these issues.
Compare how in Cyprus where sickle cell anemia is prevalent, even the Catholic church demand you get screened, before they'll marry you.)
If you really want specialised in-kind welfare, you can get people a voucher for the catastrophic version of 'unconceived baby insurance'.
Basically, you can buy insurance against insurance premiums being expensive.
FireBeyond | 11 hours ago
Well, there I entirely agree with you - health insurance as it exists in the US now is "insurance in name only".
kunai | a day ago
> large insurance pools that must span age groups and risk groups.
What you describe (community rating) has been tried and it works. But it requires that a lot of young, healthy people enroll, and seniors receive most of the care. In an inverted demographic pyramid like most Western economies have, this is a ticking time bomb, so costs will continue to rise.
> Mandate at least decent minimal coverage standards
I think a better solution is to allow the government to threaten in negotiating prices with companies as Canada does; it greatly reduces rent-seeking behavior by pharmaceutical companies while allowing them to continue earning profits and innovating. (I understand a lot of the complaints against big pharma but they are actually one of the few sectors of the economy that doesn't park their wealth and actually uses it for substantive R&D, despite what the media will tell you, and countless lives have been saved because of pharma company profits)
Essentially the gist of what I'm saying, as someone who has been involved with and studied this industry for the better part of five years, is that it's much more complex than what meets the eye.
bandrami | a day ago
kunai | a day ago
Even SpaceX's vaunted "disruption" is just clever resource allocation; despite their iterative approach to building rockets being truly novel they're not market disruptors in the same way SV usually talks about them. And their approach has some very obvious flaws relative to more traditional companies like BO, which as of now has a lower failure-to-success ratio.
I don't think you'll find many providers clamoring for an AI-assisted app that hallucinates nonexistent diseases, there are plenty of those already out there that draw the ire of many physicians. Where the industry needs to innovate is in the insurance space, which is responsible for the majority of costs, and the captive market and cartel behavior thereof means that this is a policy and government issue, not something that can be solved with rote Silicon Valley style startup-initiated disruption; that I would predict would quickly turn into dysfunction and eventual failure.
Enshittification has done a lot of damage to the concept of "disrupting" markets. It's DOA in risk-averse fields.
FireBeyond | a day ago
Her bill before "insurance negotiated prices" was $59,000. Effectively $1,000/hr, 24/7.
bandrami | 14 hours ago
harpratap | a day ago
eru | a day ago
Eg your argument would predict that healthcare price inflation is not as bad in areas with less insurance coverage. Eg for dental work (which is less often covered as far as I can tell), for (vanity) plastic surgery, or we can even check healthcare price inflation for vet care for pets.
eru | 22 hours ago
Pets typically don't have medical insurance, and any insurance that does exist there has a radically different regulatory regime than for humans.
Since 1980 for the US:
CPI has gone up by 3.16% on average per year (x4.17 in total). Human healthcare costs by 4.9% per year (x8.96 in total). And pet healthcare costs by 6.49% (or x17.87 in total).
manuelmoreale | 21 hours ago
eru | 17 hours ago
harpratap | 21 hours ago
It's similar to how AI data center buildout race is raising the prices for consumer electronics in 2026 and beyond. The suppliers have no incentive to sell lower cost products to tiny niche
eru | 17 hours ago
terminalshort | 19 hours ago
eru | 17 hours ago
But dental and vanity cosmetic surgery have gone up by that metric. Dental is less covered by insurance for most people. Vanity cosmetic insurance is covered for almost no one.
Vet care for pets has gone up a lot more than healthcare for humans.
threethirtytwo | a day ago
The real source of high medical costs is the entity that sets the hospital bill in the first place.
The explanation is much simpler than people want to admit, but emotionally uncomfortable: doctors and hospitals are paid more than the free market would otherwise justify. We hesitate to say this because they save lives, and we instinctively conflate moral worth with economic compensation. But markets don’t work that way.
Economics does not reward people based on what they “deserve.” It rewards scarcity. And physician labor is artificially scarce.
The supply of doctors is deliberately constrained. We are not operating in a free market here. Entry into the profession is made far more restrictive than is strictly necessary, not purely for safety, but to protect incumbents. This is classic supply-side restriction behavior, bordering on cartel dynamics.
See, for example: https://petrieflom.law.harvard.edu/2022/03/15/ama-scope-of-p...
We see similar behavior in law, but medicine is more insidious. Because medical practice genuinely requires guardrails to prevent harm and quackery, credentialing is non-negotiable. That necessity makes it uniquely easy to smuggle in protectionism under the banner of “safety.”
The result is predictable: restricted supply, elevated wages, and persistently high medical costs. The problem isn’t mysterious, and it isn’t insurance companies. It’s a supply bottleneck created and defended by the profession itself.
Insurance companies aren't innocent angels in this whole scenario either. When the hospital bill fucks them over they don't even blink twice when they turn around and fuck over the patient to bail themselves out. But make no mistake, insurance is the side effect, the profession itself is the core problem.
FireBeyond | a day ago
They absolutely do not.
They have their profit levels capped at 15% by law and regulation. That means if the insurer wants more absolute dollars of profit, prices must go up.
It also means that if they push prices down they necessarily have less funding to administer those plans, even if the needs are the same (same number of belly buttons, same patient demographics and state of health).
As you note there's also other variables, but this claim: "Insurance companies are structurally incentivized to minimize payouts across the board" is absolutely and categorically not so.
thepotatodude | 5 hours ago
I tell you this with certainty as a 3rd year medical student: If physician wages go down and tuition stays as is, no one will do this. Intrinsic motivation to help people evaporates as soon as you see how enshittified healthcare in the US has become.
I do agree that medical school is far too restrictive to get into (For MD schools at least). However, if you want to make medical school easier to get into: Where will all those students rotate at for their clinical years? There aren't enough spots in hospitals to jam students in.
Stop taking aim at the people that sacrifice so much to help you. Take aim at the real drivers of healthcare expenditures: administrative bloat.
sergiotapia | a day ago
sarchertech | 23 hours ago
Lack of providers isn’t what’s driving up costs.
derbOac | 20 hours ago
9% might also seem pretty big to me if it's out of all spending and doesn't include other provider compensation? What if overall healthcare costs went down, but physician compensation stayed the same? Would that then be a problem because it was an increased proportion of the total costs — fat left to be trimmed, so to speak?
There are many problems that don't have anything to do with providers per se, but I also don't think you can glean much by extrapolating to more of the same, especially compensation per se.
thepotatodude | 5 hours ago
sarchertech | 3 hours ago
We don’t have to extrapolate from physician compensation though. We know that providers per capita have increased, but costs have continued to skyrocket. Therefore a lack of providers is not the immediate cause of the increase.
In addition to increasing the number of providers, the scope of practice for non-physician providers has almost universally increased.
All of this doesn’t prove that increasing the number of physicians wouldn’t lower costs some amount, but it does show that the increases over the last 20-30 years requires some other explanation.
seanmcdirmid | a day ago
scarab92 | a day ago
It's the self-driving cars debate all over again.
thepotatodude | 6 hours ago
Before medical school, I was not so sure of the quality of your average doc. Now having spent a year in clinical practice across various settings, I am extremely reassured. I can say with certainty that a US trained doctor is miles ahead of AI right now. The system sucks really bad though and forces physicians to churn patients, giving the impression that physicians don't pay attention/don't care/etc.
bandrami | 21 hours ago
omnicognate | 20 hours ago
Not sure if this is a typo (of wringing) or a pun, but it's apt either way.
UncleMeat | 15 hours ago
If you don't have a medical issue and an AI system tells you this then you save yourself a trip to a specialist and the associated diagnostic tests. Again, this saves a bit of money but is nowhere near the bulk of medical expenses. And it has to be able to do this without any diagnostic testing, just based off of your reported symptoms.
Even if AI diagnosis works flawlessly we save a bit of money but absolutely do not revolutionize the cost of the industry.
thepotatodude | 6 hours ago
It'll be great at first while in development. But when profits need to be generated, seeing a specialist will get harder. There will be less wiggle room. I predict we will see more GP utilization.
xnx | a day ago
‘Dangerous and alarming’: Google removes some of its AI summaries after users’ health put at risk: https://www.theguardian.com/technology/2026/jan/11/google-ai...
miltonlost | a day ago
Oh, and also, the Ars article itself still contains the word "Some" (on my AB test). It's the headline on HN that left it out. So your complaint is entirely invalid: "Google removes some AI health summaries after investigation finds “dangerous” flaws"
jnamaya | a day ago
xxs | 20 hours ago
leptons | a day ago
chrisjj | a day ago
Terretta | a day ago
in a way, all overconfident guessing is a better match for the result than hallucination or fabrication would be
"confabulation", though, seems perfect:
“Confabulation is distinguished from lying as there is no intent to deceive and the person is unaware the information is false. Although individuals can present blatantly false information, confabulation can also seem to be coherent, internally consistent, and relatively normal.”
https://en.wikipedia.org/wiki/Confabulation
* insofar as “guess” conveys an attempt to be probably in the zone
terminalshort | 19 hours ago
leptons | 10 hours ago
And are you sure it's giving you good info? "AI" is famously subject to hallucinations, so you may not be getting the "good info" you think you're getting. Be careful with "AI", it's not an all-seeing-all-knowing infallible oracle.
terminalshort | 4 hours ago
thepotatodude | 6 hours ago
-Med student
terminalshort | 4 hours ago
- guy with narcolepsy who was incorrectly diagnosed by a dozen or so doctors
jeffbee | a day ago
gvedem | a day ago
what | 23 hours ago
benregenspan | a day ago
what | 23 hours ago
xxs | 20 hours ago
InMice | a day ago
yakattak | a day ago
Loughla | a day ago
Now that AI summaries exist, I have to scroll past half a page of result and nonsense about a Turkish oil company before I find the item I'm looking for.
I hate it. It's such a minor inconvenience, but it's just so annoying. Like a sore tooth.
lelandfe | a day ago
https://google.com/search?q=parkas&udm=14
ajross | a day ago
I'm pretty good at reading the original sources. But what I don't have in a lot of cases is a gut that tells me what's available. I'll search for some vague idea (like, "someone must have done this before") with the wrong jargon and unclear explanation. And the AI will... sort of figure it out and point me at a bunch of people talking about exactly the idea I just had.
Now, sometimes they're loons and the idea is wrong, but the search will tell me who the players are, what jargon they're using to talk about it, what the relevant controversies around the ideas are, etc... And I can take it from there. But without the AI it's actually a long road between "I bet this exists" and "Here's someone who did it right already".
sdenton4 | 20 hours ago
In this case, all that matters is that the outputs aren't complete hallucination. Once you know the magic jargon, everything opens up easily with traditional search.
lazide | 19 hours ago
I’ve had it ‘dream’ up entire fake products, APIs, and even libraries before.
kmeisthax | a day ago
da_chicken | a day ago
Well, some redditor had posted a comparison of a much later book in the series, and drawn all sorts of parallels and foreshadowing and references between this quite early book I was looking for and the much later one. It was an interesting post so it had been very popular.
The AI summary completely confused the two books because of this single reddit post, so the summary I got was hopelessly poisoned with plot points and characters that wouldn't show up until nearly the conclusion. It simply couldn't tell which book was which. It wasn't quite as ridiculous as having, say, Anakin Skywalker face Kylo Ren in a lightsaber duel, but it was definitely along those same lines of confusion.
Fortunately, I finished the later book recently enough to remember it, but it was like reading a fever dream.
tzs | a day ago
A couple hours later I decided to ask an LLM if it could tell me. It quickly answered, giving the same reason that I had guessed in my HN comment.
I then clicked the two links it cited as sources. One was completely irrelevant. The other was a link to my HN comment.
aucisson_masque | 19 hours ago
InMice | 4 hours ago
cortesoft | 22 hours ago
Sadly, the resource didn't actually exist. It would have been perfect if it did, though!
Tade0 | 21 hours ago
At some point in time when asked how many Kurdish people live in Poland, Google's AI would say that several million, which was true, but only in a fantasy world conjured by a certain LARP group, who put a wiki on fandom.com.
orwin | 19 hours ago
myhf | a day ago
https://www.fda.gov/medical-devices/digital-health-center-ex...
eastbound | a day ago
jessetemp | 23 hours ago
Realistically, sign a EULA waiving your rights because their AI confabulates medical advice
overfeed | 22 hours ago
With robust fines based on % revenue whenever it breaks the law, would be my preference. I'm nit here to attempt solutions to Google's self-inflicted business-model challenges.
arkh | 20 hours ago
Don't. I do not ask my mechanic for medical advice, why would I ask a random output machine?
menaerus | 20 hours ago
thisislife2 | 20 hours ago
vharish | 19 hours ago
danaris | 8 hours ago
This is really not that far off from the argument that "well, people make mistakes a lot, too, so really, LLMs are just like people, and they're probably conscious too!"
Yes, doctors make mistakes. Yes, some doctors make a lot of mistakes. Yes, some patients get misdiagnosed a bunch (because they have something unusual, or because they are a member of a group—like women, people of color, overweight people, or some combination—that American doctors have a tendency to disbelieve).
None of that means that it's a good idea to replace those human doctors with LLMs that can make up brand-new diseases that don't exist occasionally.
schiffern | 18 hours ago
thisislife2 | 16 hours ago
- Bihar teen dies after ‘fake doctor’ conducts surgery using YouTube tutorial: Report - https://www.hindustantimes.com/india-news/bihar-teen-dies-af...
- Surgery performed while watching YouTube video leaves woman dead - https://www.tribuneindia.com/news/uttar-pradesh/surgery-perf...
- Woman dies after quack delivers her baby while watching YouTube videos - https://www.thehindu.com/news/national/bihar/in-bihar-woman-...
Educating a user about their illness and treatment is a legitimate use case for AI, but acting on its advise to treat yourself or self-medicate would be plain stupidity. (Thankfully, self-medicating isn't as easy because most medication require a prescription. However, so called "alternate" medicines are often a grey area, even with regulations (for example, in India).
menaerus | 18 hours ago
thisislife2 | 16 hours ago
menaerus | 15 hours ago
TitaRusell | 6 hours ago
Nobody at Google gives a flying fuck.
close04 | 19 hours ago
By doctors. It's like handling dangerous chemicals. If you know what you're doing you get some good results, otherwise you just melt your face off.
> Should I trust the young doctor fresh out of the Uni
You trust the process that got the doctor there. The knowledge they absorbed, the checks they passed. The doctor doesn't operate in a vacuum, there's a structure in place to validate critical decisions. Anyway you won't blindly trust one young doctor, if it's important you get a second opinion from another qualified doctor.
In the fields I know a lot about, LLMs fail spectacularly so, so often. Having that experience and knowing how badly they fail, I have no reason to trust them in any critical field where I cannot personally verify the output. A medical AI could enhance a trained doctor, or give false confidence to an inexperienced one, but on its own it's just dangerous.
Timon3 | 18 hours ago
Where does "large use" of LLMs in medicine exist? I'd like to stay far away from those places.
I hope you're not referring to machine learning in general, as there are worlds of differences between LLMs and other "classical" ML use cases.
menaerus | 17 hours ago
CaptainZapp | 16 hours ago
Timon3 | 15 hours ago
menaerus | 15 hours ago
Timon3 | 14 hours ago
But your focus on the existence of this course as your only piece of evidence is evidence enough for me.
UncleMeat | 15 hours ago
dpoloncsak | 13 hours ago
- Yes. All doctors advice should be taken cautiously, and every doctor recommends you get a second opinion for that exact reason.
derbOac | 20 hours ago
Why are these products being put out there for these kinds of things with no attempt to quantify accuracy?
In many areas AI has become this toy that we use because it looks real enough.
It sometimes works for some things in math and science because we test its output, but overall you don't go to Gemini and it says "there's a 80% chance this is correct". At least then you could evaluate that claim.
There's a kind of task LLMs aren't well suited to because there's no intrinsic empirical verifiability, for lack of a better way of putting it.
lazide | 19 hours ago
totetsu | 19 hours ago
ndsipa_pomu | 19 hours ago
atoav | 18 hours ago
I have the capacity to know when it is wrong, but I teach this at university level. What worries me, are the people who are on the starting end of the Dunning-Kruger curve and needed that wrong advice to start "fixing" the spaces where this might become a danger to human life.
No information is superior to wrong information presented in a convincing way.
usefulposter | 18 hours ago
"Whether we like it or not" is LLM inevitabilism.
https://news.ycombinator.com/item?id=44567857
protocolture | 18 hours ago
schiffern | 18 hours ago
Can't put that genie back in the bottle, no matter how much the powers-that-be may wish. Such is the nature of (technological) genies.
The only way to 'stop' LLMs is to invent something better.
somewhereoutth | 10 hours ago
ipython | a day ago
So interesting to see the vastly different approaches to AI safety from all the frontier labs.
_cs2017_ | a day ago
Aren't they both searching various online sources for relevant information and feeding that into the LLM?
simianwords | 12 hours ago
thepotatodude | 6 hours ago
-Med student
simianwords | 5 hours ago
I do agree that LLM's are not there yet in the image part.
Terretta | a day ago
Notice how little this sentence says about whether anything is any good.
user34283 | 19 hours ago
Like you could have a few days of -3C, for today it goes up to +5C, and the "AI Weather report" tells you it's going to be a chilly day or something.
I never saw this feature provide any useful information whatsoever.
63stack | 15 hours ago
bjourne | a day ago
Also try "health benefits of circumcision"...
catlikesshrimp | a day ago
Going offtopic: The "health benefits of circumcision" bogus has existed for decades. The search engines are returning the results of bogus information, because the topic is mostly relevant for its social and religious implications.
I am related with the topic, and discussion is similar to topics on politics: Most people don't care and will stay quiet while a very aggresive group will sell it as a panacea.
serf | a day ago
SirIsaacGluten | a day ago
FireBeyond | a day ago
nickphx | a day ago
mannykannot | a day ago
hsuduebc2 | a day ago
Nursie | a day ago
It's still baffling to me that the world's biggest search company has gone all-in on putting a known-unreliable summary at the top of its results.
chanux | a day ago
But Alas, infinite growth or nothing is the name of the game now.
[1] Well, not entirely thanks to people investigating.
bsimpson | 22 hours ago
I wonder how accurate it is.
sitharus | 21 hours ago
Your genome is very complex and we don’t have a model of how every gene interacts with every other and how they’re affected by your environment. Geneticists are working on it, but it’s not here yet.
And remember that 23andMe, Ancestry, and most other services only sequence around 1% of your genome.
CJefferson | 21 hours ago
Part of genetics is pattern matching, and last time I checked I still can't find a model that can correctly solve hard Sudokus (well, assuming you don't pick a coding model that writes a Sudoku solver.. maybe some of them are trying to do genetics by doing correct algorithms), a trivial job if you write a program that is designed to do it.
socalgal2 | 20 hours ago
One I heard was if you make it to 80 you have a 50% chance to make it to 90. If you make it to 90 you have a 50% chance to make it to 95. From 95 to 97.5 again 50% chance. That for the general population, in a 1st world country though, not any individual.
bsimpson | 11 hours ago
It cited whatever gene it said he had that made that so.
thepotatodude | 6 hours ago
dekhn | 3 hours ago
I have a whole genome and nothing Google has built has been able to do anything useful with it, medically speaking. I could use DeepVariant to re-map all the raw reads, it would only slightly increase the accuracy of the estimate of my genome sequence. When I met with genetic counselors, they analyzed my genome and told me I had no known markers for any disease (and they also told me they Google all the unique variants that show up in the report).
(for what it's worth, I literally went to work at Google to improve their medical/health/genomics research, and after working on it a few years I concluded that the entire genomics field is about 90% fantasy. If you want actionable data, there are a small number of well-regulated tests that can help in a limited set of circumstances, but those aren't whole genome tests).
inquirerGeneral | 20 hours ago
Meanwhile Copilot launched a full bot for it:
"Dos and don’ts of medical AI While AI is a useful tool that can help you understand medical information, it’s important to clarify what it’s designed to do (and what it isn’t).
Dos: Use AI as a reliable guide for finding doctors and understanding care options. Let AI act as an always available medical assistant that explains information clearly. Use AI as a transparent, unbiased source of clinically validated health content. Don’ts: Don’t use AI for medical diagnosis. If you’re concerned you may have a medical issue, you should seek the help of a medical professional. Don’t replace your doctor or primary care provider with an “AI doctor”. AI isn’t a doctor. You should always consult a professional before making any medical decisions. This clarity is what makes Copilot safe" https://www.microsoft.com/en-us/microsoft-copilot/for-indivi...
utopiah | 18 hours ago
n8m8 | 10 hours ago
thepotatodude | 6 hours ago
There are a ton of misses. Especially on imaging. LLMs are not ready for consumer-facing health information yet. My guess is ~ 3-5 years. Right now, I see systems implementing note writing with LLMs, which is hit or miss (but will rapidly improve). Physicians want 1:1 customization. Have someone sit with them and talk through how they like their notes/set it up so the LLMs produce notes like that. Notes need to be customized at the physician level.
Also, the electronic health records any AI is trained on is loaded to the brim with borderline fraud/copy paste notes. That's going to have to be reconciled. Do we have the LLMs add "Cranial Nerves II-X intact" even though the physician did not actually assess that? The physician would have documented that... No? But then you open up the physician to liability, which is a no go for adopting software.
Building a SaaS MVP that's 80% of the way there? Sure. But medicine is not an MVP you cram into a pitch deck for a VC. 80% of the way there does not cut it here, especially if we're talking about consumer facing applications. Remember, the average American reads at a 6th grade reading level. Pause and let that sink in. You're probably surrounded by college educated people like yourself. It was a big shock when I started seeing patients, even though I am the first in my family to go to college. Any consumer-facing health AI tool needs to be bulletproof!!
Big Tech will not deliver us a healthcare utopia. Do not buy into their propaganda. They are leveraging post-pandemic increases in mistrust towards physicians as a springboard for half-baked solutions. Want to make $$$ doing the same thing? Do it in a different industry.