I think the crux of the author's argument is that LLMs are bad because people use them for offloading both their thought and their acquisition of knowledge? The example they give is for education:
Look at education. ChatGPT may be the world’s largest learning platform, but students are using it en masse to avoid the effortful thinking that’s necessary to build their durable knowledge. Evidence continues to mount demonstrating the negative impact of AI tools within education settings (and Gen Z generally hates it). A recent study from China revealed that thousands of students essentially stopped doing their homework once they started using AI (which substantially harmed their learning). There’s even evidence that when students use large language models for supposed learning purposes, they become habituated to relaxing their judgment and critical thinking in other contexts.
I think I disagree pretty strongly? It moreso seems like, if you create a game where students are forced to get a high score by any means necessary or else have their futures taken from them (either via school placement or academic achievement-linked scholarships), they're going to optimize for the presented objective. Presumably, that's why wealthy people bribe their way through the system in some cases, or pay for improved tutoring in others. They aren't learning because it's such a beautiful and wonderful way to embody the human experience, it's so that they can claw their way up out of whatever hole they were born into (see: children pursuing sports with the sole goal of escaping poverty, or joining the military in regions where no other employer exists).
Furthermore, I'd disagree with the notion that delegating thought doesn't happen widely now. People groupthink all the time, and lean on biases or stereotypes just to get through the week. The most common complaint I hear about people w.r.t. doing anything productive for society is that they don't have the spare thought for it: why research electoral candidates; they're all the same and I'm too tired -- I'll just vote the same way my parents did/I did last time. Why look for ways to reduce my carbon impact; it's all so difficult to understand, and there's already too much to do (and some bald guy on the internet said it doesn't work anyways). Why understand why anything doesn't work; it's too complicated, and the most recent attack ad I saw said brown people are causing it anyways.
So yeah, firm disagree that the problem here is the technology of LLMs. It's companies shoving it into every corner of our lives in order to optimize their personal game, which is increasing profits (or increasing the perception of a probability of future profits).
I do agree with a lot of this, especially that LLMs and social media are probably more symptoms of deeper problems than the root cause themselves. If you build systems where people are heavily incentivized to optimize for grades, productivity, money, etc. Of course they're going to use whatever tools let them do that as efficiently as possible. And obviously outsourcing thought, relying on heuristics, groupthink, authority, stereotypes, etc. existed long before LLMs. Where I disagree is with the idea that this means the technology itself isn't also part of the problem.
Something doesn't have to create a human tendency in order to amplify it. We already tend to look for easier, faster, more comfortable ways of doing things, so giving everyone a tool that can instantly research, summarize, reason through problems, and produce answers for them is obviously going to make cognitive offloading much easier and more attractive than it was before. That doesn't make LLMs inherently bad, any more than the fact that cars make it easier not to walk makes cars inherently bad. But their affordances do change our behavior, and I don't think we can completely separate the tool from the way it encourages or enables certain habits.
So to me the answer isn't "LLMs are the problem," but it also isn't "LLMs have nothing to do with the problem." We need to learn how to use them responsibly and, probably more importantly, teach people when using them helps them think better and when it's just replacing the thinking they actually need to do.
Ah, certainly! Similar to my response to creesch, I'd note that my message was laser focused on dismantling the article author's thesis rather than discussing the topic in general.
I would say that, IMO, the jury is still out on how best to apply these tools, so I'm not certain about what we should be teaching people even to start with.
If you decide to purely focus on education you might have a point. I also do agree that the root cause is companies shoving it everywhere and having (for now) unlimited funds to offer these models for next to nothing.
However, that doesn't negate the fact that using of LLMs in itself can and often has a very negative impact on us individually. Something I have seen happen all around me. Something has indeed been changing. I have seen an alarming increase of lazy non critical use of LLM tools by people who should know better. Code spanning dozens of line trying to solve something that should only take one line. Code that completely ignores and conventions or design paradigms put in place. Code that goes directly against security practices. Suddenly downgraded dependency versions (because the models training data doesn't include the latest version). Verbatim answers from co-claude and friends and much more.
It really is basic human psychology as far as I am concerned. Our minds are wired to take the easy approach wherever they can, which historically makes a lot of sense. But that also means that many people will go down the lazy route of using LLMs to one degree or another. Is this amplified by how education works and many work environments? Possibly, but I firmly believe it is still an issue even without those factors in the mix.
If you decide to purely focus on education you might have a point.
Just as an aside, it's not my preference to focus on education. I re-read the author's sprawling blog post (archive link), and "LLMs are bad for the children" is the only concrete critique I could find, hence my bringing it up. I would have crafted a more comprehensive argument were this a discussion post, and not a shared link 😅. In point form, the author wrote this:
Spun a tale to literally interpret statements about the mind == computer as being false, since it's not a Von Neumann or Harvard architecture CPU, ipso facto squishy brain != silicon calculator
Language != thought, asserted via citation
Alluding to evolutionary psychology via a metaphorical comparison to hot dogs [1] and calorie seeking behaviour, claims that AI == junk food, and since junk food == bad, AI == bad ("clogging our capacity to develop the knowledge we need [...] to navigate the world").
Pivots to how this disproves the claim that "AI is transforming education", since per the above framing, you cannot learn via AI methods (metaphorically: you can't make a balanced meal out of hot dogs)
Dismisses dismissals of AI critique by AI boosters by reframing onto education [2], which is their strongest argument of AI boosters being fallible.
(my OP; quote about how the children can't read anymore etc. etc.)
Appeals to nature [3]
Broad claim that AI assistance for education is invalid since (1) human brain != computer, and (2) some efforts have failed
Cognitive delegation is bad, but people use it
Jab at researchers by stating that they reinvented schools
Discusses school AI bans by various jurisdictions
Points out that college students have protested it
Another appeal to nature and reassertion that human brain != computer.
... hence why I was laser focused on education, since their thesis ("AI bad, humans lazy") hinges on it. Anywho.
[professional concerns around LLM usage]
That's actually super interesting! Tbh, as I'm completely out of the software dev industry now, I've been trying to come up with ways to avoid those issues. A lot of them showed up before as coworkers would just copy paste out of stack overflow: we'd end up with insecure package versions (SO suggested it), awful coding practices (it passed tests what more do you want), and so forth. I'm trying to get my personal LLM harness + tooling set up to prevent this stuff programmatically: adding local continuous testing and code coverage enforcement, automated version checking, getting a fresh agent to enforce each code guideline as part of a merge review, etc.
Thinking back, it feels like a supercharged version of the complaints I heard when switching from a majority C++ workplace to a majority Java/C#/insert-memory-managed-language-here workplace: people stopped thinking about what their code did at runtime, and started leaning on the GC to clean up their poor practices. And it worked! The GC (and python/javascript in particular) did more to democratize software development than any amount of re-reading TAOCP or Design Patterns ever could. This is obviously more of a step function than that, but I can't help but see similarities between the two. IMO the largest differences are (1) software development is now critical to the world economy, so smashing a hole in its bottom will have ... side effects, (2) there are a tonne more people to complain now, (3) as a dual to 1, there're trillions of dollars being bet that this is worth hundreds of trillions of dollars, so there's tonnes of incentive to be extra annoying about it.
It really is basic human psychology as far as I am concerned. Our minds are wired to take the easy approach wherever they can, which historically makes a lot of sense.
Maybe? I'd point out that people still play music (despite MP3 players existing), hike intentionally difficult paths (despite cars and bikes existing), and play video games on hard (despite easy modes existing).
My more broad argument from my original post could be rephrased as: when faced with a limited set of resources, and put under pressure to perform optimally with them, people will often find more efficient strategies to accomplish their objectives. However, as those pressures recede, we find ourselves pursuing art, culture, and personal improvement for their own merits (or as part of prosocial exercises), and not in the context of an external framework that demands monetary results (and threatens corporal punishment). Perhaps it speaks to the depths to which those constraints are intertwined with our current socioeconomic systems that we feel personally aligned with them. But I have hope that human nature is not to optimize laziness, and that the false perception of such is a symptom of the world we've constructed.
...
I may have had too much coffee.
[1]
AI poses a similar sort of danger to our cognitive capabilities, by clogging our capacity to develop the knowledge we need – in our heads – to navigate the world. It’s a cognitive hot dog.
The occasional hot dog won’t harm anyone, but it sure will if it becomes a regular meal at lunch or dinner. The same is true for AI — the harm is not from its occasional use, but making it part of our mental diet.
[2]
Whatever the effects of these earlier technologies, never before have we developed and broadly deployed something so explicitly intended to supplant human thinking — that’s what AI evangelists themselves argue.
Look at education. ChatGPT may be the world’s largest learning platform, but students are using it en masse to avoid the effortful thinking [...] (see previous quotes)
[3]
No other species’ brain and body develops over such an extended period as ours, and historically, we’ve complemented that with unique cultural institutions that transmit knowledge from one generation to the next.
P.S. I try not to bring up "cap----ism" anymore since it's a bit of a trigger word that instantly devolves conversations into poop slinging contests, but feel free to substitute it out for "socioeconomic system" or "external framework" if you want to imagine the first draft of this comment.
The premise that AI offloads human thought and that this is bad only works out if you use AI for the things you shouldn't be using it for. I already offload human thought about the mindless garbage tasks I have to do at work by simply deferring and half-assing them, so in a way, you could make the point that AI taking over those tasks and simply doing a better job than I can be bothered to is not only making the end product measurably better but freeing my mind to think the healthy thoughts I actually would like to think.
(Hyperbole of course, not even 30% ish percent of my job is stuff I hate doing)
I really don't think that is my problem to solve, I'll be honest. Yes, this is a thought terminating cliché, but what exactly do you want the individual to do here? This is a systemic issue to solve and the personal framing of the article as some problem we as individuals HAVE to solve and be cognisant of is a honestly kind of weirdly framed. We wouldn't have this issue if Sam Altman didn't have dollar signs in his eyes and decided to force one of the most specific and niche pieces of computer science in decades to be General Access and desirable by any means necessary.
I studied mathematics back in the day. Computers and calculators were widely available, but we almost never used them beyond a four function because the expectation was that you should build the right mental muscles to become a working mathematician. So almost everything was chalk and talk with a mixture of multi day take homes or long in class exams. There were specific electives for computing and mathematics if you were inclined that way.
This is broadly the same with education and AI - just because the tool is available for the professional, doesn't mean that it needs to be used by the student.
kacey | a day ago
I think the crux of the author's argument is that LLMs are bad because people use them for offloading both their thought and their acquisition of knowledge? The example they give is for education:
I think I disagree pretty strongly? It moreso seems like, if you create a game where students are forced to get a high score by any means necessary or else have their futures taken from them (either via school placement or academic achievement-linked scholarships), they're going to optimize for the presented objective. Presumably, that's why wealthy people bribe their way through the system in some cases, or pay for improved tutoring in others. They aren't learning because it's such a beautiful and wonderful way to embody the human experience, it's so that they can claw their way up out of whatever hole they were born into (see: children pursuing sports with the sole goal of escaping poverty, or joining the military in regions where no other employer exists).
Furthermore, I'd disagree with the notion that delegating thought doesn't happen widely now. People groupthink all the time, and lean on biases or stereotypes just to get through the week. The most common complaint I hear about people w.r.t. doing anything productive for society is that they don't have the spare thought for it: why research electoral candidates; they're all the same and I'm too tired -- I'll just vote the same way my parents did/I did last time. Why look for ways to reduce my carbon impact; it's all so difficult to understand, and there's already too much to do (and some bald guy on the internet said it doesn't work anyways). Why understand why anything doesn't work; it's too complicated, and the most recent attack ad I saw said brown people are causing it anyways.
So yeah, firm disagree that the problem here is the technology of LLMs. It's companies shoving it into every corner of our lives in order to optimize their personal game, which is increasing profits (or increasing the perception of a probability of future profits).
ToteRose | 14 hours ago
I do agree with a lot of this, especially that LLMs and social media are probably more symptoms of deeper problems than the root cause themselves. If you build systems where people are heavily incentivized to optimize for grades, productivity, money, etc. Of course they're going to use whatever tools let them do that as efficiently as possible. And obviously outsourcing thought, relying on heuristics, groupthink, authority, stereotypes, etc. existed long before LLMs. Where I disagree is with the idea that this means the technology itself isn't also part of the problem.
Something doesn't have to create a human tendency in order to amplify it. We already tend to look for easier, faster, more comfortable ways of doing things, so giving everyone a tool that can instantly research, summarize, reason through problems, and produce answers for them is obviously going to make cognitive offloading much easier and more attractive than it was before. That doesn't make LLMs inherently bad, any more than the fact that cars make it easier not to walk makes cars inherently bad. But their affordances do change our behavior, and I don't think we can completely separate the tool from the way it encourages or enables certain habits.
So to me the answer isn't "LLMs are the problem," but it also isn't "LLMs have nothing to do with the problem." We need to learn how to use them responsibly and, probably more importantly, teach people when using them helps them think better and when it's just replacing the thinking they actually need to do.
kacey | 5 hours ago
Ah, certainly! Similar to my response to creesch, I'd note that my message was laser focused on dismantling the article author's thesis rather than discussing the topic in general.
I would say that, IMO, the jury is still out on how best to apply these tools, so I'm not certain about what we should be teaching people even to start with.
creesch | 7 hours ago
If you decide to purely focus on education you might have a point. I also do agree that the root cause is companies shoving it everywhere and having (for now) unlimited funds to offer these models for next to nothing.
However, that doesn't negate the fact that using of LLMs in itself can and often has a very negative impact on us individually. Something I have seen happen all around me. Something has indeed been changing. I have seen an alarming increase of lazy non critical use of LLM tools by people who should know better. Code spanning dozens of line trying to solve something that should only take one line. Code that completely ignores and conventions or design paradigms put in place. Code that goes directly against security practices. Suddenly downgraded dependency versions (because the models training data doesn't include the latest version). Verbatim answers from co-claude and friends and much more.
It really is basic human psychology as far as I am concerned. Our minds are wired to take the easy approach wherever they can, which historically makes a lot of sense. But that also means that many people will go down the lazy route of using LLMs to one degree or another. Is this amplified by how education works and many work environments? Possibly, but I firmly believe it is still an issue even without those factors in the mix.
kacey | 5 hours ago
Just as an aside, it's not my preference to focus on education. I re-read the author's sprawling blog post (archive link), and "LLMs are bad for the children" is the only concrete critique I could find, hence my bringing it up. I would have crafted a more comprehensive argument were this a discussion post, and not a shared link 😅. In point form, the author wrote this:
... hence why I was laser focused on education, since their thesis ("AI bad, humans lazy") hinges on it. Anywho.
That's actually super interesting! Tbh, as I'm completely out of the software dev industry now, I've been trying to come up with ways to avoid those issues. A lot of them showed up before as coworkers would just copy paste out of stack overflow: we'd end up with insecure package versions (SO suggested it), awful coding practices (it passed tests what more do you want), and so forth. I'm trying to get my personal LLM harness + tooling set up to prevent this stuff programmatically: adding local continuous testing and code coverage enforcement, automated version checking, getting a fresh agent to enforce each code guideline as part of a merge review, etc.
Thinking back, it feels like a supercharged version of the complaints I heard when switching from a majority C++ workplace to a majority Java/C#/insert-memory-managed-language-here workplace: people stopped thinking about what their code did at runtime, and started leaning on the GC to clean up their poor practices. And it worked! The GC (and python/javascript in particular) did more to democratize software development than any amount of re-reading TAOCP or Design Patterns ever could. This is obviously more of a step function than that, but I can't help but see similarities between the two. IMO the largest differences are (1) software development is now critical to the world economy, so smashing a hole in its bottom will have ... side effects, (2) there are a tonne more people to complain now, (3) as a dual to 1, there're trillions of dollars being bet that this is worth hundreds of trillions of dollars, so there's tonnes of incentive to be extra annoying about it.
Maybe? I'd point out that people still play music (despite MP3 players existing), hike intentionally difficult paths (despite cars and bikes existing), and play video games on hard (despite easy modes existing).
My more broad argument from my original post could be rephrased as: when faced with a limited set of resources, and put under pressure to perform optimally with them, people will often find more efficient strategies to accomplish their objectives. However, as those pressures recede, we find ourselves pursuing art, culture, and personal improvement for their own merits (or as part of prosocial exercises), and not in the context of an external framework that demands monetary results (and threatens corporal punishment). Perhaps it speaks to the depths to which those constraints are intertwined with our current socioeconomic systems that we feel personally aligned with them. But I have hope that human nature is not to optimize laziness, and that the false perception of such is a symptom of the world we've constructed.
...
I may have had too much coffee.
[1]
[2]
[3]
P.S. I try not to bring up "cap----ism" anymore since it's a bit of a trigger word that instantly devolves conversations into poop slinging contests, but feel free to substitute it out for "socioeconomic system" or "external framework" if you want to imagine the first draft of this comment.
delphi | 10 hours ago
The premise that AI offloads human thought and that this is bad only works out if you use AI for the things you shouldn't be using it for. I already offload human thought about the mindless garbage tasks I have to do at work by simply deferring and half-assing them, so in a way, you could make the point that AI taking over those tasks and simply doing a better job than I can be bothered to is not only making the end product measurably better but freeing my mind to think the healthy thoughts I actually would like to think.
(Hyperbole of course, not even 30% ish percent of my job is stuff I hate doing)
[OP] rodrigo | 9 hours ago
That's an “you're holding it wrong” argument. Most people use AI “wrong”, this is the material reality. How do we solve this problem?
delphi | 8 hours ago
I really don't think that is my problem to solve, I'll be honest. Yes, this is a thought terminating cliché, but what exactly do you want the individual to do here? This is a systemic issue to solve and the personal framing of the article as some problem we as individuals HAVE to solve and be cognisant of is a honestly kind of weirdly framed. We wouldn't have this issue if Sam Altman didn't have dollar signs in his eyes and decided to force one of the most specific and niche pieces of computer science in decades to be General Access and desirable by any means necessary.
D_E_Solomon | 8 hours ago
I studied mathematics back in the day. Computers and calculators were widely available, but we almost never used them beyond a four function because the expectation was that you should build the right mental muscles to become a working mathematician. So almost everything was chalk and talk with a mixture of multi day take homes or long in class exams. There were specific electives for computing and mathematics if you were inclined that way.
This is broadly the same with education and AI - just because the tool is available for the professional, doesn't mean that it needs to be used by the student.