This is a very, very insightful article - he's dodged the majority of the pitfalls most writing about AI* makes, whether pro- or anti-, because he's focused on the human element and he clearly gets it. The thing everyone is angry about has very little to do with the technology itself, and a whole lot to do with the shell game of ever increasing stock prices that people somehow still pretend are correlated to real world benefits.
It's a relief to see someone else pointing out that trying to fight AI companies with expanded copyright laws would just help different megacorporations without doing anything meaningful to benefit individual creators, because the major copyright holders are just as bad as the tech companies, and beholden to just the same absurd economic incentives that we are apparently treating as the inviolable natural order of things rather than as a managing framework that we've let get wildly out of hand.
I'm a bit disappointed to see him repeating the same flawed justification about AI output not being copyrightable (tl;dr: the model isn't the monkey, as he implies. It's the camera.), but I do actually like his play through of the implications of making that ruling. I don't think it's a logically consistent ruling to make with the law and precedent as it stands, but I can at least buy his arguments about how it could theoretically play out in a beneficial way.
Most of all, I appreciate that he isn't treating "jobs" like they're an inherently good or inherently valuable thing. They aren't. But quality of work is a good and valuable thing, and again the economic incentive (and I'll stress for the third time, we are somehow accepting this insane incentive structure as "just the way it has to be") is to make cheap, substandard output and then lie about it being better than it is. We have these incredibly powerful and capable tools being created by scientists and engineers, with the possibility of genuinely using them to improve the world, and the primary thing getting in the way of that right now is the fact that making grandiose claims about the tools can make you very, very rich.
You should really read the article, because it says more and better than I can here, even accounting for the odd bit I don't agree with - but if you don't have the time, the conclusion hits the key points pretty succinctly:
AI is a bubble and it will burst. Most of the companies will fail. Most of the datacenters will be shuttered or sold for parts. So what will be left behind?
We will have a bunch of coders who are really good at applied statistics. We will have a lot of cheap GPUs, which will be good news for, say, effects artists and climate scientists, who will be able to buy that critical hardware at pennies on the dollar. And we will have the open-source models that run on commodity hardware, AI tools that can do a lot of useful stuff, like transcribing audio and video; describing images; summarizing documents; and automating a lot of labor-intensive graphic editing – such as removing backgrounds or airbrushing passersby out of photos. These will run on our laptops and phones, and open-source hackers will find ways to push them to do things their makers never dreamed of.
If there had never been an AI bubble, if all this stuff arose merely because computer scientists and product managers noodled around for a few years coming up with cool new apps, most people would have been pleasantly surprised with these interesting new things their computers could do. We would call them “plugins”.
It’s the bubble that sucks, not these applications. The bubble doesn’t want cheap useful things. It wants expensive, “disruptive” things: big foundation models that lose billions of dollars every year.
* I've said this a million times now but I'll say it again: AI is a bad term for this tech in general, and one that leads to a whole mess of confusion above and beyond that fundamental incorrectness because no two people actually use it to mean the same thing anyway. But it's the accepted way people talk about big neural net type things nowadays, so I'm going to use it as the article intends, for what I'd guess is the same reason that he didn't start with 500 words of dry explanation on why we should be saying "machine learning" and precisely what that covers.
They are necessary. That inherently means they are valuable. Because the good alternatives aren't setup and the realistic alternatives are homelessness and all that entails.
We were regressing on the idea of remote work a few years ago, and now these techno feudalists want to imagine a UBI world? I say "jobs are valuable" as a compromise, not as an idea that we need to live to work.
The difficulty here is that they're necessary in the sense of "our current economic system won't give you food and shelter if you don't have a job", but often not necessary in the sense of "the work you're doing is needed for a productive, thriving world".
I think I get where you're coming from, but the actual impact of phrasing it as "jobs are valuable" is that 99% of your audience will just hear it as support for the already ingrained idea that "having a job" is worthy, noble, or productive in and of itself. If we're going to have even the slightest hope of building support for evidence-based work reform, the first barrier to cross is breaking the idea that "40 hour per week job == good, contributing, beneficial" and "any other arrangement == bad, lazy, burdensome".
I think "jobs are not inherently valuable" is at least a clear opening statement for that conversation - as is "jobs are currently a necessary evil", if you prefer an alternative way of framing the same thing. Suggesting that their necessity gives them value, but then trying to explain to people that there's a giant asterisk there to clarify that the necessity is artificial, and "valuable" doesn't mean what the majority of them are interpreting it to mean, just seems like a very difficult way to communicate the point.
It's a relief to see someone else pointing out that trying to fight AI companies with expanded copyright laws would just help different megacorporations without doing anything meaningful to benefit individual creators,
Oh god I wish it weren't so rare to see this (utterly correct) take
I don't like that argument because yes, 99% of all initiatives disproportionately support people rich enough to argue for it (often on someone else's dime anyway). If it's not progressive taxing or welfare*, its probably something a rich person gets more out of
It feels like a form of soft defeatism to say "well the rich benefit more so we may as well not bother". We can work on that 1% while acknowledging that the 99% will still help regular people out some too.
*and even here, welfare arguably is a cost saving measure for the rich. Not having to hire security to navigate a low crime area helps immensely. The general stimulaton of economy for people who'd otherwise drops out helps everyone as well
I'm a bit disappointed to see him repeating the same flawed justification about AI output not being copyrightable (tl;dr: the model isn't the monkey, as he implies. It's the camera.)
Cameras have regulations too. So I think either way it applies. I can't use a camera to invade someone's space, and picturing copyrighted work still had loads of gray areas.
I'm not conceptually against regulating the tech, assuming it's done in a sensible way by people who understand the nuances of both the technology and the law (a big ask, I know!). I just don't like the faulty premise coming in as a foundation of the argument, because it makes it a lot harder to get to a logically consistent endpoint - even if it's coincidentally being used in support of a reasonable point on this occasion.
Same way I wouldn't want a road safety policy to be based on the idea that red cars are faster, even if it happened to end up getting to a policy I agree with, you know? If we let that kind of thing pass, it just sets up for deeper and more damaging misunderstandings later.
Good article (i actually read all of it). I particularly like the concept of reverse centaurs that he postulated; and the example of Amazon drivers being reverse centaurs, basically enslaved by technology and the billionaires who own the technology. And it makes perfect sense that they want to fire skilled workers and retain just a few 'humans in the loop' whose only real purpose will be 'accountability sinks' i.e someone to blame for AI's mistakes.
I am now fearful of how my own job will get affected by the AI bubble burst. Or worse, how my daughter's generation will get affected; she's just a toddler, and i hope the world is in a much better place by the time she is ready to enter the workforce.
I respect Cory Doctorow for remaining consistent on his copyright opinions, and I mostly enjoyed this article. If you have the utmost belief that AI will fundamentally plateau somewhere below humans, then the article seems correct. But I have to pull one quote from the article:
If there had never been an AI bubble, if all this stuff arose merely because computer scientists and product managers noodled around for a few years coming up with cool new apps, most people would have been pleasantly surprised with these interesting new things their computers could do. We would call them “plugins”.
There was never a world in which this didn't become an enormous, international phenomenon. Ever since Turing proposed the Turing test, it's been inevitable. The fact that you can now have a full conversation with a computer is strange and incredible, it would never have been relegated to something exclusively for techies.
Also, this sentence is not entirely true:
Now, AI is a statistical inference engine. All it can do is predict what word will come next based on all the words that have been typed in the past.
This is a common misconception. Modern LLMs are post-trained on reinforcement learning tasks, which means they're not only trained to predict the next word, but also trained to solve problems more effectively. And, in the case of the example used in the article, they are trained to recognize when they don't know something, in order to not hallucinate. Although, just like humans will often misremember things, this isn't perfect (but it has been getting noticeably better with new models).
This is a common misconception. Modern LLMs are post-trained on reinforcement learning tasks, which means they're not only trained to predict the next word, but also trained to solve problems more effectively.
Calling them "next token predictors" is overly simplified and arguably incorrect with transformers and attention, but the models are fundamentally massive pattern matchers. Every technique and application I've read about rely on that pattern matching capability. Machine learning is all about learning and using patterns.
Reinforcement learning and various response layers do a lot to fine-tune LLMs. They still rely on introducing, removing, identifying, encouraging, and discouraging specific patterns.
Cheers to both your post and the one you're replying to. It's a forever losing battle trying to clear up misconceptions about tech, but worth attempting nonetheless. Whatever level of understanding popular culture lands on ends up influencing policy and legislation.
Reinforcement learning and various response layers do a lot to fine-tune LLMs. They still rely on introducing, removing, identifying, encouraging, and discouraging specific patterns.
The reason I think this is important to keep pointing out is that LLMs feel like something more than advanced pattern matching tools, and I think that's going to be dangerous.
There was never a world in which this didn't become an enormous, international phenomenon. Ever since Turing proposed the Turing test, it's been inevitable. The fact that you can now have a full conversation with a computer is strange and incredible, it would never have been relegated to something exclusively for techies.
This is just true. I understand the pushback against AI, I agree with a lot of it, but it extends into the irrational at times. Not only was it always going to shake the ground, it was always going to attract trillions of dollars in investment, and that was always going to come from the megarich. Wealth consolidation was a problem long before LLMs, this is just the latest symptom. Wishful thinking is not how we'll have a shot at changing it.
A world where groundbreaking technology benefits the masses as much as it benefits the financial elite is a world with very different systems than we have now, AI or not.
I think overall Doctorow's piece was solid, he made some great points. I'm happy to give some leeway to the guy who coined enshittification. But there was one additional questionable claim:
AI is a bubble and it will burst.
I agree that AI is a bubble. I don't know if anyone can confidently predict that it will burst. Western economics is in uncharted territory, brute force capital has held off, lessoned, shortened and in some cases entirely avoided recessions, market crashes and other financial events that statistics and history tell us should have happened, or happened bigger, sooner and lasted for longer.
The AI bubble should burst dramatically. The western world should probably be in a natural and perfectly healthy recession right now. Certainly the non wealthy are experiencing something like a recession even if the financial markets don't reflect it. Not a healthy one though.
All the rules are different now. The AI bubble could pop, but the capital firehose has to run dry first, and governments, especially the US, have a stake too. Without AI the stock markets look bleak, the US looks less like a world leader and the dollar would probably be in free fall.
Which isn't to say that the bubble definitely won't pop, only that it's not a foregone conclusion. Which Doctorow might have been more willing to allow if he wasn't promoting a new book.
There's evidence that the bubble pop is being forestalled in part by massive Federal Reserve intervention in the U.S. Banks are lending money faster than they're taking it in, and there's major instability coming from multiple sources.
This doesn't take into account how vastly overvalued AI investments are in private capital markets.
Doctorow's essay should be read as a companion piece to today's Guardian interview with Ed Zitron, who's done a very cogent analysis of the circularity of lending among the hyperscalers and Nvidia, and the impossibility of the current gen AI business models.
It's not a question of "if", just "when".
Further footnote: I personally believe that the most transformative technological leaps won't come from general purpose LLMs (which really aren't doing anything humans can't do, and mostly do worse and less cost-effectively than humans), but from specialized models for narrowly specified tasks in the physical world and modeling complex systems, e.g. protein folding and receptor interactions, weather, markets, and so on. These are areas where human cognition isn't optimal - more "centaur" than job replacement.
A tricky sales pitch, though. Right now, most LLM shills are acting like LLMs can replace human labor. But who will be willing to pay the bills for LLMs that merely supplement human labor? Instead of cutting salary costs, you'd actually add subscription costs on top of your existing salaries. There's grey area here for doing more work with a similar number of employees, but the calculation is a lot more complicated than "one CEO running a company of bots" like the current industry pitch.
That is the exact reasoning for why we're calling it a bubble. It does not mean the technology is inherently useless, just that there are a lot of pie-in-the-sky expectations for it that cannot reasonably be met and that do not make financial sense for the amount of money being sunk into it.
Yeah, this is my misconception I try to rectify every other article. The dotcom bubble did not kill the internet. But it did push out a few major players and a lot of smaller, massively overvalued ones. The remnants of that still went on to become the new face of tech. Or a few old faces remained (Microsoft)
AI will probably have a similar fate.the big massive loser will likely be OpenAI unless Microsoft absorbs them in fully. Tons of others will die out, and one or two big players (Tesla is the obvious bet. Maybe Meta too) will fall from grace.
The only big difference is that the trillionaire tech companies probably won't be wiped out en masse, nor even stagnate away like the IBMs of yesterday did. They still have core products people use and will fall back to that. Maybe Antitrust finally catches up to them, but that's a different matter entirely.
Imagine business models for specialized pharma ML of operating cost plus fractional IP rights in any discoveries; or insurance industry and commodities market licensing on better weather models... I'm sure the busy imaginations of startup founders can find a way to make them profitable, but we're not talking about "One TRILLION dollars!".
If your broader social purposes are massive productivity improvement and less waste, better environmental stewardship, just distribution and abundance for the masses, improved health, more responsive and accountable government, I can see many ways that effectively applied ML would help, and even grant modest profits if we can't come up with a better model than capitalism.
And, in the case of the example used in the article, they are trained to recognize when they don't know something, in order to not hallucinate.
There are certainly lots of attempts to do this, but anyone who is remotely knowledgeable about language models knows that it is not possible to completely prevent "hallucination" with this kind of architecture atm and the best you can do is mitigate it. Those who claim otherwise are overstating the results of methods to do the latter at best and straight-up grifting you at worst. (It also has little to nothing in common with how humans "misremember things". But you're already using anthropomorphization like "realize" and "know" here, and I get what you're getting at.)
I do agree, though, that this blowing up was inevitable. I don't think people who weren't working with language models or ML before chatGPT came out realize just how big of a leap forward these LLMs are compared to even the state-of-the-art prior to them. People who are against AI for the human reasons described in the article do sometime try to downplay how good these models are as part of it, but they really are incredibly impressive and I wouldn't have believed such a huge step forward in language models would happen so quickly if you'd asked me at the beginning of 2022.
People who are against AI for the human reasons described in the article do sometime try to downplay how good these models are as part of it, but they really are incredibly impressive and I wouldn't have believed such a huge step forward in language models would happen so quickly if you'd asked me at the beginning of 2022.
A big part of it is setting completely unrealistic standards. From a single, often poorly formulated and vague prompt, this tech must understand you completely accurately, and fetch a complete and fact checked answer with no errors, on any arbitrary topic and all in under 30 seconds!
An army of human experts wouldn't be held to such a standard, let alone any single individual human! It is absolutely incredible what these tools are capable of now.
The fact that you can now have a full conversation with a computer is strange and incredible
Is it actually a conversation, though? Or just a facsimile where I contribute an input and the system produces a response that fairly convincingly emulates some sort of relatedness to my input?
I'm usually opposed to language that anthropomorphizes LLMs, but I think this particular one is a bad sticking point to have tbh. The only definition of "conversation" that would exclude what people do with tools like chatGPT would be so heavily philosophical that it would exclude huge swaths of conversations between humans. It's a language model and it's very good at producing comprehensible, relevant responses to things you say, and that allows you as a human to hold a conversation with it (and how good these models are at doing this was a genuine breakthrough). You, as a human, are doing all the same stuff you'd do in a conversation with another human, and whether the model is underlying doing those same things (it's not) isn't relevant imo. Humans can hold conversations with much less sophisticated "communicators" like parrots or magic 8-balls. There's little point in arguing the semantics of the word "conversation" rather than focusing on the actual weak points and problems with LLMs.
it would exclude huge swaths of conversations between humans
I do agree about this, but not necessarily the broader point that we shouldn't aim to tell real conversations apart from auto-piloting (machine assisted/generated or not). And I believe this to be one of the benefits of ubiquitous LLMs: more people will learn to tell the difference and why it matters.
Its clear from the hyper individualism and utter narcissism of today's society (or worse yet, the naiveness of the youth. They are an exception from the rant below) that that fascimile counts as "good enough". That's the dangerous part of the marketing of this as "intelligence".
And I don't say narcissism to be cynical. I say that because these models really do not like upsetting the user with corrections unless legally modified to. People en masse aren't using these to challenge their beliefs nor better themselves. It's just one giant Yes man that wants to validate preconceived notions to try and lure them into eventual financial relationships. I'd call it manipulative if people weren't looking to be manipulated.
Like spock_vulcan i found the reverse-centaur concept very useful (and I read the whole too).
But, being as I'm uneducated on the current use of LLMs, please can someone suggest how I could begin to get to grips with them? Where do I start and what do I?
ways that leverage the fact that they're language models
ways that mitigate any problems from inaccurate information in the output by either:
making it very easy for you to notice mistakes quickly on skimming the result
being an application where the result doesn't matter
Language learning is one such application, as it can be very useful to practice having conversations in a target language. It might not necessarily be right if you ask it deeper theoretical questions about the language, but native speakers also usually aren't. As long as your target language is big enough to have a sufficient presence in the training data (which is definitely true of most of the languages that are popular to learn, at least), this is an excellent way to just get yourself practice at fluidly using your target language with another speaker without the nerves of looking stupid in front of someone else or wasting a real human's time.
Writing tedious things that are a waste of your time like cover letters is also a good bet, because you'll know pretty quickly if it's making up something about you and you can freely edit the result before you actually send any result out into the wild. Bureaucratic things like this are really a place where the ability to avoid the tediousness of writing it yourself shines. You definitely still need to check the output for accuracy and tweak the wording on occasion, but that's far less work on your part.
In a combination of the two, I needed to write a letter to challenge the German Jobcenter's denial of my unemployment benefits, and chatGPT was very helpful on that front. Something with legal information like that can be a little more fraught, so I'd advise others to proceed with caution, but luckily the German legal code is online and it was thus easy enough to check that its citations weren't straight-up wrong. I don't think you should take legal advice from an LLM, but they are extremely competent when it comes to the actual composition of a letter like this, especially with human editing. I wouldn't have done this if I couldn't read German well enough to understand the output, but it helped a lot with using more formal legal language than I usually can and brainstorming a list of legal arguments to include therein.
As a former manager, cover letters weren't a waste of time in my experience, but a first-pass filter on whether the attached resumé was worth reading.
Last week, I was coaching a friend through resumé writing and application cover letters. I had to explain, in detail, why an AI-written cover letter and resumé weren't going to get her foot in the door. Aside from the unnecessary verbosity (you don't put 5 paragraphs in a cover letter!), the model's output didn't prioritise relevant experience or provide the personal "why" for the application.
The LLM could target job description keywords, yet didn't question back about whether a functional, chronological, or hybrid resumé would be the most effective presentation for someone who's had career gaps for education and major changes in professional direction.
It sounds like you've used the chatGPT output as thoughtfully as possible, but it's not a substitute for human experience and professional input.
I have found Claude to be useful in generating cover letters that I then thoroughly rewrite, but I have something of a mental block regarding language for praising myself.
I don't think that setting forth concrete achievements is praising yourself gratuitously, and that's what I tell the people who've asked me for resumé advice. The target audience wants to know you're capable of doing the work with skill and efficiency, adapting to new processes, and getting along.
The frustrating thing about job applications is that HR doesn't really know the entailments of the jobs they're posting, and they get to adjust the published language. You have to put yourself in the shoes of the hiring manager, who (at least theoretically) knows the true requirements and nice-to-haves for the position, then tailor your cover/resumé for their attention. LLMs aren't capable of doing that for you - I doubt that the LLM training team is tuning the results to increase hiring as opposed to merely generating text output responsive to the published job.
What specifically are you trying to learn how to do? Generally, I would recommend making an account with one of the big 3 LLM providers (OpenAI/ChatGPT, Google/Gemini, or Anthropic/Claude), and just talking to it. LLMs are pretty good at giving advice on how they should be used.
Is there anything you do that’s relatively simple or straightforward but also tedious and repetitive? LLMs are pretty good for those kinds of tasks.
I’ve used LLMs to generate .ics files to import into my calendar. I can quickly check its work to see if it did it correctly and I get to not populate my calendar manually.
I’ve used LLMs for generating cover letter for job applications which hit all the key points listed in the job description. I obviously proofread what it generates and even prompt it to interview me on experiences I feel like may be relevant to include.
If there’s something you actually enjoy doing or is complex enough that you want to handle it yourself, don’t try to shoehorn LLMs into it. If there’s anything that’s tedious and annoying that you have always wished you could delegate to someone else, try delegating to an LLM and see how it does.
For general LLM use, it's functional to treat GPTs like smart search engines with artificial persona templates. If you use an LLM as a universal conversational partner, that's where things can go off the rails - it will make dangerous, hard-to-check assertions with complete confidence, doing the equivalent of a mentalist's cold reading to feed your own identity and thought processes back to you with distortions.
Free-tier ChatGPT isn't a terrible place to start, or Google Gemini. This is a decent intro. If you want, say, a travel itinerary to an unfamiliar destination, you can ask the GPT to assume the role of a travel agent, then put some boundaries on your questions. Something like, "I'm staying for X days, with Y budget, and I'm interested in Z. Generate an itinerary that lets me visit as many Z sites as I can, with restaurants along the way that serve {gluten-free, vegan, etc.} food, without exceeding my budget."
There are all kinds of resources on "prompt engineering", but for general personal use, it's fine to jump in and explore. Just hold onto the understanding that GPT is the epitome of an unreliable narrator. You're best off sticking with queries for concrete, checkable factual information or generating files (text, code, images, spreadsheets, etc.) that you're willing and able to re-edit. Also understand that nothing you type, speak or upload is genuinely private with a consumer LLM - avoid disclosing personal or sensitive information.
Close the chat when you're done, so that it doesn't maintain a memory record that can be distorted by iterating within a continuing chat, like "Since you're interested in Z, it means you're a fascinating person and I'd be happy to keep you engaged in thinking about Z."
LLMs are useful, but not substitutes for critical thinking or subject matter knowledge and experience. They provide "garbage in, garbage out" on steroids - if the data you're asking for doesn't exist or is unreliable, the LLM can and often will make something up.
The snarky side of me says "avoid them". But in all honesty the one thing you need is skepticism. Approach any and all answers as if it came from some random bystander on the street. It can be correct, it also might 'feel correct' even if it's a subjective take. If it's not some throwaway trivia, make sure to gather more sources to reinforce the statement heard.
Make sure to review any writing, code, or art generated for imperfections, awkwardness, or simply uncanny details. If you don't have the skills to identify such imperfections, you probably shouldn't use it for those tasks.
Depends on what you mean by "get to grips with them." If you're asking how to get the most value out of them, I can't help you. But if you're asking how to figure out if they're worth using, I can answer that one:
They're not.
I'm not going to argue that an LLM don't have any utility whatsoever. But when balanced against the many and multifaceted harms they cause, the many risks they pose directly to you, and the fact that they are so frequently and plainly bad at solving the problems given, I find it very safe to say that you do not actually need to learn how to use this tool at all.
At the very least, I recommend saving the test-run for if/when the tool becomes ethical.
Can you explain what ethical harms you're concerned about from using local models? Lots of text prediction, grammar, translation, and content warning systems use LLMs nowadays, so you're likely already using them somewhere.
I have found them helpful for idea generation. Coming up with a group name, list ingredients I want to use and suggesting potential recipes, that kind of thing. They will generally list a good number of options, and of you want them to go a different direction or just keep generating along the same lines you can just ask. Then pick the one(s) that you like the best, or get inspired by one to think of something you might have taken much longer to think of, or not thought of at all.
Greg | a month ago
This is a very, very insightful article - he's dodged the majority of the pitfalls most writing about AI* makes, whether pro- or anti-, because he's focused on the human element and he clearly gets it. The thing everyone is angry about has very little to do with the technology itself, and a whole lot to do with the shell game of ever increasing stock prices that people somehow still pretend are correlated to real world benefits.
It's a relief to see someone else pointing out that trying to fight AI companies with expanded copyright laws would just help different megacorporations without doing anything meaningful to benefit individual creators, because the major copyright holders are just as bad as the tech companies, and beholden to just the same absurd economic incentives that we are apparently treating as the inviolable natural order of things rather than as a managing framework that we've let get wildly out of hand.
I'm a bit disappointed to see him repeating the same flawed justification about AI output not being copyrightable (tl;dr: the model isn't the monkey, as he implies. It's the camera.), but I do actually like his play through of the implications of making that ruling. I don't think it's a logically consistent ruling to make with the law and precedent as it stands, but I can at least buy his arguments about how it could theoretically play out in a beneficial way.
Most of all, I appreciate that he isn't treating "jobs" like they're an inherently good or inherently valuable thing. They aren't. But quality of work is a good and valuable thing, and again the economic incentive (and I'll stress for the third time, we are somehow accepting this insane incentive structure as "just the way it has to be") is to make cheap, substandard output and then lie about it being better than it is. We have these incredibly powerful and capable tools being created by scientists and engineers, with the possibility of genuinely using them to improve the world, and the primary thing getting in the way of that right now is the fact that making grandiose claims about the tools can make you very, very rich.
You should really read the article, because it says more and better than I can here, even accounting for the odd bit I don't agree with - but if you don't have the time, the conclusion hits the key points pretty succinctly:
* I've said this a million times now but I'll say it again: AI is a bad term for this tech in general, and one that leads to a whole mess of confusion above and beyond that fundamental incorrectness because no two people actually use it to mean the same thing anyway. But it's the accepted way people talk about big neural net type things nowadays, so I'm going to use it as the article intends, for what I'd guess is the same reason that he didn't start with 500 words of dry explanation on why we should be saying "machine learning" and precisely what that covers.
BuckyMcMonks | a month ago
Yuuuuuuuup.
I'll concede that people need to occupy their time. I refuse to concede that they need to sell their time under threat of starvation.
raze2012 | a month ago
They are necessary. That inherently means they are valuable. Because the good alternatives aren't setup and the realistic alternatives are homelessness and all that entails.
We were regressing on the idea of remote work a few years ago, and now these techno feudalists want to imagine a UBI world? I say "jobs are valuable" as a compromise, not as an idea that we need to live to work.
Greg | a month ago
The difficulty here is that they're necessary in the sense of "our current economic system won't give you food and shelter if you don't have a job", but often not necessary in the sense of "the work you're doing is needed for a productive, thriving world".
I think I get where you're coming from, but the actual impact of phrasing it as "jobs are valuable" is that 99% of your audience will just hear it as support for the already ingrained idea that "having a job" is worthy, noble, or productive in and of itself. If we're going to have even the slightest hope of building support for evidence-based work reform, the first barrier to cross is breaking the idea that "40 hour per week job == good, contributing, beneficial" and "any other arrangement == bad, lazy, burdensome".
I think "jobs are not inherently valuable" is at least a clear opening statement for that conversation - as is "jobs are currently a necessary evil", if you prefer an alternative way of framing the same thing. Suggesting that their necessity gives them value, but then trying to explain to people that there's a giant asterisk there to clarify that the necessity is artificial, and "valuable" doesn't mean what the majority of them are interpreting it to mean, just seems like a very difficult way to communicate the point.
sparksbet | a month ago
Oh god I wish it weren't so rare to see this (utterly correct) take
raze2012 | a month ago
I don't like that argument because yes, 99% of all initiatives disproportionately support people rich enough to argue for it (often on someone else's dime anyway). If it's not progressive taxing or welfare*, its probably something a rich person gets more out of
It feels like a form of soft defeatism to say "well the rich benefit more so we may as well not bother". We can work on that 1% while acknowledging that the 99% will still help regular people out some too.
*and even here, welfare arguably is a cost saving measure for the rich. Not having to hire security to navigate a low crime area helps immensely. The general stimulaton of economy for people who'd otherwise drops out helps everyone as well
raze2012 | a month ago
Cameras have regulations too. So I think either way it applies. I can't use a camera to invade someone's space, and picturing copyrighted work still had loads of gray areas.
Greg | a month ago
I'm not conceptually against regulating the tech, assuming it's done in a sensible way by people who understand the nuances of both the technology and the law (a big ask, I know!). I just don't like the faulty premise coming in as a foundation of the argument, because it makes it a lot harder to get to a logically consistent endpoint - even if it's coincidentally being used in support of a reasonable point on this occasion.
Same way I wouldn't want a road safety policy to be based on the idea that red cars are faster, even if it happened to end up getting to a policy I agree with, you know? If we let that kind of thing pass, it just sets up for deeper and more damaging misunderstandings later.
spock_vulcan | a month ago
Good article (i actually read all of it). I particularly like the concept of reverse centaurs that he postulated; and the example of Amazon drivers being reverse centaurs, basically enslaved by technology and the billionaires who own the technology. And it makes perfect sense that they want to fire skilled workers and retain just a few 'humans in the loop' whose only real purpose will be 'accountability sinks' i.e someone to blame for AI's mistakes.
I am now fearful of how my own job will get affected by the AI bubble burst. Or worse, how my daughter's generation will get affected; she's just a toddler, and i hope the world is in a much better place by the time she is ready to enter the workforce.
tesseractcat | a month ago
I respect Cory Doctorow for remaining consistent on his copyright opinions, and I mostly enjoyed this article. If you have the utmost belief that AI will fundamentally plateau somewhere below humans, then the article seems correct. But I have to pull one quote from the article:
There was never a world in which this didn't become an enormous, international phenomenon. Ever since Turing proposed the Turing test, it's been inevitable. The fact that you can now have a full conversation with a computer is strange and incredible, it would never have been relegated to something exclusively for techies.
Also, this sentence is not entirely true:
This is a common misconception. Modern LLMs are post-trained on reinforcement learning tasks, which means they're not only trained to predict the next word, but also trained to solve problems more effectively. And, in the case of the example used in the article, they are trained to recognize when they don't know something, in order to not hallucinate. Although, just like humans will often misremember things, this isn't perfect (but it has been getting noticeably better with new models).
Minori | a month ago
Calling them "next token predictors" is overly simplified and arguably incorrect with transformers and attention, but the models are fundamentally massive pattern matchers. Every technique and application I've read about rely on that pattern matching capability. Machine learning is all about learning and using patterns.
Reinforcement learning and various response layers do a lot to fine-tune LLMs. They still rely on introducing, removing, identifying, encouraging, and discouraging specific patterns.
post_below | a month ago
Cheers to both your post and the one you're replying to. It's a forever losing battle trying to clear up misconceptions about tech, but worth attempting nonetheless. Whatever level of understanding popular culture lands on ends up influencing policy and legislation.
The reason I think this is important to keep pointing out is that LLMs feel like something more than advanced pattern matching tools, and I think that's going to be dangerous.
This is just true. I understand the pushback against AI, I agree with a lot of it, but it extends into the irrational at times. Not only was it always going to shake the ground, it was always going to attract trillions of dollars in investment, and that was always going to come from the megarich. Wealth consolidation was a problem long before LLMs, this is just the latest symptom. Wishful thinking is not how we'll have a shot at changing it.
A world where groundbreaking technology benefits the masses as much as it benefits the financial elite is a world with very different systems than we have now, AI or not.
I think overall Doctorow's piece was solid, he made some great points. I'm happy to give some leeway to the guy who coined enshittification. But there was one additional questionable claim:
I agree that AI is a bubble. I don't know if anyone can confidently predict that it will burst. Western economics is in uncharted territory, brute force capital has held off, lessoned, shortened and in some cases entirely avoided recessions, market crashes and other financial events that statistics and history tell us should have happened, or happened bigger, sooner and lasted for longer.
The AI bubble should burst dramatically. The western world should probably be in a natural and perfectly healthy recession right now. Certainly the non wealthy are experiencing something like a recession even if the financial markets don't reflect it. Not a healthy one though.
All the rules are different now. The AI bubble could pop, but the capital firehose has to run dry first, and governments, especially the US, have a stake too. Without AI the stock markets look bleak, the US looks less like a world leader and the dollar would probably be in free fall.
Which isn't to say that the bubble definitely won't pop, only that it's not a foregone conclusion. Which Doctorow might have been more willing to allow if he wasn't promoting a new book.
patience_limited | a month ago
There's evidence that the bubble pop is being forestalled in part by massive Federal Reserve intervention in the U.S. Banks are lending money faster than they're taking it in, and there's major instability coming from multiple sources.
This doesn't take into account how vastly overvalued AI investments are in private capital markets.
Doctorow's essay should be read as a companion piece to today's Guardian interview with Ed Zitron, who's done a very cogent analysis of the circularity of lending among the hyperscalers and Nvidia, and the impossibility of the current gen AI business models.
It's not a question of "if", just "when".
Further footnote: I personally believe that the most transformative technological leaps won't come from general purpose LLMs (which really aren't doing anything humans can't do, and mostly do worse and less cost-effectively than humans), but from specialized models for narrowly specified tasks in the physical world and modeling complex systems, e.g. protein folding and receptor interactions, weather, markets, and so on. These are areas where human cognition isn't optimal - more "centaur" than job replacement.
DynamoSunshirt | a month ago
A tricky sales pitch, though. Right now, most LLM shills are acting like LLMs can replace human labor. But who will be willing to pay the bills for LLMs that merely supplement human labor? Instead of cutting salary costs, you'd actually add subscription costs on top of your existing salaries. There's grey area here for doing more work with a similar number of employees, but the calculation is a lot more complicated than "one CEO running a company of bots" like the current industry pitch.
donn | a month ago
That is the exact reasoning for why we're calling it a bubble. It does not mean the technology is inherently useless, just that there are a lot of pie-in-the-sky expectations for it that cannot reasonably be met and that do not make financial sense for the amount of money being sunk into it.
raze2012 | a month ago
Yeah, this is my misconception I try to rectify every other article. The dotcom bubble did not kill the internet. But it did push out a few major players and a lot of smaller, massively overvalued ones. The remnants of that still went on to become the new face of tech. Or a few old faces remained (Microsoft)
AI will probably have a similar fate.the big massive loser will likely be OpenAI unless Microsoft absorbs them in fully. Tons of others will die out, and one or two big players (Tesla is the obvious bet. Maybe Meta too) will fall from grace.
The only big difference is that the trillionaire tech companies probably won't be wiped out en masse, nor even stagnate away like the IBMs of yesterday did. They still have core products people use and will fall back to that. Maybe Antitrust finally catches up to them, but that's a different matter entirely.
patience_limited | a month ago
Imagine business models for specialized pharma ML of operating cost plus fractional IP rights in any discoveries; or insurance industry and commodities market licensing on better weather models... I'm sure the busy imaginations of startup founders can find a way to make them profitable, but we're not talking about "One TRILLION dollars!".
If your broader social purposes are massive productivity improvement and less waste, better environmental stewardship, just distribution and abundance for the masses, improved health, more responsive and accountable government, I can see many ways that effectively applied ML would help, and even grant modest profits if we can't come up with a better model than capitalism.
donn | a month ago
If any of us could we'd honestly be rich enough to not care about the next bubble
sparksbet | a month ago
There are certainly lots of attempts to do this, but anyone who is remotely knowledgeable about language models knows that it is not possible to completely prevent "hallucination" with this kind of architecture atm and the best you can do is mitigate it. Those who claim otherwise are overstating the results of methods to do the latter at best and straight-up grifting you at worst. (It also has little to nothing in common with how humans "misremember things". But you're already using anthropomorphization like "realize" and "know" here, and I get what you're getting at.)
I do agree, though, that this blowing up was inevitable. I don't think people who weren't working with language models or ML before chatGPT came out realize just how big of a leap forward these LLMs are compared to even the state-of-the-art prior to them. People who are against AI for the human reasons described in the article do sometime try to downplay how good these models are as part of it, but they really are incredibly impressive and I wouldn't have believed such a huge step forward in language models would happen so quickly if you'd asked me at the beginning of 2022.
Lobachevsky | a month ago
A big part of it is setting completely unrealistic standards. From a single, often poorly formulated and vague prompt, this tech must understand you completely accurately, and fetch a complete and fact checked answer with no errors, on any arbitrary topic and all in under 30 seconds!
An army of human experts wouldn't be held to such a standard, let alone any single individual human! It is absolutely incredible what these tools are capable of now.
Macha | a month ago
I think the reason that often ends up being the standard is that's what the most prolific boosters (especially on linkedin) claim it be capable of.
Lobachevsky | a month ago
I don't think this is attributable to any particular actor or group of actors. Let's just say that laymen don't accurately describe the technology.
Lia | a month ago
Is it actually a conversation, though? Or just a facsimile where I contribute an input and the system produces a response that fairly convincingly emulates some sort of relatedness to my input?
sparksbet | a month ago
I'm usually opposed to language that anthropomorphizes LLMs, but I think this particular one is a bad sticking point to have tbh. The only definition of "conversation" that would exclude what people do with tools like chatGPT would be so heavily philosophical that it would exclude huge swaths of conversations between humans. It's a language model and it's very good at producing comprehensible, relevant responses to things you say, and that allows you as a human to hold a conversation with it (and how good these models are at doing this was a genuine breakthrough). You, as a human, are doing all the same stuff you'd do in a conversation with another human, and whether the model is underlying doing those same things (it's not) isn't relevant imo. Humans can hold conversations with much less sophisticated "communicators" like parrots or magic 8-balls. There's little point in arguing the semantics of the word "conversation" rather than focusing on the actual weak points and problems with LLMs.
Lia | a month ago
I do agree about this, but not necessarily the broader point that we shouldn't aim to tell real conversations apart from auto-piloting (machine assisted/generated or not). And I believe this to be one of the benefits of ubiquitous LLMs: more people will learn to tell the difference and why it matters.
boxer_dogs_dance | a month ago
Had a lot of fun with magic 8-balls when I was in grade school.
raze2012 | a month ago
Its clear from the hyper individualism and utter narcissism of today's society (or worse yet, the naiveness of the youth. They are an exception from the rant below) that that fascimile counts as "good enough". That's the dangerous part of the marketing of this as "intelligence".
And I don't say narcissism to be cynical. I say that because these models really do not like upsetting the user with corrections unless legally modified to. People en masse aren't using these to challenge their beliefs nor better themselves. It's just one giant Yes man that wants to validate preconceived notions to try and lure them into eventual financial relationships. I'd call it manipulative if people weren't looking to be manipulated.
blivet | a month ago
Thank you. This was exactly my reaction. Are you really having a conversation if there is no one on the other end?
anadem | a month ago
Like spock_vulcan i found the reverse-centaur concept very useful (and I read the whole too).
But, being as I'm uneducated on the current use of LLMs, please can someone suggest how I could begin to get to grips with them? Where do I start and what do I?
sparksbet | a month ago
The best ways to use LLMs, in my opinion are:
Language learning is one such application, as it can be very useful to practice having conversations in a target language. It might not necessarily be right if you ask it deeper theoretical questions about the language, but native speakers also usually aren't. As long as your target language is big enough to have a sufficient presence in the training data (which is definitely true of most of the languages that are popular to learn, at least), this is an excellent way to just get yourself practice at fluidly using your target language with another speaker without the nerves of looking stupid in front of someone else or wasting a real human's time.
Writing tedious things that are a waste of your time like cover letters is also a good bet, because you'll know pretty quickly if it's making up something about you and you can freely edit the result before you actually send any result out into the wild. Bureaucratic things like this are really a place where the ability to avoid the tediousness of writing it yourself shines. You definitely still need to check the output for accuracy and tweak the wording on occasion, but that's far less work on your part.
In a combination of the two, I needed to write a letter to challenge the German Jobcenter's denial of my unemployment benefits, and chatGPT was very helpful on that front. Something with legal information like that can be a little more fraught, so I'd advise others to proceed with caution, but luckily the German legal code is online and it was thus easy enough to check that its citations weren't straight-up wrong. I don't think you should take legal advice from an LLM, but they are extremely competent when it comes to the actual composition of a letter like this, especially with human editing. I wouldn't have done this if I couldn't read German well enough to understand the output, but it helped a lot with using more formal legal language than I usually can and brainstorming a list of legal arguments to include therein.
patience_limited | a month ago
As a former manager, cover letters weren't a waste of time in my experience, but a first-pass filter on whether the attached resumé was worth reading.
Last week, I was coaching a friend through resumé writing and application cover letters. I had to explain, in detail, why an AI-written cover letter and resumé weren't going to get her foot in the door. Aside from the unnecessary verbosity (you don't put 5 paragraphs in a cover letter!), the model's output didn't prioritise relevant experience or provide the personal "why" for the application.
The LLM could target job description keywords, yet didn't question back about whether a functional, chronological, or hybrid resumé would be the most effective presentation for someone who's had career gaps for education and major changes in professional direction.
It sounds like you've used the chatGPT output as thoughtfully as possible, but it's not a substitute for human experience and professional input.
boxer_dogs_dance | a month ago
I have found Claude to be useful in generating cover letters that I then thoroughly rewrite, but I have something of a mental block regarding language for praising myself.
patience_limited | a month ago
I don't think that setting forth concrete achievements is praising yourself gratuitously, and that's what I tell the people who've asked me for resumé advice. The target audience wants to know you're capable of doing the work with skill and efficiency, adapting to new processes, and getting along.
The frustrating thing about job applications is that HR doesn't really know the entailments of the jobs they're posting, and they get to adjust the published language. You have to put yourself in the shoes of the hiring manager, who (at least theoretically) knows the true requirements and nice-to-haves for the position, then tailor your cover/resumé for their attention. LLMs aren't capable of doing that for you - I doubt that the LLM training team is tuning the results to increase hiring as opposed to merely generating text output responsive to the published job.
tesseractcat | a month ago
What specifically are you trying to learn how to do? Generally, I would recommend making an account with one of the big 3 LLM providers (OpenAI/ChatGPT, Google/Gemini, or Anthropic/Claude), and just talking to it. LLMs are pretty good at giving advice on how they should be used.
ackables | a month ago
Is there anything you do that’s relatively simple or straightforward but also tedious and repetitive? LLMs are pretty good for those kinds of tasks.
I’ve used LLMs to generate .ics files to import into my calendar. I can quickly check its work to see if it did it correctly and I get to not populate my calendar manually.
I’ve used LLMs for generating cover letter for job applications which hit all the key points listed in the job description. I obviously proofread what it generates and even prompt it to interview me on experiences I feel like may be relevant to include.
If there’s something you actually enjoy doing or is complex enough that you want to handle it yourself, don’t try to shoehorn LLMs into it. If there’s anything that’s tedious and annoying that you have always wished you could delegate to someone else, try delegating to an LLM and see how it does.
patience_limited | a month ago
For general LLM use, it's functional to treat GPTs like smart search engines with artificial persona templates. If you use an LLM as a universal conversational partner, that's where things can go off the rails - it will make dangerous, hard-to-check assertions with complete confidence, doing the equivalent of a mentalist's cold reading to feed your own identity and thought processes back to you with distortions.
Free-tier ChatGPT isn't a terrible place to start, or Google Gemini. This is a decent intro. If you want, say, a travel itinerary to an unfamiliar destination, you can ask the GPT to assume the role of a travel agent, then put some boundaries on your questions. Something like, "I'm staying for X days, with Y budget, and I'm interested in Z. Generate an itinerary that lets me visit as many Z sites as I can, with restaurants along the way that serve {gluten-free, vegan, etc.} food, without exceeding my budget."
There are all kinds of resources on "prompt engineering", but for general personal use, it's fine to jump in and explore. Just hold onto the understanding that GPT is the epitome of an unreliable narrator. You're best off sticking with queries for concrete, checkable factual information or generating files (text, code, images, spreadsheets, etc.) that you're willing and able to re-edit. Also understand that nothing you type, speak or upload is genuinely private with a consumer LLM - avoid disclosing personal or sensitive information.
Close the chat when you're done, so that it doesn't maintain a memory record that can be distorted by iterating within a continuing chat, like "Since you're interested in Z, it means you're a fascinating person and I'd be happy to keep you engaged in thinking about Z."
LLMs are useful, but not substitutes for critical thinking or subject matter knowledge and experience. They provide "garbage in, garbage out" on steroids - if the data you're asking for doesn't exist or is unreliable, the LLM can and often will make something up.
raze2012 | a month ago
The snarky side of me says "avoid them". But in all honesty the one thing you need is skepticism. Approach any and all answers as if it came from some random bystander on the street. It can be correct, it also might 'feel correct' even if it's a subjective take. If it's not some throwaway trivia, make sure to gather more sources to reinforce the statement heard.
Make sure to review any writing, code, or art generated for imperfections, awkwardness, or simply uncanny details. If you don't have the skills to identify such imperfections, you probably shouldn't use it for those tasks.
LukeZaz | a month ago
Depends on what you mean by "get to grips with them." If you're asking how to get the most value out of them, I can't help you. But if you're asking how to figure out if they're worth using, I can answer that one:
They're not.
I'm not going to argue that an LLM don't have any utility whatsoever. But when balanced against the many and multifaceted harms they cause, the many risks they pose directly to you, and the fact that they are so frequently and plainly bad at solving the problems given, I find it very safe to say that you do not actually need to learn how to use this tool at all.
At the very least, I recommend saving the test-run for if/when the tool becomes ethical.
Minori | a month ago
Can you explain what ethical harms you're concerned about from using local models? Lots of text prediction, grammar, translation, and content warning systems use LLMs nowadays, so you're likely already using them somewhere.
Lyrl | a month ago
I have found them helpful for idea generation. Coming up with a group name, list ingredients I want to use and suggesting potential recipes, that kind of thing. They will generally list a good number of options, and of you want them to go a different direction or just keep generating along the same lines you can just ask. Then pick the one(s) that you like the best, or get inspired by one to think of something you might have taken much longer to think of, or not thought of at all.