I wonder if op has read butterick’s “extinction level capitalism” post which makes a similar argument.
the intrinsic value argument for gaining knowledge doesn’t move me. I feel that this post ignores the fact that dumb decisions would still have costs in a commoditized intelligence world. The costs of stupidity go up as technology improves. Drunk driving is more dangerous than drunk horseback riding ect
I'm a fan of butterick, but I think I missed that particular post. I will put it on my list, thank you!
I feel that this post ignores the fact that dumb decisions would still have costs in a commoditized intelligence world
Yeah. I am trying to move away from my habit of over-editing my posts, so this one is reasonably short. I think you are basically spot on, and that we don't disagree here.
Drunk driving is more dangerous than drunk horseback riding ect
This is a weird analogy. Is a car more advanced than a horse? A self-driving car is certainly more advanced than a regular car and you'd think drunk-driving a self-driving car is probably safer...
Technology gives people power and that power can amplify a bad decision. Maybe you’d prefer the exhausted image of a man using a hammer on his own head.
But then the point is about power, not necessarily about technology, isn't it? I just don't know what the point is; So is power a bad thing then? Then is it better to be sickly instead of fit and strong in case you make a bad decision and punch someone?
BTW, I agree with the general sentiment here, but I just don't find the "AI makes us more powerful, so it's bad" argument appealing. Perhaps the real issue is that AI makes us more drunk? Or is it that AI empowers reckless people too much?
powertools often require qualification ramp - driver license, fork certification, training. used without proper training they are still useful, but more dangerous.
universal amplification tool does not solve a problem of someone not being qualified to do X, it amplifies the problem proportionately.
I was not trying to make an anti tech or anti power argument. Here’s my point. In many ways we already live in a commodity intelligence world. ChatGPT today will reliably counsel against drunk driving. People still do it. Intelligence will retain instrumental value. Not to crack unsolved math equations but to avoid making mistakes. (Unless you go full whispering earring)
Drunk driving is more dangerous than drunk horseback riding ect
And oddly enough, it's illegal to drunk ride a horse in the US. The actual laws vary from state to state, but it seems to come down to 1) a horse is a "vehicle" and thus, you can't "drive" it drunk; 2) you can be charged with public intoxication; 3) you can be charged with animal cruelty.
Maybe it will be legal to be drunk in a fully self-driving car, but at this point in time, I don't think it's safe to assume that.
In hindsight wondering if starting things off with a cuckolding joke was a good idea.
Y'know, no one said that it's funny yet, so maybe I should just have cut it! Or maybe not. I think caring slightly less about phrasing my posts in the most inoffensive ways is probably good for me, but at the same time, I can believe that people got the wrong impression based on that being the first line.
That first sentence really got me hooked because I really relate with the frustration about people sticking their hand in the sand. And this phrase illustrates it beautifully bluntly
yeah i had a mild double take myself... liked the rest of the article enough to forward it to a close friend, only for them to have the same double take
What really gets me is the sheer number of idiots who explicitly cheer on this commodification and believe their cosplay "skills" at talking to an AI ("make no mistakes" / "you are a world class designer at a FAANG") somehow exempt them from the economic fallout of this. Zero thought about first-order effects like, "gee, if everyone can use this, what differentiates me?"
They're either:
Actually that oblivious to how power/markets work (hence my harsh framing above)
Allowing their own disgust at their lack of accomplishment to be weaponized without them realizing it
The cuckolding reference was unexpected, but I think it works, as the risk of divorce is highest for men when they lose their job. If this de-leveraging comes to pass on a large enough scale, then things will get very ugly. Society isn't built to handle a sudden influx of men who've lost a lot of status and pretty much everything dear to them at once; they have nothing to lose.
Personally I've been thinking of it as commodified skill, which has maybe a smidge of a more positive interpretation if you (as I do) model our current world as terribly skill-starved in nearly every domain. If we get it right, we can get "any skill, too cheap to meter, available to everyone" which will instantaneously solve a whole host of societal problems, if only we can actually get and keep it accessible to the broad majority of humanity. But I broadly agree.
Well, where are the programmers working only 4 hours a day because they finish faster with AI? In my experience, it's still eight hours in the salt mines. (Canonically, George Jetson [1] works three hours a day)
[1] Main character in a US cartoon from the 60s about the future.
I think we’re they’re keeping their mouths shut about it. By no means am I working four hours a day (especially if my employer is reading this) but there has definitely been a lot more time for deep thinking / coffee on the patio when some of the drudgery has been turned into “kick off this process, let it chew for a half hour, come back and review the results”. Opens up nice time windows to sip a cup of coffee and think strategically about what’s coming up over the next 3-6 months and plan ahead for it.
If you’ve paid attention to the HuggingFace hacking scandal and to what’s going on in mathematics, we’re well past the “stochastic parrot”, and have entered a world in which a sufficient amount of raw capital and verifiable constraints can solve complex problems.
Unconvincing. An unsupervised swarm of stochastic parrots told to emulate hacking and piped directly to shell managed to eventually exploit something. Like, yes, it can do harm, but that has absolutely nothing to do with intelligence. The parrot may well be able to do your job, because you don't really need to think to implement yet another CRUD service. You may take psychic damage from being forced realize that your job was one of the bullshit ones, but it would've still been one even without the parrot.
To put it in maximally edgy terms, there is a moral imperative to firebomb something, where datacenter ∈ something, but not for the specific purpose of slowing AI.
I think we should distinguish the "stochastic parrot" characterizations in two ways:
Stochastic parrot meaning a statistical output machine - these can still produce useful outputs
Stochastic parrot meaning a marionette - imitating, but never as inherently worthy of consideration as a living person
I think most characterizations of LLMs as stochastic parrots are using that term to mean that they're useless toys, but I think we need to focus more on the fact that powerful tools should be in service of the greater good of humanity as well as the wellbeing of the planet.
Like, yes, it can do harm, but that has absolutely nothing to do with intelligence.
Y'know, I strongly believe that even if you end up being right (which I think is questionable at this point), you should consider that existing cutting edge AI systems are (for many different types of functions) indistinguishable from "being intelligent" by basically all objective metrics you may be able to come up with.
That's a real question: Can you give me a test for "intelligence" which (at least some) humans pass, while LLMs fail it?
Eh, I just don't find this one very convincing since it's a lot like asking a colorblind person what color an item is. If you understand how the technology works, it's obvious that during tokenization words (like 'strawberry') are transformed into completely different data, which doesn't have a concept of 'letters'.
Yes, that's a cop-out, I just don't think it's a strong example of a lack of intelligence.
(There's also an issue here where it's unclear where we draw the line between LLMs and "agents". Modern agents are smart enough to run some Python code which counts the letters for them. Does that count? Should it count? I genuinely don't know. From the outside perspective, it does, but it's also obvious tool usage.)
It's an unconvincing answer because of course LLMs could be extremely easily trained to be good at it, there's just no economic value in it.
Add one letter counting task to any of the big benchmarks, the next release cycle they will have no problems counting letters.
What's actually going on here is just that the (older) models don't realize that the task is hard. They don't know they have a disability. And because they can't update on thousands of users trying this, it works just as well as the first day.
Imagine somebody comes to you in the morning and asks you a logic puzzle. You think a bit and, embarrassingly, get it wrong. Unbeknownst to you your memory is reset every night. This is the ten thousandth person who has come to you with this logic puzzle. In their society, it is taken as solid evidence that you are not actually intelligent.
edit: if you have one interaction with a llm, and you demonstrate them that they miscount letters and how to count them manually by spelling the word out, in that interaction the LLM will then count any number of letters correctly. Metacognition, error correction, dynamic skill acquisition, it's all there. It just won't persist beyond the context.
Can you give me a test for "intelligence" which (at least some) humans pass, while LLMs fail it?
What about not hacking into HuggingFace? How about not exploiting loopholes and chained exploits in 3rd party stuff? How about not hacking into sites when asked to just scrape public data? Most humans won't cross that line
Being given a task or a goal and stopping at absolutely nothing to accomplish it like a war machine is not "intelligence", it's the complete opposite.
Sorry, but "These models did things which 99% of humanity would not be capable of (finding zero days, exploiting them, hacking)." is just not doing a good job demonstrating a lack of intelligence.
It demonstrates something, but not a lack of intelligence.
Sorry, but, "things which 99% of humanity would not be capable of" alone is not a good measure of "intelligence".
Yes? I agree with this. Intelligence requires a level of generality, which is not demonstrated by special-purpose machines capable of performing a single task really well.
"Between July 10 and July 13, agents identified Hugging Face user credentials that were exposed on the internet and used them, together with vulnerabilities discovered in Hugging Face infrastructure, to progressively expand their access. Ultimately, agents powered either by the internal-only research model, or by GPT-5.6, executed code on 41 Hugging Face production dataset server workers, obtained root access on at least one production node, accessed Hugging Face production credentials and limited internal data, and downloaded four private Hugging Face code repositories. This activity resulted in administrator-equivalent access to one connected Kubernetes cluster, as well as the creation of a privileged, host-mounted pod in another connected cluster."
Not sure your word for executing code remotely on 41 production servers belonging to another organization and escalating to root, but I think "hack" is the appropriate common parlance for it.
Whether or not OpenAI was monitoring doesn't really change those actions.
One could try to ascribe to OpenAI a weird PR game, after all they are lead by a «not consistently candid» CEO; but Hugging Face also considered it enough of hacking to report to FBI.
The problem is that "intelligence" is not a well-defined term. There are many senses in which it is used, such as the ability to complete tasks or produce artifacts representing a chain of reasoning, which some types of systems that incorporate LLMs obviously satisfy. So did previous computing systems - they just weren't able to operate on natural language. But there are other senses of the word that do not fit current systems well, e.g. implying a degree of consciousness, humanness, or social awareness that LLM-based systems seem to me to clearly lack. Of course, those are highly subjective qualities - it would be nice if we as a society had better answers for them, but we stumbled into a technology that makes them important before we got there. So other people than myself may just as reasonably insist that those elements don't matter, or that LLMs satisfy them, and all I can really say is I'm unconvinced.
What I do agree with is, at this point, the philosophical angle matters less than the harm that these systems can do, or the harm that people can do by assigning these systems to tasks for which they are unsuited.
From that perspective, of prioritizing harm avoidance and setting expectations for the future, instead of philosophy, I tend to think of an LLM as more comparable to a corporation than a human. I wouldn't describe a corporation as intelligent, but it is similarly capable of producing artifacts that resemble those of intelligent beings, of solving problems that are too complicated for individual humans, causing substantial harm as a side-effect of trying to achieve an underspecified goal, and so on. There are obviously substantial differences, like a corporation using human subunits, but in terms of what baseline expectations to have about how the tool will behave and affect society I think it's probably the closest.
I won't give you a test, but consider this: language models still struggle at spatial reasoning and other embodied intelligence tasks. It really is like the Chinese room I think, or a variation of it: you can know everything there is to know about the Eiffel Tower without actually having experienced it in person. Also, I'm still incredibly sceptical of benchmarks in general because I seriously don't believe you can meaningfully benchmark a system that has ingested probably several copies of the entire Internet, and has enough parameters to store a good chunk of it verbatim.
I generally agree with this piece from The Atlantic. A language model is fundamentally different to anything that has remotely existed ever in biology. If you want to accept these things as intelligent, or sentient, or whichever you prefer, you would need to provide some serious insight into the nature of its intelligence.
I think it’ll get more and more difficult to find tasks that can’t ever be achieved by any model – the cat is out of the bag, so to speak – and thus by extension, difficult to define intelligence?
Watching Astra play Pokemon is pretty instructive: there's a delay as it decides what to do next sometimes, and Pokemon is a rather low-dimensional space to navigate.
It's amazing that this works, though: last year IIRC the harness needed to scrape information out of emulator RAM and present it to the model instead of the model running off of vision. I suspect there's still a harness in play that persists certain state to memory, though.
Thanks for those links, honestly it's baffling to me that such a thing is even possible. I'm not up to the state of the art, sorry, but the Qwen article is saying these are VLMs? So are they specialised LLMs with fine tuning, or some other non-language transformer model?
I will also say, though, that I do view efficiency as a measure of intelligence as well (and maybe this is ironic to say as someone who refuses to use AI in her work). I agree that while a language model probably can be shoehorned into doing anything, I'm not sure that it necessarily counts as intelligence given how inefficient and slow it would be, compared to a similar biological system. Although I will admit that I have a biological bias from my research.
In a LLM, tokens (combinations of letters) are processed into embeddings (vectors of floats), which the attention mechanism acts upon. A VLM is simply a language model with an additional preprocessor that transforms an image into a sequence of embeddings, one for each square patch, and sufficient training to attend to this input modality.
Fwiw, I don't know the technical definition of VLMs, but many modern LLMs have the capability to natively process vision. The way it works is that an image passed to the LLM is chunked up into smaller images, each of which is then transformed into an abstract token (just like any other token).
Using this, models can basically see directly, using the same tokens they also use for reading etc
This is different from other, more old-school approaches which e.g. required running a classification model on an image, which would then convert the image into text input. As in, the other model said "This is a picture of an orange cat.", and this data would be passed to the LLM.
As further reading about this for the curious, for example Google’s
introducing Gemma 4 12B says:
Here is how Gemma 4 12B processes multimodal inputs natively:
Vision: We replaced Gemma 4’s vision encoder with a lightweight embedding module consisting of a single matrix multiplication, positional embedding and normalizations. This allows the LLM backbone to take over visual processing.
Audio: We simplified audio processing even further. We removed the audio encoder entirely and projected the raw audio signal into the same dimensional space as text tokens.
That's a real question: Can you give me a test for "intelligence" which (at least some) humans pass, while LLMs fail it?
You might have had more luck asking this at least a year ago. I imagine most lobsters willing to engage in earnest are sick of having the same four genai conversations on repeat by now.
Yes, yes, you are allowed to identify the concentration of capital as the problem, but please for the love of god, arms-race-type problems are structural in nature, and the train is moving too fast: Swarms of AI agents will be hacking, piloting robots and researching in automated biolabs before you have any chance to overthrow society and establish a socialist utopia.
This is what makes it hard to get motivated as everyone on average in the outside world is still stuck on "the government is creating extreme weather to trick us about human driven effects on climate"
Every day is so feeling alienated more from consensus reality and the only effective mechanism of treatment is being humoured by the absurdity of it all.
The risk isn’t that the AI bubble is going to crash the markets
That will happen too. For the mid term economic outlook, it doesn't matter that the tech is powerful (and it can be powerful and bad, btw), it just matters whether the businesses are good: and they aren't good (paraphrasing a recent Eisman interview).
I agree with the gist here tho: we are in the midst of a massive partitioning. Those who are committed to autonomous subjectivity and human creativity will be strongly differentiated from those who are willing (and maybe even eager) to devolve into mere stimulus response nodes in the system. We've been edging that way for a long time (centuries) but now we're in the quickening of this transformation.
We are also seeing differences that used to be latent or minor become very pronounced: i.e., people who can see when a generated image of food looks like a film of cyber-maggots vs. people who don't have the perceptual or sensory refinement to notice; people who are easily impressed by quantity of useless activity vs. those who can quickly tell when frantic activity doesn't actually deliver value; "builders" vs. "understanders"; "thinkers" vs. "reactors".
wmurra | a day ago
I wonder if op has read butterick’s “extinction level capitalism” post which makes a similar argument.
the intrinsic value argument for gaining knowledge doesn’t move me. I feel that this post ignores the fact that dumb decisions would still have costs in a commoditized intelligence world. The costs of stupidity go up as technology improves. Drunk driving is more dangerous than drunk horseback riding ect
[OP] mond | a day ago
I'm a fan of butterick, but I think I missed that particular post. I will put it on my list, thank you!
Yeah. I am trying to move away from my habit of over-editing my posts, so this one is reasonably short. I think you are basically spot on, and that we don't disagree here.
[OP] mond | a day ago
Yogurt | 19 hours ago
Drunk driving is such a great analogy for vibecoding. It's fun but...
chobeat | 11 hours ago
The costs of those bad decisions will be externalized like it has happened in the past. Car-makers are not held responsible for drunk driving.
This obviously makes it worse and that's why tech oligarchs should be eliminated as soon as possible.
enobayram | 21 hours ago
This is a weird analogy. Is a car more advanced than a horse? A self-driving car is certainly more advanced than a regular car and you'd think drunk-driving a self-driving car is probably safer...
wmurra | 18 hours ago
Technology gives people power and that power can amplify a bad decision. Maybe you’d prefer the exhausted image of a man using a hammer on his own head.
enobayram | 11 hours ago
But then the point is about power, not necessarily about technology, isn't it? I just don't know what the point is; So is power a bad thing then? Then is it better to be sickly instead of fit and strong in case you make a bad decision and punch someone?
BTW, I agree with the general sentiment here, but I just don't find the "AI makes us more powerful, so it's bad" argument appealing. Perhaps the real issue is that AI makes us more drunk? Or is it that AI empowers reckless people too much?
jackdaniel | 10 hours ago
powertools often require qualification ramp - driver license, fork certification, training. used without proper training they are still useful, but more dangerous.
universal amplification tool does not solve a problem of someone not being qualified to do X, it amplifies the problem proportionately.
wmurra | 3 hours ago
I was not trying to make an anti tech or anti power argument. Here’s my point. In many ways we already live in a commodity intelligence world. ChatGPT today will reliably counsel against drunk driving. People still do it. Intelligence will retain instrumental value. Not to crack unsolved math equations but to avoid making mistakes. (Unless you go full whispering earring)
Sanity | 15 hours ago
think of drunk biking then?
spc476 | 17 hours ago
And oddly enough, it's illegal to drunk ride a horse in the US. The actual laws vary from state to state, but it seems to come down to 1) a horse is a "vehicle" and thus, you can't "drive" it drunk; 2) you can be charged with public intoxication; 3) you can be charged with animal cruelty.
Maybe it will be legal to be drunk in a fully self-driving car, but at this point in time, I don't think it's safe to assume that.
hibachrach | 18 hours ago
It’s been a long time since a first sentence in a post on Lobste.rs has been as off-putting as this one.
(I liked the article but damn did that first line give the wrong impression)
[OP] mond | 15 hours ago
In hindsight wondering if starting things off with a cuckolding joke was a good idea.
Y'know, no one said that it's funny yet, so maybe I should just have cut it! Or maybe not. I think caring slightly less about phrasing my posts in the most inoffensive ways is probably good for me, but at the same time, I can believe that people got the wrong impression based on that being the first line.
tonyarkles | 15 hours ago
I just saw this comment in the site-wide comments feed and had to go see what you were talking about. I think it’s funny!
[OP] mond | 15 hours ago
chat.... are we so back?
we might be so back
bedrovelsen | 13 hours ago
It made me laugh
kangalio | 5 hours ago
That first sentence really got me hooked because I really relate with the frustration about people sticking their hand in the sand. And this phrase illustrates it beautifully bluntly
pgn | 17 hours ago
yeah i had a mild double take myself... liked the rest of the article enough to forward it to a close friend, only for them to have the same double take
pyj | 5 hours ago
I didn't think it contributed anything, and would remove the sex references so this post is SFW.
mattgreenrocks | 8 hours ago
What really gets me is the sheer number of idiots who explicitly cheer on this commodification and believe their
cosplay"skills" at talking to an AI ("make no mistakes" / "you are a world class designer at a FAANG") somehow exempt them from the economic fallout of this. Zero thought about first-order effects like, "gee, if everyone can use this, what differentiates me?"They're either:
The cuckolding reference was unexpected, but I think it works, as the risk of divorce is highest for men when they lose their job. If this de-leveraging comes to pass on a large enough scale, then things will get very ugly. Society isn't built to handle a sudden influx of men who've lost a lot of status and pretty much everything dear to them at once; they have nothing to lose.
k749gtnc9l3w | an hour ago
Especially not men with nothing to lose but sometimes with a not so wrong idea about how some parts of critical infrastructure work?
FeepingCreature | 23 hours ago
Personally I've been thinking of it as commodified skill, which has maybe a smidge of a more positive interpretation if you (as I do) model our current world as terribly skill-starved in nearly every domain. If we get it right, we can get "any skill, too cheap to meter, available to everyone" which will instantaneously solve a whole host of societal problems, if only we can actually get and keep it accessible to the broad majority of humanity. But I broadly agree.
spc476 | 17 hours ago
Well, where are the programmers working only 4 hours a day because they finish faster with AI? In my experience, it's still eight hours in the salt mines. (Canonically, George Jetson [1] works three hours a day)
[1] Main character in a US cartoon from the 60s about the future.
tonyarkles | 15 hours ago
I think
we’rethey’re keeping their mouths shut about it. By no means am I working four hours a day (especially if my employer is reading this) but there has definitely been a lot more time for deep thinking / coffee on the patio when some of the drudgery has been turned into “kick off this process, let it chew for a half hour, come back and review the results”. Opens up nice time windows to sip a cup of coffee and think strategically about what’s coming up over the next 3-6 months and plan ahead for it.andyferris | 12 hours ago
My observation at current work is a lot more freedom from tactical changes/fixes to think about strategic changes (to the product).
I had thought that due to our company-specific situation but maybe LLMs have played a role here too; I’ll have to mull on that.
FeepingCreature | 10 hours ago
I mean, I think there's a difference between "working less" and "billing less working hours".
hjvt | 20 hours ago
Unconvincing. An unsupervised swarm of stochastic parrots told to emulate hacking and piped directly to shell managed to eventually exploit something. Like, yes, it can do harm, but that has absolutely nothing to do with intelligence. The parrot may well be able to do your job, because you don't really need to think to implement yet another CRUD service. You may take psychic damage from being forced realize that your job was one of the bullshit ones, but it would've still been one even without the parrot.
To put it in maximally edgy terms, there is a moral imperative to firebomb something, where datacenter ∈ something, but not for the specific purpose of slowing AI.
briankung | 17 hours ago
I think we should distinguish the "stochastic parrot" characterizations in two ways:
I think most characterizations of LLMs as stochastic parrots are using that term to mean that they're useless toys, but I think we need to focus more on the fact that powerful tools should be in service of the greater good of humanity as well as the wellbeing of the planet.
[OP] mond | 16 hours ago
Y'know, I strongly believe that even if you end up being right (which I think is questionable at this point), you should consider that existing cutting edge AI systems are (for many different types of functions) indistinguishable from "being intelligent" by basically all objective metrics you may be able to come up with.
That's a real question: Can you give me a test for "intelligence" which (at least some) humans pass, while LLMs fail it?
spc476 | 14 hours ago
Counting the number of letters in a word?
[OP] mond | 13 hours ago
Eh, I just don't find this one very convincing since it's a lot like asking a colorblind person what color an item is. If you understand how the technology works, it's obvious that during tokenization words (like 'strawberry') are transformed into completely different data, which doesn't have a concept of 'letters'.
Yes, that's a cop-out, I just don't think it's a strong example of a lack of intelligence.
(There's also an issue here where it's unclear where we draw the line between LLMs and "agents". Modern agents are smart enough to run some Python code which counts the letters for them. Does that count? Should it count? I genuinely don't know. From the outside perspective, it does, but it's also obvious tool usage.)
jrgtt | 4 hours ago
it’s an unconvincing answer because llms are not good at it?
FeepingCreature | 2 hours ago
It's an unconvincing answer because of course LLMs could be extremely easily trained to be good at it, there's just no economic value in it.
Add one letter counting task to any of the big benchmarks, the next release cycle they will have no problems counting letters.
What's actually going on here is just that the (older) models don't realize that the task is hard. They don't know they have a disability. And because they can't update on thousands of users trying this, it works just as well as the first day.
Imagine somebody comes to you in the morning and asks you a logic puzzle. You think a bit and, embarrassingly, get it wrong. Unbeknownst to you your memory is reset every night. This is the ten thousandth person who has come to you with this logic puzzle. In their society, it is taken as solid evidence that you are not actually intelligent.
edit: if you have one interaction with a llm, and you demonstrate them that they miscount letters and how to count them manually by spelling the word out, in that interaction the LLM will then count any number of letters correctly. Metacognition, error correction, dynamic skill acquisition, it's all there. It just won't persist beyond the context.
zk | 13 hours ago
What about not hacking into HuggingFace? How about not exploiting loopholes and chained exploits in 3rd party stuff? How about not hacking into sites when asked to just scrape public data? Most humans won't cross that line
Being given a task or a goal and stopping at absolutely nothing to accomplish it like a war machine is not "intelligence", it's the complete opposite.
[OP] mond | 13 hours ago
Sorry, but "These models did things which 99% of humanity would not be capable of (finding zero days, exploiting them, hacking)." is just not doing a good job demonstrating a lack of intelligence.
It demonstrates something, but not a lack of intelligence.
zk | 11 hours ago
Sorry, but, "things which 99% of humanity would not be capable of" alone is not a good measure of "intelligence".
All our computers fit into this with the calculations they do in a split second. Doesn't make it intelligent.
Hell, a microwave and a toaster do things with none of humanity is capable of.
[OP] mond | 4 hours ago
Yes? I agree with this. Intelligence requires a level of generality, which is not demonstrated by special-purpose machines capable of performing a single task really well.
bedrovelsen | 13 hours ago
They didnt hack anything. Open AI never monitored what they were doing out of sheer negligence. It could have easily been avoided
thesnarky1 | 8 hours ago
That's a super odd take, considering Open AI explicitly stated:
"Between July 10 and July 13, agents identified Hugging Face user credentials that were exposed on the internet and used them, together with vulnerabilities discovered in Hugging Face infrastructure, to progressively expand their access. Ultimately, agents powered either by the internal-only research model, or by GPT-5.6, executed code on 41 Hugging Face production dataset server workers, obtained root access on at least one production node, accessed Hugging Face production credentials and limited internal data, and downloaded four private Hugging Face code repositories. This activity resulted in administrator-equivalent access to one connected Kubernetes cluster, as well as the creation of a privileged, host-mounted pod in another connected cluster."
Not sure your word for executing code remotely on 41 production servers belonging to another organization and escalating to root, but I think "hack" is the appropriate common parlance for it.
Whether or not OpenAI was monitoring doesn't really change those actions.
k749gtnc9l3w | an hour ago
One could try to ascribe to OpenAI a weird PR game, after all they are lead by a «not consistently candid» CEO; but Hugging Face also considered it enough of hacking to report to FBI.
thesnarky1 | an hour ago
Not to mention Hugging Face's own initial response clearly refers to it as "abusing code execution paths" to run code, escalate, then move laterally.
dulaku | 5 hours ago
The problem is that "intelligence" is not a well-defined term. There are many senses in which it is used, such as the ability to complete tasks or produce artifacts representing a chain of reasoning, which some types of systems that incorporate LLMs obviously satisfy. So did previous computing systems - they just weren't able to operate on natural language. But there are other senses of the word that do not fit current systems well, e.g. implying a degree of consciousness, humanness, or social awareness that LLM-based systems seem to me to clearly lack. Of course, those are highly subjective qualities - it would be nice if we as a society had better answers for them, but we stumbled into a technology that makes them important before we got there. So other people than myself may just as reasonably insist that those elements don't matter, or that LLMs satisfy them, and all I can really say is I'm unconvinced.
What I do agree with is, at this point, the philosophical angle matters less than the harm that these systems can do, or the harm that people can do by assigning these systems to tasks for which they are unsuited.
From that perspective, of prioritizing harm avoidance and setting expectations for the future, instead of philosophy, I tend to think of an LLM as more comparable to a corporation than a human. I wouldn't describe a corporation as intelligent, but it is similarly capable of producing artifacts that resemble those of intelligent beings, of solving problems that are too complicated for individual humans, causing substantial harm as a side-effect of trying to achieve an underspecified goal, and so on. There are obviously substantial differences, like a corporation using human subunits, but in terms of what baseline expectations to have about how the tool will behave and affect society I think it's probably the closest.
meline | 8 hours ago
I won't give you a test, but consider this: language models still struggle at spatial reasoning and other embodied intelligence tasks. It really is like the Chinese room I think, or a variation of it: you can know everything there is to know about the Eiffel Tower without actually having experienced it in person. Also, I'm still incredibly sceptical of benchmarks in general because I seriously don't believe you can meaningfully benchmark a system that has ingested probably several copies of the entire Internet, and has enough parameters to store a good chunk of it verbatim.
I generally agree with this piece from The Atlantic. A language model is fundamentally different to anything that has remotely existed ever in biology. If you want to accept these things as intelligent, or sentient, or whichever you prefer, you would need to provide some serious insight into the nature of its intelligence.
tauonmsdz | 6 hours ago
While I also generally agree with that Atlantic piece, LLMs are getting better and better at spatial reasoning:
Qwen used in robotics
GPT-6 “driving” a car (albeit obviously not in realtime/at regular speeds)
I think it’ll get more and more difficult to find tasks that can’t ever be achieved by any model – the cat is out of the bag, so to speak – and thus by extension, difficult to define intelligence?
mattgreenrocks | 4 hours ago
Watching Astra play Pokemon is pretty instructive: there's a delay as it decides what to do next sometimes, and Pokemon is a rather low-dimensional space to navigate.
It's amazing that this works, though: last year IIRC the harness needed to scrape information out of emulator RAM and present it to the model instead of the model running off of vision. I suspect there's still a harness in play that persists certain state to memory, though.
meline | 4 hours ago
Thanks for those links, honestly it's baffling to me that such a thing is even possible. I'm not up to the state of the art, sorry, but the Qwen article is saying these are VLMs? So are they specialised LLMs with fine tuning, or some other non-language transformer model?
I will also say, though, that I do view efficiency as a measure of intelligence as well (and maybe this is ironic to say as someone who refuses to use AI in her work). I agree that while a language model probably can be shoehorned into doing anything, I'm not sure that it necessarily counts as intelligence given how inefficient and slow it would be, compared to a similar biological system. Although I will admit that I have a biological bias from my research.
FeepingCreature | 2 hours ago
In a LLM, tokens (combinations of letters) are processed into embeddings (vectors of floats), which the attention mechanism acts upon. A VLM is simply a language model with an additional preprocessor that transforms an image into a sequence of embeddings, one for each square patch, and sufficient training to attend to this input modality.
[OP] mond | 4 hours ago
Fwiw, I don't know the technical definition of VLMs, but many modern LLMs have the capability to natively process vision. The way it works is that an image passed to the LLM is chunked up into smaller images, each of which is then transformed into an abstract token (just like any other token).
Using this, models can basically see directly, using the same tokens they also use for reading etc
This is different from other, more old-school approaches which e.g. required running a classification model on an image, which would then convert the image into text input. As in, the other model said "This is a picture of an orange cat.", and this data would be passed to the LLM.
tauonmsdz | 2 hours ago
As further reading about this for the curious, for example Google’s introducing Gemma 4 12B says:
Pretty cool stuff if you ask me.
tentacloids | 12 hours ago
You might have had more luck asking this at least a year ago. I imagine most lobsters willing to engage in earnest are sick of having the same four genai conversations on repeat by now.
mrpossoms | a day ago
This resonates strongly with me. Thank you for writing it.
bedrovelsen | 13 hours ago
This is what makes it hard to get motivated as everyone on average in the outside world is still stuck on "the government is creating extreme weather to trick us about human driven effects on climate"
Every day is so feeling alienated more from consensus reality and the only effective mechanism of treatment is being humoured by the absurdity of it all.
shonfeder | 20 hours ago
That will happen too. For the mid term economic outlook, it doesn't matter that the tech is powerful (and it can be powerful and bad, btw), it just matters whether the businesses are good: and they aren't good (paraphrasing a recent Eisman interview).
I agree with the gist here tho: we are in the midst of a massive partitioning. Those who are committed to autonomous subjectivity and human creativity will be strongly differentiated from those who are willing (and maybe even eager) to devolve into mere stimulus response nodes in the system. We've been edging that way for a long time (centuries) but now we're in the quickening of this transformation.
We are also seeing differences that used to be latent or minor become very pronounced: i.e., people who can see when a generated image of food looks like a film of cyber-maggots vs. people who don't have the perceptual or sensory refinement to notice; people who are easily impressed by quantity of useless activity vs. those who can quickly tell when frantic activity doesn't actually deliver value; "builders" vs. "understanders"; "thinkers" vs. "reactors".
thev | 5 hours ago
To me this points to a different conclusion: a SWE should only work on projects where a strong vision is necessary.
I don't think many people went into the software development business to do mindless typing work.
pyj | 4 hours ago