The title seems misleading, and reading the article explains the reason more clearly. There's nonsense OKR's and objectives at these companies to burn as many tokens as possible. It turns out that when you make a metric out of token usage, it unsurprisingly ends up becoming extremely expensive.
Inference is affordable, and you don't need a SOTA proprietary model to get a lot of use out of this technology. While you likely will still need a human engineer for quite a while longer, I don't agree that some number of humans + an LLM is going to be (or will ever remain) more expensive than just hiring more humans.
They may as well have just said:
Company institutes an OKR that the IT division must spend over $1000/day/developer (fictious number).
Company is surprised when IT division is costing far more than it did before. Company increases this to $1500/day/developer to build a system to identify why this has happened.
I feel like vibe coding is less of an issue than vibe leadership at this point, and vibe leadership has nothing inherently to do with AI. These people are getting a vague feeling in their giblets, and then chasing it to the illogical conclusion no matter the cost or outcome.
I'm not sure that vibe leadership is a new thing and in fact may be a redundant term. I've worked for enough companies to get the sense that doggedly following vague feelings in their giblets is what leadership has been since 2008.
I won't deny that sometimes it works but there's much more coverage on when it does that when it fails which only serves to amplify the survivorship bias around it.
The media seems hellbent on torching AI. My news feeds are nothing but stories about the evils of data centers, how useless AI is, and how much everyone hates it.
The media is hellbent on torching it, and on propping it up against all reason too, both things can be true. HN is no exception. It's another noisy room problem where the distortion in dialogue is rapidly leading us into a distorted reality. https://thenoisyroom.com/
For people who are actually interested in reality, participation in the mainstream discourse either way is a strategic error. The best thing to do is to check out from all of it, actually read the literature and listen to the technical heros who are working at the edge, and stop reading the pro/anti marketing noise from the media or corporate PR
Again you're probably thinking of the forum discussion / booster blogs / executive interviews that I'm suggesting you should leave behind. Papers are the only place left where nuance is even allowed and might actually be encouraged. Just try it, you'll be surprised. Depends on the research but.. a lot of it is kind of incentivized to align with AI skeptics actually because it leaves many things open for invention, study, and fixes
> and listen to the technical heros who are working at the edge
that's terrible advice. those guys dedicate their lives to the advancement of this field. there's no way you will get a tempered, balanced answer from them. none of them will gravitate towards "yeah, maybe we should stop or slow down for a while".
From AI companies’ perspective, it’s free press… why would they even think about stopping people talking about it!
This about it like this - if you were a CEO of a company that ONLY made garden gnomes, would you rather a) nobody ever talk about garden gnomes, or b) garden gnomes be in the news every day, people protesting because they’re losing their jobs because of garden gnomes, companies making billions and collectively investing trillions to making garden gnomes, people starting startups to support the garden gnomes pipeline, consumer electronics prices having huge variance because of the demand to support garden gnomes etc.
When you’re one of the largest garden gnomes companies in the world, you want garden gnomes to saturate the zeitgeist
Seems like a strategy that could backfire, if Congress passes legislation outlawing the manufacture, sale, distribution, possession, and admiration for garden gnomes. PT Barnum only thought there was not such thing as bad publicity because he was pulling up the stakes and leaving town before anyone woke up.
I am afraid that the TL would be uncomfortable if they have no human team members but only agents, which means they have no space to pass the bulk and have to take responsibilities for the business results.
And in a way this feels like a good thing (from a corporate strategy perspective). If MS really wants to compete with Claude Code they will need to dogfood to have even a hope of ever catching up.
As much as I may dislike MS, their software or their practices I have to admit that they have pulled this off at least once before. Back in 2019/2020 their Teams web client was absolutely atrocious and utterly unusable on Linux. Sometime in 2023/2024 it had become quite tolerable and worked mostly better than Google Meet. (Screen sharing options in Teams suck to this day, though.)
But aren’t the revenue numbers that have investors foaming at the teeth based on that “tokens as a metric” world? It can’t be both an explosive growth business and also only ROI with more disciplined spend.
The premise of this article is incorrect - MS isn't cancelling Claude code internal usage because of AI costs too much, they're cancelling it because GitHub copilot is the compete product and they want their employees to use their product.
It's the same reason Teams got so much attention during lockdown.
Yeah, they conflate Microsoft's actions (which are not about cost) with a random quote from the "vice president of applied deep learning at Nvidia," who says that compute costs more than people on his team -- but his team isn't using LLMs for software development, they're literally a deep learning team that is burning compute in deep learning development ways.
If people would do even a little bit of math, they'd see that Microsoft can't possibly be paying more for AI than for developers: They have about 80K employees in product development roles. Senior developers probably cost them $400K all-in.
Do they have a $32 billion Claude bill? I suspect they do not.
Both things can be right: Claude costing too much and migrating employees to copilot, which hopefully will decrease cost as it owns the product, which will in turn increase usage and feedback.
> It's the same reason Teams got so much attention during lockdown
Not sure I see the parallel with the point you were making
Anecdotally from people I know from MS is that adhoc slack usage was popular up until lockdown, at which point more resources was poured into Teams due to everything going remote and internal slacks were frowned upon.
I see a parallel here where a competitor's product is taking over and MS leaders see it becoming an existential problem and are putting their foot down and pushing internal users to the company's products.
Also if half the company don't even want to use something made by the other half it's a bad look lol
They've made a hardware LLM that reaches over 14k TPS on Llama 3.1 8B, and you can try it here: https://chatjimmy.ai/
So clearly hardware LLMs are the future, and the cost will be drastically reduced. But I know that all the AI labs want to create a perception of high prices forever.
The 'tokenmaxxing' trend is probably the more inane ideas emanating out of this whole AI wave. It goes in the opposite direction of efficiency and productivity maximization. Yet, it has wide acceptance.
Burning tokens is as easy as throwing dollars in a furnace.
Token usage is not a good measure of productivity.
Problem is nobody has really been able to figure out how to gauge productive AI engagement.
Are your developers maximizing productivity or are they burning tokens or resisting change.
Somehow this 'ai companies will never be profitable' is believed by so many. It's often used by those who don't like AI. There is no doubt that ai is the most lucrative business currently out there. It will only get better. Faster hardware, better algorithms
The usage of AI has to be put in context for cost analysis.
A lot of people I see are using AI to beautify their documents, their slack conversations, emails, generating big enough documents with small prompts. Sending a slack message or email should not have required AI within the company. Its wastage of resources and time, just to make it sound better without changing much of the meaning.
No! - me after reading the title. I am unable to hire any more interns and junior devs. Simply because there is no way I can justify the time and money spent behind them. Money is not the issue - time taken to make them productive is not worth now.
1. Article does not quote anyone at Microsoft saying AI is more expensive than employees.
2. Article says the nvidia deep learning team spends more on AI than employees, but that makes sense - the goal of that team is deep/heavy AI use, not agentic coding etc.
> But with a token-based pricing system, the work gets more expensive with more use and better efficiency.
Why do they claim costs increase with “better efficiency”?
[OP] nreece | 15 hours ago
missedthecue | 15 hours ago
tovej | 14 hours ago
The fact that AI is more expensive still comes through, even though Microsoft does not state this explicitly.
hsbauauvhabzb | 13 hours ago
xigoi | 13 hours ago
raincole | 13 hours ago
scronkfinkle | 15 hours ago
Inference is affordable, and you don't need a SOTA proprietary model to get a lot of use out of this technology. While you likely will still need a human engineer for quite a while longer, I don't agree that some number of humans + an LLM is going to be (or will ever remain) more expensive than just hiring more humans.
leecommamichael | 14 hours ago
no-name-here | 11 hours ago
andwur | 14 hours ago
I feel like vibe coding is less of an issue than vibe leadership at this point, and vibe leadership has nothing inherently to do with AI. These people are getting a vague feeling in their giblets, and then chasing it to the illogical conclusion no matter the cost or outcome.
juancb | 13 hours ago
I won't deny that sometimes it works but there's much more coverage on when it does that when it fails which only serves to amplify the survivorship bias around it.
baron816 | 14 hours ago
ian_j_butler | 14 hours ago
For people who are actually interested in reality, participation in the mainstream discourse either way is a strategic error. The best thing to do is to check out from all of it, actually read the literature and listen to the technical heros who are working at the edge, and stop reading the pro/anti marketing noise from the media or corporate PR
parineum | 13 hours ago
Sounds like a great way to get the rose colored view.
ian_j_butler | 59 minutes ago
boca_honey | 13 hours ago
that's terrible advice. those guys dedicate their lives to the advancement of this field. there's no way you will get a tempered, balanced answer from them. none of them will gravitate towards "yeah, maybe we should stop or slow down for a while".
sysguest | 14 hours ago
for ML training loads, it just doesn't make sense to build them near residential areas for few millisecs
cicko | 13 hours ago
Absolutely f'ing not
no-name-here | 11 hours ago
alfiedotwtf | 13 hours ago
This about it like this - if you were a CEO of a company that ONLY made garden gnomes, would you rather a) nobody ever talk about garden gnomes, or b) garden gnomes be in the news every day, people protesting because they’re losing their jobs because of garden gnomes, companies making billions and collectively investing trillions to making garden gnomes, people starting startups to support the garden gnomes pipeline, consumer electronics prices having huge variance because of the demand to support garden gnomes etc.
When you’re one of the largest garden gnomes companies in the world, you want garden gnomes to saturate the zeitgeist
NoMoreNicksLeft | 12 hours ago
alexwwang | 14 hours ago
olalonde | 14 hours ago
bostik | 13 hours ago
As much as I may dislike MS, their software or their practices I have to admit that they have pulled this off at least once before. Back in 2019/2020 their Teams web client was absolutely atrocious and utterly unusable on Linux. Sometime in 2023/2024 it had become quite tolerable and worked mostly better than Google Meet. (Screen sharing options in Teams suck to this day, though.)
repeekad | 14 hours ago
Paradigma11 | 12 hours ago
phyzix5761 | 13 hours ago
verisimi | 13 hours ago
Readerium | 14 hours ago
Or train your own power efficient stack.
chopete3 | 14 hours ago
Hearsay information and click bait.
charles_f | 13 hours ago
lacy_tinpot | 14 hours ago
What if companies both don't see a large return on investment, and at the same time can't reduce their AI spend?
bentcorner | 14 hours ago
It's the same reason Teams got so much attention during lockdown.
mkozlows | 14 hours ago
If people would do even a little bit of math, they'd see that Microsoft can't possibly be paying more for AI than for developers: They have about 80K employees in product development roles. Senior developers probably cost them $400K all-in.
Do they have a $32 billion Claude bill? I suspect they do not.
charles_f | 13 hours ago
> It's the same reason Teams got so much attention during lockdown
Not sure I see the parallel with the point you were making
bentcorner | 12 hours ago
I see a parallel here where a competitor's product is taking over and MS leaders see it becoming an existential problem and are putting their foot down and pushing internal users to the company's products.
Also if half the company don't even want to use something made by the other half it's a bad look lol
charles_f | 11 hours ago
upbeatlinux | 14 hours ago
nullbio | 14 hours ago
They've made a hardware LLM that reaches over 14k TPS on Llama 3.1 8B, and you can try it here: https://chatjimmy.ai/
So clearly hardware LLMs are the future, and the cost will be drastically reduced. But I know that all the AI labs want to create a perception of high prices forever.
coredev_ | 11 hours ago
henry2023 | 14 hours ago
ChicagoDave | 14 hours ago
With discipline, it’s an aggregator.
https://devarch.ai/
hsbauauvhabzb | 13 hours ago
ChicagoDave | 10 hours ago
baigy | 14 hours ago
Shitty-kitty | 14 hours ago
philipallstar | 14 hours ago
polski-g | 14 hours ago
Opus is expensive. And almost always unnecessary.
holoduke | 14 hours ago
parineum | 13 hours ago
The important part is how much and over what time period.
startpage_com | 14 hours ago
sandeepkd | 14 hours ago
A lot of people I see are using AI to beautify their documents, their slack conversations, emails, generating big enough documents with small prompts. Sending a slack message or email should not have required AI within the company. Its wastage of resources and time, just to make it sound better without changing much of the meaning.
loeg | 13 hours ago
DevKoala | 13 hours ago
It is probably more expensive for Microsoft now since the Anthropic tokens were subsidized.
vittore | 13 hours ago
wileydragonfly | 13 hours ago
SebastianSosa | 13 hours ago
ghoshbishakh | 13 hours ago
mbernstein | 13 hours ago
It’s also about an influx of new ideas, different lenses to view problems, and connections to people like them, amongst a number of other reasons.
tomhow | 13 hours ago
https://news.ycombinator.com/item?id=48238896
(Note article says "Microsoft has reportedly begun canceling most of its direct Claude Code licenses, according to The Verge.")
no-name-here | 11 hours ago
1. Article does not quote anyone at Microsoft saying AI is more expensive than employees.
2. Article says the nvidia deep learning team spends more on AI than employees, but that makes sense - the goal of that team is deep/heavy AI use, not agentic coding etc.
> But with a token-based pricing system, the work gets more expensive with more use and better efficiency.
Why do they claim costs increase with “better efficiency”?