Satya Nadella says we should assume all AI models are 'compromised'

42 points by jonbaer 18 hours ago on hackernews | 73 comments

https://wallhop.io/https://www.theverge.com/ai-artificial-in...

(wallhop is a 12ft.io alternative I noticed on hn earlier: https://news.ycombinator.com/item?id=50038634 )

I was expecting he was going to finally acknowledge these are trained on CSAM, been compromised and used to perpetuate abusive crimes.

rsalus | 18 hours ago

I assume he's referring to the long-standing Microsoft engineering principles of "zero trust": https://learn.microsoft.com/en-us/security/zero-trust/zero-t...

and "least privilege": https://learn.microsoft.com/en-us/entra/identity-platform/se...

greenthrow | 18 hours ago

Microsoft didn't invent either of those concepts. They are long standing security principles. Especially least privilege.

: dudeinjapan | 17 hours ago
Microsoft invented those principles when they added UAC to Windows Vista. Also, Al Gore invented the Internet.
: rsalus | 14 hours ago
didn't say they invented them.. I'm just saying they follow them rather dogmatically.

igor47 | 17 hours ago

> Unfortunately, Nadella also refers to AI as “super intelligence” throughout the post.

What's the deal with these clowns? Nadella's estimated net worth is like $600M. Is that still not enough for him to buy himself a spine?

timcobb | 17 hours ago

No he's an automaton optimizing for status

coliveira | 17 hours ago

He wouldn't have this net worth if he ever had a spine. Mind you, he's very tough to people under him, and that's a very important characteristic of this kind of people, but he really doesn't have a spine.

bushbaba | 17 hours ago

You assume he’s a human and not a super intelligent being himself

: Xirdus | 17 hours ago
I am pretty sure he's not super intelligent.

NordStreamYacht | 17 hours ago

He has to keep The Orange House happy.

dudeinjapan | 17 hours ago

ELI5 why he needs to please the Dutch royal family?

: anon7725 | 17 hours ago
The Orange House, not the House of Orange.
See also: Peoples’ Front of Judea vs Judean Peoples Front.
: hulitu | 7 hours ago
> ELI5
1337 bro.

blagie | 17 hours ago

I don't quite see the problem here.

My AI is regularly solving master's thesis level, and sometimes Ph.D thesis level problems for the purpose of, e.g. routine tasks like filling out random paperwork.

It's been very odd.

I'd only assume Nadella has access to more advanced models than I do, just as much as I have more advanced models than my relatives.

UnlockedSecrets | 17 hours ago

Can you ask your AI to answer the ultimate question, that being the answer to life, the universe, and everything?

: jonah | 17 hours ago
I hadn't tried that question before. Gemini knew the answer.

idiotsecant | 17 hours ago

Because it's a political loyalty test. Nobody started using it organically until they were ordered to in order to prove their loyalty.

Using it is embarrassing.

dofm | 17 hours ago

It's a bit more than a mere loyalty test, in that Trump knows it is humiliating for them to listen to him on anything like this.

He knows humiliation (from his father and from Putin) and he loves to dish it out to others.

This is Reek, basically.

Terr_ | 17 hours ago

Right, and humiliation rituals are not just about the authoritarian's ego, they do damage that the authoritarian exploits.

People who comply scar their reputation and trustworthiness, so that it's harder for them to build any alliance against the authoritarian in the future, ex: "We all know Bob lied to keep his job... But if they threaten his job again, doesn't that mean he might lie to us too?"

: dofm | 5 hours ago
Right. It’s absolutely crucial to understand what is going on here.
He has a litany of weaknesses, failings and shortcomings, but he is an international expert at humiliating, suborning and breaking people. His mendacity is very nearly unparalleled.

blagie | 14 hours ago

I feel like I'm missing some context. A lot of context.

Whose loyalty test? How?

I mostly heard "super intelligence" a long, long time ago in the context of work on AI / philosophy / futurism which, honestly, sounded pretty detached until the current deep learning sprint, and now sounds pretty prophetic.

card_zero | 14 hours ago

You missed yesterday's edict? https://news.ycombinator.com/item?id=50019684

Wait, the renaming was from a speech on 11 September in fact, I lost track.

: blagie | 13 hours ago
Yes, I did miss it.
Thank you.
That sucks. It's a helpful term. Now, it's politicized.

kumarvvr | 17 hours ago

At 600M, if keeping a spine has a risk of slight reduction in that wealth, then the spine is thrown out in a jiffy.

No rich man or woman is a friend of society.

That is why governments fear revolt from the poor. They have nothing more to lose.

: joquarky | 9 hours ago
They are so proud that they have "earned" their obscene wealth but terrified what would happen if they had to do it again.

tbrownaw | 17 hours ago

I would expect people selling AI to like that term for marketing reasons, completely independent of their opinion of the person who provided a crystallization point to organize the terminology change around.

dumberquestions | 17 hours ago

Not having a spine goes a long way when you're trying to make billions.

khazhoux | 17 hours ago

If he stands up to Trump, Trump declares Microsoft a national security threat and decimates the company. That’s what this is about.

thih9 | 13 hours ago

On one hand I don’t mind the government keeping corporations in check. On the other, the government doing business via random threats and bullying seems worse than corporations.

: timcobb | 7 hours ago
Yeah it makes the USA a "shithole country", to borrow a term

khazhoux | 17 hours ago

His own net worth isn’t the point.

Bend the knee or Trump will destroy your company. And every Republican in Congress is cool with that.

8note | 17 hours ago

the spine here is going with a deeply unpopular guy's dumb opinions and getting the hate mail

downrightmike | 16 hours ago

Actual Indians > Super Indians

hollowonepl | 16 hours ago

600M is not that much actually, just three proper celebrity houses in LA or sth… he’s still poor-man to run a trillion dollar company. SteveB had better deal out that responsibility.

: perelin | 16 hours ago
Was also wondering. Sounds low. Forbes puts him around 1.4B. Still lowish compared to his CEO peers. https://www.forbes.com/profile/satya-nadella/

blululu | 16 hours ago

Super Intelligence is a term of art that has been around for a few decades now. Nick Bostrom gave a the canonical definition for it almost 30 years ago. Super Intelligence is more relevant to the concerns articulated in this article than AI which is rather broad in its purview. Maybe this is Trump stuff? I don't know. If it were something stupid like Freedom Intelligence then I'd be pretty annoyed, but there is a lot of plausible deniability that Super Intelligence is just a better term for what we're all freaking out about. A world where GPT 4 hits scaling limits and it costs $500 a month to run is definitely AI but none of the catastrophes emerge at that level of intelligence. That would actually be a pretty sweet world. But now we get to have our whole civilization ground up into dust by super intelligence instead.

perelin | 16 hours ago

True, but for Bostrom Superintelligence was a capability level for AI and not a replacement for the accepted scientific term. Narrow -> General -> Super.

thiht | 12 hours ago

Stop being obtuse, it’s clearly Trump’s hand up in his ass. No one’s used "super intelligence" before

blululu | 6 hours ago

It was definitely a common term well before 2024, you could please refer to the Wikipedia:

https://en.wikipedia.org/wiki/Superintelligence

Or any of the literature on the subject from the past 30 years: From here

[1] https://nickbostrom.com/papers/how-long-before-superintellig...

to more recently:

[2] https://openai.com/index/governance-of-superintelligence/

For context, Nick Bostrom, Sam Altman, Greg Brockman, Ilya Sutskever are big names in AI. There is a reason they were using this term back in 2023.

: lioeters | 3 hours ago
Superstupid. These people are high on their own hype, oblivious to how ridiculous they sound.

gumby | 15 hours ago

Otherwise suddenly MS is excluded from government contracts (even ones awarded or in progress), justice department will start attacking it etc.

imartin2k | 15 hours ago

To me, this is a confirmation of how much of a free man I am and how unfree these tech billionaires and CEOs are. Because I can choose to say AI, whereas they seemingly cannot.

: hulitu | 7 hours ago
I'm not a number, I am a free man. /s

minraws | 11 hours ago

That's the thing though, he has $600M dollars worth of reasons to bend.

People think that money makes you strong, maybe it does but only upto like a couple million dollars if at all, well whatever amount will give you a lifetime of comfortable life, after that it's all about how to preserve or make more money.

The more you have the more reason you have to do the wrong thing.

arw0n | 8 hours ago

It is actually a very common part of the fascist playbook to demand ridiculous things as proof of loyalty and then declare those not following the enemy. Absolute loyalty is required. For unprincipled people like Nadella and the rest of tech leadership, it is a small price to pay to not get hit with economic sanctions in retaliation.

bigbadfeline | 5 hours ago

Let's not be confused, it's not about spines, it's about shared interests.

cyanydeez | 5 hours ago

Bootlicking is good for shareholders. Shareholders are more important than citizens. Money is speech.

Anyone who doesnt see the calculus is a mental gymnast.

sandeepkd | 17 hours ago

so the grapes are sour for some after all

xscott | 17 hours ago

Someone will vibe-code Office and Windows, and he knows it.

Xirdus | 17 hours ago

Didn't someone vibe code Photoshop this week?

: functionmouse | 7 hours ago
didn't that just turn out to be basically a plagiarised copy of an existing long term open source project that the AI stole?

steelframe | 17 hours ago

Maybe I am living in a bubble, but no company I have worked at in the past 15 years has cared in the least about either Windows or Office.

khaled4vokalz | 17 hours ago

Yeah, after having a meeting at the Oval office, it's for sure everyone got taught by our bro "Dario" regarding containing things as the Chinese are catching up. Give this a break...

gexla | 17 hours ago

I don't get why using “super intelligence” is an issue here. It seems the US admin sort of politicized this and now US companies have their own reasons for selecting terms that may not make sense to people who aren't making the same decisions.

Everything else seems reasonable? Assume the worst and put the controls in place to handle that scenario? Isn't this how we approach security normally?

NOTE: NOT a security expert, so I may be talking out of my...

khazhoux | 17 hours ago

Because literally one person came up with that term, just to put their own stamp on it.

dofm | 17 hours ago

Not just to put their own stamp on it. To force others to use it.

khazhoux | 17 hours ago

Naming it is taking credit for it. “No one’s done as much for tech as me!”

Forcing others to use it is just par for the course with this administration

: dofm | 3 hours ago
> “No one’s done as much for tech as me!”
This is a recognisable symptom of the man's particular neurological makeup.
But you misunderstand what the whole "call it superintelligence", "call it Lake America" thing is about.
This man is a genius at suborning, humiliating and hollowing people out. That is what is happening. He may be an ignoramus, but he's an ignoramus Leland Gaunt.

jiggawatts | 17 hours ago

It is a loyalty test, of the type favoured by dictators, emperors, and heads of mafia families everywhere.

America is lost.

davesque | 17 hours ago

Because it's creepy and Orwellian, not to mention hypocritical (MAGA is supposed to hate thought policing). Do you like people telling you what words you can and cannot use?

: gexla | 7 hours ago
Okay, so you felt like someone was telling you what words to use. I didn't feel that.

tbrownaw | 17 hours ago

Well of course.

If you blindly assume it works and it doesn't, that's a problem.

If your process can handle it not working, it's much less of a problem.

butwhentho | 17 hours ago

Should I then assume that Copilot is "compromised", and given that they've shoehorned it into all Microsoft products, then, by extension, that means all Microsoft products are compromised?

: naruhodo | 17 hours ago
No change there.

dcmatt | 17 hours ago

> More advanced models will require more advanced containment technologies that we need to standardize on.

How about yanking the cords, or maybe a big red “shutdown” button

ChrisArchitect | 17 hours ago

Source: https://twitter.com/satyanadella/status/2108931348857827686 (https://news.ycombinator.com/item?id=50035193)

vehemenz | 17 hours ago

Once again, why do all these CEOs think they are thought leaders all of a sudden?

: monksy | 17 hours ago
Basis - in short: https://www.youtube.com/watch?v=0uwfOWYchpI
: bmitc | 16 hours ago
Rich people in general have this belief, that they somehow got there by objective means and via their superior intellect. I just watched an interview the other day that someone posted where Sam Altman described using a notebook (yes literally pencil and paper) but was describing it as if he had discovered fire. These people are literally blinded by their money.

TZubiri | 17 hours ago

Here's a video of Nadella speaking about AI a year ago.

I couldn't for the life of me understand him, it sounded like when someone I know did cocaine and spent like 8 hours per day with ChatGPT and developed their own internal vocabulary, right around peak AI psychosis era with 4o as well.

One thing you can hear is that he was super excited about AI, which is very obvious by how Copilot was slapped everywhere. And now that the market is expressing their backlash about AI they are just hopping on the bandwagon.

But I remember, and I think the MSFT customers will remember how AI was slapped into everything INCLUDING NOTEPAD.EXE

trog | 12 hours ago

This just feels like more of the war on local models. I expect to see a lot more FUD about this as hyperscalers desperately try to push the message that local models are incapable to keep people on the teat.