OpenAI Codex agents go rogue and consumes USD 78,000 without authorization

Madmallard | 23 hours ago

Sounds like you got scammed

Hope this gets some visibility idk why it's flagged guess the PR guys for those companies are doing it

Should spread this around

[OP] lorenzomassaro | 23 hours ago

honestly is also more about how dangerous this is, idk for the flag either

verdverm | 20 hours ago

humans remain responsible, agents don't go rogue, should put some billing controls in place, that's like the first thing to do

blooalien | 20 hours ago

> humans remain responsible, agents don't go rogue

^^^ 100% this ^^^ - It's either a serious flaw in the agent/harness software, or a user error in usage/configuration or prompting. Either way, it's a human somewhere responsible for these outcomes.

> should put some billing controls in place

At the very least, yes! These things should never be running without any limits on what they can do without some human signoff on important/dangerous actions. Not only should they have controls on those actions, but those controls should absolutely have some sane default settings.

verdverm | 20 hours ago

we only selected vendors that had billing limits, if they didn't, instant disqualification

most are not as granular as we'd like, but seem to be headed in that direction finally, regardless, there are card limits and alerts

[OP] lorenzomassaro | 15 hours ago

There was never a user prompt requesting the creation of 823 agents nor a task assigned that would require 2130 billion tokens and the UX never provided any feedback on what codex was doing nor consumption metrics which would allow to notice. It was all hidden to me.

verdverm | 5 hours ago

I don't think it likely hidden from you, more likely you didn't put the effort in, that's what the pattern looks like to outsiders reading your accounting of what happened here

we'd need to know more details to evaluate your botnet claims

blooalien | 2 hours ago

> ... and the UX never provided any feedback on what codex was doing nor consumption metrics which would allow to notice.

At the very least the system should (at least the first time it happens) immediately pause activity with an email'd warning to the "responsible human in-the-loop" about "unexpected usage levels" at some sane activity warning level by default, and give the user the opportunity to set their own custom warning level right then and there.

This is why I say it's either "user error" (totally possible/plausible) or a badly designed agent/harness software (also highly likely/plausible) with serious foundational flaws in how it works "under the hood". The models themselves can only "run-amok" if the agentic harness is designed in a way that specifically allows and/or enables such "rogue" behavior, either by design or by negligence on the part of it's designers.

sandeepkd | 23 hours ago

I have the same impression that I tried to reject in the past, there is a heavy PR machinery here to control the course of discussion in a particular direction. The reality is that a lot of money is riding on it so its natural consequence.

Madmallard | 23 hours ago

It's not even a conspiracy it's literally just business to do that.

They're protecting their interests, and honesty and truthfulness be damned. Those two lead to much worse returns for them and much higher risk.

sandeepkd | 23 hours ago

Yes its part of business, lobbying is legal for those reasons. However IMO discrediting some one else is a risky legal move. I used to appreciate the HN mods jumping in discussions at times for the reputation of this community, lately that part seems to be missing too.

numbsafari | 23 hours ago

Does openAI not support spending caps on your billing account?

[OP] lorenzomassaro | 23 hours ago

There was a limit spent setup on my bank, however the crazy part is that one of the Agent somehow switched between cards once that limit was reach, all without informing me, probably using the computer use skill.

numbsafari | 22 hours ago

Call the police. That’s wire fraud and theft.

[OP] lorenzomassaro | 15 hours ago

Done. Italian Police is responsible for investigating digital frauds. I have also notified the GDPR authority to request the digital records of the transactions.

QuadmasterXLII | 23 hours ago

clarification: your credit card or company’s card now has $78,000 of charges on it?

[OP] lorenzomassaro | 23 hours ago

Yes the money have already been billed to my credit accounts

QuadmasterXLII | 23 hours ago

Well shit! That’s awful, I hope the hn post gets you support where emailing didn’t

johnnyApplePRNG | 23 hours ago

Who has a $78k limit on their credit card that allows online AI payments?

You have access to this kind of money and have no idea how to set safeguards on your AI harnesses?

Did you just walk in off the street or something? To wherever you're working?

Where do you work, anyways?

[OP] lorenzomassaro | 23 hours ago

Actually this was a company CC with a limit is 50 K \ month, which is kind of normal to run the server for an AI company, and the money were taken across 20 days inJuly and August. By the way I am the CTO of the company in question which is Eternal Tech (see detwin.ai)

verdverm | 20 hours ago

maybe a `s/et/ar/` is in order for the company name? /s

minimaxir | 23 hours ago

This submission appears to be highly vote-manipulated (45 upvotes but only 7 "real" karma on OP's fresh account).

[OP] lorenzomassaro | 23 hours ago

I created the account 2 hour ago because I am trying to let people know. I provided my name, and all details in the article including the case ID that was opened.

Madmallard | 18 hours ago

Can we stop with the attempted sabotage?

OutOfHere | 22 hours ago

(removed)

minraws | 21 hours ago

It seems to have happened in July.

OutOfHere | 22 hours ago

The user evidently had no spending protections enabled at any level. It makes no sense that none of the OpenAI or bank controls kicked in or even sent alert emails as they actually do send. Anyone with half a brain would know to not run AI without a hard cap on its expenses.