Rumors of this were going around for a few days already which is why I'm a lot less surprised than I should be. But still it's worth reflecting on how little we're surprised at this point that mathematicians given access to agents are now able to solve some really long standing issues.
I think it's worth keeping in mind that the pure math community is pretty small and, up until now, was fairly artisanal - i.e., as far as I know, you mostly didn't have well-funded "mathematics labs" throwing millions of dollars in compute at problems on a whim. In particular, the Navier-Stokes thing appears to have cost OpenAI more than the prize they possibly stand to win. So I guess there are two interesting hypotheticals. First, if we had the same drive and funding in the pre-LLM era, would have gotten comparable advances out of it? And second, will the AI labs keep spending that kind of money on math proofs in the long haul?
I think you can still answer the first question in the negative if you look at my own field. Specifically, vulnerability research has a lot of parallels to math (a good chunk of it is discrete mathematics by another name) but also far more practitioners, far more money thrown at it, and a robust history of large-scale automation. And even here, AI is coming up with a lot of good vulns. None of them are "new science" - things that are qualitatively different from what humans can think of - but it's real. And it's worth $$$. I'm not sure if it's worth all the $$$ in the world, but we'll find out eventually.
The numbers for sure are staggering. Assuming a 95% cache hit rate and prices similar to Astra, it would come to 18M USD for that run. My intuition is that after a while people will no longer be interested to throw that much money at problems.
In general right now the actual unsubsidized token prices are too high for most of the problem we are throwing at them. The economics in general are quite off and everyone is hoping for that order of magnitude improvement in token prices which so far has not emerged.
You could make your 18 million dollars invest back in a week if you solved Navier Stokes and kept it secret for your application, the implications for this kind of stuff is pretty mind boggling.
OpenAI did this for marketing, the next company that pays OpenAI for this kind of work and keeps it secret is going to be the scary stuff. Imagine proofs for P=NP or any other groundbreaking problems being hoarded by the technocrats. Even having access to that kind of a thing for hours could make you millions of dollars
I don't understand what you're saying. How do you monetize the knowledge that a system of partial differential equations has a singularity? This isn't "we discovered a secret formula that lets you develop a better airplane".
[OP] ni5arga | 16 hours ago
also, it is very much worth mentioning that:
tristan buckmaster alleges openai's proof builds on his and levent alpöge's recent work on related equations after learning of it. openai denies this.
mitsuhiko | 15 hours ago
Rumors of this were going around for a few days already which is why I'm a lot less surprised than I should be. But still it's worth reflecting on how little we're surprised at this point that mathematicians given access to agents are now able to solve some really long standing issues.
lcamtuf | 14 hours ago
I think it's worth keeping in mind that the pure math community is pretty small and, up until now, was fairly artisanal - i.e., as far as I know, you mostly didn't have well-funded "mathematics labs" throwing millions of dollars in compute at problems on a whim. In particular, the Navier-Stokes thing appears to have cost OpenAI more than the prize they possibly stand to win. So I guess there are two interesting hypotheticals. First, if we had the same drive and funding in the pre-LLM era, would have gotten comparable advances out of it? And second, will the AI labs keep spending that kind of money on math proofs in the long haul?
I think you can still answer the first question in the negative if you look at my own field. Specifically, vulnerability research has a lot of parallels to math (a good chunk of it is discrete mathematics by another name) but also far more practitioners, far more money thrown at it, and a robust history of large-scale automation. And even here, AI is coming up with a lot of good vulns. None of them are "new science" - things that are qualitatively different from what humans can think of - but it's real. And it's worth $$$. I'm not sure if it's worth all the $$$ in the world, but we'll find out eventually.
mitsuhiko | 14 hours ago
The numbers for sure are staggering. Assuming a 95% cache hit rate and prices similar to Astra, it would come to 18M USD for that run. My intuition is that after a while people will no longer be interested to throw that much money at problems.
In general right now the actual unsubsidized token prices are too high for most of the problem we are throwing at them. The economics in general are quite off and everyone is hoping for that order of magnitude improvement in token prices which so far has not emerged.
ondreian | 13 hours ago
You could make your 18 million dollars invest back in a week if you solved Navier Stokes and kept it secret for your application, the implications for this kind of stuff is pretty mind boggling.
OpenAI did this for marketing, the next company that pays OpenAI for this kind of work and keeps it secret is going to be the scary stuff. Imagine proofs for P=NP or any other groundbreaking problems being hoarded by the technocrats. Even having access to that kind of a thing for hours could make you millions of dollars
lcamtuf | 13 hours ago
I don't understand what you're saying. How do you monetize the knowledge that a system of partial differential equations has a singularity? This isn't "we discovered a secret formula that lets you develop a better airplane".
gerikson | 15 hours ago
@mods, please fold into https://lobste.rs/s/ki3ylq/