Dueling proofs from OpenAI and Anthropic astound scientists

182 points by scientificamerican 15 hours ago on reddit | 49 comments

sarabjeet_singh | 14 hours ago

Some of this stuff will break our basic assumptions, though I wonder what will change practically.

The implications of this kind of math, what are they ?

Spunge14 | 12 hours ago

Well if we keep attacking P=NP and it doesn't go the way we expect, and it's not galactic, then all assumptions about cryptography break and society collapses.

NoNameSwitzerland | 12 hours ago

I would bet N!=NP and see the current result rather as a confirmation.

aboothe726 | 10 hours ago

That’s certainly my preferred outcome. The ETH result is interesting and seems related to me. But it’s not a lower bound, merely proof of non-exponentiality. What’s your intuition regarding P=NP?

Illustrious-Lime-878 | 12 hours ago

Depends. Polynomial time can still be really hard and impracticable. And many crypto designs like one time pads are unaffected if P=NP.

Epic_Tea | 13 hours ago

Ai might know. Lol

loungesinger | 13 hours ago

Gemini prompt: AI written mathematics proofs are breaking human assumptions (regarding math), potentially leading to theoretical and practical changes. Explore and analyze the possible implications of this new math. Provide a detailed summary with examples.

Follow-up Gemini prompt: provide a summary of the above-described analysis in less than 300 words.

UnwaveringThought | 14 hours ago

I just wish people would explain this in lay terms. Given that there are various ways to reach the same result in school level math problems, it is not surprising there are several ways to reach the answer in professional math problems.

I also don't understand the criticism of "brute forcing" some of the "unsolvable" problems, vs... what's the alernative? (Inventing/"discovering" a new function or number or something? )

Its always these flat, uninformative complaints that frankly aren't landing with people who have no reason to care

elehman839 | 12 hours ago

There is an infinite collection of equations called 3-SAT.

Turns out, if you can solve 3-SAT equations, then you can solve LOTS of other problems. So if you get great at solving 3-SAT equations, you are awesome!

But the Exponential Hypothesis claimed that large equations in 3-SAT are intrinsically hard to solve. There is no way to get great at solving 3-SAT equations. No one can ever be awesome. That sucks.

But now AI figured out how to solve the big equations in 3-SAT faster than anyone thought possible, which is awesome! The method is apparently complicated, and I'm not sure anyone understands it yet.

So the Exponential Hypothesis is wrong. Admittedly, the AI approach to solving 3-SAT equations still isn't *very* fast. So maybe this new advance only qualifies as semi-awesome.

Mostly people care about this AI discovery, because this feels like a crack in the dike: if computers can solve 3-SAT equations faster than anyone expected with this new AI method, then maybe there are even better methods that can solve 3-SAT *really* fast. That would be *really* awesome and world-changing.

TeamWorkTom | 12 hours ago

From my understanding it seems like while the science community is reeling from this, it's not because it's actually solved but in part because OpenAI won't even release their proofs so no one can actually verify if they are real.

Seems like it's more than just a solving of the 3-SAT equations.

elehman839 | 12 hours ago

There is a technical side to "reeling" and an emotional side.

Technically, this is a huge deal. The central question in theoretical computer science is, "What problems can computers solve efficiently?" And 3-SAT is sort of the canonical collection of problems for which no efficient solution was believed to exist. But... apparently there is a somewhat efficient solution. So this means that the dominant belief in the field regarding this central question was wrong. Computers, when instructed correctly, are more capable than we thought in a fundamental way.

Emotionally, this is also a huge deal. What is the life purpose now of people who spent decades training their minds to master some intellectual realm (such as complexity theory, in this case), when AI can out-perform every human on earth in that realm by such a large margin? Who is going to employ or even respect such scholars in the future? Pretty scary situation. Some people get angry, some excited, some depressed.

gravy_maker | 9 hours ago

"What problems can computers solve efficiently?" is a common layman-friendly formulation of complexity problems, but it omits something very important.

The hierarchy of complexity classes defines the rate of growth in running time of an algorithm as the input approaches infinity. This does not, however, mean that every algorithm with lower asymptotic complexity is necessarily faster, for realistic inputs and on realistic machines, than an algorithm with higher complexity.

Consider an algorithm which takes O(n^1000), and an algorithm which takes O(2^n). The former is polynomial, and the latter is exponential. Yet, the former yields a greater value - ie. a greater running time - than the latter until well over n=10,000.

This is a known and well-understood phenomenon in computer science. Such an algorithm - which is in a lower complexity class, but which is nevertheless highly impractical - is called a "galactic algorithm": so named because they would never be practical for any data on Earth.

This is what this result offers - a galactic algorithm for solving 3-SAT. So while it might be a theoretically significant finding, I wouldn't call it technically significant, necessarily, because that sounds to me like you mean technololgically/practically.

In much the same way, if we found tomorrow that P=NP, but that the bounding polynomial for solving NP-hard problems was something ridiculous like O(n^100,000,000), encryption would probably be pretty safe if that algorithm couldn't be improved upon, because for any realistic problem, the resultant algorithm would be less practical than the theoretically-more-costly higher-complexity approaches.

WillingShilling_20 | 11 hours ago

Okay but where are the proofs for this equation?

tending | 11 hours ago

This isn't the first time they've done this, and there was always a proof release eventually. It's very likely true.

IAmNotMyName | 10 hours ago

What are their applications?

elehman839 | 10 hours ago

Qualitatively, stuff closely related to 3-SAT has a flavor of packing, scheduling, route-finding, logic verification, data compression, logistics, etc.

UnwaveringThought | 10 hours ago

Excellent. So then why not find ways to collapse the complexity and let that be the breakthrough? Eg, work in tandem with the computer

Cool_Address_2571 | 9 hours ago

They’ve haven’t released the actual “solution”, nor the prompts and computational power it took to get there. So far all we’ve heard is that AI solved it. There’s no reason to think we won’t get this info eventually, but right now it’s being gate kept for (I would assume) marketing purposes (“only OUR AI was able to figure it out!”)

This is Scientific American, it is written for the layperson. It's not really possible to write an article involving higher-dimensional geometry at the ELI5 level, but it's about as simple as it can be.

loungesinger | 13 hours ago

I just wish someone would copy and paste the article here since it’s behind a pay wall.

Hostilis_ | 12 hours ago

Dueling proofs from OpenAI and Anthropic astound scientists

The results resolve major problems in math, physics and computer science. But like other recent AI-generated “slop drops,” the releases are already controversial

By Joseph Howlett & Peter Hall edited by Lee Billings

Just a few weeks after dropping hundreds of artificial-intelligence-generated math breakthroughs on a field that’s already in crisis, OpenAI is piling on more.

Yesterday the company dropped three massive proofs that touched on math, physics and computer science, sending shockwaves rippling through those fields. Experts are still staggering.

These latest releases from OpenAI continue to contradict the recommendations of the Advisory Group on Mathematics and Artificial Intelligence (AGMAI), the expert panel the company organized in response to widespread criticism from mathematicians. The large language model (LLM) that produced the new proofs remains proprietary, and the exact prompts used, time spent computing and “chain of thought outputs” involved have not been made public. Two of the results coincide with closely related work involving OpenAI’s competitor Anthropic.

OpenAI’s approach to two of the three results also directly violates a clarifying statement AGMAI issued just a day before they were released by designating specific experts to shepherd the results into the public sphere. Among the work of such experts, a paper relating to the “million-dollar” Yang-Mills problem (one of seven Millennium Prize Problems set by the Clay Mathematics Institute in 2000) has still not been made public. Writing that “there is no scientific reason for delaying ... any further,” AGMAI has already called on OpenAI to release the paper.

Here’s a quick breakdown of what the three new results are and why they matter.

Kakeya The Kakeya conjecture is about swirling a needle or stylus around in the air cleverly—so that it points in every direction once but sweeps out as little volume as possible. Specifically, the conjecture says that the volume cannot be less than a certain number. Surprisingly, this simple geometry problem is one of the most important conjectures in harmonic analysis, the study of how all kinds of mathematical objects can be built from waves.

Last year mathematicians Hong Wang and Joshua Zahl proved the conjecture in three dimensions in one of the biggest math breakthroughs of this century. The feat won Wang a Fields Medal—math’s highest honor—at this summer’s International Congress of Mathematicians in Philadelphia. But the problem remained open in higher-dimensional spaces.

In last month’s “slop drop,” OpenAI’s internal model solved the four-dimensional Kakeya problem. Now it’s solved the problem in every dimension, proving that there’s a limit to how small a space a high-dimensional stylus can take up.

Yang-Mills The Yang-Mills problem, one of Clay’s five million-dollar Millennium Prize Problems that remain open, is about particle physics. Specifically, it asks whether the mathematics that underpins the “Standard Model” of particle physics is actually sound.

This question comes down to two parts, referred to as “existence” and “mass gap.” The Clay Mathematics Institute’s definition of the problem asks for answers to both. Today OpenAI claimed to prove the first of the two—that the “Yang-Mills theory” is mathematically consistent in all relevant cases: in other words, it “exists.” The part of the Clay problem that remains open is more quantitative, asking whether a certain number in the theory, called the “mass gap,” is always bigger than zero.

In the late 1980s, a little more than a decade before the Clay Mathematics Institute existed, mathematician Tadeusz Balaban published a series of massive, dense papers that sketched out a path to prove the “existence” part of the problem. As voluminous as “Balaban’s program” is, it only codifies the problem’s broad contours—filling in the gaps to construct a mathematically rigorous proof would demand far more laborious and exhaustive calculations. The project is so immense—and Balaban’s papers are so difficult to parse—that no one has carried it out more than three decades later. “He basically did it,” says Roland Bauerschmidt, a mathematician at New York University. “But I think there’s no one who understands these papers and can tell whether they’re correct or not.”

Recently, sensing that a race was on to finish Balaban’s program, some experts began using LLMs to churn out the necessary tomes of tedious mathematical steps. Working with Anthropic employee Levent Alpöge, Ethan Sussman, a mathematician at Northwestern University, created and publicized a public GitHub repository to do just that. They were making considerable progress and believed they were near to finishing the program. OpenAI’s claim—with no publicly available proof—appears to have beaten them to the finish line.

Exponential Time Hypothesis The exponential time hypothesis (ETH) is about computational complexity—the science of ranking computer science problems by how hard they are.

Specifically, ETH suggests that a famous task called the 3-satisfiability, or 3-SAT, problem cannot be solved in less than exponential time. This means that as the number of variables in the problem increases, any algorithm that solves it will run exponentially slower. OpenAI’s proof provides an algorithm to solve 3-SAT that is just a tiny bit faster than exponential time, refuting ETH.

ETH is important because of its relationship to “P versus NP,” which is another of the Clay institute’s Millennium Prize Problems and widely considered to be one of the most difficult, with major implications for modern encryption.

First suggested in 1999, ETH can be thought of as an even stronger version of P ≠ NP (the conjecture that P does not equal NP, which the majority of people in the field believe) and has been treated as a bedrock tenet of computer science by most experts. Because 3-SAT is a problem computer scientists know to be central to the debate, ETH would show that there were problems that were not only fundamentally hard to solve but that would also scale up so brutally that they would be effectively unassailable.

Although the new result doesn’t necessarily translate to a transformative breakthrough about P versus NP, the potential effects are so sweeping that OpenAI has tapped outside experts to vet its claim. Russell Impagliazzo and Ramamohan Paturi, two computer scientists at the University of California, San Diego, who first suggested ETH, as well as Ryan Williams, a computer scientist at the Massachusetts Institute of Technology, are now working with OpenAI to perform an independent investigation of the result. The company will also sponsor a computer science conference about the proof as part of this investigation, according to Williams.

The result hasn’t been without controversy, either. Mere hours after OpenAI’s announcement, Anthropic posted its own independent result disproving ETH, based on work by an internal Anthropic model.

UnwaveringThought | 10 hours ago

Yeah i read the cut and pasted version below. It purported to explain spinning the needle around and how that was 2d and this is 4d. But the COMPLAINERS have failed to explain why they don't like the solution in a cognizable way. Do you have a condescending answer for that?

It's an editorial decision. Scientific American was founded in 1845 so this is not just random blogspam, but actual professional technical writers. This might be a you problem.

UnwaveringThought | 8 hours ago

You aren't addressing the call of the question. What are mathematicians even complaining about?

I understand, but you asked for a condescending answer.

Sorry. But also I don't know how to answer. There was a specific proof that worked in a limited number of spatial dimensions, and they've managed to generalise that to higher dimension spaces. I would not expect it to be immediately useful for any practical application. Could be, I don't know.

hustl3tree5 | 11 hours ago

Discovering/inventing a new function/math are why these problems are pursued

UnwaveringThought | 10 hours ago

Ok so they don't like that in fact they can be solved with old fashioned math? I would posit, these weren't worthy problems if they didn't actually require something new; Right?

Johnny20022002 | 11 hours ago

The brute force “criticism” can’t even land because the problems can’t be brute forced the solution space is too large.

Apprehensive-Ad9523 | 11 hours ago

It comes directly from the ai companies themselves

NaiveComfortable2738 | 8 minutes ago

Developing new, comprehensible concepts and interesting ideas to solve a problem is far more valuable than solving the problem itself.

For example, attempts to prove Fermat's Last Theorem in the nineteenth century led Ernst Kummer to develop the theory of ideal numbers, which eventually became one of the foundations of modern algebraic number theory. The theorem itself was finally proved by Andrew Wiles, but the mathematical significance of the ideas developed along the way extends far beyond the original problem.

The history of mathematics is full of similar examples. More often than not, the detours, dead ends, and failed attempts to solve a problem turn out to be far more valuable than the solution itself.

FrogLaner | 14 hours ago

While these examples have their problems, it sounds like AI could form a new type of supercomputing tool to solve complex mathematics with a greater degree or flexibility

xeoron | 14 hours ago

We know models lie with just basic math, so I hope people are checking the work

FrogLaner | 14 hours ago

If you read the article you would know that they are

eliminate1337 | 14 hours ago

They’re having AI write proofs in source code (the Lean theorem prover) which checks them for correctness in a non-AI way.

versacealexander | 10 hours ago

Isn't the question whether the Lean proofs match the problem? AI writes plain language proof -> Humans don't understand it -> AI says the proof is Navier-Stokes -> Humans ask it to formalize the proof in lean -> AI formalizes a proof that resembles NS but actually breaks down to 1==1 -> "see I proved it"

Is that not a risk?

eliminate1337 | 10 hours ago

Yes it’s a risk and one that exists for human written proofs as well. Human review is needed for making sure the Lean statement matches the actual problem but that’s far easier than verifying the entire proof.

versacealexander | 10 hours ago

Yes but humans have the deterrent of losing credibility esp for a willfully deceptive proof. And the benefit that at least one human should be able to understand the plain language proof. It seems like xeoron is making a valid point.

xeoron | 11 hours ago

I am surprised it is not using Snowball

True-Grab-5288 | 14 hours ago

They are literally writing lean proofs and if you don’t know what lean is you shouldn’t be making this comment

Ok_Cabinet2947 | 10 hours ago

I think you might still be living in 2022.

RobBobPC | 12 hours ago

And how much of this is AI generating a false answer ?

Bowl_of_Cham_Clowder | 11 hours ago

These proofs are formalized in Lean, so if they are wrong it’d be because both models modified the direction of the proof independently. Possible but not super likely.

Anecdotally and by the data, frontier models are pretty good at not generating false answers. They are using far more powerful models + order of magnitude more agents than anyone can use publicly. It’s unlikely they one shot the result either, they just aren’t publishing the hundreds of failed attempts, nor exposing the amount of input from professional mathematicians.

Trick_Pin_2139 | 3 hours ago

lol.

Quakedogg | an hour ago

I’ve worked long enough with AI to understand this: it’s a probabilistic system that can smart guess/ brute force it’s way to an answer. Testable tasks, like solving math problems are its daily bread. Give it enough time, GPU’s and electricity, and it will stumble its way onto an answer.
If humans solved these problems, the reasoning and explanation would make sense because we are efficient.
Mathematicians can’t make sense of these proofs because the LLM and agents tried all kinds of nonsense and stumbled upon the answer to unlock the problem. The chain of thought is an incoherent mess of decisions and plans, probably Gigabytes worth. Sifting through it to come up with a proper reasoning chain that helps us understand the problem is difficult .