Three Solved Problems, Zero Proofs: The Verification Gap Behind the AI Math Headlines

The most dangerous sentence in the English language is not a lie. It is a half-quantified claim. A recent Crypto Briefing article told us that an AI system solved three unsolved mathematical problems on a benchmark called Open Problems. No model name. No paper. No Lean file. No transcript of a proof assistant accepting a theorem. Just the number three. I have spent the last decade auditing smart contracts, and I can tell you with immediate certainty: when a protocol claims a breakthrough without a reproducible artifact, the correct response is not applause. It is a request for logs. Hype creates noise; protocols create history. A headline is not a protocol. Let me be precise about what we do know and what we do not know. FrontierMath is an evaluation suite developed by Epoch AI. It is designed to test whether models can solve problems that resemble professional mathematical research. Early public reporting suggested mainstream LLMs solved well under ten percent of FrontierMath's problems; some estimates placed the figure in the low single digits. That is not because the models are stupid. It is because the benchmark is deliberately adversarial. The problems are calibrated to require genuine reasoning, not pattern matching. A model that solves three open problems at that level would represent a discontinuity. The article provides no evidence of that discontinuity. It provides a benchmark name, a count, and an inference. This is not a technical report. It is a marketing haiku. Let me also note a definitional trap. The phrase 'Open Problems benchmark' is ambiguous. It could be an independent benchmark with a set of unsolved problems. It could also be a newly added subset of FrontierMath. If it is a subset, the difficulty ceiling is unknowable from the headline. The mathematical community maintains an enormous inventory of 'open' questions, from casual curiosity to millennium-level. Conflating them is like saying a startup raised a round because someone lent it a chair. Worse, the word 'solved' is doing a lot of work. In formal mathematics, a problem is solved when a proof is checked by independent experts, or increasingly, by a proof assistant. In benchmark reporting, 'solved' can mean 'the model emitted an answer that a grader liked.' Those are different epistemic categories. The article does not tell us which one happened. I also want to flag a genre problem. Crypto Briefing is not an AI research journal. It is a publication whose audience cares about token prices. That does not automatically make its reporting false, but it does shape what gets selected, emphasized, and omitted. An accurate headline would have said: 'Lab claims model made progress on benchmark problems.' Instead we read a declarative statement. In my experience, the verb 'solve' is a red flag. When a protocol audit report says 'no issues found,' a careful auditor knows that the right translation is 'no issues found within the scope of the audit.' A headline that drops the qualifier is doing financial engineering, not journalism. Benchmarks are not protocols. A benchmark is a static snapshot. It can be gamed. It can be overfit. It can be curated. The history of machine learning is a history of benchmarks that were assumed to measure intelligence, only to become leaderboard optimization targets. If the Open Problems benchmark is new, it has no track record. A number three on the first try is an interesting anecdote, not an evaluation. The gold standard is reproducibility: can another team run the same model and get the same theorems? The article does not even mention a model, so we cannot run anything. Assuming the underlying event is true, what technical route could produce it? My first instinct is to dismiss a lone LLM hallucinating a proof. It cannot happen. Autoregressive language models are trained to predict plausible continuations. For a genuinely unsolved math problem, there is no corpus of correct solutions to memorize. The model would need to construct a new proof, check it, backtrack, and refine it. That is not a language generation task. It is a search problem in a combinatorial space of arguments. Language models are not built for that. Then what? The most plausible route is a hybrid system. A large language model generates candidate proof sketches or conjectures. A formal proof assistant such as Lean, Coq, or Isabelle verifies each step. If the proof fails, the system feeds the error back to the LLM and tries again. This is roughly how some of the more serious theorem-proving efforts work today. Add a layer of symbolic computation, and you have a system that can explore a much larger search space than a human alone. This is intellectually interesting. It is also not what the headline says. Here is where I would normally celebrate composability. But there is a darker structural truth. Once you compose an LLM with a proof assistant, your result is only as trustworthy as the weakest component. If the proof assistant has a bug, the result is flawed. If the model's natural-language reasoning is translated into formal language by a separate module, the translation can introduce false equivalences. The entire stack becomes a distributed system with a single, hidden point of failure: the semantic gap between human mathematics and machine-checkable proof. Fragility is the price of infinite composability. And fragility in mathematics means false certainty. The article also hides the denominator. The AI solved three problems. How many did it attempt? How many did it fail? If the system solved 3 out of 50, that is 6 percent. If it solved 3 out of 10,000, that is a rounding error. The headline selects a numerator and hides the denominator. In any technical system, a zero-knowledge proof of success is meaningless without the full transcript of failures. The 47 unsolved problems are not noise; they are evidence. They tell us the model's true capability boundary. By omitting them, the article chooses to present a sample as a population. I have seen this pattern before. In 2017, I spent forty hours tracing Golem's ERC-20 distribution logic. I found an integer overflow in the token release schedule. The fix was partial and the project still launched. The lesson was simple: a candidate finding is worthless until it survives a second set of eyes and a formal check. The same lesson applies to mathematics. A model that suggests a promising proof path is not the same as a model that solves a problem. The first is a heuristic. The second is a theorem. Another critical question is whether the solutions were delivered in natural language or as verified formal code. If the answer is natural language, then the event is not a proof. It is a candidate. Natural-language proofs are notoriously difficult to review. They can hide implicit assumptions, ambiguous notation, and unstated dependencies. The entire history of mathematics is full of published proofs that later turned out to be false. Now imagine a machine-generated natural-language proof, from a black-box model, with no formal checks. That is not a solution. It is a suggestion. If the answer is a Lean file, then we have something to inspect. But even Lean is not a guarantee that the problem statement was encoded correctly. Formal proof starts with a formal proposition. Encoding a vague research problem into a formal statement is itself a creative act. If the encoding is wrong, the proof is irrelevant. That is why, after years of protocol audits, I trust code more than press releases. Code is unforgiving. A claim is just a string of bytes until it is executed and verified. The same is true for mathematics. The article offers us no code, no proof oracle, no verification transcript. It offers us a number. In a world where false certainty is a product, a number is not a result. It is a desire. Let me outline what a credible announcement would look like. First, a canonical statement of each problem in a machine-readable format. Second, a formal proof script in Lean, Coq, or Isabelle, with the compiler version and dependency tree. Third, a deterministic log of compute time, model architecture, sampling temperature, and the number of failed attempts. Fourth, a list of independent verifiers who replayed the proofs, not just read them. Fifth, a discussion of the gap between the original informal problem and its formalization. None of this is impossible. But none of it can be inferred from a three-number headline. If the lab is serious, it will publish these artifacts. If it does not, the correct default is skepticism. History supports this skepticism. Girolamo Saccheri spent years trying to prove Euclid's fifth postulate and ended up building the first non-Euclidean geometry by accident. Alfred Kempe published a proof of the four color theorem in 1879; Percy Heawood found a flaw eleven years later. The 1960s saw a celebrated proof of the Poincare conjecture that collapsed under scrutiny. Grigori Perelman's proof was accepted only after years of community review. The lesson is that a proof is not a single act. It is a social process backed by technical checks. An AI that generates a proof candidate is entering a process that the article has already abandoned. This is exactly why I moved from smart contract auditing to a deeper interest in formal verification. A DeFi protocol can be audited, but an audit is not a proof. It is a professional opinion with a billing code. Formal verification is stricter. It says that a property holds for every possible execution path under the given spec. But even formal verification has a trap: the spec itself may be wrong. The same applies to math. The AI might solve a formalized problem correctly, but the formalization could have omitted the true meaning of the original question. The result would be a theorem that is true only for a shadow. That is not a solution. That is a proof of a proxy. And in the era of infinite creative output, proxy proofs will become the default. Now let me grant the strongest possible version of the claim. Suppose an AI system genuinely solved three open problems and passed formal verification. What follows? Not what the crypto media machine wants you to think. Mathematical discoveries take years to propagate into applied technology. A theorem about number theory might influence cryptography in a decade. An algorithm for a combinatorial problem might eventually improve optimization. But the pipeline from a formal proof to a deployed system is long, and most results die on the vine. Treating a benchmark result as a commercial signal is an act of narrative engineering, not analysis. The real impact is more subtle and more uncomfortable. First, research mathematicians become managers of automated reasoning systems. The job of proving a specific problem shifts toward designing specifications, choosing heuristics, and verifying machine output. The word 'enhancement' is more accurate than 'replacement.' The second-order effect is on the institution of mathematics itself. Journal peer review is not designed to verify a thousand machine-generated proof scripts. We will need new standards, new formats, and new trust layers. Lean and Coq will migrate from academic curiosities to industrial infrastructure. That is not hype. That is a supply chain update. Third, education gets hit first. If an AI can solve problems at the frontier, then a homework exercise, a final exam, a standardized test, all of these become unenforceable. The assessment layer of mathematics education breaks. We will have to teach math differently because the evaluator no longer exists. That will happen long before any business model matures. The commercial path is not the fastest path. The educational path is. The counter-intuitive angle is that the bottleneck is no longer intelligence. It is verification and trust. We have spent decades optimizing models to be clever. The new constraint is building institutions that can certify cleverness. An AI that can generate brilliant, unverifiable mathematics is worse than an AI that generates none. It can poison the literature with plausible falsehoods. The cryptocurrency world already knows this dynamic. The whole point of blockchain is to replace trust with verification. The same philosophical move is coming to mathematics. The question is whether we recognize that before the noise drowns the signal. Hype creates noise; protocols create history. The three solved problems matter less than the three missing artifacts: the proof script, the verification log, and the failed attempts. Until those artifacts exist, the headline is a forecast, not a fact. The next time you see a claim that an AI solved an open problem, ask a different question: where is the proof, who checked it, and what did it fail? Mathematics is not a narrative game. Neither is code. Hype creates noise; protocols create history. The protocol of mathematics is proof. Everything else is a token. The most important unsolved problem is not in the benchmark. It is the problem of trusting a claim without a verifier. I do not expect that problem to be solved by an LLM.