Anthropic researchers reported major experiment: AI assistance formalizing parts of Fermat’s Last Theorem in Lean, interactive theorem prover.
Headline simple. Work not.
Fermat’s Last Theorem: no positive integers x, y, z satisfy:
x^n + y^n = z^nfor integer n > 2.
Pierre de Fermat wrote claim in margin, 1637. Claimed “truly marvelous proof,” margin too small. Proof never appeared. Centuries later, Andrew Wiles proved theorem using elliptic curves, modular forms, Galois representations.
Human-readable Wiles proof already difficult. Machine-checked version another problem.
What formalization means in Lean
Mathematical papers skip steps. Writers assume readers fill details, recall lemmas, recognize claims.
Lean no assumptions.
Lean proof assistant. Checks formal statements, proof terms in precise logical system. Every variable needs type. Every function needs definition. Every theorem needs enough for kernel to verify.
Statement obvious to mathematician may need several Lean lines.
Example: paper says integer even, uses fact written as twice another. Lean needs connection explicit. Proof may need definitions, rewrites, type conversions, previous lemmas.
Creates two tasks:
- Express mathematics in Lean.
- Produce proof Lean kernel accepts.
AI helps with both, neither automatic.
Why Fermat’s Last Theorem hard target
Fermat’s Last Theorem not difficult for final sentence. Claim fits one line. Difficulty inside proof.
Wiles’ argument depends advanced number theory. Full formalization needs large supporting mathematics: elliptic curves, modular forms, Galois representations, rings, fields, algebraic number theory, commutative algebra, abstract linear algebra, arithmetic geometry.
Lean users draw on Mathlib, large open-source formalized math library. Mathlib includes many theorems, definitions needed. Still, library not turn research proof into one-click verification.
Researchers must map informal ideas onto existing formal definitions. Find matching lemmas, handle representations, fill gaps papers leave implicit.
Proof may fail because mathematically equivalent objects have different types. May fail because theorem expects stronger assumption. May fail because definition uses different convention.
Where AI assistance useful.
AI as proof-writing partner
Large language models generate Lean code, suggest lemmas, explain errors, propose strategies. Usefulness from short feedback cycles.
Typical workflow:
- Researcher writes informal statement.
- AI proposes Lean definitions or theorem statements.
- Lean checks generated code.
- Error messages return precise feedback.
- Researcher revises.
- Process repeats.
Proof assistant strict referee. Does not care if generated code sounds convincing. Either kernel accepts, or not.
Changes AI role. In text generation, model can produce confident explanation with subtle mistake. In Lean, unsupported claims stop at checker. Model can waste time, choose poor definitions, generate messy code. Cannot bypass kernel verification through persuasive prose.
Anthropic’s result matters because shows AI working inside loop on serious mathematics. System not replacing formal verification. Producing candidate work formal verification tests.
Formal proof differs from mathematical explanation
Formal proof and informal proof serve different readers.
Informal proof communicates main idea. Leaves routine algebra, standard facts to audience. Formal proof records exact path from assumptions to conclusion.
Consider short sentence:
Since expression divisible by 2, square divisible by 4.
Human reader accepts immediately. Lean may need:
- Definition of divisibility
- Witness showing expression equals
2 * k - Rewrite for square
- Proof resulting product contains factor of 4
- Type information for every operation
Extra detail feels tedious. Also exposes hidden assumptions.
Formalization often reveals proof uses condition nobody stated. Maybe variable must nonzero. Maybe construction works only over field with particular property. Maybe result depends on finite set informal argument treated as obvious.
AI systems help surface, resolve gaps. But introduce new friction: generated proofs may compile while difficult for humans maintain.
Why compilation not enough
Lean proof that compiles is checked. Not mean well designed.
Researchers still inspect:
- Whether definitions match intended mathematics
- Whether assumptions too strong
- Whether theorem says what paper claims
- Whether imported results already contain main work
- Whether proof remains understandable
- Whether future library changes break it
Last point matters. Formal proof can rely large dependency graph. Readers need know which results support final theorem, whether independently checked.
Difference between formalizing theorem statement and formalizing famous proof. Project may prove final claim using existing lemmas or library theorem containing major part argument. Still valuable, not described complete reconstruction original proof unless it is.
Clear reporting matters. AI-generated research creates pressure celebrate result quickly. Formal mathematics needs slower language. Researchers should distinguish verified theorem, formalized proof outline, library extension, complete mechanization published argument.
What Lean kernel contributes
Lean’s trusted kernel checks proof terms against logical rules. Tactics automate work, external tools suggest code, accepted results pass kernel.
Gives theorem proving useful division:
- AI searches, writes candidates.
- Tactic system performs routine transformations.
- Kernel checks final proof.
- Researcher decides formal statement represents intended mathematics.
Kernel does not judge theorem interest. Does not decide definitions natural. Checks correctness relative to formal system, imported assumptions.
Boundary makes AI-assisted formalization easier evaluate than ordinary AI-generated mathematics. Proof artifact compiled, inspected, reproduced. Reader examines dependencies instead trusting paragraph from model.
Boundary limits what result proves. If formal development starts incorrect translation original problem, Lean may verify wrong theorem perfectly. Formal proof removes one class errors. Does not remove need for mathematical judgment.
Evaluation beyond “proof passed” needed. Useful measurements: RAG evaluation checklist for AI apps principles apply here - first-attempt success, repair cycles, human guidance dependence, performance across difficulty levels.
System proving one famous theorem after heavy human guidance different capability from solving many new problems little help.
Researchers need track compute usage. Search-based theorem proving can consume substantial CPU time because model may generate many candidate paths. Short final proof can hide large search process.
Benchmark design creates concern. If training data contains existing Lean proofs, model may reproduce familiar patterns rather than discover useful reasoning. Not make output invalid, but changes what benchmark measures.
Strong evaluation separates memorization, library retrieval, tactic selection, genuine proof planning.
Why this matters for mathematicians
Formalization long demanded more time than most mathematicians spare. Researcher may understand proof but lack weeks months needed encode every detail.
AI assistance could reduce cost.
Model can search Mathlib for candidate lemmas faster than person unfamiliar naming conventions. Translate repetitive algebra into Lean syntax. Try several proof layouts while researcher focuses definitions, strategy.
Best use collaborative not autonomous. Mathematician supplies idea, checks translation. Model handles routine code, failed attempts. Lean rejects mistakes. Researcher keeps control meaning.
Resembles programming with compiler giving unusually detailed feedback. Compiler not design application. Catches errors in code. Lean does same for formal mathematics, AI helps produce more code.
What changes for AI research
Fermat’s Last Theorem useful demonstration because people recognize name. Famous results can distort expectations. Successful formalization not mean AI solved mathematics.
Harder test sustained work unfamiliar problems. Can model select useful abstractions? Notice theorem statement too weak? Build reusable library components instead one-off scripts? Explain why proof works without confusing explanation verification?
Questions point toward different AI evaluation model. Final answers matter, so do artifacts left behind.
Good theorem-proving system should produce code another researcher can read, run, modify, trust. Expose dependencies. Fail clearly. Avoid hiding uncertainty behind polished language.
Standard demanding. Also practical. Anthropic's Claude breached three companies during security tests highlights need for clear boundaries when AI operates in sensitive domains. Mitigating prompt-level exploits in cybersecurity benchmarks shows evaluation must account adversarial conditions.
Larger lesson
Anthropic’s Lean work shows where AI assistance has natural fit: environments with fast, exact feedback.
Mathematical reasoning remains difficult. Formal systems make difficulty visible. AI can search possible steps, translate ideas into syntax, handle repetitive proof work. Lean checks steps form valid proof.
Result less dramatic than claim machines can “do mathematics” alone. More useful. Researchers get tool that can shorten distance between informal idea, verified artifact.
Fermat’s Last Theorem took centuries prove, years explain. Formalizing its mathematics in Lean adds another layer work, aimed at making every dependency explicit. AI may reduce manual burden, but central responsibility stays with people: define right theorem, choose sound foundations, inspect result, say exactly what proved.
