I was scrolling through Hacker News last night when a post literally pulled me out of bed — Mistral released Leanstral 1.5, a model purpose-built for mathematical proof and code verification. Apache-2.0 license, free API, 6B active parameters, and it crushes models orders of magnitude larger on formal verification benchmarks.
I'll be honest — I've always been skeptical about the whole "AI does math proofs" thing. It felt like academic circle-jerking, something that has zero relevance to the average developer. But after reading through Mistral's release notes, I changed my mind. Not because of the math contest scores, but because it actually found real bugs in real open-source projects — 5 of them that nobody had reported before.
Here's my take on what Leanstral 1.5 actually does, why it matters for developers, and whether it's useful enough today.
Why I actually paid attention
I've been running this site for three months now, tracking every AI tool launch I can find. And honestly? I'm burned out on announcements. Another model dropped, another funding round, another "we're changing the world" press release. It all starts blending together.
Leanstral 1.5 is different.
First, it does something genuinely novel — not another "code-writing AI" but a "code-verifying AI." Second, it's from Mistral, a company that refuses to play the same game as everyone else. While everyone else is stacking parameters and burning cash on compute, Mistral is shipping 6B-parameter MoE models under Apache-2.0 with free APIs and achieving SOTA in a niche nobody else is fighting over.
That kind of anti-mainstream approach makes me sit up and pay attention.
What is Leanstral?
Mistral launched the first Leanstral last year as a theorem-proving assistant for Lean 4. Lean 4 is an interactive theorem prover and functional programming language originally developed at Microsoft Research. It's been used mostly in academia and safety-critical software verification.
Leanstral 1.5 is the second generation, and my reaction is: the jump is absurd.
A model with only 6B active parameters (119B total via MoE architecture) achieving SOTA on multiple math proof benchmarks. And not through brute compute — through smart training.
Quick specs
- Architecture: MoE, 119B total / 6B active parameters
- License: Apache-2.0, fully open-source, commercial-friendly
- Availability: HuggingFace weights + free API endpoint
- Training: Three-stage — mid-training → SFT → RL with CISPO
CISPO sounds fancy but it's basically reinforcement learning inside a proof environment. The model submits a proof, the Lean compiler checks it — pass = reward, fail = compiler error feedback. The model iterates like a human working through a problem set.
How it stacks against competitors
Mistral's release is refreshingly honest about positioning — they don't claim to be the absolute best, just "good enough + cheap + open." On FATE-H and FATE-X, Leanstral 1.5 beats Seed-Prover 1.5 and Goedel-Architect. On PutnamBench it edges out Seed-Prover 1.5 high (587 vs 580 problems solved). And the cost per problem? About $4, compared to Seed-Prover's $300+ — a 75x savings.
The few provers that score higher (like Aleph Prover at $54-68/problem) operate under very different conditions. Some get natural-language proof guidance. Others burn 10 H20-days per problem. Leanstral's edge isn't raw power — it's being practically useful at a price that doesn't make you wince.
Training: Three stages to build a proof expert
Stage 1: Mid-Training
They continued pretraining the model on massive amounts of Lean code and math proofs. The goal: teach the model Lean's syntax, proof patterns, and library conventions. Mistral didn't disclose the data volume, but based on results, this step was thorough — the model's understanding of Lean 4 rivals someone who's studied it for six months.
Stage 2: Supervised Fine-Tuning
Standard SFT but on proof-engineering data. Input is a theorem or proposition, output is a complete Lean verification. Same as coding SFT, just in the formal verification domain.
Stage 3: RL with CISPO
This is where it gets interesting. Two training environments:
Multiturn Environment: The model gets a theorem and tries to prove it. The Lean compiler validates the proof — compile = reward, fail = error message + retry. Like doing LeetCode problems, except the compiler is your judge.
Code Agent Environment: This one mimics real developer workflow. The model can:
- Edit files
- Run bash commands
- Query the Lean language server for type info, goals, errors
- Complete partial proofs across a repository
- Build auxiliary lemmas
- Persist through multiple rounds of context compaction
It's a full agentic sandbox — not a simple input-output loop, but an environment where the model operates like a human dev in the filesystem.
Why Lean is the perfect training ground
Lean's community is smaller than Python's, but it has one huge advantage: every proof is compiler-verified. The training data is inherently high-quality. No human labeling needed, no LLM-as-Judge — the Lean compiler provides an objective binary signal: compiles = correct.
This is why Mistral could build such a capable proof model with relatively modest resources. The compiler acts as the reward model, eliminating the need for expensive human annotations.
Benchmark results: Actually impressive
Leanstral 1.5's results on major math proof benchmarks:
- miniF2F: 100% (full saturation). Cross-system math proof benchmark from elementary to IMO level. No prior model has ever saturated this.
- PutnamBench: 587/672 (87.4%). Putnam math competition — the ceiling of undergraduate math.
- FATE-H: 87% (SOTA). Graduate-level abstract algebra.
- FATE-X: 34% (SOTA). PhD-level abstract algebra. Solving a third of PhD-level problems is beyond coincidence.
FLTEval: Real-world capability
Beyond academic benchmarks, Mistral released FLTEval — a benchmark built from actual pull requests on the Fermat's Last Theorem repository. These are real, complex proof-engineering tasks, not contest problems.
Leanstral 1.5 hits pass@1 of 28.9 (up from 21.9 in the previous generation) and pass@8 of 43.2, beating Opus 4.6's 39.6 at one-seventh the cost.
Test-time scaling that actually works
One thing that genuinely impressed me: Leanstral 1.5's test-time scaling curve is beautiful.
What this means in practice: give it more compute (tokens to think), and performance increases monotonically. From 44 PutnamBench problems at 50k tokens to 244 at 200k, 493 at 1M, and 587 at 4M. No plateau in sight.
This isn't a model that gets lucky sometimes — it has sustained reasoning capability. You can trade compute for quality, which is enormously valuable in engineering. Instead of "solve or don't solve," you get "solve more with more resources."
The part that actually excites me: Code verification
Leanstral 1.5 can verify real-world Rust code. Here's the pipeline:
- Aeneas translates Rust code into Lean
- Leanstral analyzes the code and infers correctness properties from the code itself
- It tries to prove those properties
- If all 4 attempts fail, it tries to prove the negation — that the code actually has a bug
- Proving the negation = confirmed bug
Mistral's team ran this pipeline across 57 open-source repos. They flagged 47 violated properties, 11 pointing to genuine bugs — 5 of which were previously unreported.
Case study: The varinteger overflow
One concrete example: the datrs/varinteger Rust library's zigzag decode function. On Std.U64.MAX input, the expression (value + 1) overflows — crashing in debug mode and silently corrupting data in release mode.
This kind of bug is nasty. Test suites might not check u64::MAX, and fuzzing might take forever to hit that exact path. Leanstral found it through formal verification at the structural level.
Some HN commenters pushed back — "that's literally the first edge case any decent fuzzer would catch." Fair point for a single bug. But Leanstral isn't manually checking one function — it's automatically, indiscriminately scanning entire repos. 57 repos, 5 undiscovered bugs. No human can do that at scale.
AVL tree time complexity proof
Another case study: proving O(log n) time complexity for a real AVL tree implementation. Leanstral spent 2.7 million tokens across 22 compaction cycles, systematically unfolding each layer of the TimeM monad. It established a tight bound of 48 steps per height unit plus a constant for insertion, then connected height to tree size via a logarithmic relationship.
In plain English: it proved the code maintains O(log n) performance in the worst case, regardless of input.
Cost comparison: $4 vs $300+
On PutnamBench, Leanstral 1.5 averages ~$4 per problem. Seed-Prover 1.5 high mode? $300+ per problem (10 H20-days of compute). Even against Aleph Prover's $54-68/problem, Leanstral is 10x cheaper.
Mistral's strategy is increasingly clear: don't compete on raw capability — compete on efficiency for specific domains. Their OCR model follows the same logic. Put $100 in your account and it runs for a year.
HN community reactions
The discussion on HN (251 points, 74 comments) had some genuinely insightful takes:
Someone challenged the bug-finding claim: "isn't this just basic fuzzing?" Yes, if you know what to test. Leanstral doesn't need to know — it scans the whole repo, infers correctness properties, and finds inconsistencies.
Someone asked "can I use this without knowing Lean?" The answer is yes, via Aeneas for Rust → Lean translation. You don't need to understand Lean syntax — just understand what properties you want to verify.
The most valuable comment came from a developer who's been using Lean for six months: "Going from zero knowledge to writing useful programs took about 6 months, and AI assistance was a massive help. The language itself is beautiful, theorem proving aside."
How to get started
Mistral provides a complete toolchain. If you have uv:
| 1 | |
| 2 | |
| 3 | |
| 4 | |
| 5 | |
| 6 | |
| 7 | |
| 8 | |
| 9 | |
| 10 | |
For better DX, install Lean LSP MCP — it lets the model query the Lean language server for real-time type info:
| 1 | |
| 2 | |
| 3 | |
| 4 | |
| 5 | |
| 6 | |
| 7 | |
Or just call the API directly:
| 1 | |
| 2 | |
| 3 | |
I ran a quick test
Signed up for Mistral's API, grabbed the free credits, and threw a simple proof at it — forall n : Nat, n + 0 = n. Response time: about 2 seconds. Clean Lean proof:
| 1 | |
| 2 | |
| 3 | |
| 4 | |
| 5 | |
Then I asked it to analyze a Rust function similar to the varinteger case. It correctly flagged the edge case. The Aeneas translation pipeline I haven't fully wired up yet — that'll be a follow-up post.
A few gotchas
- Leanstral only speaks Lean 4, not Lean 3. Different syntax, incompatible.
- Great at math proofs, limited on code with complex memory models. Rust lifetimes and unsafe code depend heavily on Aeneas translation quality.
- $4/problem is an average. Complex agentic loops can burn tokens. Start with the free API tier.
- No Chinese optimization. Prompts and outputs are in English + Lean code.
What this means for the average developer
The short answer: not much yet, but soon.
If you write Rust or work on safety-critical software (embedded, finance, crypto, protocols), Leanstral 1.5 can already help find issues you'd otherwise miss. Especially with the Aeneas pipeline — Rust code gets translated to Lean automatically.
What I expect in the next year
- CI/CD integration. Every PR triggers formal verification on critical paths — as natural as running linters today.
- IDE integration. VSCode plugins that highlight "this function overflows on input X" in real time as you type.
- Costs keep dropping. $4/problem is already cheap. It'll get cheaper.
The big picture
I've always believed that 100% test coverage doesn't prove your code is bug-free — it only proves your tests don't crash. Formal verification is different: it mathematically proves your code satisfies its specification for all possible inputs.
The barrier has always been cost and complexity. You had to learn Lean, write specs, and manually construct proofs. Leanstral 1.5 lowers both. It infers correctness properties from the code itself and automates the proof search.
If testing is security guard, formal verification is FBI background check. You couldn't afford the FBI before. Now the price is dropping.
Fuzzing vs formal verification: my testing
When I dug into Leanstral, the HN comments made a valid point — the varinteger overflow is trivially findable with fuzzing. I tested it. Using Rust's proptest, a round-trip test found the overflow in under a second:
| 1 | |
| 2 | |
| 3 | |
| 4 | |
| 5 | |
But there are two critical differences:
-
You need to know there's a bug to test for it. Leanstral scans the entire repo without you telling it what to look for. Fuzzing requires you to define test properties.
-
Formal verification proves absence of bugs. Testing proves absence of bugs in the cases you tested. That's not a technical nuance — it's a fundamental difference in quality assurance.
My take
Mistral is doing something genuinely different. They're not trying to out-AGI OpenAI and Anthropic. They're building small, focused, open-source models that do one thing well at a fraction of the cost.
You can argue Leanstral 1.5 isn't SOTA. Fine. $4/problem vs $300/problem — I'll take the 75x cost savings and be happy with results that are 95% as good.
And those 5 real-world bugs? Those aren't benchmark numbers. They're actual bugs in real projects that could crash your software or corrupt your data. That alone makes this model practically valuable — it doesn't need to be the smartest AI, it just needs to scan your codebase while you sleep and find things you missed.
I think formal verification is on a trajectory similar to early linters. Everyone thought linters were toys. Now ESLint and Prettier ship with every modern project. TypeScript went from "why would I need types in JS" to default choice in five years. Formal verification may be at that inflection point.
I'm planning to hook Leanstral into my Rust projects' CI pipeline and see how it performs in practice. If it works out, I'll write a detailed tutorial covering the full pipeline: Aeneas translation, Lean verification, CI integration.
Questions? Drop them in the comments.