LLM Tests Expose Cracks in Ethereum Infrastructure

98 bugs across all 11 Ethereum clients were found with LLM-generated differential testing, according to BlackHat Asia 2026 on LLM-empowered differential testing for Ethereum infrastructure.

The team paired ISSTA 25 work on the EVM with OOPSLA 2026 work on JSON-RPC APIs. Both target the part of Ethereum users never see.

How does the testing work?

The team built a local testnet of 30 nodes, every combination of six consensus clients and five execution clients, and sent identical inputs to each. LLMs generated minimal, semantically valid bytecode to cover single opcodes, then extracted interprocedural control flow graphs to mutate args and cover all paths in the spec. Differences in return data, gas usage and storage flag the bug, and the LLM then locates the implementation function across Go, Java, Rust and other languages.

What did they actually find?

The count was 26 EVM bugs and 72 API bugs, more than 85% confirmed or fixed, affecting all 11 clients. Case one was PREVRANDAO handling where Geth-derived clients dereferenced a nil random pointer and crashed the EVM, a denial of service that also propagated to forks like BSC. Case two was Besu returning a payload status of invalid instead of JSON-RPC error -32062, which can confuse consensus clients reading execution responses.

Why do Geth and Prysm examples matter?

The talk opened with two motivating failures. Geth's precompiled dataCopy at 0x00...04 did a shallow copy where others did a deep copy, a consensus vulnerability reported in 2020 that can split the chain. Prysm's Fusaka incident on Dec. 4 triggered resource exhaustion by replaying past epoch blocks on atestations from lagging nodes, dropping participation to about 75% and costing validators 382 ETH.

Why does client diversity alone not save you?

Ethereum deliberately maintains multiple clients to a common spec so they interoperate, and ideal distribution avoids any client over 33%. In practice Lighthouse holds about 42.7% of consensus and Prysm about 30.9%, while Geth still dominates execution, so a bug in a majority client threatens finality. The Prysm drop to 75% stayed above the 66% finality threshold, but the authors noted a Lighthouse bug at that scale would have halted finalization.

StartupHub.ai data shows Ethereum scores 31/100, well below the competitors we track including Solana at 61/100, Near at 58/100, Cardano at 55/100, Elrond Network at 54/100 and Aptos at 54/100. That gap reflects how hard infrastructure reliability weighs on ecosystem health, not just smart contract tooling, which accounts for roughly 60% of academic Ethereum security papers cited in the talk.

Why this matters for AI and startup teams

Spec-driven LLM generation solves a hard scaling problem. Handwritten RPC tests and DSLs do not keep up when specs change, and under 1% of the 90 million deployed contracts are open source, so corpus fuzzing misses corner cases. The second contribution is LLM as judge for API diffs, filtering allowed differences like peer IDs and semantically equivalent error strings such as unable to decode versus cannot decode before filing.

The specless is not safe either. The team flagged Beacon API spec bugs where block gas used was missing for a year and proof size was defined as 32 instead of the correct 33, which the maintainers fixed within an hour. If your wallet, explorer or rollup reads different APIs for the same transaction, as Etherscan did with debug_traceTransaction versus traceTransaction, you ship the inconsistency to users.