Your browser can run LLMs now. So what?

Stateofutopia shipped a lab that runs small language models directly in your browser. No server, no account, no API bill.

The pitch is pure edge.

MicroLLM lab loads Q4 quantized models in the 25M to 360M parameter range and executes them with WebGPU, the W3C standard that hits Apple Silicon Metal, DirectX 12 and Vulkan from inside the browser window. Weights are compressed from 16-bit to 4-bit, cutting memory by 75% so a 100M+ model fits in about 50 to 84 MB of browser memory. The site claims sub-10ms time-to-first-token, infinite concurrency on client GPUs, and 100% private prompts that never leave the device.

That last part is the real sell, and the thinnest proof.

The lab gives you three steps: click Load to cache a model to IndexedDB, chat in a panel that shows live tok/s, then run benchmarks and generate a shareable performance certificate with peak and sustained tokens per second. It will run a sustained 256-token decode, show wall time per test, JS heap, GPU buffers, and adapter info, and it falls back to WASM then JS if WebGPU is not available. The downloadable zip is listed at 589 MB, with a current build at 591 MB, and it must be served over HTTP, not file://. For checks it is explicit: the suite uses objective regex and exact-token tests, not writing quality. As the UI notes, a 135M model is allowed to fail, that is the measurement.

That honesty is useful but it leaves the core questions unanswered. Stateofutopia reports near-lossless generation quality for Q4, but shows no side-by-side quality retention numbers, no calibration set, and no comparison to the cloud models this edge layer is supposed to triage. Triage and intent extraction live or die on accuracy at low latency, and this lab measures speed first. It also frames cost as zero, which is true for the host but not for the user. Client GPU time, memory pressure, battery drain and IndexedDB quota are real limits, especially on mobile. Browser support is conditional. WebGPU is still rolling out, so many devices will hit the slower WASM fallback.

Context helps place it. In-browser inference via WebGPU is no longer novel. Stacks like WebLLM run models directly in the browser with zero setup, and engineering proposals now describe a browser SLM engine using @mlc-ai/web-llm or onnxruntime-web as Tier 2 progressive enhancement inside a 3-Tier hybrid inference architecture. MicroLLM lab fits that Tier 2 idea, fast local classification and filtering to decide if a frontier call is needed, rather than a replacement for GPT-4 or Claude.

Until there are device-diverse results, a clear Q4 quality curve, and a task-accurate routing demo that saves a cloud call without hurting outcomes, this remains a polished harness, not a migration path. The certificate is verifiable only to your own hardware.