Better Search Pushed an Agent From 36% to 98%

Can search alone make a bad agent good? Aamir Shakir said it can, and he showed how on AI Engineer, where the Mixedbread co-founder demoed a retrieval system that lifted a hedge fund agent from 36% to 98% answer accuracy without changing the underlying language model.

The fix targets budget and workflow, not intelligence.

Shakir framed the problem as access, not reasoning. Retrieval-augmented generation is a method that fetches your own documents to ground an agent answer, but the usual stack is what he called a Frankenstein stack. You parse PDFs, run OCR on scans, chunk pages, embed each chunk into a single vector, then search a vector database and rerank. Each step is lossy, he said, and combining vendors for OCR, chunking, embeddings and storage means teams spend six to eight months evaluating tools and assigning four to five people to build what remains brittle. He illustrated it with an equity analyst asking why Intel's new D270 GPU uses LPDDR5 instead of HBM. With lexical search like grep or a single embedding point per chunk, a system returns many near-identical hits and the model must read every candidate. At billions of documents, Shakir said, that pushes a single simple question to $10 or $20 in token cost, and PDFs with tables, figures and scans often are not grepable at all. A single embedding also struggles when documents look alike, he argued, like Nvidia SEC filings from 1996, 2000 and 2026 whose points collapse together, so quality degrades as the corpus scales and agents compensate by spinning up tens of sub-agents and flooding the context window.

What changed is the embedding design. A late interaction model is a retriever that keeps multiple embeddings per page for the document and multiple embeddings per query, so fine grained facts stay distinct while remaining contextualized. Shakir said Mixedbread's next model, previewed for release next week and already live with some customers, creates those embeddings from the visual page itself rather than OCR text, which preserves tables and figures. In a live demo he queried Intel foundry segment operating margin for Q2 2025 and the system returned the exact page at sub-200 millisecond latency with bounding boxes to cite the source. For complex qualitative questions like whether Intel's foundry profitability is improving and why, he said Mixedbread trained an agentic search primitive that decomposes the request into multi-hop queries, which he described as as good as Astra and Fable while being 40 times faster and cheaper. The team also exposes layout and OCR outputs for downstream models that are not yet multimodal, so builders can choose to feed the screenshot or text.

The impact shows up in deployment, not just benchmarks. Shakir cited a large hedge fund with over 1 billion documents that onboarded in about one and a half weeks and builds internal agents for analysts and portfolio managers; human-graded accuracy rose from 36% with chunking plus a generic embedding and vector database to 98% after switching to Mixedbread, he said. Multimodality accounted for a large share of the gain, and because better context reduces the reading load, he said large language model spend fell by 3x. Mixedbread is pitching that as an unbundling of intelligence and data, where teams pick any frontier model but rely on a single indexing layer. That layer is sold with zero data retention and what Shakir called bring your own bucket, plus ISO compliance that he said helped onboard banks and hedge funds in the last three months despite five months since launch. For context on the lab's prior text retriever, Mxbai-embed-large uses a BERT-large backbone with 335 million parameters and 1,024-dimensional embeddings and scores 64.68 overall on MTEB with 54.39 on retrieval, per InnovativeAIs.

The demo also showed the limits. The first page load failed on slow venue wifi, which Shakir flagged himself, and the 98% figure is a single customer case study with human grading, not a public benchmark. If you run agents over private filings, podcasts and research notes, the near-term move to watch is whether visual late interaction at scale holds that accuracy and token saving without adding operational overhead, and whether teams that budgeted months for OCR and vector store tuning can shift that headcount to application logic instead.