Technology

Your agent scolding is training data

Brian Douglas says every agent pushback is training data if teams capture and label traces.

StartupHub.ai Staff

By StartupHub.ai Staff, reviewed by Daniel Singer, Editor

2 min read

What happens after you correct your agent? Brian Douglas told AI Engineer to save the correction and label it.

Your agent scolding is training data - AI Engineer
Your agent scolding is training data, AI Engineer

Douglas works at Paper Compute. He said he spent six to eight months capturing agent sessions with tapes.dev. Tapes.dev is open source session capture that writes prompts, tool calls and diffs to a searchable store. He runs agents in a sandbox runtime his team calls stereos. Traces flow to a central console where teammates tag pushback, apologies and regressions.

He still cannot write Rust on the job.

That gap drives his demo, Douglas said. Senior engineers push back when his agent duplicates services or writes Rust that compiles but breaks Paper API conventions. He collects each pushback from an engineer named Matt. He post-trained a small reviewer that comments on pull request 381 as Matt would. The pattern needs no GitHub scrape, Douglas said, because most review now happens inside the harness and never reaches GitHub.

His advice is to start with skills, which are reusable checks built from labeled traces, before paying for post-training. Specialized fine-tuning adds small adapters on top of a base model, Douglas said. It worked for him on a 4B Qwen model quantized on a single 5090. Direct preference optimization needed 70B-plus scale he could only reach on rented Brev GPUs. Do not confuse the tooling with Tape to Tape, a roguelite hockey game from Excellent Rectangle that left Early Access in October, per its Steam listing.

Team leads who let agents write in languages juniors do not know will have to budget for trace storage and labeling, or keep paying seniors to catch the same errors in review. Douglas kept his reviewer private because it holds sensitive data, and Paper console remains waitlisted.

Mentioned in this story

Sources

Get the free daily brief every morning

Every morning the day's AI news, new AI startups and new funding rounds. Free and takes a few minutes to read, unsubscribe at any time.

Join 64,000 other subscribers.

TopicsBrian DouglasPaper ComputeTapes.devStereosQwenFireworksBrevMatt

StartupHub.ai Staff

Written by

StartupHub.ai StaffEditorial team

The staff writers of StartupHub.ai, ranging from investment analysts to avid AI tool users, early adopters and critical enthusiasts. Backgrounds span engineering, business and the arts. We hold every piece to rigorous standards of research and review.

Daniel Singer

Reviewed by

Daniel Singer is an AI expert who has been writing about, covering and building AI technology since 2018.

Test your Agent Readiness now

Paste your website. How well ChatGPT, Claude and the other agents can read and use a site, in about a minute.

Watch

The videos behind the news, one flip at a time

The feed mixes the videos from our articles with startup demos and you flip through them like TV channels for free with no signup.