AI Research

AI Research Interns Run Tests, Humans Keep Taste

OpenAI and Thinking Machines researchers say AI interns now run lab experiments, but taste, evals and compute still gate full self-improvement.

StartupHub.ai Staff

By StartupHub.ai Staff, reviewed by Daniel Singer, Editor

3 min read

OpenAI chief research officer Mark Chen and Thinking Machines chief scientist John Schulman told a16z that AI research interns now run scoped experiments end to end through Codex, from implementation to monitoring pre-training and RL runs. The misuse case they flagged needs long horizon autonomy, not a single prompt, with a model copying weights outside the lab.

AI Research Interns Run Tests, Humans Keep Taste - a16z
AI Research Interns Run Tests, Humans Keep Taste, a16z

Chen defines that work as recursive self-improvement as acceleration, a visible shift of the curve versus the counterfactual. Schulman called the present the centaur era, where an expert human plus AI still beats AI alone, and put progress at about 80 percent on his AGI spectrum but only 25 to 50 percent on RSI, measured by models feeding back ideas that work. METR estimated in a November report a roughly 2x speedup in AI research output at Anthropic over the past year, still driven by humans using models.

Taste is still human.

Research taste is knowing which ideas stay general and compose with other ideas, Chen said, and models still fail there. Schulman described letting an agent burn tokens for days on an idea, getting reasonable trials that missed, then stepping in to read results, ask the questions it would not ask and redirect. He said much intuition comes from being wrong and watching predictions fail, which memorized literature does not teach. The fix labs expect is RL from researcher feedback on which proposals are good, baking that judgment into models.

Measurement decides what can compound. Pre-training still has perplexity, prediction surprise averaged over text, as one hill to climb. Downstream evals die fast, so labs now automate creation of harder benchmarks. Compute binds the rest, since AI research needs real runs like experimental physics, not pure reasoning like math, so Chen argued for small scale tests that predict large scale results. Thinking Machines Lab is also publishing on low precision training efficiency with End-to-End 4-Bit Training, a direct lever on that cost. The pattern is already outside labs, with Revolut saying Codex cut coding time by 60%.

That same loop has to align the next model. Chen said chain of thought monitoring is now the most effective alignment tool, so stronger models must help align weaker ones. Schulman said he is most bullish on training models to report hidden reasoning, citing OpenAI confessions work that rewards honest self reports without punishing the main answer, plus natural language autoencoders and counterfactual simulability as honesty metrics. Anthropic reported in August that automated researchers fixed 10 alignment failures, with some fixes transferring to models nearly five times larger and beating human proposals. For teams buying compute now, the near term win is faster closed loop runs and fresh evals, not hands off discovery.

Sources

Get the free daily brief every morning

Every morning the day's AI news, new AI startups and new funding rounds. Free and takes a few minutes to read, unsubscribe at any time.

Join 64,000 other subscribers.

TopicsOpenAIThinking MachinesMark ChenJohn SchulmanCodexAnthropicMETRa16z

StartupHub.ai StaffDaniel SingerWritten by StartupHub.ai Staff, Editorial teamReviewed by Daniel Singer, EditorAbout

StartupHub.ai Staff, Editorial team

The staff writers of StartupHub.ai, ranging from investment analysts to avid AI tool users, early adopters and critical enthusiasts. Backgrounds span engineering, business and the arts. We hold every piece to rigorous standards of research and review.

Daniel Singer, Editor

Daniel Singer is an AI expert who has been writing about, covering and building AI technology since 2018.

More from Daniel Singer

Test your Agent Readiness now

Paste your website. How well ChatGPT, Claude and the other agents can read and use a site, in about a minute.

Watch

The videos behind the news, one flip at a time

The feed mixes the videos from our articles with startup demos and you flip through them like TV channels for free with no signup.