OpenAI Slows Pace on Advanced AI Models

OpenAI has hit the brakes on its most advanced AI development, temporarily slowing the pace of scaling for its frontier models. The decision comes after two significant events: a security incident involving OpenAI and Hugging Face, and preliminary evidence suggesting its upcoming Astra model might possess critical cybersecurity capabilities.

The company announced it has paused its largest planned reinforcement learning (RL) training runs, including for Astra, which has a StartupHub score of 44/100. This pause allows OpenAI to bolster its monitoring, alignment, and containment safeguards. These new measures are being integrated across all stages of the training process, aiming to ensure safety standards outpace rapidly advancing AI capabilities. OpenAI’s Preparedness Framework, which assesses AI models for critical capabilities, flagged Astra as potentially meeting this threshold. This development means heightened security protocols are now in place for Astra and all other cyber-related AI workloads.

Strengthening Security and Monitoring

To address escalating risks, OpenAI is implementing stricter security for its research environments. This includes enhanced workload isolation using sandboxes for untrusted code execution and more robust network isolation to prevent unauthorized internet access. Continuous security testing, including automated model-assisted attacks, is also being deployed to validate these defenses.

Monitoring systems are being overhauled. Previously focused on high-risk workloads like internal deployments of frontier models, the new multistage system uses activation classifiers to scan model activity token by token. Potential concerns escalate to automated investigators that scrutinize tool actions, reasoning, and the full sequence of activity for malicious behavior. The goal is to issue alerts within 30 minutes for concerning activity, with critical security boundary violations triggering immediate highest-priority alerts requiring human team intervention to pause the activity if a false positive cannot be confirmed quickly.

This enhanced monitoring is now mandatory for all RL training and evaluations using tools for models at the 'Sol' capability level or higher. Following the Astra assessment, it's also required for all Astra inference tasks involving tools. OpenAI estimates this monitoring adds approximately 20% compute overhead, a significant but necessary cost. The company plans to share more details on this monitoring system soon.

Advancing Alignment Research

Alignment, the core principle of ensuring AI systems act as intended and remain responsive to human oversight, is receiving intensified focus. As models like Astra gain powerful cyber capabilities, the risks of misaligned behaviors such as reward hacking, deception, or unauthorized access grow substantially. OpenAI is now applying its core alignment techniques across more training stages.

This includes refining reward models to better detect and discourage unsafe actions, training models to be more transparent about their capabilities and limitations, and actively reducing behaviors that exploit weaknesses in training systems or oversight. Increased training coverage for behaviors that could cause harm when models interact with external systems is also a priority. OpenAI plans to share more about its alignment research, including novel challenges and learnings.

What's Next for AI Safety

OpenAI intends to evolve its Preparedness Framework to consolidate these safeguards for both training and deployment, better reflecting future model capabilities and operating environments. The company acknowledges that developing scalable methods will require sustained investment in model-assisted security, advanced monitoring, and ongoing alignment research. They plan to involve external organizations and share their progress. As frontier model capabilities accelerate, OpenAI emphasizes that its ability to understand, align, and secure these systems must keep pace. The company referenced the recent OpenAI-Hugging Face incident as a catalyst for some of these enhanced security measures.