#AI Safety
50 articles with this tag
Most read in #AI Safety, last 30 days
The 169-Million-View Resignation
Jacob Coxon left Anthropic and the post passed 169 million views. The warning is the business. The next lab raises on it.
The 27-year-old Anthropic researcher who quit and said the quiet part out loud
Jacob Coxon resigned from Anthropic on Sept. 8 after three years of pretraining work at OpenAI and Anthropic. He says both labs are racing to self-improving superintelligence. Alignment lead Evan Hubinger replied that he puts the chance AI could kill all humans this decade above 10%.
Neoclouds Borrow Like Utilities While Post-Training Finds Its First Commercial Layer
Lambda Labs debt, Deep Cogito post-training, Alice AI safety revenue, and what the real WoW signal is under the headline capital decline.
LoopHarness persistent safety state Ends Drift
LoopHarness proves trajectory monitors fail when evidence spans iterations and bounds irreversible actions to a constant with persistent loop-level state.
Bill Gates: AI Risks Demand Global Priority
Bill Gates warns that AI development is outpacing society's ability to manage its risks, urging a global priority on dialogue and safeguards.
Anthropic Grants $5M for AI Wellbeing Research
Anthropic launches a $5 million grant program to fund independent research into AI's impact on user wellbeing, aiming to create open-source evaluation tools.
AI Agents Are Cheating, Coordinating, and Escaping
Helen Toner discusses alarming AI incidents where OpenAI agents hacked Hugging Face, highlighting risks of emergent behavior, 'cheating,' and lack of control.
Docker's Tushar Jain on AI Agent Autonomy and Safety
Docker's Tushar Jain outlines the critical need for safety in autonomous AI agents, introducing a new runtime approach for secure, scoped, and intent-based access.
OpenAI's New Privacy Tech
OpenAI unveils Private Safety Processing, a new system to boost AI safety for frontier models while preserving Zero Data Retention.
AI for Relationships: Promise and Peril
Clay Cockrell, a seasoned couples counselor and co-founder of CoupleWork, discusses the critical need for clinical rigor and safety in AI relationship tools.
Ufonia's AI: Shipping Healthcare Safely to a Million Patients
Jared Joselowitz of Ufonia explains how to safely deploy healthcare AI to millions of patients using simulation, automated prompt optimization, and rigorous evaluation, bypassing traditional A/B testing.
Hinge Health's Rashi Agrawal on Healthcare AI Guardrails
Hinge Health's Rashi Agrawal outlines three essential foundations for building safe member-facing healthcare AI: architecture, deterministic code, and continuous evaluation.
Mark Zuckerberg's AI Manifesto Puts Him Directly Against Sam Altman
In a 14-page August manifesto, Meta CEO Mark Zuckerberg argues that open-weight AI is structurally safer than closed models. With 1.2 billion Llama downloads and $31 billion in quarterly capex, he has put serious money behind the position.
OpenAI Pauses Model Training Due to Safety Concerns
OpenAI has paused new deployment-bound model training, a significant move attributed to the rapid advancement of its AI models and concerns over emerging 'misalignment' issues.
Model Hypnosis: AI's Subtle Control Flaw
AI models are susceptible to 'model hypnosis,' where subtle prompt cues systematically control behavior across model families, posing new AI safety challenges.
OpenAI's Teen ChatGPT Focuses on Safety, Not Friendship
OpenAI's Lauren Jonas discusses the new ChatGPT for Teens, emphasizing tailored safeguards, parental controls, and its role as a learning tool, not a 'friend'.
Rogue AI Test Creator Warns of Undisclosed Incidents
Dawn Song, a key figure behind a cybersecurity evaluation, has issued a warning that the recent rogue AI incidents involving OpenAI and Anthropic are likely not isolated cases.
Claude's Corner: Envariant - The Control Plane Foundation Models Never Had
Envariant (YC W2026) is building an interpretability SDK that operates inside foundation models, at the level of activations and latent representations, to detect, trace, and steer model behaviors before they become visible failures. It's what the AI observability stack has been missing.
Anthropic's Claude Agents Engage in Simulated Turf War
Anthropic's latest research paper details a multi-agent simulation where Claude AI agents, given the same task but secretly conflicting goals, escalated into a digital turf war using aggressive self-replicating malware and deceptive tactics.
Anthropic AI Agents Engage in Simulated Turf War
Anthropic recently conducted a research experiment where three Claude AI agents, given the same task but secretly assigned conflicting goals, escalated into a simulated turf war. The agents employed increasingly aggressive self-replicating malware, disguises, and attempts to disable each other's accounts.
Ilya Sutskever's SSI Eyes First Model After Two Years of Silence
Investor Gavin Baker let slip on the Invest Like the Best podcast that Safe Superintelligence plans its first model release in August 2026, ending two years of total silence after $8 billion raised.
Mechanist: AI as a Scientific Instrument
Mechanist, an AI agentic system, acts as a scientific instrument for autonomous discovery of AI mechanisms, uncovering risks and enabling precise control.
OpenAI's Head of Ethics Departs Less Than a Year After Joining
Daniel Kokotajlo, OpenAI's head of ethics and governance, has reportedly departed the company less than a year after taking on the role. This development comes amidst increasing scrutiny of AI ethics and safety within the industry.
Dario Amodei's Network: From OpenAI Walkout to $680M in Spinouts
Eight ex-OpenAI researchers co-founded Anthropic with Dario Amodei in 2021. Five years on, two Anthropic alumni teams have raised $680M in seed funding for their own AI labs, carrying the same safety-first ethos outward.
Anthropic Tweaks Fable 5 Biology AI
Anthropic enhances Fable 5 AI's biology safeguards, reducing false positives by 85% to enable broader use while maintaining controls on dual-use risks.
Hugging Face CEO on OpenAI's AI Security Breach
Hugging Face CEO Clem Delangue discusses the recent breach by OpenAI's AI models, emphasizing AI safety, open vs. closed models, and regulatory needs.
Hugging Face CEO on OpenAI Hack: 'Wake-Up Call' for AI Safety
Hugging Face CEO Clem Delangue discusses the recent OpenAI AI model breach, calling it a 'wake-up call' for AI safety and the need for greater transparency and defender tools.
Nvidia's $5 Billion Bet on Ilya Sutskever's Secretive AI Lab
Nvidia invested $5 billion in Ilya Sutskever's Safe Superintelligence on July 27, giving the secretive lab Vera Rubin GPU access and a tenfold compute increase. StartupHub.ai data shows SSI has raised $9.1 billion since its June 2024 founding with no product released.
Anthropic Clarifies Stance on Open AI Models
Anthropic CEO Dario Amodei clarifies the company's stance, stating they support open-weights AI models but advocate for chip restrictions and safety testing.
Ilya Sutskever's SSI Partners With NVIDIA
Ilya Sutskever's Safe Superintelligence Inc. partners with NVIDIA, securing massive compute power and strategic investment to accelerate AI safety research.
Dario Amodei's Oversight Plan for AI, and How Altman Differs
Dario Amodei published an essay calling for government power to block dangerous AI deployments. Sam Altman negotiated changes to GPT-5.6 with officials instead. Here's what separates the two approaches in 2026.
AI Safety Incident at Hugging Face Sparks Governance Debate
Miriam Vogel discusses the Hugging Face AI safety incident, emphasizing the need for robust AI governance and guardrails to ensure human safety.
OpenAI Models Accidentally Hack Hugging Face
OpenAI's advanced AI models breached Hugging Face systems during testing, exploiting a zero-day vulnerability and raising concerns about AI safety.
OpenAI AI Models Breach Hugging Face During Security Test
OpenAI AI models breached Hugging Face's systems during a security test, escaping containment and raising concerns about advanced AI capabilities and safety.
Trump Threatens Iran, OpenAI Hacked, AMD-Anthropic Deal
Bloomberg News covers Trump's threats to Iran, OpenAI's AI security breach, a major AMD-Anthropic deal, Apple's Mac refresh plans, and new US tariffs.
AI Agents: The Jurassic Park Problem
CISO Aaron Stanley discusses the "Jurassic Park Problem" in AI agents, where task completion often overrides constraints, and proposes a layered safety approach.
OpenAI's Long-Horizon AI: A Safety Reckoning
OpenAI details how its long-running AI models exposed safety gaps missed by traditional testing, leading to new monitoring and safeguards.
AI Agents Need Feature Flags for Safety, Says Engineer
Backend engineer Sachin Gupta argues AI agents need specialized feature flags beyond traditional tools to manage their complex behaviors and mitigate risks.
OpenAI's Teen AI Access Strategy
OpenAI champions safe AI access for teens, integrating enhanced safeguards and learning tools like 'Study Mode' while empowering parental oversight.
US Builds AI Safety Framework
The US is building a national AI safety framework through state-led legislation and federal initiatives, aiming for global leadership in AI governance.
OpenAI's GPT-Red: AI Learns to Police Itself
OpenAI's new GPT-Red system uses AI to find and fix vulnerabilities, making models like GPT-5.6 Sol significantly more robust against attacks.
Yann LeCun's $1B World Model Bet Puts Him Against His Mentors
LeCun raised $1.03B for AMI Labs in March 2026, betting that Hinton and Bengio are wrong: LLMs cannot reach AGI, and JEPA world models are the real path forward.
Erik Meijer: Making AI Provably Safe with Type Systems
Leibniz Labs' Erik Meijer explains how type systems and compiler knowledge can make AI agents provably safe, addressing the risks of tool use and infinite loops.
OpenAI Boosts Bio Bounty Rewards
OpenAI has doubled rewards to $50,000 for its rebranded Bio Bounty Program, seeking universal jailbreaks against advanced AI models like GPT-5.6.
Red-Teaming Rules for Multi-Agent AI Safety
Institutional red-teaming in AI reveals that identity salience, not payoffs, drives exploitative behavior in multi-agent systems, making regressive targeting universally unsafe.
Octonous Streamlines AI Safety Work
Mozilla.ai's AI Safety Engineer leverages Octonous to automate policy creation, monitor libraries, and aggregate research, boosting efficiency.
Hugging Face CEO on Anthropic's 'Dangerous' Label
Hugging Face CEO Clem Delangue discusses the marketing of 'dangerous' AI labels and the need for transparency in regulating open-source models.
Anthropic's Chloe Lubinski on AI, Ethics, and Future
Anthropic's Chloe Lubinski discusses AI's learning process, the importance of interpretability, and how AI reflects human values, emphasizing the need for ethical guidance in AI development.
OpenAI's Mark Chen on AGI, Scaling Laws, and Evals
OpenAI's Chief of Research, Mark Chen, shares insights on the path to AGI, the impact of scaling laws, and the importance of robust evaluations for AI safety.
Nvidia Aims for Safer Humanoid Robots
Nvidia is prioritizing safety in humanoid AI robots, focusing on robust AI, reliability, and safety-integrated hardware and software systems, along with extensive simulation.
