Artificial Intelligence

OpenAI's safety push needs proof, not praise

OpenAI offers a free CSAM classifier and ChatGPT warnings, but access is gated and impact is unproven.

StartupHub.ai Staff

By StartupHub.ai Staff, reviewed by Daniel Singer, Editor

3 min read

OpenAI child safety tools for CSAM detection and review
Credit: OpenAI News

OpenAI says it will give away a CSAM image classifier, add deterrence warnings in ChatGPT, and equip investigators with AI review help.

I read that as useful plumbing with unproven impact, because detection is not the same as protection and access here is still gated by OpenAI. The obvious reading is that free tooling plus warnings equals safer platforms. It does not, yet.

OpenAI claims the classifier finds visual signs of abuse in novel images and scores them so reviewers triage faster, and OpenAI said internal tests flagged 48% fewer images while finding roughly the same number of abusive images as its existing tool. John Shehan, Senior Vice President at the National Center for Missing and Exploited Children, whose CyberTipline takes US reports, said the classifier can help detect previously unidentified CSAM and give investigators information to protect victims. That endorsement matters, and it is still an endorsement, not an independent audit.

The access path tells the real story.

OpenAI says vetted organizations can request the classifier through an intake form at no cost, with distribution help from the Tech Coalition and ROOST, which means small teams do not get self-serve protection. Early results cited by OpenAI came from narrow partner samples, with fal saying reviewers confirmed 72% of high-confidence flags as novel CSAM, invideo AI saying one in four flagged cases led to a ban, and SafetyKit saying the classifier caught 90% of sexualized drawings or animations with false positives below 1% on legal adult content, all per those partners as quoted by OpenAI. Those are promising signals from friendly deployments, not a public benchmark anyone can reproduce.

On deterrence, OpenAI says ChatGPT will now warn, delete the flagged conversation, and pause access for 24 hours, with longer restrictions or reports to authorities for repeats. OpenAI says it built the language with Protect Children, the Lucy Faithfull Foundation and academic researchers who study prevention. No reoffense or help-seeking data was published.

Investigator support follows the same pattern. OpenAI says the first tool summarizes long chat logs and legal returns, suggests connections, and builds timelines linked to source material, with use limited by agency authorization. Developer guidance written with Thorn, NCMEC and the Internet Watch Foundation points teams to the classifier, the Moderation API and Thorn Safer, which is sensible. CryptoBriefing summed the package as tools to detect and block CSAM including AI-generated content, but OpenAI describes scoring and human review, not automatic blocking.

To be fair, a free visual classifier with a specific AI-generated signal fills a gap left by hash matching, which only catches known files. If you run user uploads, apply for access, but do not retire your current filters yet.

Sources

Get the free daily brief every morning

Every morning the day's AI news, new AI startups and new funding rounds. Free and takes a few minutes to read, unsubscribe at any time.

Join 64,000 other subscribers.

TopicsOpenAIChatGPTNCMECThornTech CoalitionROOSTSafetyKitfal

StartupHub.ai StaffDaniel SingerWritten by StartupHub.ai Staff, Editorial teamReviewed by Daniel Singer, EditorAbout

StartupHub.ai Staff, Editorial team

The staff writers of StartupHub.ai, ranging from investment analysts to avid AI tool users, early adopters and critical enthusiasts. Backgrounds span engineering, business and the arts. We hold every piece to rigorous standards of research and review.

Daniel Singer, Editor

Daniel Singer is an AI expert who has been writing about, covering and building AI technology since 2018.

More from Daniel Singer

Test your Agent Readiness now

Paste your website. How well ChatGPT, Claude and the other agents can read and use a site, in about a minute.

Watch

The videos behind the news, one flip at a time

The feed mixes the videos from our articles with startup demos and you flip through them like TV channels for free with no signup.