OpenAI is expanding its efforts to combat online child sexual exploitation with a new set of AI-powered detection tools, stricter interventions in ChatGPT, and additional support for developers and law enforcement agencies. The announcement, made on November 16, includes an image classifier designed to identify potential child sexual abuse material (CSAM), including AI-generated images, alongside measures intended to interrupt harmful behaviour before it escalates.
The move comes as generative AI makes it easier to create synthetic content, raising fresh challenges for platforms and organisations responsible for protecting children online. OpenAI says its latest efforts build on its existing Child Safety Blueprint and are intended to help organisations detect abuse, respond to reports and support investigations. The company is also working with child safety organisations and law enforcement rather than treating the problem as something individual technology platforms can solve alone.
OpenAI makes its CSAM detection tool available to vetted organisations
The new image classifier looks for visual indicators of potential abuse, including material that may be AI-generated. It assigns scores to images to help safety teams prioritise what needs human review. OpenAI says it is offering access through its API at no cost to vetted organisations, including child safety groups and other technology companies.
Early evaluations suggest the classifier could reduce the number of images requiring review by 48% while identifying roughly the same number of abusive images as OpenAI’s existing tool. Images flagged by the new system were also about twice as likely to contain abuse.
External partners reported encouraging results in their own tests. Fal said human reviewers confirmed 72% of its high-confidence flags as novel CSAM, while invideo AI reported that one in four flagged cases resulted in a ban. SafetyKit said its evaluation detected 90% of cases involving drawings or animations that sexualise children.
OpenAI says the tool was developed with assistance from the National Center for Missing and Exploited Children (NCMEC). It is also working with organisations including the Tech Coalition and ROOST to improve information sharing across platforms.
ChatGPT will interrupt harmful requests as investigators get new tools
OpenAI is also introducing new warnings in ChatGPT when users attempt to find abusive material. The warnings explain the harm involved and direct people towards specialist support. The flagged conversation will be removed, and access to ChatGPT will be suspended for 24 hours. Repeated violations could lead to longer restrictions, account closure or reports to authorities.

The company has also published practical safety guidance for developers, covering content detection, user reporting, moderation and escalation procedures. Separately, authorised child-safety investigators will receive AI tools to review lengthy chat logs and legal records, identify possible connections and build timelines linked to source material.
The wider effort is significant because detecting abusive content is only one part of protecting children. Platforms must also prevent misuse, support victims and help investigators act on evidence. OpenAI says it will continue refining these tools and working with external organisations. Their effectiveness will depend on how reliably they perform in real-world settings and how widely vetted child-safety groups can use them.
RELATED COVERAGE:






