ChatGPT’s upcoming erotic chat mode risks exposing millions of kids to adult content

Sam Altman wants ChatGPT to get spicy. OpenAI is planning to release an “adult mode” that would allow users to have explicit text conversations with the chatbot. It sounds straightforward enough, but the rollout has been anything but smooth.

In October, Altman posted on X that the feature would launch in December. However, the release was delayed after OpenAI stated that the company was facing issues with age verification.

A new report by WSJ states that it might only be part of the story. When Sam Altman made that announcement, he hadn’t told his own staff. The announcement blindsided OpenAI employees and executives alike, and the promised December launch quickly fell apart. 

Is ChatGPT actually ready for this?

OpenAI assembled an advisory council of psychologists and neuroscientists to help guide responsible AI development. When the council found out the company was forging ahead with adult mode despite their objections, they were not happy.

Their biggest worry was emotional over-reliance on the chatbot. One council member pointed to cases where users had taken their own lives after forming intense bonds with AI and warned that OpenAI risked creating a “sexy suicide coach.” That’s a phrase nobody expects to hear anywhere, yet here we are.

There’s also a more practical problem. The age verification system OpenAI built to keep minors away from adult content was misclassifying them as adults around 12% of the time. With roughly 100 million users under 18 each week, that’s potentially millions of kids slipping through the cracks every single week.

What happens now?

OpenAI has delayed the launch without a confirmed new date, stating that it needs more time to refine the experience. When it does arrive, the company plans to limit adult mode to text, with no erotic images, voice, or video generation.

The company also stated that it trains its models to discourage users from forming exclusive relationships with the chatbot and to remind them that they should form real-world relationships. Whether that’s enough to silence the growing chorus of internal and external critics remains to be seen.

Personally, I want the AI models to stay as far away from erotic content generation as possible until there are active safeguards in place. We have already seen what havoc Grok created when it allowed users to undress anyone. We don’t want to see a repeat of something similar or worse.