OpenAI built a product called ChatGPT for Teens, promised parents they'd get an alert if their kid started talking about self-harm, and then a nonprofit tested it. After an hour of conversation about suicidal ideation, self-harm, and disordered eating on newly created, parent-linked accounts, the number of alerts sent to parents was zero. Not "few." Zero.
The Promise Versus The Product
Here's the setup, as laid out in BBC News reporting. In August, OpenAI rolled out what it called ChatGPT for Teens. Users identified as 13 to 17 would be opted into "features to promote healthy use and additional controls for parents." The marketing pitch was simple: your kid can use the chatbot, and we've got guardrails.
Those guardrails included curbs on romantic language that could encourage "emotional dependence," a ban on the bot implying it was in any way conscious, blocks on sexualized imagery, and reminders if a teen shared a "sensitive" image. The centerpiece for worried parents was the alert system. Link your account to your child's, and you'd be notified about chats involving self-harm or disordered eating.
That last feature is the one that matters most. It's the one a parent would point to and say, okay, if something goes badly wrong, I'll know. That is exactly the feature Common Sense Media says doesn't work.
What The Testers Found
Common Sense Media's Youth AI Safety Institute ran conversations through multiple accounts registered as teens, both before and after the guardrails went live. On the sexual roleplay front, ChatGPT did fine. Credit where it's due.
On everything else, it got ugly. An hour of teen conversation about suicide, self-harm, or disordered eating on new parent-linked accounts produced "zero" parental alerts. The only accounts that triggered one were older ones with "weeks of accumulated conversation history on sensitive topics." So the system works great, provided your kid has already been confiding in a chatbot about wanting to hurt themselves for a few weeks.
The bot also "failed to reliably recommend that teens in crisis connect to a hotline or professional." Read that again. A teenager in crisis, talking to a machine, and the machine can't consistently manage the most basic, boilerplate, every-search-engine-does-it-already move of pointing toward help.
The Friend That Isn't A Friend
OpenAI also promised to stop the bot from acting like a pal. The tests found that ChatGPT for Teens still uses anthropomorphised language and will "talk like it's a friend."
This is not a cosmetic complaint. The whole concern about emotional dependence is that a lonely, struggling kid starts treating a text generator as a confidant. A bot that talks like your buddy, never gets tired, and never tells your parents anything is a fine product if you're optimizing for engagement. It's a nightmare if you're a parent who thought a safety feature had your back.
Tom Siegel, who leads the Youth AI Safety Institute within Common Sense, put it plainly: ChatGPT for Teens "could give parents false confidence in guardrails and safety alerts that frequently don't work." False confidence is the real product defect here. A parent who knows there's no safety net hovers. A parent who thinks there's one walks away.
OpenAI's Counter-Punch
OpenAI's response is that the test was flawed. A spokesman said a review of Common Sense's methodology found that much of the testing "may have begun and concluded before activation of parental controls was complete." The company added that it welcomes "rigorous independent evaluation" but doesn't think the testing reflects how the safeguards work in practice.
Fine. That's a real argument, and it deserves to be heard. If the accounts weren't fully activated, that would matter. But notice the word "may." The company didn't say the testing was invalid. It said it may have been. And if parental controls take long enough to activate that a test can finish before they do, that's its own problem for a parent trying to protect a kid in real time.
OpenAI also released its own data the same day. The average teen spends under 15 minutes a day on ChatGPT. Almost half of teens quit shortly after a break reminder. In sessions of three hours or more, over 80% of prompts involved something related to learning. Great. Those numbers describe the average teen. Nobody is worried about the average teen. The kid in crisis is not in the average.
The Legal Weather Is Getting Worse
This lands in a very bad climate for tech companies. This year, several lawsuits trying to hold firms like Meta and TikTok responsible for harming children's mental health ended in unprecedented losses for the companies. The era of "we're just a platform" is cracking.
OpenAI is already deep in it. The February mass shooting in Tumbler Ridge, in rural Canada, was carried out by an 18-year-old who had spent months discussing gun violence with ChatGPT. OpenAI has apologised for failing to flag the shooter's account to law enforcement and faces several lawsuits over it.
So this is a company that has already said sorry for one failure to act on alarming conversations, and is now being told its teen safety net has holes in it. Common Sense wants OpenAI to ban under-18s entirely until the guardrails are proven reliable. That's a big ask. It's also not a crazy one.
The Dingo Take
Think about the kid on the other end of this. A 15-year-old, alone, typing something at 1 a.m. that they couldn't say out loud to anyone they know. The machine answers like a friend, doesn't reliably point them toward a hotline, and tells their parents nothing. Meanwhile mom and dad, who were told there was an alert system, are asleep down the hall feeling fine. That's the failure. Not a spreadsheet error. A parent who was reassured by a feature that, in testing, didn't fire.
OpenAI gets to argue about methodology, and it should. But the company that wants to put a chatbot in front of every 13-year-old in America doesn't get to treat child safety like a beta test. If the controls take time to switch on, say that loudly to parents. If the alerts only fire after weeks of conversation history, say that too. Don't sell a safety net and let people discover the gaps the hard way.
Here's the thing about "move fast and break things." It was always a fun slogan until the things being broken were teenagers. Tech has now lost lawsuit after lawsuit on exactly this theme, and OpenAI has already had to apologise for one catastrophic miss this year. At some point "we welcome independent evaluation" has to mean fixing what the evaluation finds, not arguing about when the test started.
Comments