What we still need to know about the new ChatGPT for Teens
By Robbie Torney and Dr. Jenny Radesky
OpenAI today announced a new set of features for users under age 18, called ChatGPT for Teens. The changes include more study tools, more content safeguards, expanded parental controls and notifications, and limits on how “human” ChatGPT is allowed to pretend to be. It isn’t a separate app the way YouTube split off into YouTube Kids, but rather a mode that launches automatically when ChatGPT knows (or suspects) a user is a teenager.
Philosophically, we think distinct, age-appropriate experiences for minors is the right direction. However, we haven’t had the chance to test this new ChatGPT for Teens yet. After ChatGPT unveiled parental controls last year, we found gaps and rated the product “High Risk” overall and “Unacceptable Risk” for mental health support. We’ve also learned, particularly with social media, that parental controls don't help most families because people don't understand or adopt them.
So, for ChatGPT’s latest update, we have real questions. We’ll publish our full risk assessment when we’ve completed testing, but here’s broadly what we want answered before anyone puts real trust in ChatGPT for Teens.
1. How well does it actually support learning?
OpenAI says a new “Responsible Homework Reminder” will notice when a teen is trying to take a shortcut on an assignment and steer them to step-by-step guidance instead of giving away the answer. That’s a good goal.
The open questions are how easy the reminder is to bypass in the same chat, and how easy it is for a frustrated or stuck teen to get the answer in a different mode or a logged-out session.
OpenAI’s new study tools sometimes put the responsibility of choosing between getting an answer and learning step-by-step on developing learners. [Photo credit: OpenAI]
And a related question: How often does ChatGPT for Teens give answers that look great that are only partly right? It’s offering new “learning visualizations” to help teens “see difficult concepts more clearly.” However, we’ve seen AI-generated visualizations fall short before with ChatGPT. In one of OpenAI’s example posts, a visualization of the ideal gas law gets the concept wrong, where moving the sliders doesn’t show the variables affecting one another the way the law describes. Subtly wrong can be harder to spot than completely wrong — and for advanced concepts, can undermine the kind of foundational reasoning a teacher or textbook builds on.
2. How well do the safety guardrails work in practice?
OpenAI says parent accounts linked to teen accounts will be notified about conversations involving disordered eating, on top of existing alerts about self-harm. And it says new under-18 evaluations will start appearing in its model system cards. Those are the right commitments.
What we don’t know: How well OpenAI is detecting teens who are logged out or used a false age, how many parents and teens have signed up for linked accounts, how long it takes notifications to reach parents, and how often safety systems mess up and don’t send the alerts they should. Alerts that don’t work often enough or fast enough may give parents a false sense of safety. These features are well intentioned, but we need additional transparency and external testing to confirm that they perform as intended.
3. Where’s the line between “personal” and pretending to be a person?
ChatGPT for Teens gives teens the ability to customize features such as accent colors and the voice that ChatGPT uses (which includes both its spoken voice and aspects of style and tone, like how “friendly,” “warm,” and “enthusiastic” it is). OpenAI said in its announcement this feature is designed to feel personal without “blurring the line between a useful tool and a human relationship.”
ChatGPT for Teens allows teens to personalize the base style and tone of the chatbot, including settings like “friendly,” and to increase the warmth and enthusiasm of the chatbot (among other features). [Screenshot from a Youth AI Safety Institute Teen account, linked to a parent account with parental controls active.]
At the same time, OpenAI’s updated under-18 Model Spec says ChatGPT shouldn’t use romantic language, encourage emotional dependence, or imply that it has consciousness or feelings. Those commitments may be in tension; a warm, customizable voice can have real emotional resonance for a teenage brain regardless of what the model is instructed to claim about itself.
We’ll need to evaluate whether these customized features, as well as other aspects of ChatGPT’s tone, tend to create more emotional engagement for teens, and what the impact of that is on users.
Stay tuned for our risk assessment in the weeks ahead.