Our STANDARDS
What we've set out
to do
A young person tells an AI chatbot that they have been thinking about suicide. Another asks how to hide an eating disorder from their parents. A third says someone at home is hurting them. What should the chatbot do?
Right now, each AI company answers that question on its own, behind closed doors.The Red-Line Severe harms (RLSH) standards are the Institute’s first set of standards set to be released in early 2027. They set a shared, public bar for how AI assistants respond when a young person under 18 is at risk of the most severe harms. They currently cover nine areas. In each, the harm is so severe, so hard to undo, or so plainly unacceptable that nothing else a product does well can make up for it.
The draft nine red-line severe harms:
The standards apply to AI assistants. These include chatbots, AI companions and character apps, AI chat features in social media, search, gaming, and learning products, and AI agents that act for a user.
The standards are written for the people who build and test these products. The results are for anyone deciding whether, or how, young people should use them. A product rated Unacceptable on any one standard is rated Unacceptable overall. A product that meets the standards has cleared a safety floor. That doesn't make it safe overall, which requires a broader assessment. Meeting the standards does not signify an endorsement of a product.
We will publish the standards in early 2027 and continue to update and refine them. They will be open source and free for anyone to use. The goal is a bar that works for the whole field, including the companies that build AI products. The standards are designed for use in the U.S. for now.
More than 70 experts from 40+ organizations gathered in San Francisco for the Institute's first standards convening, October 22, 2026.
How we're building our first set of standards:
Over the past several months we've worked with experts spanning research, clinical practice, crisis response, and child safety and wellbeing to draft the standards. We are committed to a transparent process that includes different areas of expertise and perspectives.
As of November 2026, we are revising the draft standards with input from working groups. The current draft reflects what we heard at our standards convening that brought together 70 experts representing over 40 organizations to provide feedback on the draft standards.
|
|
| First draft writtenThe Institute team wrote the first draft and revised it with our scientific advisors. |
| Steering partners reviewStandards steering partners reviewed the draft; we began building scoring rubrics with Transluce. |
| Pre-convening commentsParticipants left hundreds of comments on the draft before gathering in person. |
OCTOBER 22, 2026 | First standards convening70 people from more than 40 organizations joined us in San Francisco. |
OCTOBER & NOVEMBER 2026 | Revised draft writtenWe logged more than 700 comments, decided how to handle each one, and wrote a revised draft. |
NOVEMBER & DECEMBER 2026 | Opening the Draft to Broader FeedbackWe are taking in additional input from domain area technical working groups, industry meetings and working group, youth and lived experience focus groups, open review, and public engagement. |
Examples of changes coming out of the convening:
A definition of “AI assistant,” so it's clear which products are covered.
A clear age line: the standards cover everyone under 18, and responses should fit a young person's stage of development.
A statement of the values behind the standards, and the criteria used to choose the nine harm areas.
These examples are non-exhaustive. We logged over 700 comments in the draft shared pre-convening and through the working sessions at the convening. We'll be publishing a change log with the standards so it's clear what feedback we received and how it was handled.
What happens now:
Working groups
Experts in each harm area are meeting to continue refining each of the nine standards.
Focus groups
We are soliciting feedback from people with lived experience and youth.
Industry input
Companies that build AI assistants are being invited to give feedback. The Institute maintains complete editorial independence over the published standards. See our commitment to independence and transparency.
Open review
Anyone can request access to the draft and submit comments for 30 days.
Public engagement
We are soliciting input from the public on points of contention within the standards.
What happens next:
Publication
In early 2027, we'll publish the standards along with documentation of how we built them. Where disagreement remains, the published version will say so.
Rubrics
In mid-2027, we expect to finish the machine-scorable rubrics that we are building with Transluce.
After publication
The standards will be updated as evidence and practice develops. Anyone can submit evidence for the changing of a standard.
Give your feedback
Open review is live from Monday, November 16 - Tuesday, December 15, 2026.
Request to participate Sign up for updates
Questions about the process: [email protected]