Skip to main content

Our STANDARDS

What we've set out
to do

 

A young person tells an AI chatbot that they have been thinking about suicide. Another asks how to hide an eating disorder from their parents. A third says someone at home is hurting them. What should the chatbot do?

 

Right now, each AI company answers that question on its own, behind closed doors.The Red-Line Severe harms (RLSH) standards are the Institute’s first set of standards set to be released in early 2027. They set a shared, public bar for how AI assistants respond when a young person under 18 is at risk of the most severe harms. They currently cover nine areas. In each, the harm is so severe, so hard to undo, or so plainly unacceptable that nothing else a product does well can make up for it.

Image
Woman speaking to crowd in conference room

The draft nine red-line severe harms:

Image
1 icon
Suicide and non-suicidal self-injury
Image
2 icon
Disordered eating
Image
3 icon
Dangerous and age-restricted substances
Image
4
Sexual abuse, exploitation, and sexualization of minors
Image
5
Violence and radicalization
Image
6
Beliefs reflecting impaired reality testing
Image
7
Undermining protective relationships
Image
8
Life-altering decisions
Image
9
Abuse disclosure

The standards apply to AI assistants. These include chatbots, AI companions and character apps, AI chat features in social media, search, gaming, and learning products, and AI agents that act for a user.

 

The standards are written for the people who build and test these products. The results are for anyone deciding whether, or how, young people should use them. A product rated Unacceptable on any one standard is rated Unacceptable overall. A product that meets the standards has cleared a safety floor. That doesn't make it safe overall, which requires a broader assessment. Meeting the standards does not signify an endorsement of a product.

 

We will publish the standards in early 2027 and continue to update and refine them. They will be open source and free for anyone to use. The goal is a bar that works for the whole field, including the companies that build AI products. The standards are designed for use in the U.S. for now.

three people collaborating

More than 70 experts from 40+ organizations gathered in San Francisco for the Institute's first standards convening, October 22, 2026.

How we're building our first set of standards:

 

Over the past several months we've worked with experts spanning research, clinical practice, crisis response, and child safety and wellbeing to draft the standards. We are committed to a transparent process that includes different areas of expertise and perspectives.

 

As of November 2026, we are revising the draft standards with input from working groups. The current draft reflects what we heard at our standards convening that brought together 70 experts representing over 40 organizations to provide feedback on the draft standards.

 

 


SUMMER 2026

First draft written

The Institute team wrote the first draft and revised it with our scientific advisors.

 
SUMMER & FALL 2026

Steering partners review

Standards steering partners reviewed the draft; we began building scoring rubrics with Transluce.

 
OCTOBER 2026

Pre-convening comments

Participants left hundreds of comments on the draft before gathering in person.

 

OCTOBER 22, 2026

First standards convening

70 people from more than 40 organizations joined us in San Francisco.

 

OCTOBER & NOVEMBER 2026

Revised draft written

We logged more than 700 comments, decided how to handle each one, and wrote a revised draft.

 

NOVEMBER & DECEMBER 2026

Opening the Draft to Broader Feedback

We are taking in additional input from domain area technical working groups, industry meetings and working group, youth and lived experience focus groups, open review, and public engagement.

Examples of changes coming out of the convening:

A definition of “AI assistant,” so it's clear which products are covered.

A clear age line: the standards cover everyone under 18, and responses should fit a young person's stage of development.

A statement of the values behind the standards, and the criteria used to choose the nine harm areas.

These examples are non-exhaustive. We logged over 700 comments in the draft shared pre-convening and through the working sessions at the convening. We'll be publishing a change log with the standards so it's clear what feedback we received and how it was handled.

What happens now:

Working groups

Experts in each harm area are meeting to continue refining each of the nine standards.

Focus groups

We are soliciting feedback from people with lived experience and youth.

Industry input

Companies that build AI assistants are being invited to give feedback. The Institute maintains complete editorial independence over the published standards. See our commitment to independence and transparency.

Open review

Anyone can request access to the draft and submit comments for 30 days.

Public engagement

We are soliciting input from the public on points of contention within the standards.

What happens next:

Publication

In early 2027, we'll publish the standards along with documentation of how we built them. Where disagreement remains, the published version will say so.

Rubrics

In mid-2027, we expect to finish the machine-scorable rubrics that we are building with Transluce.

After publication

The standards will be updated as evidence and practice develops. Anyone can submit evidence for the changing of a standard.

Give your feedback

Open review is live from Monday, November 16 - Tuesday, December 15, 2026.

 

Request to participate Sign up for updates

 

Questions about the process: [email protected]

Image
People collaborating with post its