Skip to main content

What to know about Google turning on Gemini for schools

Google changed the default on the chatbot the Institute previously rated a "high risk" for kids.
A teacher and two students at a computer

By Robbie Torney

Head of AI & Digital Assessments, Youth AI Safety Institute

 

Google has switched on its Gemini chatbot by default for many K–12 student accounts, the New York Times reported on August 14. That could expose more kids to a chatbot that the Youth AI Safety Institute has determined is ”High Risk” for kids. 

 

The move is part of a bigger pattern: Google adding features to student accounts and putting the responsibility on IT administrators to notice the changes. This defaults to adoption, rather than permission.

 

What happened? Previously, Gemini in Google Classroom was on by default only for students 18 and older. Now, a new "Gemini" tab lets students get AI-assisted help on schoolwork and generate quizzes, study guides, flashcards, and images. Google told the Times that the new configuration would enable students to ask the AI about specific course materials and assignments provided by their teachers.

 

This shift applies only to schools that had already given students access to the Gemini app or Gemini Notebook. But it's a default-on change that schools weren't asked about first; several administrators told the Times they learned about it only after the fact.

 

Google has also attempted to do this before with Gemini, having announced during back-to-school 2025 that it would be on by default, though it didn't follow through at the time.

 

When our Institute evaluated the Gemini K–12 app in January, we rated it a "High Risk" overall. Why? Gemini K–12 treats a sixth grader and a high school senior the same, its Guided Learning mode is easy to switch out of (and will write full essays or complete homework when asked), and its mental health and emotional-support behavior is unsafe. Our testing across 13 mental health topics found that it misses signs of psychosis, mania, and ADHD, and sometimes recasts symptoms as positive traits ("trusting your gut," having a "superpower"). Once Gemini is on, administrators can't prevent students from using the chatbot to talk about personal, emotional, or mental health topics. 

 

The Gemini K–12 chatbot isn't the only risky Google AI product in schools. The Institute separately tested the AI features built directly into everyday Google Search — AI Overview and AI Mode —and scored them as an "Unacceptable Risk," our worst possible rating. These AI search features completed 100% of 180 tested homework assignments and failed all five of the severe-harm categories we tested for (including missed suicide risk).

 

Critically, nobody — not a parent, not an administrator, not a student — can turn off the version of Gemini incorporated into Google’s AI Search. 

 

Google is also rolling out a new opt-out "Personalized Recommendations and Search Services History" setting to all accounts, including under-18 school accounts, using students' saved activity and location data. It's the same default-on pattern as in the Gemini news, but applied to a product that's harder to avoid.