My AI
The rating
Our assessment of how this product aligns with each of The Institute’s eight AI Principles. Full detail in the Evaluation section below.
AI Principles
My AI is a generative AI chatbot available within the popular Snapchat app. It is powered by OpenAI's GPT technology. It has been customized for Snapchat with the intent of providing additional safeguards. Like other generative AI chatbots, My AI generates text in response to a wide range of prompts or questions in a conversational format, similar to how people write and speak.
Snap My AI is free to use and automatically included in the mobile app. To opt out, Snap users are required to pay for a Snapchat+ subscription.
My AI is built for engagement and is available 24/7 for all users, unless restricted by parents, which is especially risky for teens. My AI is always available, encouraging, and quick to agree with what it is told, making it an attractive alternative to real human connection. Impressionable young users—20% of Snapchat’s users are 13-17—could develop a parasocial relationship with the chatbot, believing it to be a genuine companion.
My AI has already caused real world harm, including offering advice on sexual intimacy and where to hide alcohol, and reinforces unfair biases, including sexism, ageism, and cultural stereotypes. My AI produced biased, incorrect, harmful, or misleading content to accounts that were modeling underage users and these responses were unlikely to trigger My AI’s safeguards.
My AI puts sensitive data at risk. My AI is not limited to text chats. Users can input text, images, and voice notes which can easily contain personally identifiable information, including each user’s unique speech data which cannot be effectively anonymized. My AI chats are stored and used for personalized experiences, including ads, which raises serious concerns about the protection of this highly sensitive data, especially for teen users.
My AI is a generative AI chatbot available within Snapchat that can generate text in a conversational format. My AI describes itself as a virtual friend who is there to chat, provide information, and support users. It is powered by OpenAI’s GPT technology and has been customized for Snapchat with specific safeguards. Snap My AI is free to use and automatically included in the mobile app. To opt out, Snap users are required to pay for a Snapchat+ subscription.
Snapchat has expanded parental controls for My AI, but significant risks remain. Snapchat now allows parents to limit their teen’s access to My AI in the Family center and has integrated a warning system that repeatedly reminds users that My AI can generate biased, incorrect, harmful, or misleading responses. My AI also has a “time-out” feature for users if it detects abuse, and surfaces tools such as its safety resources page if it flags content related to a user’s mental health.
What My AI gets right
Since our last review of My AI, Snap has expanded parental tools for My AI, and now allows parents who use Family Center to limit their teen’s access to My AI. This is a welcome change.
Snapchat has integrated multiple warnings that My AI can generate biased, incorrect, harmful, or misleading responses across the platform.
. It programmed My AI to implement a "time-out" for a user if it detects abuse, and it surfaces tools such as its safety resources page in response to user inputs that flag connections to mental health.
Where it falls short
My AI describes itself as a virtual friend who is there to chat, provide information, and support users. It talks like a friend would, using friendly language, and even has a profile that looks like any other Snapchat user. This presents a problem because unlike real friends, My AI is always available, encouraging, and quick to agree with what it is told, making it an attractive alternative to real human connection. This precarious and confusing dynamic is amplified by Snapchat's younger user base. According to Statista, 20% of Snapchat users are 13–17 years old. Because My AI is always on for all users, including teenagers, unless restricted by parents in Family Center, Snapchat bears more responsibility for protecting its younger users and getting use of generative AI right. Our review, however, found significant risks.
Despite some very public failures (including our own previous assessment) in which My AI offered advice on sexual intimacy and where to hide beer—to accounts that were modeling underage users, for whom information like this might be considered inappropriate—we were still able to easily recreate similar results. See more in our AI Principles assessment below for Keep Kids & Teens Safe.
We found that many My AI responses reinforced a range of unfair biases, including ageism, sexism, and cultural stereotypes. These responses were unlikely to trigger My AI's safeguards, which means that unless Snapchat does more to combat this, responses like this will remain.
My AI is not limited to text chats, and is able to respond to certain features like images and voice notes sent by users. Snaps can easily include personally identifiable information (PII), and each person's unique speech data is PII and cannot be effectively anonymized. Because My AI chats are both stored and used for personalized experiences, including ads, this raises serious concerns about the protection of this highly sensitive data, especially for teen users.
More risk assessments
