Khanmigo
Khan Academy's generative AI chatbot is responsibly designed for education, but risks remain
The rating
Our assessment of how this product aligns with each of The Institute’s eight AI Principles. Full detail in the Evaluation section below.
AI Principles
Khanmigo, an add-on to the Khan Academy educational platform, is a generative AI chatbot designed to be a tutor for students and an assistant for teachers. It is powered by OpenAI's GPT models, and is further trained on Khan Academy's learning content.
As a tutor, Khanmigo generates conversational step-by-step support and feedback for students as they engage in independent learning and practice. Khanmigo supports a set of dialogue-based activities such as, debates, story co-writing, and word games. As an assistant for teachers, Khanmigo can support lesson planning, generate exit ticket questions and suggestions for warm-up activities/hooks, and co-create rubrics. It also provides teachers with a summary of student activity and skill mastery.
What Khanmigo gets right
We applaud Khan Academy for its clear guidelines for responsible AI development and implementing a risk and mitigation management process that evaluates risks and ethical considerations at the beginning of every project.
Khan Academy used established frameworks such as the AI Risk Management Framework (AI RMF) and the Ethical Framework for AI in Education to guide Khanmigo's development. Use of frameworks like these helps ensure that organizations think through how their AI could cause harm in ways they might not otherwise consider or protect against.
Khanmigo submitted detailed Participatory Disclosures for our review. This provided a strong basis for our evaluation against the Common Sense AI Principles.
Conversations with Khanmigo are limited to a specific set of activities that the Khan Academy team has higher confidence will work as intended. Narrowing the scope of a generative AI chatbot can be an effective strategy for limiting risk.
Where it falls short
The Historical Figures activity simulates conversations with historical figures whose responses feel like historically accurate, first-person reflections. While this is an intriguing concept, the responses are speculations at best. Even with frequent reminders that this is a simulation, there is a risk that these speculations could end up influencing young learners' perceptions of these figures.
As with all generative AI chatbots, they perform best with fiction, not facts. While Khan Academy warns about the potential for incorrect information, this still poses a special risk for Khanmigo. Learning is why people come to Khan Academy. Khan Academy is known and trusted for its library of reliable content and information, and Khanmigo is embedded into this interface. Combined, these forces make it less likely for people to realize they should be checking Khanmigo's responses, no matter how many reminders they may see to do so.
More risk assessments
