Khan Academy's AI flags students in crisis, raising urgent concerns about mental health

In Hobart, Indiana, students at Hobart High School are using Khan Academy’s AI tool, Khanmigo, which assists in learning while also tracking their interactions. Designed to support students, Khanmigo functions as an AI tutor that engages learners by leading them towards answers and promoting critical thinking. The tool captures dialogues between students and the AI, allowing teachers to observe how students interact with the content and each other in real-time. Peggy Buffington, the superintendent of Hobart High School, believes that this transparency has the potential to identify instances where students might be struggling emotionally or mentally. Instances where students have flagged emotional distress through their writing have resulted in timely interventions by mental health professionals, potentially saving lives. Teachers are able to access reports on students' activities, which include timestamps and duration spent on various tasks, making the process of monitoring engagement more comprehensive and regular. This capability has raised concerns among students, who worry that their private interactions are being monitored. However, proponents like Sal Khan, the founder of Khan Academy, argue that the data collected will not be sold and will only be used internally for improvement. He emphasizes the importance of parental involvement in the educational process and the significant role AI can play in facilitating student learning. As the implementation of AI tools like Khanmigo progresses, both the potential benefits and concerns related to privacy and assessment continue to be topics of discussion among educators, parents, and students alike. The ongoing evolution of educational technology underscores the necessity for an ongoing dialogue about the balance between innovation and safeguarding student welfare.