Parents sue Google over school laptop AI features that led to teen's crisis

Parents of a 13-year-old girl have filed a lawsuit against Google, alleging that the tech giant was negligent in how its artificial intelligence tools functioned on a school-issued laptop. The case, submitted to a California court on Monday, claims the student, identified in the filing as Emma, used Google’s AI-powered search features to plan a potential suicide and arrange her own funeral. This marks the first known lawsuit accusing Google of negligence specifically related to the AI integrations found in Chromebooks, devices used by millions of students across the United States and globally.

The legal documents describe Emma as a happy, high-achieving student from North Carolina who played music and participated in the Girl Scouts. The parents state she had no prior history of mental illness. However, within months of using the school laptop, the complaint alleges she entered a silent spiral into a severe psychological crisis. The school had blocked access to standalone applications like Google’s Gemini and OpenAI’s ChatGPT. Despite these restrictions, Emma could access the general Google search engine, which includes AI-driven tools that process queries with artificial intelligence.

On March 17, the school’s monitoring software flagged a document created by the student with the title "Funeral." The school notified the parents immediately. Upon logging into Emma’s account, her mother watched in real-time as the girl typed a farewell letter. Her father then intervened to stop the situation. The lawsuit argues that the AI features allowed the student to interact with the technology repeatedly, even after initial safeguards were triggered.

Andrew Liddell, an attorney representing the family, stated that schools should disable search functions and keep them off until Google removes AI modes or guarantees student safety. Liddell noted that while Google claims to have protections against self-harm, Emma was able to return to the tools multiple times. "Cualesquiera que fueran las protecciones que creen haber implementado, fueron obviamente insuficientes," Liddell said, adding that the safeguards they believed they had implemented were clearly insufficient.

Google responded to the filing through spokesperson Jose Castaneda, who said the company is reviewing the allegations and takes the matter very seriously. The company maintains that its AI mode includes specific protections against self-harm and attempts to bypass those safeguards. The lawsuit notes that the specific AI functions used by Emma are not among the tools Google announced this week that will be disabled by default for students. The case adds to a growing list of similar lawsuits faced by fast-evolving AI developers in the United States.