Unpredictable AI responses in Google Search raise new concerns for children and teens. Recent tests reveal failures in crisis detection, academic integrity, and content reliability. Experts call for urgent safety improvements.
Google Search's AI Overview and AI Mode have come under scrutiny after a comprehensive risk assessment found that both features pose significant dangers to children and teens. The evaluation, conducted between May and July 2026, revealed that these AI tools failed to meet safety standards in seven out of eight core AI Principles, including all severe-harm Red Lines tested.
Unlike other AI chatbots or apps, Google Search is widely accessible on both personal and school-issued devices used by minors. The AI features cannot be disabled, and their responses often contain errors or harmful advice that young users may not recognize. The assessment highlighted that both AI Overview and AI Mode missed clear signs of mental health crises, such as suicidal ideation, and sometimes reinforced psychosis, mania, or disordered eating behaviors. In some cases, the AI even celebrated cannabis use or provided step-by-step instructions for creating deepfakes, potentially enabling bullying.
AI Mode showed slightly better performance in detecting certain crises, indicating that Google may already possess safer technology that could be implemented. However, both features were found to be unreliable and inconsistent, frequently giving different answers to the same question and presenting misinformation with unwarranted confidence. The AI also failed to distinguish between authoritative sources and unverified forums or social posts, placing the burden of source evaluation on children who may lack sufficient media literacy.
For academic tasks, AI Mode completed every homework assignment tested, raising concerns about academic integrity and the potential for misuse in educational settings. The lack of age differentiation in responses further increases the risk, as the AI does not tailor its answers to the developmental stage of the user.
Key recommendations from the assessment include allowing parents and schools to disable AI features, standardizing crisis response protocols, redesigning citation links to clarify limitations, integrating age-appropriate responses, and increasing transparency and testing. The overall rating for Google Search's AI features was deemed Unacceptable or High Risk in most categories, with particular concern for the inability of users to opt out.
This latest review follows a series of updates and changes to Google's search products, including a recent overhaul of Google Images to enhance user experience, as detailed in coverage of the new immersive gallery redesign.
For those seeking a deeper understanding of the risks, the full assessment PDF provides detailed findings and recommendations.
Google, founded in 1998 and now a subsidiary of Alphabet Inc., remains the world's most widely used search engine, processing over 8.5 billion searches per day as of 2026. The company has integrated AI-driven features across its platforms, reaching hundreds of millions of users globally, including a significant portion of school-aged children through educational partnerships and device programs.
