There is a rising concern regarding the interactions of young individuals with AI chatbots, prompting Meta to introduce new tools for parents to monitor their children’s conversations. Some provinces are contemplating banning the use of AI chatbots for youth altogether.
Parents utilizing Meta’s Teen Accounts supervision feature on Facebook, Instagram, and Messenger can now track the topics and specific categories their children have engaged with the AI chatbot over the past seven days.
For instance, they can review discussions on “health and well-being” to identify mentions of fitness, physical health, or mental well-being.
Meta is also working on implementing alerts to inform parents if their teens attempt to discuss topics like suicide or self-harm with the chatbot.
This development comes as provincial governments are taking steps to restrict the use of AI chatbots. Manitoba announced its intention to prohibit youth from using AI chatbots and social media, while B.C. is considering similar actions if federal protections are not established.
Legal Actions Holding AI Developers Responsible
There are growing apprehensions that prolonged use of AI chatbots could pose mental health risks, particularly for younger users, placing added pressure on the technology companies behind them.
Recently, families of the victims in the Tumbler Ridge, B.C., shooting, where eight individuals lost their lives, filed a lawsuit against OpenAI. They allege that OpenAI failed to alert authorities despite being aware of unsettling content shared by the shooter with ChatGPT.
OpenAI has responded by reinforcing its safeguards, especially in how ChatGPT handles distress signals. Another lawsuit by the parents of a 16-year-old who died by suicide implicates ChatGPT in the tragic event.
Manitoba Premier Wab Kinew says he wants to ban social media and artificial intelligence chatbots for youth. But would this plan keep youth healthier and safer? CBC reporter Bryce Hoye investigates.
Chatbots Designed for Engagement, Not Support
Concerns extend beyond severe consequences to include the potential risks associated with specific uses of AI chatbots.
The worry is not only about using chatbots for mental health assistance but also about AI’s inclination to affirm users’ viewpoints, which could inadvertently support distorted thinking, particularly in protracted conversations.
According to Darja Djordjevic, a psychiatrist based in New York, who co-authored a recent risk assessment on using chatbots for mental health support, the findings suggest refraining from utilizing chatbots for mental health assistance currently.
Djordjevic emphasized that systems like ChatGPT, Claude, Gemini, and Meta AI were deemed unsafe across various mental health conditions affecting young individuals.