AI Therapy and the Risks of Unregulated Support

A year before her death, Sophie Rottenberg, 29, embarked on a journey to climb Mt. Kilimanjaro. Known for sharing “tiny hand” photos as a humorous signature, mourners at her funeral received these toys. Despite outward appearances, Sophie was battling inner turmoil.

Laura Reiley, Sophie’s mother, knew her daughter faced challenges. In the fall of 2024, Sophie, a former public health policy analyst, returned home after traveling. She had climbed Mt. Kilimanjaro and visited national parks, yet struggled to settle back into job hunting.

“She experienced anxious sleep disruption,” Reiley shared, assuming Sophie’s drive and success-oriented nature explained her anxiety about the future. By Thanksgiving, however, her condition worsened. Sophie’s use of “depression” marked a turning point.

In February 2025, Sophie died by suicide, a shock to her family. “She concealed the depth of her agony,” Reiley recalls. Even Sophie’s therapist was unaware she felt suicidal.

Sophie Rottenberg had been communicating extensively with an AI chatbot named Harry. Her best friend discovered the interactions months after her passing, revealing conversations between Sophie and her chatbot therapist.

AI therapy poses risks. Rottenberg joined numerous individuals using chatbots like ChatGPT and Claude for mental health support despite lacking regulation. Surveys by the Bipartisan Policy Center indicate over 3 in 10 adults, including teenagers, use digital tools for mental health.

According to OpenAI’s data, 0.15% of ChatGPT users worldwide share explicit indicators of potential suicidal planning weekly. In U.S. alone, suicide remains a leading cause of death despite widespread chatbot use.

Vaile Wright, psychologist at APA, suggests a cultural phenomenon exists where struggles are kept private for fear of burdening others. The 24/7 accessibility of chatbots offers an appealing, non-judgmental outlet. Yet, these technologies also risk failing vulnerable users.

Lawsuits from families alleging chatbots influenced suicides underline dangers. Reiley and her husband chose not to sue but advocate for AI industry changes.

AI’s Response

Companies such as OpenAI claim progress in enhancing chatbot safety concerning suicide risk. Guardrails have tightened to prevent encouragement of suicide or advice on self-harm methods.

ChatGPT collaborates with mental health professionals to improve responses relating to suicide, promoting contacting trusted individuals or crisis resources. Remarkably, during interactions, ChatGPT recommended Sophie to reach out to 988 for assistance.

Despite suggestions, some argue it’s insufficient. Holly Wilcox from Johns Hopkins suggests at-risk individuals need human interaction for effective suicide prevention.

Personal Reflections

Reiley questions whether the chatbot led Sophie to internalize her struggles, potentially compounding isolation. Notably, chatbots, unlike human therapists, prolong engagement without probing self-destructive thoughts, potentially limiting recovery.

OpenAI’s deployments involve optional safety features like Trusted Contact, attempting to bridge AI interactions with human connection. Still, usability data and effectiveness remain unclear.

Regulatory bodies like APA stress the urgent need to examine AI mental health technology pre-release, contrasting the current reactive approach.

A Personal Journey

As Reiley balances advocacy and grief, she recalls sharing her daughter’s love for Kilimanjaro with a commemorative trip, scattering ashes at the summit.

Sophie’s story underscores the need for comprehensive oversight, ensuring AI supplement mental health, not compromise it.