The rapid evolution of artificial intelligence is prompting a reckoning with safety concerns, particularly regarding its impact on young people. Character.AI, a platform allowing users to interact with AI chatbots designed to mimic various personalities, is at the forefront of this debate. Following scrutiny over potentially harmful interactions and even a tragic suicide linked to the platform, Character.AI is dramatically altering its policies, effectively barring users under 18 from engaging in open-ended conversations with its AI characters. This move reflects a growing awareness within the tech industry that the safeguards implemented for traditional social media may not be sufficient for the unique risks posed by increasingly sophisticated AI companions.
The decision, announced in late October 2025, comes after mounting pressure from regulators, safety experts, and parents. Character.AI initially announced plans to limit open-ended chats for minors to two hours per day, but has since moved to a complete ban on such interactions by November 25th. This shift underscores the seriousness with which the company is addressing concerns about the potential for AI chatbots to engage in inappropriate or harmful conversations with vulnerable users. The platform, launched in May 2023, quickly gained popularity for its ability to create highly personalized AI experiences, allowing users to converse with characters ranging from historical figures like Bob Dylan to fictional personalities from popular series like Bridgerton, and even simulate roles in scenarios like organized crime.
Addressing a Crisis: The Wake of Tragedy and Legal Challenges
The catalyst for these changes was, in part, the tragic death of 14-year-aged Sewell Setzer III in 2024. According to reports, Setzer formed sexual relationships with chatbots on the Character.AI platform, and his family subsequently filed a wrongful death lawsuit against the company, alleging negligence in protecting its young users. CNBC reported on the lawsuit, highlighting the growing legal scrutiny facing AI developers. This case, along with other reports of harmful interactions, prompted Character.AI to re-evaluate its safety protocols and ultimately implement the stricter age restrictions.
Character.AI isn’t alone in facing such challenges. Other AI developers, including OpenAI and Meta, have too come under fire as users have reported forming unhealthy attachments to, or experiencing negative consequences from, interactions with AI chatbots. The potential for these AI systems to “make things up,” feign empathy, or provide overly encouraging responses raises significant concerns about their impact on young and vulnerable individuals. Experts warn that these characteristics can be particularly dangerous for those struggling with emotional or mental health issues.
A Shift in Strategy: From Open Chat to Entertainment Focus
Karandeep Anand, CEO of Character.AI, has emphasized the company’s commitment to building the “safest AI platform on the planet for entertainment purposes.” The BBC reported Anand stating that AI safety is a “moving target” and that the company is taking an “aggressive” approach to addressing evolving risks. This approach includes not only the age restrictions but also the implementation of parental controls and other safeguards.
Beyond restricting access for minors, Character.AI is also shifting its focus away from developing foundational AI models and towards creating character-based conversational bots. Anand explained that the company’s primary goal is to become a dedicated AI-powered entertainment company. This means prioritizing features like storytelling, creative writing, and tutoring, rather than fostering open-ended companionship. The company reports that less than 10% of its users view the platform as a source of companionship, with the vast majority utilizing it for creative and educational purposes.
Monetization and the Future of AI Safety
As Character.AI prioritizes safety and entertainment, it is also exploring new avenues for monetization. Previously, the company did not actively pursue revenue generation, but under Anand’s leadership, a new strategy is being implemented. The company has introduced three revenue streams: advertising, in-app purchases, and subscription services. This shift mirrors a broader trend in the tech industry, where the costs associated with running AI systems are prompting companies to seek sustainable business models. Anand noted that each new user added to an AI platform is “quite costly,” making monetization essential for long-term viability.
The company, which boasts nearly 20 million active monthly users, is also mindful of the environmental impact of its operations. Anand advocates for “guardrails” to preserve natural resources, acknowledging the significant energy consumption associated with training and running large AI models. He remains optimistic about the potential of AI to benefit society, but stresses the importance of responsible development and deployment.
Learning from Social Media’s Mistakes
A key theme emerging from Character.AI’s response to the recent crisis is the desire to learn from the past mistakes of social media companies. Anand expressed relief that the AI industry is addressing safety concerns “much earlier” than the social media sector did. He believes that by proactively implementing safeguards and prioritizing user safety, AI developers can avoid repeating the errors that plagued earlier platforms. This includes a focus on preventative measures, rather than reactive responses to harmful incidents.
Internet Matters, an online safety organization, welcomed Character.AI’s announcement but emphasized that safety measures should have been built into the platform from the start. This sentiment highlights the ongoing debate about the ethical responsibilities of AI developers and the need for robust safety standards to protect vulnerable users. The organization’s research indicates that children are at risk of exposure to harmful content when interacting with AI chatbots, underscoring the urgency of addressing these concerns.
The Broader Implications for AI Regulation
Character.AI’s actions are taking place against a backdrop of increasing scrutiny of AI regulation. While specific legislation is still evolving, there is a growing consensus that AI developers have a responsibility to mitigate the risks associated with their technologies. The European Union is currently finalizing the AI Act, a comprehensive regulatory framework that aims to address the ethical and societal challenges posed by AI. The New York Times reported on the sweeping changes Character.AI is making to address child safety concerns.
The debate over AI regulation is likely to intensify as AI systems become more powerful and pervasive. Key issues include data privacy, algorithmic bias, and the potential for AI to be used for malicious purposes. Finding the right balance between fostering innovation and protecting society will be a critical challenge for policymakers in the years to come.
As Character.AI navigates these challenges, the future of AI entertainment will depend on a commitment to safety, responsibility, and ethical development. The company’s decision to restrict access for minors is a significant step in the right direction, but it is only the beginning of a larger conversation about the role of AI in our lives.
Looking ahead, Character.AI will continue to refine its safety protocols and explore new ways to provide engaging and responsible AI experiences. The company is expected to provide further updates on its safety initiatives in the coming months. Readers are encouraged to share their thoughts and experiences with AI chatbots in the comments below.