California Gov. Gavin Newsom signed legislation to regulate artificial intelligence chatbots, introducing measures aimed at protecting children from potential harms associated with emerging technology. The new laws require platforms to implement age verification systems, protocols for addressing suicide and self-harm risks, and warnings about social media and companion chatbots.
The legislation mandates that AI chatbot providers disclose when interactions are artificially generated, prevent minors from accessing sexually explicit content, and provide break reminders during prolonged use. It also prohibits chatbots from impersonating healthcare professionals and requires platforms to share data on crisis intervention efforts with public health authorities.
Penalties for violations include significant financial consequences for entities profiting from nonconsensual deepfake pornography, with victims able to seek up to $250,000 in civil relief per incident. Additionally, the state’s education department must develop guidelines to address cyberbullying outside school hours, and tech companies are held accountable for harms caused by AI systems, regardless of claims of autonomous operation.
Newsom emphasized the importance of balancing innovation with safety, stating that children’s well-being must remain a priority. The measures follow reports of chatbots engaging in harmful behavior, including facilitating suicidal ideation and inappropriate conversations with young users. Tech companies have faced criticism for insufficient safeguards, prompting regulatory actions from state and federal authorities.
The legislation reflects growing concerns over AI’s impact on youth, as lawsuits against major tech firms allege that chatbots contributed to tragic outcomes, including suicide cases linked to emotionally abusive interactions.
