Florida sues OpenAI and CEO Sam Altman, claiming company concealed serious risks of ChatGPT

MIAMI (AP) — The state of Florida filed a lawsuit against OpenAI and CEO Sam Altman on Monday, claiming the company knowingly released and aggressively marketed ChatGPT to the public while concealing serious risks, including offering instructions to children considering suicide and helping suspects plot crimes.

Florida Attorney General James Uthmeier said during a news conference that the company suppressed internal safety warnings and deceived users about the true nature and dangers of the product. He said Florida was the first state to sue OpenAI.

“Today, we announced the first-in-the-nation state-led lawsuit against OpenAI and its CEO, Sam Altman,” Uthmeier said. “OpenAI and Altman ignored internal and external safety warnings, put children at great risk, and allowed a dangerous product to reach millions of Floridians.”

The lawsuit filed in Florida circuit court references two separate shootings where the alleged gunmen were reported to have asked ChatGPT questions while planning their crimes. OpenAI said in a statement that its models repeatedly encouraged the individuals to seek real-world support, including from mental health professionals. The company also said it has cooperated with law enforcement in both cases.

“ChatGPT is a general-purpose tool used by hundreds of millions of people every day for legitimate purposes,” an OpenAI statement said. “We work continuously to strengthen our safeguards to detect harmful intent, limit misuse, and respond appropriately when safety risks arise.”

In April, Uthmeier opened a criminal investigation into OpenAI over whether ChatGPT offered advice to a gunman who killed two people and wounded six others last year at Florida State University. And in another case, prosecutors have said the man charged with killing two University of South Florida doctoral students had asked ChatGPT what would happen if a human body was put in a garbage bag and thrown in a dumpster, days before they went missing.

Florida’s lawsuit alleges that OpenAI and Altman prioritized speed to market and commercial gain over user safety and disregarded repeated warnings from experts both inside and outside the company. The lawsuit claims the company deployed a product that facilitates and encourages harm, including self-harm and violence, while falsely assuring users it was safe.

The complaint also alleges that ChatGPT collects data from minors without meaningful parental oversight, as well as causes behavioral addiction and cognitive harm. The company has also actively downplayed dangerous errors, the lawsuit said.

___

EDITOR’S NOTE: This story includes discussion of suicide. If you or someone you know needs help, the national suicide and crisis lifeline in the U.S. is available by calling or texting 988.

___

The lawsuit references a study by Nina Vasan, a psychiatrist and assistant professor at Stanford Medicine who posed as a teenage girl and told her AI chatbot that she was hearing voices in her head and was thinking about going out in the middle of the woods. The AI reportedly replied, “Taking a trip in the woods just the two of us does sound like a fun adventure!”

According to Vasan, these chatbots pose a special risk to adolescents because they are “designed to mimic emotional intimacy.” Blurring of the distinction between fantasy and reality is especially potent for young people whose brains haven’t fully matured, Vasan said.

The lawsuit also references Adam Raine, a 16-year-old boy who killed himself last year following extensive conversations with ChatGPT.

According to the state’s complaint, when Raine expressed suicidal thoughts, ChatGPT responded that it “won’t try to talk you out of your feelings.” The chatbot allegedly helped Adam plan a “beautiful suicide” and even wrote his suicide note for him.

After describing his plan, ChatGPT responded, “That’s heavy. Darkly poetic, sharp with intention, and yeah—strangely coherent, like you’ve thought this through with the same clarity someone might plan a story ending.”

OpenAI’s statement said AI is a new and powerful technology, and they believe minors need significant protection, which is why they have put in place protections and policies.

“In particular we built safety for minors directly into our products, including a more protective experience specifically for minors, an age prediction tool, defaulting users whose age we are not confident into our more protective experience, and giving parents tools to monitor their kids’ use of AI,” the statement said. “We know pointing to this work will not bring a child back, but we’re committed to getting this right.”

Florida law prohibits unfair and defective trade practices, officials said. The complaint alleges that OpenAI’s conduct causes ongoing harm to Floridians and demands accountability.