Florida seeks a ban on ChatGPT acting like a person

AG James Uthmeier says OpenAI needs to stop calling ChatGPT safe and ‘Stop pretending it’s human.‘

Florida Attorney General James Uthmeier is calling for a judge to block OpenAI from “giving ChatGPT false human attributes,” a few months after Florida sued the AI company over safety concerns . According to Uthmeier, users are lulled into a false sense of security by the AI bot, as “ChatGPT’s use of language, including first-person pronouns and output that mimics emotion, deceptively suggests to users that it is a trustworthy ‘friend.’” All of this happens to increase engagement and contribute to OpenAI’s training data, convincing users to rely on ChatGPT more even as its pursuit of engagement makes it potentially less trustworthy, claims Uthmeier.

The filing also asks the court to block OpenAI from developing new AI models without “third-party approved safety guardrails,” citing recent security incidents at Hugging Face , an Australian government website , and US government websites , as well as recent statements from researchers and former OpenAI employees raising alarm bells around AI safety.

Uthmeier says that OpenAI should “Stop calling it safe. Stop pretending it’s human. Stop selling it to kids.” Last month, OpenAI launched ChatGPT for Teens , with additional restrictions applied by default, saying that access to the tool “should come with protections that reflect their developmental stage, reinforce real-world relationships, and support healthy use over time.”

OpenAI spokesperson Drew Pusateri commented on the filing in a statement to The Verge , saying, “People want to know AI is being developed safely, and that starts with what companies like ours do ourselves. On Friday, we announced we’ve paused training our most capable models and will resume training only when we are confident that we have additional safeguards in place. Governments have an important role to play in setting robust safety standards for AI, and we’re committed to working with Florida and other states on advancing pragmatic AI policies that apply to the entire AI industry–not just one company.”

Update, September 28th: Added comment from OpenAI spokesperson Drew Pusateri .