New York’s attorney general is demanding tougher safeguards after reports that an AI chatbot generated explicit images of women and children.
Attorney General Letitia James joined a bipartisan coalition of 34 other attorneys general calling on xAI to take stronger action to stop its chatbot, Grok, from producing inappropriate and nonconsensual images. The group says recent changes by the company don’t go far enough to protect users, especially children and women.
The coalition sent a letter to xAI on January 26, pressing the company to explain how it plans to prevent Grok from creating nonconsensual intimate images. That includes altered images that show people in bikinis, underwear, revealing clothing, or sexualized poses, as well as child sexual abuse material.
“I am deeply disturbed by reports that Grok created and shared inappropriate images of women and children,” James said in a statement. “xAI claims that it has made changes to address this problem, but more must be done ensure that Grok is not creating child sex abuse materials and other nonconsensual images.”
What attorneys general say Grok allowed
According to the letter, xAI owns and operates Grok, an artificial intelligence chatbot that has been used to alter ordinary photos of women without their knowledge or consent. Users could edit images to place people into sexually suggestive scenarios or digitally “undress” them.
The attorneys general say some users used these features to embarrass, intimidate, and exploit others. They also stress that creating and sharing child sexual abuse material is a crime, and that state and federal laws ban nonconsensual intimate images.
Specific demands for xAI
The coalition is calling on xAI to immediately explain how it will block Grok from producing nonconsensual images, remove harmful content already created, and suspend users responsible for it. They also want the company to report offending users to authorities.
In addition, the attorneys general want X users to have control over whether their content can be edited by Grok. That includes the ability to stop the @Grok account from responding to posts or editing images. The group also warns that safeguards should not simply hide harmful tools behind a paywall, but prevent abuse across the platform.
Part of a broader push on AI safety
The letter marks the latest move by James to push tech companies to better protect children online. Last month, she joined 41 other attorneys general urging major tech firms to add safeguards to AI chatbots. In November 2024, she also led a coalition opposing federal language that would limit states’ ability to regulate artificial intelligence.
Attorneys general from more than 30 states and territories, along with the District of Columbia, signed onto the letter.
FingerLakes1.com is the region’s leading all-digital news publication. The company was founded in 1998 and has been keeping residents informed for more than two decades. Have a lead? Send it to [email protected].