Elon Musk’s platform X has announced new restrictions on its AI chatbot Grok after facing global scrutiny, moving to block the generation and editing of sexualised images of real people.
The measures come after Grok was widely criticised for allowing users to create non-consensual sexualised images of women and children using simple text prompts, sparking investigations and temporary bans in several countries.
X moves to restrict image manipulation
X said it will now prevent Grok from editing images of real people in revealing clothing, including bikinis and underwear, in jurisdictions where such content is illegal.
The restrictions apply to all users, including paid subscribers.
“We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis,” X’s safety team said.
“This restriction applies to all users, including paid subscribers.”
As an additional safeguard or “extra layer of protection,” X said image creation and photo-editing features through Grok will now only be available to paid subscribers.
Elon Musk
It still works
But even after the added security, users were reportedly still generating sexualised images of real people via requests, as per The Washington Post.
Musk came out in defence of the tool and said: “When asked to generate images, it will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state,” he said.
“There may be times when adversarial hacking of Grok prompts does something unexpected. If that happens, we fix the bug immediately,” He added.
Investigations and regulatory pressure mount
The changes follow the launch of an investigation by California Attorney-General Rob Bonta into xAI, the developer of Grok, over the creation and spread of sexually explicit material.
“The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking,” Bonta said, adding that authorities have “zero tolerance” for AI-generated non-consensual intimate images or child sexual abuse material.
Gavin Newsome
California Governor Gavin Newsom described the situation as “vile” and urged authorities to hold the company accountable.
In Europe, the European Commission said it had noted the additional steps taken by X and would assess whether they adequately protect users.
UK media regulator Ofcom has also opened an investigation into whether X breached local laws.
International bans and scrutiny
Indonesia became the first country to block access to Grok entirely, with Malaysia following shortly after.
Malaysian Communications Minister Fahmi Fadzil said restrictions would only be lifted if X fully prevents the generation of harmful content.
India has also required X to remove thousands of posts and hundreds of user accounts linked to Grok-generated imagery.
Further pressure has come from a coalition of 28 civil society groups, which has called on Apple and Google to remove Grok and X from their app stores.
Ongoing concerns around AI misuse
An analysis by Paris-based non-profit AI Forensics found that more than half of over 20,000 Grok-generated images depicted individuals in minimal attire, predominantly women, with around two per cent appearing to be minors.
Despite X’s latest changes, regulators and advocacy groups continue to question whether the platform’s safeguards go far enough as governments worldwide grapple with how to respond to the rapid spread of AI-generated sexual content.
Top image: A Grok AI-generated image of Elon Musk (Getty Images)