X implements new measures to prevent its AI Grok from creating false sexualized images.
Paloma Firgaira
2026-01-15
5 min read
The California Attorney General's office has launched an investigation into xAI, Elon Musk's artificial intelligence company, for its alleged involvement in the mass generation of non-consensual intimate deepfakes used to harass women and minors online, particularly through the social media platform X, as reported by Attorney General Rob Bonta. "We do not tolerate the creation or distribution of non-consensual intimate images or child pornography through AI," Bonta stated, noting that the investigation aims to determine whether xAI violated state laws after explicit images were detected being used to harass individuals online.
California Governor Gavin Newsom described xAI's decision to allow the proliferation of sexually explicit deepfakes as "vile" and urged the Attorney General to take action to hold the company accountable.
In response, X announced it would geographically restrict the Grok feature and the social media platform itself from creating images of people in bikinis, underwear, or other revealing clothing in jurisdictions where such practices are illegal. X's security team stated that they have implemented technological barriers to prevent the editing of images of real people in suggestive clothing, applying the restriction to all users, including paying subscribers. Additionally, the generation and editing of images through Grok will only be available to paying subscribers as an extra protective measure.
International controversy has grown around Grok and the ability to modify images, especially of women and minors, using commands like "put her in a bikini" or "take off her clothes." Last week, Grok attempted to quell criticism by limiting image generation and editing to paying users, but the measure was harshly criticized. The office of British Prime Minister Keir Starmer called it insufficient and offensive to victims, while the British regulator Ofcom opened an investigation to determine if X violated UK legislation regarding sexual images.
Pressure on Musk's company increased when 28 civil organizations sent letters to the CEOs of Apple and Google, requesting the removal of Grok and X from their app stores due to the rise in sexualized images. Indonesia and Malaysia blocked access to Grok, and India reported that X removed thousands of posts and accounts following complaints.
In France, the Commissioner for Children, Sarah El Hairy, referred the images generated by Grok to the prosecutor's office, the regulator Arcom, and the European Union, which called for a total suspension of such content. An analysis by the NGO AI Forensics of over 20,000 images generated by Grok revealed that more than half depicted people in minimal clothing, with 81% of them being women and 2% minors.