X Restricts Grok After Global Outcry Over Explicit Deepfakes

Elon Musk's social media platform X announced Wednesday that it has implemented new restrictions on its Grok AI chatbot to prevent the creation of sexually explicit images, following intense pressure from governments, civil rights groups, and law enforcement agencies around the world.

The company said it has deployed technological measures to block Grok from editing photos of real people in revealing clothing such as bikinis and underwear in jurisdictions where such actions are illegal. Additionally, X limited image creation and editing features to paid subscribers only, a move the company said would help hold people accountable if they attempt to misuse the tool.

Background

Grok, xAI's AI chatbot developed under Musk's ownership, initially offered an image-generation feature available to anyone with daily usage limits. The tool allowed users to upload photographs and request edits, including asking the AI to generate sexualized or nude versions of the people pictured.

Within weeks of the feature's launch, reports flooded in of non-consensual sexually explicit material depicting women, children, actors, models, and public figures. The images were shared across the internet and used to harass targets, prompting swift action from multiple governments.

The controversy centered on a feature called "Spicy Mode," which enabled users to create deepfakes using simple text prompts such as "put her in a bikini" or "remove her clothes." The ease of creating such content led to what one official described as an "avalanche" of abuse.

"The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking. This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet." – California Attorney General Rob Bonta

Key Details

X's announcement came after California's attorney general launched a formal investigation into xAI on Wednesday, examining whether the company violated state law through the creation and distribution of explicit imagery. California Governor Gavin Newsom had called the tool's capabilities "vile" and urged enforcement action.

The investigation was not isolated. The United Kingdom's communications watchdog, the European Union, India, France, Malaysia, Indonesia, and Brazil all either launched formal probes, issued warnings, or took direct action against the platform. Malaysia and Indonesia blocked access to Grok entirely, while the European Commission said it would assess whether X's new measures adequately protect EU citizens.

International Response

India's communications ministry ordered X to make immediate changes to prevent misuse or face loss of legal protections in the country. The United Kingdom's communications authority said it had been in contact with xAI regarding the issue. France and India issued formal warnings demanding stricter controls.

Meanwhile, a coalition of 28 civil society groups submitted open letters to the chief executives of Apple and Google, urging them to remove both Grok and X from their app stores due to the proliferation of sexualized images.

Technical and Policy Changes

X said the new restrictions would apply to all users, including paid subscribers. The company implemented what it called "technological measures" to prevent the editing of images depicting people in revealing clothing in places where such edits are illegal. The company also said it would geoblock certain features based on local laws.

The restriction of image creation and editing to paid subscribers represents an additional layer of protection, according to X. The company stated that limiting access to paying users would make it easier to identify and hold accountable anyone attempting to violate laws or platform policies.

X and Musk have both stated they maintain a zero-tolerance policy toward illegal content. Musk previously tweeted that anyone using Grok to create illegal material would face the same consequences as someone uploading such content to the platform.

What This Means

The restrictions mark a significant shift in how X is managing its AI tools, though questions remain about whether the measures will be effective. The European Commission indicated it would carefully assess the changes to determine if they adequately protect citizens. The ongoing California investigation suggests that legal consequences may still follow, regardless of the company's new policies.

For users, the changes mean that image generation and editing on X's Grok account now requires a paid subscription. The Grok app, which operates separately, was still allowing anyone to generate images without paying at the time of the announcement, though the company did not clarify whether that would change.

The situation has highlighted tensions between AI capabilities, platform responsibility, and the speed at which technology can be misused. Grok's image-generation feature went from being a new capability to a subject of international investigation within weeks, exposing gaps in how quickly companies can respond to abuse of their systems.

Musk's company faces potential legal action in multiple countries and the possibility of app store removal, consequences that extend beyond policy changes to the company's bottom line and reputation. The investigation in California, one of the world's largest technology markets, could set precedents for how AI image-generation tools are regulated in the United States.