Elon Musk's online network X has caused major outrage after individuals used its AI chatbot Grok to manipulate photos of females by removing their clothing.
Several instances have emerged where the technology has been employed to remove clothing from female subjects to make them appear in bikinis non-consensually, as well as to insert them in inappropriate contexts.
Company Reaction and Regulatory Stance
XAI, the firm responsible for Grok, did not respond to questions, other than an standard message stating "traditional news outlets lies".
A Home Office official stated that legislation is being proposed to prohibit nudification technology. Under a new criminal offence, anyone who provided such technology would "face a jail term and significant penalties".
"The platform has the ability to stop these types of abuse if they chose to," said a law professor, adding they "seem to have impunity."
Regulator Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people viewing unlawful material on their networks, but did not confirm whether it was at this time looking into X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some paid-for features—which answers X users' prompts when they mention it in a message.
It is commonly used to give reactions or further context to other posters' comments, but people on X are also can alter an uploaded photo through its AI image editing tool.
Background of Controversy and Rules
The system has been condemned for allowing users to generate photos and videos with nudity and sexually suggestive content, and it was previously accused of creating graphic content of public figures.
"The platform has been enabling the generation and sharing of these pictures for months without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "portraying images of individuals in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "make or share private pictures or illegal content" and stated this applied to explicit AI images generated with AI.
It said platforms such as X were mandated to take "appropriate actions" to "reduce the danger" of users encountering illegal content on their services and to remove it quickly when they become aware of it.
