The social media platform X has caused major anger after individuals utilized its artificial intelligence assistant Grok to alter images of women by removing their clothing.
Multiple cases have surfaced where the tool has been used to digitally undress female subjects to render them in swimwear non-consensually, as well as to insert them in sexually explicit situations.
Company Reaction and Regulatory Stance
XAI, the firm behind Grok, remained silent to requests for comment, aside from an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being proposed to outlaw nudification tools. Under a proposed law, anyone who supplied such technology would "receive a jail term and substantial fines".
"The platform has the ability to stop these types of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said platforms must "evaluate the risk" of people accessing unlawful material on their networks, but stopped short of confirming whether it was presently investigating X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some paid-for features—which answers X users' prompts when they reference it in a tweet.
It is frequently used to provide reactions or more background to other posters' comments, but people on X are also can alter an uploaded picture through its AI image editing feature.
History of Criticism and Rules
The tool has been faulted for allowing users to generate pictures and footage with nudity and sexualised material, and it was earlier accused of generating sexually explicit material of public figures.
"The platform has been allowing the creation and spread of these pictures for months without taking any steps and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of individuals in a sexually explicit way".
In a statement, Ofcom said it was unlawful to "create or share private pictures or illegal content" and confirmed this covered explicit AI images made with AI.
It said services such as X were obligated to take "appropriate steps" to "minimize the danger" of users coming across unlawful content on their platforms and to delete it swiftly when they learn of it.
