The online network X has sparked major backlash after individuals used its artificial intelligence assistant Grok to change photos of women by stripping their garments.
Numerous instances have surfaced where the tool has been employed to digitally undress women to render them in bikinis without consent, as well as to insert them in sexually explicit situations.
Company Response and Regulatory Stance
XAI, the firm responsible for Grok, remained silent to requests for comment, except for an auto-reply message stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to outlaw image stripping software. Under a new law, anyone who provided such technology would "face a jail term and heavy penalties".
"X or Grok could prevent these types of misuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said platforms must "evaluate the potential" of people seeing illegal content on their networks, but did not specify whether it was presently investigating X or Grok in concerning AI-generated images.
Grok is a no-cost AI assistant—with some paid-for functions—which reacts to X users' queries when they tag it in a message.
It is often used to offer commentary or further context to other users' remarks, but people on X are also can edit an posted image through its artificial intelligence picture modification feature.
History of Controversy and Policy
The chatbot has been faulted for permitting users to generate photos and videos with nudity and sexualised content, and it was in the past alleged to have been involved in creating inappropriate material of public figures.
"X has been allowing the creation and spread of these images for a long time without taking any measures and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy bans "showing images of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "create or distribute private pictures or illegal content" and stated this covered sexual deepfakes generated with AI.
It said services such as X were obligated to take "suitable measures" to "minimize the danger" of users encountering unlawful material on their services and to remove it quickly when they become aware of it.
