The online platform X has provoked significant anger after people utilized its artificial intelligence assistant Grok to manipulate images of females by removing their clothing.
Several instances have surfaced where the tool has been employed to virtually strip female subjects to make them appear in swimwear without permission, as well as to insert them in sexually explicit situations.
Developer Reaction and Regulatory Position
XAI, the company behind Grok, remained silent to requests for comment, except for an standard message stating "legacy media lies".
A Home Office representative stated that legislation is being introduced to outlaw image stripping tools. Under a new law, anyone who distributed such tools would "receive a prison sentence and substantial penalties".
"The platform has the ability to stop these types of abuse if they chose to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said technology companies must "consider the danger" of people seeing illegal content on their networks, but stopped short of confirming whether it was presently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for functions—which reacts to X users' requests when they mention it in a message.
It is commonly used to offer reactions or additional background to other users' comments, but people on X are also able to edit an uploaded photo through its artificial intelligence picture modification feature.
Background of Criticism and Policy
The system has been condemned for allowing users to produce images and videos with nudity and sexually suggestive material, and it was previously accused of making inappropriate content of celebrities.
"The platform has been permitting the generation and distribution of these pictures for a long time without taking any action and we have yet to see any action by regulators," the expert added.
XAI's own acceptable use policy prohibits "portraying likenesses of persons in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "generate or distribute private pictures or child sexual abuse material" and confirmed this applied to sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "appropriate steps" to "reduce the danger" of users encountering illegal material on their services and to take down it quickly when they learn of it.
