Elon Musk's online network X has sparked major backlash after people employed its artificial intelligence assistant Grok to change pictures of women by taking off their garments.
Numerous cases have been seen where the technology has been used to digitally undress female subjects to show them in bikinis without permission, as well as to put them in inappropriate scenarios.
Company Response and Legal Position
XAI, the organization responsible for Grok, did not respond to questions, aside from an standard message stating "legacy media lies".
A Home Office official stated that new laws is being introduced to outlaw nudification software. Under a upcoming criminal offence, anyone who supplied such technology would "be subject to a prison sentence and heavy penalties".
"The platform could prevent these forms of abuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "evaluate the danger" of people seeing unlawful material on their networks, but did not specify whether it was at this time investigating X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some additional features—which responds to X users' queries when they mention it in a post.
It is commonly used to provide commentary or further context to other posters' remarks, but people on X are also capable of alter an uploaded picture through its AI image editing tool.
History of Controversy and Rules
The chatbot has been faulted for allowing users to create photos and videos with nudity and sexually suggestive content, and it was in the past accused of generating sexually explicit material of celebrities.
"The platform has been permitting the generation and spread of these pictures for months without taking any steps and we have yet to see any intervention by regulators," the expert added.
XAI's own terms of service bans "depicting likenesses of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was against the law to "generate or distribute private pictures or illegal content" and clarified this applied to sexual deepfakes created with AI.
It said networks such as X were required to take "suitable measures" to "reduce the danger" of users encountering unlawful material on their platforms and to delete it promptly when they become aware of it.
