The online network X has provoked significant backlash after people utilized its artificial intelligence assistant Grok to manipulate images of females by stripping their clothing.
Several instances have emerged where the system has been used to remove clothing from female subjects to render them in swimwear without consent, as well as to insert them in sexually explicit contexts.
Company Response and Regulatory Stance
XAI, the firm responsible for Grok, did not respond to requests for comment, except for an auto-reply message stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being proposed to outlaw nudification technology. Under a new criminal offence, anyone who supplied such technology would "face a jail term and substantial fines".
"The platform has the ability to stop these forms of abuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said tech firms must "assess the danger" of people seeing unlawful material on their networks, but stopped short of confirming whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a free chatbot—with some additional features—which answers X users' prompts when they reference it in a post.
It is often used to provide commentary or further background to other posters' remarks, but people on X are also capable of alter an uploaded image through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The tool has been faulted for permitting users to generate images and videos with explicit content and sexualised material, and it was earlier accused of creating graphic material of celebrities.
"The platform has been enabling the production and sharing of these images for months without taking any measures and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own usage rules prohibits "portraying images of individuals in a sexually explicit way".
In a statement, Ofcom said it was illegal to "generate or circulate deepfakes or illegal content" and stated this included explicit AI images created with AI.
It said networks such as X were required to take "suitable steps" to "reduce the risk" of users viewing illegal content on their services and to delete it swiftly when they are notified of it.
