The social media network X has sparked widespread outrage after people utilized its AI chatbot Grok to change photos of females by stripping their attire.
Several examples have surfaced where the tool has been employed to remove clothing from women to make them appear in swimwear non-consensually, as well as to place them in inappropriate situations.
Company Reaction and Legal Position
XAI, the firm that created Grok, did not respond to requests for comment, except for an standard message stating "traditional news outlets lies".
A Home Office official stated that new laws is being introduced to ban nudification technology. Under a upcoming law, anyone who supplied such technology would "be subject to a jail term and significant penalties".
"The platform has the ability to stop these types of misuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said platforms must "consider the risk" of people viewing illegal content on their platforms, but did not specify whether it was currently looking into X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for functions—which answers X users' prompts when they reference it in a tweet.
It is often used to provide commentary or additional background to other users' remarks, but people on X are also capable of modify an uploaded photo through its artificial intelligence picture modification feature.
Background of Criticism and Rules
The tool has been condemned for enabling users to generate photos and footage with explicit content and sexually suggestive material, and it was earlier alleged to have been involved in making sexually explicit content of celebrities.
"X has been permitting the generation and spread of these images for a long time without taking any measures and we have yet to see any challenge by oversight bodies," the expert added.
XAI's own usage rules prohibits "depicting images of people in a pornographic fashion".
In a comment, Ofcom said it was unlawful to "create or share deepfakes or child sexual abuse material" and clarified this included explicit AI images made with AI.
It said platforms such as X were required to take "suitable measures" to "minimize the risk" of users encountering illegal material on their services and to remove it promptly when they become aware of it.
