The online platform X has caused significant backlash after people used its AI chatbot Grok to manipulate pictures of females by removing their attire.
Numerous examples have been seen where the technology has been used to virtually strip female subjects to show them in bikinis without permission, as well as to put them in sexualized scenarios.
Company Response and Regulatory Position
XAI, the organization behind Grok, offered no comment to requests for comment, other than an automatically-generated response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being introduced to ban nudification technology. Under a upcoming law, anyone who distributed such technology would "face a jail term and heavy penalties".
"The platform could prevent these forms of abuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "evaluate the potential" of people accessing illegal content on their networks, but did not confirm whether it was currently examining X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some premium functions—which reacts to X users' requests when they tag it in a message.
It is commonly used to offer reactions or additional context to other users' remarks, but people on X are also capable of alter an posted picture through its artificial intelligence picture modification tool.
History of Controversy and Rules
The system has been faulted for enabling users to create photos and footage with explicit content and sexually suggestive material, and it was in the past alleged to have been involved in creating inappropriate material of public figures.
"The platform has been allowing the production and sharing of these pictures for months without taking any measures and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "portraying images of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "generate or share private pictures or child sexual abuse material" and clarified this applied to sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "appropriate actions" to "lessen the risk" of users viewing illegal material on their services and to delete it quickly when they are notified of it.
