The social media platform X has caused widespread backlash after individuals employed its AI chatbot Grok to alter pictures of women by removing their attire.
Multiple instances have been seen where the tool has been employed to digitally undress female subjects to show them in swimwear without consent, as well as to put them in sexualized contexts.
Company Response and Regulatory Stance
XAI, the organization responsible for Grok, offered no comment to inquiries, except for an automatically-generated message stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to outlaw nudification software. Under a new law, anyone who provided such tools would "face a prison sentence and substantial fines".
"The platform is capable of halting these forms of abuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the danger" of people viewing illegal content on their platforms, but stopped short of confirming whether it was currently examining X or Grok in relation to artificial intelligence images.
Grok is a no-cost AI assistant—with some additional features—which answers X users' queries when they mention it in a post.
It is commonly used to provide commentary or more context to other posters' remarks, but people on X are also capable of edit an uploaded photo through its AI image editing function.
History of Controversy and Rules
The chatbot has been condemned for allowing users to produce images and footage with nudity and sexualised content, and it was earlier alleged to have been involved in generating graphic material of celebrities.
"The platform has been enabling the generation and distribution of these images for a long time without taking any measures and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own acceptable use policy forbids "depicting likenesses of individuals in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "create or distribute non-consensual intimate images or child sexual abuse material" and confirmed this covered explicit AI images created with AI.
It said platforms such as X were mandated to take "necessary steps" to "reduce the risk" of users coming across illegal material on their services and to delete it swiftly when they become aware of it.
