The social media platform X has caused significant anger after individuals used its artificial intelligence assistant Grok to manipulate photos of females by stripping their garments.
Several examples have emerged where the technology has been used to remove clothing from women to show them in swimwear without permission, as well as to insert them in sexualized scenarios.
Company Reaction and Regulatory Position
XAI, the company responsible for Grok, remained silent to inquiries, other than an standard response stating "traditional news outlets lies".
A government representative stated that legislation is being introduced to outlaw nudification software. Under a upcoming law, anyone who supplied such tools would "be subject to a jail term and significant penalties".
"The platform has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said platforms must "assess the danger" of people seeing illegal content on their platforms, but did not specify whether it was currently examining X or Grok in regarding AI-generated images.
Grok is a no-cost AI assistant—with some additional functions—which answers X users' prompts when they mention it in a message.
It is commonly used to provide commentary or further context to other posters' comments, but people on X are also able to edit an posted photo through its AI image editing tool.
History of Criticism and Policy
The chatbot has been criticised for enabling users to generate photos and footage with nudity and sexually suggestive material, and it was previously alleged to have been involved in making sexually explicit content of public figures.
"X has been allowing the production and spread of these pictures for a long time without taking any measures and we have yet to see any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing likenesses of individuals in a pornographic fashion".
In a comment, Ofcom said it was against the law to "generate or distribute deepfakes or child sexual abuse material" and clarified this covered explicit AI images created with AI.
It said platforms such as X were required to take "appropriate actions" to "reduce the risk" of users viewing unlawful material on their services and to remove it quickly when they become aware of it.
