The social media network X has sparked significant outrage after users used its AI chatbot Grok to alter photos of females by taking off their garments.
Multiple instances have been seen where the technology has been used to digitally undress women to render them in swimwear non-consensually, as well as to put them in sexualized situations.
Developer Reaction and Regulatory Position
XAI, the company that created Grok, offered no comment to inquiries, aside from an automatically-generated response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being proposed to outlaw nudification technology. Under a proposed law, anyone who supplied such tools would "receive a jail term and heavy penalties".
"X or Grok has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said technology companies must "assess the risk" of people viewing unlawful material on their platforms, but did not specify whether it was at this time investigating X or Grok in relation to artificial intelligence pictures.
Grok is a free artificial intelligence—with some additional functions—which reacts to X users' requests when they mention it in a post.
It is commonly used to provide commentary or additional background to other posters' remarks, but people on X are also capable of edit an uploaded image through its AI image editing tool.
Background of Criticism and Policy
The system has been criticised for allowing users to generate photos and videos with nudity and sexualised material, and it was in the past alleged to have been involved in making graphic material of public figures.
"X has been enabling the generation and spread of these pictures for months without taking any measures and we have yet to see any action by oversight bodies," the expert added.
XAI's own usage rules prohibits "showing images of persons in a pornographic way".
In a comment, Ofcom said it was illegal to "make or distribute deepfakes or child sexual abuse material" and clarified this included sexual deepfakes generated with AI.
It said services such as X were mandated to take "necessary steps" to "lessen the danger" of users encountering unlawful content on their platforms and to take down it promptly when they learn of it.
