The social media network X has caused widespread backlash after people used its AI chatbot Grok to change photos of females by removing their garments.
Multiple examples have been seen where the technology has been used to digitally undress female subjects to show them in bikinis non-consensually, as well as to insert them in sexualized situations.
Company Response and Legal Stance
XAI, the company that created Grok, offered no comment to requests for comment, except for an standard response stating "traditional news outlets lies".
A Home Office official stated that legislation is being introduced to prohibit nudification software. Under a new law, anyone who provided such technology would "receive a prison sentence and substantial fines".
"The platform could prevent these types of misuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "consider the danger" of people viewing unlawful material on their networks, but stopped short of confirming whether it was presently investigating X or Grok in relation to AI-generated images.
Grok is a free chatbot—with some paid-for functions—which answers X users' prompts when they mention it in a post.
It is frequently used to offer commentary or more background to other posters' remarks, but people on X are also can alter an uploaded photo through its AI image editing tool.
Background of Controversy and Policy
The system has been faulted for allowing users to produce pictures and clips with explicit content and sexually suggestive content, and it was earlier accused of making graphic material of public figures.
"The platform has been enabling the generation and distribution of these pictures for months without taking any steps and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying images of people in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "make or circulate non-consensual intimate images or illegal content" and stated this applied to explicit AI images made with AI.
It said services such as X were mandated to take "necessary measures" to "reduce the risk" of users encountering unlawful material on their platforms and to remove it swiftly when they learn of it.
