The online network X has caused significant anger after individuals used its AI chatbot Grok to alter photos of women by taking off their attire.
Multiple instances have emerged where the system has been used to remove clothing from female subjects to show them in bikinis without consent, as well as to place them in sexually explicit contexts.
Company Response and Regulatory Position
XAI, the company that created Grok, did not respond to inquiries, other than an automatically-generated message stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being proposed to outlaw image stripping software. Under a upcoming law, anyone who provided such tools would "face a jail term and substantial penalties".
"The platform has the ability to stop these types of misuse if they wanted to," said a law professor, adding they "operate with impunity."
Oversight Body Stance and Tool Capabilities
The communications watchdog Ofcom said tech firms must "consider the risk" of people accessing illegal content on their services, but did not confirm whether it was presently investigating X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some additional functions—which answers X users' requests when they mention it in a message.
It is often used to give commentary or further context to other posters' remarks, but people on X are also able to modify an posted image through its artificial intelligence picture modification tool.
Background of Criticism and Rules
The tool has been faulted for permitting users to create photos and clips with explicit content and sexualised material, and it was earlier alleged to have been involved in creating sexually explicit content of celebrities.
"X has been permitting the generation and spread of these pictures for months without taking any action and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own usage rules forbids "depicting likenesses of people in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "generate or circulate private pictures or illegal content" and stated this included explicit AI images generated with AI.
It said networks such as X were obligated to take "necessary actions" to "minimize the danger" of users viewing illegal content on their platforms and to take down it quickly when they learn of it.
