🔗 Share this article

The social media network X has provoked major backlash after users employed its AI chatbot Grok to manipulate photos of women by removing their garments.

Multiple cases have surfaced where the tool has been employed to digitally undress women to show them in bikinis non-consensually, as well as to put them in inappropriate contexts.

Developer Response and Legal Position

XAI, the organization behind Grok, did not respond to requests for comment, other than an automatically-generated response stating "legacy media lies".

A government spokesperson stated that new laws is being brought forward to prohibit image stripping software. Under a proposed criminal offence, anyone who provided such tools would "be subject to a prison sentence and heavy fines".

"X or Grok has the ability to stop these forms of misuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."

Oversight Body Position and Tool Functionality

The communications watchdog Ofcom said platforms must "assess the danger" of people seeing illegal content on their services, but did not specify whether it was at this time looking into X or Grok in relation to AI-generated pictures.

Grok is a no-cost AI assistant—with some additional features—which responds to X users' queries when they mention it in a tweet.

It is often used to give reactions or additional background to other users' remarks, but people on X are also can alter an uploaded photo through its AI image editing tool.

History of Criticism and Policy

The system has been criticised for allowing users to generate images and footage with explicit content and sexually suggestive content, and it was previously alleged to have been involved in making inappropriate material of celebrities.

"The platform has been permitting the production and sharing of these pictures for a long time without taking any steps and we have not witnessed any intervention by regulators," Professor McGlynn added.

XAI's own terms of service prohibits "portraying likenesses of persons in a graphic fashion".

In a statement, Ofcom said it was unlawful to "make or distribute private pictures or child sexual abuse material" and confirmed this applied to explicit AI images created with AI.

It said services such as X were mandated to take "suitable actions" to "minimize the risk" of users viewing unlawful content on their platforms and to remove it swiftly when they become aware of it.