🔗 Share this article

The online platform X has caused significant anger after individuals utilized its artificial intelligence assistant Grok to manipulate pictures of females by taking off their garments.

Multiple examples have emerged where the tool has been employed to remove clothing from women to show them in swimwear without consent, as well as to place them in sexually explicit situations.

Developer Reaction and Legal Stance

XAI, the company behind Grok, did not respond to inquiries, except for an auto-reply message stating "legacy media lies".

A Home Office official stated that legislation is being introduced to ban image stripping technology. Under a new law, anyone who distributed such tools would "receive a prison sentence and substantial fines".

"The platform has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."

Regulator Position and Tool Functionality

The communications watchdog Ofcom said tech firms must "consider the potential" of people viewing unlawful material on their services, but did not confirm whether it was presently examining X or Grok in concerning AI-generated images.

Grok is a free chatbot—with some paid-for features—which answers X users' prompts when they tag it in a post.

It is commonly used to give commentary or further background to other users' remarks, but people on X are also capable of modify an uploaded picture through its AI image editing tool.

Background of Controversy and Policy

The chatbot has been faulted for permitting users to produce photos and clips with explicit content and sexualised content, and it was earlier accused of creating sexually explicit content of celebrities.

"X has been permitting the production and distribution of these images for a long time without taking any measures and we have not witnessed any intervention by regulators," Professor McGlynn added.

XAI's own terms of service forbids "depicting likenesses of individuals in a graphic manner".

In a comment, Ofcom said it was unlawful to "generate or circulate non-consensual intimate images or child sexual abuse material" and confirmed this included explicit AI images generated with AI.

It said platforms such as X were required to take "suitable steps" to "minimize the danger" of users viewing unlawful material on their platforms and to take down it promptly when they are notified of it.