🔗 Share this article

The social media platform X has sparked significant outrage after users utilized its AI chatbot Grok to manipulate photos of females by removing their attire.

Numerous cases have surfaced where the system has been used to digitally undress women to show them in swimwear non-consensually, as well as to insert them in sexually explicit scenarios.

Company Reaction and Legal Stance

XAI, the company that created Grok, remained silent to inquiries, aside from an standard response stating "legacy media lies".

A Home Office representative stated that new laws is being introduced to prohibit nudification software. Under a new criminal offence, anyone who supplied such tools would "be subject to a prison sentence and significant penalties".

"X or Grok is capable of halting these forms of abuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."

Oversight Body Stance and Tool Capabilities

The communications watchdog Ofcom said technology companies must "consider the risk" of people accessing illegal content on their networks, but did not specify whether it was presently investigating X or Grok in regarding artificial intelligence pictures.

Grok is a free artificial intelligence—with some additional functions—which answers X users' queries when they tag it in a post.

It is commonly used to provide reactions or more context to other users' remarks, but people on X are also capable of alter an uploaded photo through its AI image editing tool.

History of Controversy and Policy

The chatbot has been faulted for permitting users to create photos and footage with explicit content and sexualised material, and it was previously alleged to have been involved in generating inappropriate material of public figures.

"The platform has been enabling the creation and distribution of these images for a long time without taking any action and we have not seen any action by oversight bodies," the expert added.

XAI's own usage rules bans "portraying images of individuals in a sexually explicit fashion".

In a comment, Ofcom said it was against the law to "generate or share non-consensual intimate images or illegal content" and confirmed this covered sexual deepfakes created with AI.

It said services such as X were obligated to take "appropriate steps" to "reduce the risk" of users viewing illegal material on their platforms and to delete it quickly when they learn of it.