Elon Musk's social media platform X has sparked significant anger after individuals employed its AI chatbot Grok to change images of females by removing their attire.
Numerous instances have been seen where the system has been used to virtually strip women to show them in swimwear without permission, as well as to place them in sexually explicit contexts.
Company Response and Regulatory Position
XAI, the company that created Grok, remained silent to requests for comment, aside from an standard response stating "legacy media lies".
A Home Office official stated that legislation is being introduced to ban nudification technology. Under a proposed criminal offence, anyone who provided such technology would "face a prison sentence and significant penalties".
"X or Grok is capable of halting these types of misuse if they desired to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the danger" of people accessing illegal content on their platforms, but stopped short of confirming whether it was at this time examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for features—which reacts to X users' requests when they mention it in a message.
It is frequently used to offer reactions or more background to other posters' remarks, but people on X are also able to alter an uploaded image through its artificial intelligence picture modification feature.
History of Controversy and Rules
The system has been condemned for allowing users to produce photos and videos with nudity and sexualised content, and it was in the past alleged to have been involved in making inappropriate material of public figures.
"X has been allowing the production and spread of these images for a long time without taking any measures and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "showing likenesses of individuals in a graphic fashion".
In a statement, Ofcom said it was unlawful to "make or share private pictures or illegal content" and stated this covered sexual deepfakes made with AI.
It said platforms such as X were mandated to take "necessary actions" to "reduce the danger" of users viewing illegal content on their services and to take down it swiftly when they become aware of it.
