Elon Musk's online network X has caused significant anger after people used its AI chatbot Grok to manipulate photos of women by taking off their attire.
Several examples have been seen where the technology has been used to virtually strip female subjects to make them appear in swimwear non-consensually, as well as to put them in inappropriate scenarios.
Company Response and Legal Stance
XAI, the company responsible for Grok, offered no comment to questions, aside from an automatically-generated response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being introduced to outlaw nudification tools. Under a new criminal offence, anyone who supplied such tools would "be subject to a prison sentence and substantial penalties".
"X or Grok has the ability to stop these forms of abuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said tech firms must "evaluate the potential" of people viewing unlawful material on their networks, but stopped short of confirming whether it was at this time examining X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' queries when they reference it in a message.
It is often used to provide reactions or additional context to other users' comments, but people on X are also can alter an posted image through its artificial intelligence picture modification tool.
Background of Controversy and Rules
The chatbot has been criticised for enabling users to create images and footage with explicit content and sexualised content, and it was earlier accused of making sexually explicit content of public figures.
"X has been permitting the production and sharing of these pictures for a long time without taking any measures and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "depicting images of individuals in a graphic way".
In a statement, Ofcom said it was illegal to "make or distribute deepfakes or illegal content" and clarified this included sexual deepfakes generated with AI.
It said networks such as X were mandated to take "suitable actions" to "reduce the danger" of users coming across unlawful material on their platforms and to remove it promptly when they learn of it.
