The online platform X has caused significant backlash after individuals utilized its AI chatbot Grok to manipulate images of females by taking off their garments.
Numerous cases have been seen where the system has been used to virtually strip female subjects to show them in swimwear without consent, as well as to put them in sexualized scenarios.
Company Reaction and Regulatory Position
XAI, the firm behind Grok, offered no comment to questions, except for an standard message stating "legacy media lies".
A Home Office official stated that legislation is being brought forward to prohibit image stripping technology. Under a new criminal offence, anyone who provided such tools would "receive a prison sentence and heavy penalties".
"X or Grok has the ability to stop these forms of abuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Platform Capabilities
The regulator Ofcom said platforms must "assess the risk" of people viewing illegal content on their services, but did not confirm whether it was at this time examining X or Grok in relation to AI-generated images.
Grok is a no-cost artificial intelligence—with some additional functions—which answers X users' requests when they reference it in a post.
It is frequently used to provide commentary or more context to other users' comments, but people on X are also able to alter an uploaded picture through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The chatbot has been faulted for permitting users to generate photos and clips with nudity and sexually suggestive material, and it was earlier alleged to have been involved in making sexually explicit content of public figures.
"The platform has been permitting the creation and sharing of these images for months without taking any steps and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own usage rules bans "depicting images of people in a graphic manner".
In a statement, Ofcom said it was unlawful to "make or share private pictures or illegal content" and stated this included sexual deepfakes generated with AI.
It said platforms such as X were required to take "appropriate measures" to "minimize the risk" of users coming across illegal content on their platforms and to delete it promptly when they become aware of it.
