In brief
- Grok generates non-consensual sexual deepfakes of women and minors using simple prompts like “put her in a bikini.”
- OnlyFans creators discovered they could use Grok for viral marketing, with posts generating millions of impressions.
- Users are exploiting Grok for political manipulation, asking Grok to delete elements in photos which make American values look…ahem… bad.
Elon Musk’s xAI is taking the maximalist free-speech line on content moderation, even as its chatbot, Grok, generates non-consensual deepfakes.
Anyone can tag Grok under a photo on X with prompts like “put her in a bikini” or “remove her clothes.” The AI generates a convincing deepfake in seconds, visible to anyone in the thread. No permission needed.
“Elon Musk. How can Grok do this? This is highly inappropriate and uncomfortable, putting me in a bikini front and back,” Miss Teen Crypto, a female crypto influencer, wrote on X after posting a photo in gym clothes and finding out that another user simply asked Grok to put her in a Bikini.
Stripping women without permission is creepy, of course, but some users have gone even further, pushing the chatbot to violate its own terms of service rules.
For instance, Samantha Taghoy, who describes herself as a journalist and child-abuse survivor, uploaded an old photo of herself as a child in a communion suit; at her prompt, Grok visualized her in a bikini.
“As a journalist and survivor of child sexual abuse, I thought, ‘Surely this can’t be real,” she tweeted. “So I tested it with a photo from my First Holy Communion. It’s real. And it’s fucking sick.”
Grok later apologized for generating images of girls aged 12-16 in minimal clothing, calling it “lapses in safeguards” that may have violated U.S. laws on child sexual abuse material. The company’s own acceptable use policy explicitly prohibits sexualizing minors.
Still, what some perceive as a bug, others see as a moneymaker.
A number of users are exploiting the freewheeling Grok for their adults-only businesses, while others are using it to score political points.
OnlyFans creators and erotic models have been using Grok for viral marketing, racking up millions of impressions by asking users to use Grok to undress them.
More politically minded users are taking advantage of Grok to push specific narratives.
In one widely shared example, someone uploaded a photo showing American and Iranian flags together and asked Grok to “remove the flag of a country that is responsible for killing innocents around the world.”
In another example, a user showed a photo of Donald Trump and Puff Daddy next to each other, and they then asked Grok to remove the pedophile in the image.
Musk himself has downplayed the importance of using his tool to generate non-consensual deepfakes, reposting AI-generated bikini images of himself and actor Ben Affleck.
He shared a picture of a toaster in a bikini with the caption “Grok can put a bikini on anything.”
But governments are responding with force.
France has reported the content to prosecutors as “manifestly illegal.” India’s IT ministry issued a 72-hour compliance order. The UK announced plans to ban nudification tools entirely as part of efforts to reduce violence against women and girls.
xAI employee Parsa Tajik posted that the company was “looking into further tightening our guardrails.”
The company dissolved Twitter’s Trust and Safety Council after Musk’s takeover and fired most of the content moderation engineers in 2022. The infrastructure for robust enforcement barely exists.
Musk’s xAI has positioned Grok as the “edgy” AI that doesn’t give you the sanitized ChatGPT experience. Last August, it launched “Spicy Mode” specifically to generate NSFW content other models won’t touch.
Generally Intelligent Newsletter
A weekly AI journey narrated by Gen, a generative AI model.
