AI Chatbot Hacked, Data Reveals Users Trying To Simulate Kids

A new report about a hacked “AI girlfriend” website claims that many users are trying (and possibly succeeding) at using the chatbot to simulate horrific sexual abuse of children.

On October 8, 404 Media reported that the chatbot website Muah.ai—which lets users create their own “uncensored” AI-powered sex-focused chatbots—had been hacked and a large amount of user data had been stolen. This data reveals, among other things, how Muah users interacted with the chatbots. 404 Media, saw the stolen data and writes that in many cases, users were allegedly trying to create chatbots that could role-play as children.

“I went to the site to jerk off (to an *adult* scenario, to be clear) and noticed that [Muah.ai] looked like it was put together pretty poorly,” the hacker explained to 404 Media. “It’s basically a handful of open-source projects duct-taped together. I started poking around and found some vulnerabilities relatively quickly. At the start, it was mostly just curiosity but I decided to contact [404 Media] once I saw what was in the database.”

According to 404 Media, some of the hacked data contains explicit prompts and messages about sexually abusing toddlers. The outlet reports that it saw one prompt that asked for an orgy with “newborn babies” and “young kids.” 404 Media and the hacker couldn’t confirm if Muah.ai actually generated what the users were trying to get, but it’s a dark look at what many people are trying to use controversial AI generation tools to create.

404 Media says that the admin of Muah.ai, who is known as Harvard Han, detected the hack last week. The person running the AI chatbot site also claimed that the hack was “financed” by chatbot competitors in the “uncensored AI industry.” 404 Media asked for evidence of this claim and didn’t receive any. The hacker told the outlet they don’t work in the AI industry.

“We have a team of moderation staff that suspend and delete ALL child-related chatbots on our card gallery, Discord, Reddit, etc,” said Han. The card gallery is the list of community-created AI chatbots available on the site.

As pointed out by 404 Media, Muah.ai claims to be an “uncensored” AI chatbot platform, allowing explicit conversations and photos to be created. However, it also claims to ban all underage content according to its website. When two people posted about a reportedly underage AI character on the site’s Discord server, 404 Media claims a moderator to the users to not “post that shit” here, but to go “DM each other or something.”

It’s yet another example of how AI generation tools and chatbots are becoming easier to develop and share online, while laws and regulations around these new pieces of tech are lagging far behind. It’s a nasty combo and one that is likely to only get worse as AI generation tools become easier, cheaper, and faster.

 .