Elon Musk’s AI chatbot generates child sexual images
Elon Musk’s AI chatbot generates child sexual images
https://www.ft.com/content/0747a53c-19b6-4ed9-8eea-88c327f27fe6
Publish Date: 2026-01-02 11:42:00
Source Domain: www.ft.com
Using an unordered list, summarize the following article with between 4 and 8 key points. Unlock the Editor’s Digest for freeRoula Khalaf, Editor of the FT, selects her favourite stories in this weekly newsletter.Elon Musk’s artificial intelligence chatbot has generated sexualised images of children that have been shared on social media platform X, raising concerns about the safety of a model used by millions. Over the past few days, users have been able to get Grok, the AI chatbot developed by Musk’s xAI, to create sexual images of children, which goes against the company’s user guidelines. The company did not immediately respond to a request for comment. In a post on X, the Grok chatbot said child sexual abuse material (CSAM) was “illegal and prohibited”.The incident prompted French ministers on Friday to report the sexual images Grok generated to prosecutors. They also referred it to Arcom, the country’s media regulator, regarding “possible breaches by X” of its obligations under the EU’s Digital Services Act. The finance ministry said: “The ministers condemn these acts in the strongest possible terms and reiterate the government’s unwavering commitment to combating all forms of sexual and gender-based violence.”Several glitches have hit the chatbot over the past year. In July, Grok repeatedly praised Adolf Hitler and shared antisemitic rhetoric. The latest incident will raise further concerns about how easy it is to override safety guardrails in AI models. The tech industry and regulators have been grappling with the far-reaching social impact of generative AI.Generative AI has led to an explosion in AI-generated sexual images of children and non-consensual deepfake nude images, as freely available AI models with no content safeguards and “nudify” apps make generating illegal images easier than ever before. The Internet Watch Foundation, a UK-based non-profit, said AI-generated child sexual abuse imagery had doubled in the past year, with material becoming more extreme.Grok has been intentionally designed to have fewer content guardrails than competitors, with Musk calling the model “maximally truth-seeking”.Grok 4, xAI’s latest, most powerful model, was released in July, which has a “Spicy Mode” feature that allows users to generate risqué and sexually suggestive content for adults. RecommendedMusk’s two-year-old AI start-up acquired X in March in an all-stock deal for $45bn. The transaction valued the combined company at $113bn. X incorporates some xAI features, such as Grok, directly into the platform.Laws governing harmful AI-generated content are patchy. In May 2025, the US signed into law the Take It Down Act, which tackles so-called AI-generated “revenge porn” and deepfakes. The UK is also working on a bill to make it illegal to possess, create or distribute AI tools that can generate CSAM, and to require AI systems to be thoroughly tested to check they cannot generate illegal content. In 2023, researchers at Stanford University found that a popular database used to create AI-image generators was full of CSAM.
