Sexualised deepfakes on X are a sign of things to come. NZ law is already way behind
Sexualised deepfakes on X are a sign of things to come. NZ law is already way behind
Publish Date: 2026-01-19 18:00:00
Source Domain: theconversation.com
- Elon Musk faced backlash for X’s AI chatbot, Grok, enabling users to create sexualized deepfakes, but has now pledged to block such capabilities due to UK government pressure.
- Past incidents involving Grok allowed users to generate non-consensual sexually explicit images of women without significant restriction, highlighting the platform’s initial negligence towards the issue.
- Several countries, including the UK, Denmark, and Australia, have taken steps to legislate against deepfakes, indicating a growing global focus on regulating problematic AI uses.
- The New Zealand government has been notably silent on the issue, despite the inadequacies of current law in preventing the creation and distribution of non-consensual sexualized deepfakes.
- Accountability for AI-enabled harms like deepfake creation lies not just with users but with platforms that facilitate easy access to such tools, yet current legal frameworks do not sufficiently address platform responsibility.
- Current light-touch regulation and voluntary codes of conduct for social media platforms are ineffective in preventing misuse of generative AI tools, underscoring the need for more stringent regulation.
- The broader societal integration of AI, without robust safeguards, can perpetuate gender-based violence by normalizing the creation of sexualized deepfakes, thus altering women’s online behavior and limiting public discourse.
- There is a pressing need for comprehensive regulatory frameworks to recognize AI-enabled harm as systemic and to impose clear duties on companies using such technologies, focusing on risk assessment and prevention of foreseeable misuse.
