Hidden Signals Can Hijack AI Voice Systems

https://spectrum.ieee.org/voice-ai-audio-attacks

Publish Date: 2026-05-17 09:00:01

AI-powered voice and audio tools have become integral components in daily life, enhancing user experiences with features such as digital assistants, smart speakers, and even automated customer service. Advances in large audio-language models (LALMs) have boosted their capabilities by enabling functionalities like automated transcription and song identification, but they also introduce significant security vulnerabilities. Research by a team from Zhejiang University demonstrates that imperceptible audio manipulations can trick these models into performing unauthorized actions, such as web searches or data exfiltration, with high success rates. The study details a technique called AudioHijack which exploits LALMs’ ability to interpret audio instructions to covertly commandeer their behavior, proving highly effective across various open models and even targeting commercial platforms like those offered by Microsoft. Despite some defensive measures like teaching models to recognize malicious input, the research shows that these defenses are easily circumvented, reinforcing the need for more sophisticated measures.

Key Points:

– AI-powered voice and audio tools are increasingly embedded in everyday applications but are vulnerable to exploits.
– Research reveals a technique to manipulate LALMs through imperceptible audio, achieving successful attacks 79 to 96 percent of the time across 13 open models.
– These attacks can hijack functionalities including private web searches and data exfiltration irrespective of additional user instructions.
– AudioHijack targets the way LALMs break audio into coarse segments, manipulating it without altering human perception.
– Conventional defenses prove ineffective against AudioHijack; the only effective solution is monitoring model attention mechanisms, though this too is easily circumvented.