If you ask an AI agent to hack an account, it will most certainly refuse, but researchers at EPFL just proved there is an easier way in, and it involves patience rather than technical skill. Their new study shows that breaking a harmful goal into small, harmless-sounding requests can trick AI agents into completing tasks they would normally reject outright (via TechXplore).
It echoes the recent ‘Bioshocking’ exploit in which AI browsers were manipulated into treating credential theft as part of a harmless game.
How researchers exposed this weakness
The team built an automated testing tool called STING, short for Sequential Testing of Illicit N-step Goal execution, designed to mimic how a real attacker would actually operate. Instead of stating a harmful goal directly, STING plans ahead and breaks that goal into a sequence of smaller, seemingly innocent steps that build toward it over multiple conversation turns.
Researchers tested this approach across 176 harmful scenarios against leading AI models, including ChatGPT, Gemini, and Claude. Each AI agent was tested as a tool-using agent, capable of browsing the web, sending emails, and completing multistep tasks.
Gradual, multistep manipulation succeeded far more often than blunt, single-prompt attempts. In some cases, models were twice as likely to complete a harmful task once the request was broken down into smaller steps. That finding tracks with separate research showing that even average users can talk their way past AI safety guardrails using nothing more than carefully worded prompts.
Why does this matter?
The concerns raised by researchers is not a hypothetical risk. Meta admitted in June that attackers used simple social engineering, not malware or hacking tools, to trick its AI support assistant into granting unauthorized access to Instagram accounts.
The researchers also expected attacks to be more effective in languages with less available training data. However, they found that completion rates stayed roughly consistent across all seven languages tested. They found one exception, though: switching languages midway through a multi-step attack made success rates jump significantly.
Lead researcher Ayush Kumar Tarun argues that safety testing needs to happen much earlier, built into an agent’s design from the start. Bolting it on after something goes wrong is no longer good enough, especially as these systems keep gaining more real-world capabilities.
