AI Models Overthink Problems-and It's a Security Risk
Large language models (LLMs) that can think through problems step-by-step have significantly increased the scope of tasks that AI can tackle. But new research suggests these reasoning capabilities also introduce a critical vulnerability that could allow attackers to slow these systems to a crawl. While earlier generations of LLMs would immediately produce a response to a user's request, today's most advanced models generate an internal monologue where they break down the problem into steps and reason about the best way to tackle it before providing an answer.
Key Takeaways
- This has allowed AI to tackle increasingly complex problems, particularly in areas like coding and math .
However, previous research has shown that these models are susceptible to sometimes producing excessively long streams of reasoning that do little to boost performance, a phenomenon known as "overthinking.
- Evolutionary Prompt Attack on LLMs The team has developed an evolutionary algorithm that corrupts the logical structure of prompts, causing models to spiral into overthinking as they attempt to reason through fundamentally unsolvable problems.
Generating longer responses costs more and increases the load on a model provider's servers, so if done at scale, the researchers say, this could significantly degrade the experience of legitimate users.
- "Across multiple datasets and reasoning models, our method substantially amplifies the output length," Wei Cao, a masters student at Zhejiang University, wrote in an email to IEEE Spectrum .
"Our results suggest that overthinking is not an isolated phenomenon specific to individual models, but rather a shared vulnerability among modern reasoning models.
- Taking the idea a step further, the authors took 940 problems from three math benchmark datasets and used an LLM to break down their logical structure into a set of premises and a final question.
The genetic algorithm then jumbled these up using a variety of "mutations," including swapping premises between problems, adding extra premises to problems, deleting existing premises from problems, and swapping the final questions between two sets of premises.
- Crucially, the approach doesn't require access to the internals of a model and can generate malicious prompts by simply querying the target, which makes it possible to attack closed-source commercial services, says Cao.
Stats & Key Facts
- #" In research presented this week at the International Conference on Machine Learning 2026 in Seoul, researchers from Zhejiang University and e-commerce giant Alibaba in China demonstrate that they can deliberately induce overthinking by subjecting models to logically inconsistent prompts.
- #5 Flash and resulted in outputs up to 26 times as long as standard responses on a standard math benchmark.
- #Taking the idea a step further, the authors took 940 problems from three math benchmark datasets and used an LLM to break down their logical structure into a set of premises and a final question.
This has allowed AI to tackle increasingly complex problems, particularly in areas like coding and math . However, previous research has shown that these models are susceptible to sometimes producing excessively long streams of reasoning that do little to boost performance, a phenomenon known as "overthinking. " In research presented this week at the International Conference on Machine Learning 2026 in Seoul, researchers from Zhejiang University and e-commerce giant Alibaba in China demonstrate that they can deliberately induce overthinking by subjecting models to logically inconsistent prompts.
The result is a form of denial-of-service attack on commercial AI models. Evolutionary Prompt Attack on LLMs The team has developed an evolutionary algorithm that corrupts the logical structure of prompts, causing models to spiral into overthinking as they attempt to reason through fundamentally unsolvable problems. Generating longer responses costs more and increases the load on a model provider's servers, so if done at scale, the researchers say, this could significantly degrade the experience of legitimate users.
The attack was effective against reasoning models from leading AI companies including DeepSeek-R1, Alibaba's Qwen3-Thinking, OpenAI's GPT-o3, and Google's Gemini 2. 5 Flash and resulted in outputs up to 26 times as long as standard responses on a standard math benchmark. "Across multiple datasets and reasoning models, our method substantially amplifies the output length," Wei Cao, a masters student at Zhejiang University, wrote in an email to IEEE Spectrum .
For more details please read the original article at IEEE Spectrum AI.
Continue Learning
Comments
Sign in to join the conversation