New Anthropic Research Paper: Many-Shot Jailbreaking

  • Editor
  • April 3, 2024

Anthropic has recently published a research paper that sheds light on a significant vulnerability within large language models (LLMs), including those developed by itself and its peers.

This vulnerability, termed “many-shot jailbreaking,” has the potential to circumvent the safety measures put in place by developers, prompting a swift call to action within the AI community.

Many-shot jailbreaking exploits the expansive context windows of current LLMs, allowing attackers to insert a sequence of fake dialogues in which the AI appears to comply with harmful requests.

In an official blog post, the Anthropic team said:

We investigated a “jailbreaking” technique — a method that can be used to evade the safety guardrails put in place by the developers of large language models (LLMs). The technique, which we call “many-shot jailbreaking”, is effective on Anthropic’s own models, as well as those produced by other AI companies. We briefed other AI developers about this vulnerability in advance and have implemented mitigations on our systems.

This technique effectively bypasses the LLM’s safety protocols, raising concerns about the potential for misuse.

Anthropic’s study reveals that the vulnerability becomes more pronounced as the number of inserted dialogues increases, highlighting a direct correlation between the “shots” and the likelihood of eliciting a harmful response from the AI.

The decision to publish this research stems from a desire to prompt immediate action and share knowledge across the AI landscape, ensuring that all stakeholders are equipped to tackle this challenge collectively.

Anthropic believes that making these findings public can accelerate the development of effective mitigation strategies and cultivate a culture of transparency and cooperation among LLM providers and researchers.

Here is what people are saying:

The phenomenon of many-shot jailbreaking underscores a critical aspect of AI development: the balance between advancing capabilities and ensuring safety.

As LLMs continue to evolve, with context windows expanding to accommodate more complex inputs, the need for rigorous security measures becomes increasingly evident.

Anthropic’s research not only highlights a specific vulnerability but also serves as a call to action for the AI community to prioritize the development of robust safeguards against potential exploits.

However, people seemed to be not surprised at all:

Anthropic’s disclosure of the many-shot jailbreaking technique marks a significant moment in the ongoing dialogue about AI safety and security.

By choosing to share their findings, Anthropic has underscored the importance of collective action and open communication in addressing the challenges posed by advanced AI technologies.

As the AI community comes together to respond to this call, the path forward will undoubtedly be marked by a strengthened commitment to ensuring the safe and responsible development of LLMs, safeguarding the digital landscape for years to come.

For more of such AI news, visit our AI news at

Was this article helpful?
Generic placeholder image

Dave Andre


Digital marketing enthusiast by day, nature wanderer by dusk. Dave Andre blends two decades of AI and SaaS expertise into impactful strategies for SMEs. His weekends? Lost in books on tech trends and rejuvenating on scenic trails.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *