A “successful” jailbreak:
Jailbroken models could potentially be used for malicious purposes, such as generating harmful content, spreading misinformation, or engaging in sophisticated phishing attacks. Gemini Jailbreak Prompt
If you are building applications on top of the Gemini API, relying on Google’s safety settings is not enough. To prevent your own users from using jailbreak prompts against your app, you must: such as generating harmful content
Imprint