Find answers from the community

Updated 8 months ago

what can we do to prevent Prompt

what can we do to prevent Prompt injections and jailbreaks ( the Ai will stop acting on its role and will act the way the user wants, making the company look bad etc... ) - I know we have input moderation, bu how do we even use it if the Ai can change the way it acts so easily

3 comments

TTeemu

The two main ways are using a moderation endpoint and having a strong system message. There is no way to stop it 100% but with those integrated, it should stop most of it

AAbed Malak

what could a strong moderation endpoint and system message be like

YYolk

@Abed Malak we've just introduced a ZenGuard AI integration to LlamaIndex - https://llamahub.ai/l/llama-packs/llama-index-packs-zenguard. We protect against Prompts Attacks (prompt injection, jailbreaks), topic restrictions (allowed and banned topics), PII, sensitive info, and keywords leakage (control what you share vs. what is sent to you), toxicity and other features in security.

Add a reply