Prompt Engineering Risks and Misuse

Three major risk categories documented by the Prompt Engineering Guide.

Adversarial Prompting (confidence: high)

Techniques that attempt to manipulate LLMs into producing unintended or harmful outputs:

• Prompt Injection — Embedding malicious instructions within user input that override system prompts
• Prompt Leaking — Extracting the system prompt itself from the model, revealing proprietary instructions
• Jailbreaking — Crafting inputs that bypass safety filters and content moderation (e.g., role-play scenarios, hypothetical framing)
• Prompt Injection and Prompt Leaking are specific sub-types covered in the Prompt Hub

Factuality (confidence: high)

LLMs can produce confident-sounding but incorrect information (hallucinations). Mitigation strategies include:

• Using retrieval-augmented generation to ground outputs in verified sources
• Prompting for uncertainty expressions (“Say ‘I’m not sure’ if uncertain”)
• Self-consistency and multi-path verification

Biases (confidence: high)

LLMs can amplify or exhibit social biases present in training data. Key concerns:

• Gender, racial, and cultural stereotypes in model outputs
• Political and ideological biases in content generation
• Mitigation through careful prompt design, bias-aware evaluation, and diverse training data

Implications

Risk awareness is essential for responsible LLM deployment. The Prompt Engineering Guide’s inclusion of this section signals that prompt engineering is not just about maximizing capability but also about managing safety, reliability, and fairness.

prompt-engineering-guide | prompt-engineering-techniques | prompt-hub-catalog

Sources

• raw/prompts/articles/promptingguide-pt-risks.md
• raw/prompts/articles/promptingguide-pt-risks-adversarial.md
• raw/prompts/articles/promptingguide-pt-risks-factuality.md
• raw/prompts/articles/promptingguide-pt-risks-biases.md