Fix: Added content for Prompt Engineering: Prompt Hacking (#7318)

* fix: added content for Prompt Hacking

* fix: formatted the roadmap content according to the guidelines
pull/7327/head
Satyam Vyas 2 weeks ago committed by GitHub
parent aa76db78da
commit 6461ccaf59
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
  1. 5
      src/data/roadmaps/prompt-engineering/content/107-prompt-hacking/100-prompt-injection.md
  2. 7
      src/data/roadmaps/prompt-engineering/content/107-prompt-hacking/101-prompt-leaking.md
  3. 7
      src/data/roadmaps/prompt-engineering/content/107-prompt-hacking/102-jailbreaking.md
  4. 5
      src/data/roadmaps/prompt-engineering/content/107-prompt-hacking/103-defensive-measures.md
  5. 5
      src/data/roadmaps/prompt-engineering/content/107-prompt-hacking/104-offensive-measures.md

@ -1,3 +1,8 @@
# Prompt Injection
Prompt injection exploits vulnerabilities in AI systems by inserting malicious instructions into user inputs. Attackers manipulate the model's behavior, potentially bypassing safeguards or extracting sensitive information. This technique poses security risks for AI-powered applications.
Visit the following resources to learn more:
- [@article@Prompt Injection](https://learnprompting.org/docs/prompt_hacking/injection)
- [@article@IBM Article](https://www.ibm.com/topics/prompt-injection)

@ -1,3 +1,8 @@
# Prompt Leaking
- [@article@Prompt Leaking](https://learnprompting.org/docs/prompt_hacking/leaking)
Prompt leaking occurs when attackers trick AI models into revealing sensitive information from their training data or system prompts. This technique exploits model vulnerabilities to extract confidential details, potentially compromising privacy and security of AI systems.
Visit the following resources to learn more:
- [@article@Prompt Leaking](https://learnprompting.org/docs/prompt_hacking/leaking)
- [@opensource@Adversarial Prompting - Leaking](https://github.com/dair-ai/Prompt-Engineering-Guide/blob/main/guides/prompts-adversarial.md#prompt-leaking)

@ -1,3 +1,8 @@
# Jailbreaking
- [@article@Jailbreaking](https://learnprompting.org/docs/prompt_hacking/jailbreaking)
Jailbreaking bypasses AI models' ethical constraints and safety measures. Attackers use carefully crafted prompts to manipulate models into generating harmful, biased, or inappropriate content, potentially leading to misuse of AI systems.
Visit the following resources to learn more:
- [@article@Jailbreaking](https://learnprompting.org/docs/prompt_hacking/jailbreaking)
- [@opensource@Jailbreaking](https://github.com/dair-ai/Prompt-Engineering-Guide/blob/main/guides/prompts-adversarial.md#jailbreaking)

@ -1,3 +1,8 @@
# Defensive Measures
Defensive measures protect AI models from prompt attacks. Techniques include input sanitization, model fine-tuning, and prompt engineering. These strategies aim to enhance AI system security, prevent unauthorized access, and maintain ethical output generation.
Visit the following resources to learn more:
- [@article@Defensive Measures](https://learnprompting.org/docs/prompt_hacking/defensive_measures/overview)
- [@opensource@Prompt Injection Defenses](https://github.com/tldrsec/prompt-injection-defenses?tab=readme-ov-file#prompt-injection-defenses)

@ -1,3 +1,8 @@
# Offensive Measures
Offensive measures in prompt hacking actively test AI systems for vulnerabilities. Researchers use techniques like adversarial prompts and model probing to identify weaknesses, enabling improved defenses and highlighting potential risks in deployed AI models.
Visit the following resources to learn more:
- [@article@Offensive Measures](https://learnprompting.org/docs/prompt_hacking/offensive_measures/overview)
- [@article@Definitions and Types](https://www.gyata.ai/prompt-engineering/offensive-measures)

Loading…
Cancel
Save