Fix: Added content for Prompt Engineering: Prompt Hacking (#7318)
* fix: added content for Prompt Hacking * fix: formatted the roadmap content according to the guidelinespull/7327/head
parent
aa76db78da
commit
6461ccaf59
5 changed files with 27 additions and 2 deletions
@ -1,3 +1,8 @@ |
||||
# Prompt Injection |
||||
|
||||
Prompt injection exploits vulnerabilities in AI systems by inserting malicious instructions into user inputs. Attackers manipulate the model's behavior, potentially bypassing safeguards or extracting sensitive information. This technique poses security risks for AI-powered applications. |
||||
|
||||
Visit the following resources to learn more: |
||||
|
||||
- [@article@Prompt Injection](https://learnprompting.org/docs/prompt_hacking/injection) |
||||
- [@article@IBM Article](https://www.ibm.com/topics/prompt-injection) |
||||
|
@ -1,3 +1,8 @@ |
||||
# Prompt Leaking |
||||
|
||||
- [@article@Prompt Leaking](https://learnprompting.org/docs/prompt_hacking/leaking) |
||||
Prompt leaking occurs when attackers trick AI models into revealing sensitive information from their training data or system prompts. This technique exploits model vulnerabilities to extract confidential details, potentially compromising privacy and security of AI systems. |
||||
|
||||
Visit the following resources to learn more: |
||||
|
||||
- [@article@Prompt Leaking](https://learnprompting.org/docs/prompt_hacking/leaking) |
||||
- [@opensource@Adversarial Prompting - Leaking](https://github.com/dair-ai/Prompt-Engineering-Guide/blob/main/guides/prompts-adversarial.md#prompt-leaking) |
@ -1,3 +1,8 @@ |
||||
# Jailbreaking |
||||
|
||||
- [@article@Jailbreaking](https://learnprompting.org/docs/prompt_hacking/jailbreaking) |
||||
Jailbreaking bypasses AI models' ethical constraints and safety measures. Attackers use carefully crafted prompts to manipulate models into generating harmful, biased, or inappropriate content, potentially leading to misuse of AI systems. |
||||
|
||||
Visit the following resources to learn more: |
||||
|
||||
- [@article@Jailbreaking](https://learnprompting.org/docs/prompt_hacking/jailbreaking) |
||||
- [@opensource@Jailbreaking](https://github.com/dair-ai/Prompt-Engineering-Guide/blob/main/guides/prompts-adversarial.md#jailbreaking) |
@ -1,3 +1,8 @@ |
||||
# Defensive Measures |
||||
|
||||
Defensive measures protect AI models from prompt attacks. Techniques include input sanitization, model fine-tuning, and prompt engineering. These strategies aim to enhance AI system security, prevent unauthorized access, and maintain ethical output generation. |
||||
|
||||
Visit the following resources to learn more: |
||||
|
||||
- [@article@Defensive Measures](https://learnprompting.org/docs/prompt_hacking/defensive_measures/overview) |
||||
- [@opensource@Prompt Injection Defenses](https://github.com/tldrsec/prompt-injection-defenses?tab=readme-ov-file#prompt-injection-defenses) |
||||
|
@ -1,3 +1,8 @@ |
||||
# Offensive Measures |
||||
|
||||
Offensive measures in prompt hacking actively test AI systems for vulnerabilities. Researchers use techniques like adversarial prompts and model probing to identify weaknesses, enabling improved defenses and highlighting potential risks in deployed AI models. |
||||
|
||||
Visit the following resources to learn more: |
||||
|
||||
- [@article@Offensive Measures](https://learnprompting.org/docs/prompt_hacking/offensive_measures/overview) |
||||
- [@article@Definitions and Types](https://www.gyata.ai/prompt-engineering/offensive-measures) |
||||
|
Loading…
Reference in new issue