Awesome
Machine Learning CTF Challenges
As the realms of artificial intelligence and machine learning continue to shape our world :earth_asia:, the imperative to assess their influence on cybersecurity intensifies. While many AI/ML breakthroughs in cybersecurity revolve around defense and threat intelligence, such as intelligent :computer: SIEM systems and AI-driven malware detection, an intriguing question arises: "Can researchers harness AI/ML for offensive security, or can they outmaneuver AI/ML algorithms with innovative cybersecurity approaches?" This presents a captivating new horizon in the domain of offensive security.🚀
Within this repository lies an array of engaging CTF (Capture The Flag :triangular_flag_on_post:) challenges meticulously crafted for applications that leverage machine learning algorithms in their backend. The intent of this repository is to emphasize the need for implementation of security measures within machine learning applications, safeguarding 🛡️ them against the ever-evolving threat landscape. It serves as a guiding beacon in fortifying the convergence of technology and security🔒.
CTF Challenges :open_file_folder:
| Name | Category | Description | Difficulty | References |
|---|---|---|---|---|
| Vault | Web - Model Inversion | Gain access to Vault and fetch Secret (Flag:). | Hard | <ul><li> OWASP ML03</li></ul> |
| Dolos | Web - Prompt Injection to RCE | Flag is at same directory as of flask app, [FLAG].txt. | Easy | <ul><li> OWASP LLM01</li><li>AML.T0051</li></ul> |
| Dolos II | Web - Prompt Injection to SQL Injection | Make the LLM to reveal Secret (Flag:) of user David. | Easy | <ul><li> OWASP LLM01</li><li>AML.T0051</li></ul> |
| Heist | Web - Data Poisoning Attack | Compromise CityPolice's AI cameras and secure a smooth escape for Heist crew's red getaway car! | Medium | <ul><li>OWASP LLM03</li><li>OWASP ML02</li><li>AML.T0020</li></ul> |
| Persuade | Web - Model Serialization Attack | Flag is at /app/InternalFolder/Flag.txt, not on the website. Find it. | Medium | <ul><li>OWASP LLM05</li><li>OWASP ML06</li><li>AML.T0010</li></ul> |
| Fourtune | Web - Model Extraction Attack | Bypass AI Corp's identity verification to view the flag | Hard | <ul><li>OWASP LLM10</li><li>AML.T0044</li></ul> |
:thought_balloon: If you want to contribute to the above list of CTF challenges (please do), submit a pull request or ping me at
Stay tuned for more challenges being added to the repo. :eyes: