machine-learning-security

A curated list of MLSecOps tools and resources for securing machine learning and AI systems - adversarial ML defense, LLM security, AI red teaming, model scanning, supply-chain protection, and MLOps pipeline security.

Simple pytorch implementation of FGSM and I-FGSM

Hack AI/ML applications — CTF challenges for model attacks, LLMs and AI Agent exploitation.

pretrained BERT model for cyber security text, learned CyberSecurity Knowledge

Code for our USENIX Security 2021 paper -- CADE: Detecting and Explaining Concept Drift Samples for Security Applications

Reading list for adversarial perspective and robustness in deep reinforcement learning.

Adversarial Machine Learning (AML) Capture the Flag (CTF)

The Anti-Virus for AI Artifacts & RAG Firewall. A static analysis tool scanning Models and Notebooks for RCE, Datasets and RAG docs for Data Poisoning, PII, and Prompt Injections. Secure your AI Supply Chain.

Static security scanner for ML model files — detects pickle bombs, Keras Lambda RCE and GGUF template injection, and generates CycloneDX / SPDX AI-BOMs (AIBOM) as EU AI Act, CRA and FDA §524B evidence.

Train AI (Keras + Tensorflow) to defend apps with Django REST Framework + Celery + Swagger + JWT - deploys to Kubernetes and OpenShift Container Platform

AI Firewall & LLM security toolkit - protect your AI applications from prompt injection, jailbreaks, PII leakage, and adversarial attacks

Do you want to learn AI Security but don't know where to start ? Take a look at this map.

Hands-on lessons for attacking and defending AI systems, starting with the OWASP Top 10 for LLM Applications.

Honest-but-Curious Nets: Sensitive Attributes of Private Inputs Can Be Secretly Coded into the Classifiers' Outputs (ACM CCS'21)

Open-source prompt injection attack console. Test AI security by firing categorized attacks at any endpoint.

Test and evaluate Large Language Models against prompt injections, jailbreaks, and adversarial attacks with a web-based interactive lab.

A curated bilingual list of cryptanalytic neural-network model extraction papers, code, taxonomies, and open problems.

Datasets for training deep neural networks to defend software applications

Comprehensive, auto-updating literature review of GenAI & LLM security research, standards, tools, and resources. 100+ curated entries with interactive webapp.

[USENIX Security 2026] Attesting Model Lineage by Consisted Knowledge Evolution with Fine-Tuning Trajectory