AI red teaming tools: open-source scanners and frameworks compared

Open-source AI red teaming tools send large sets of attack prompts to a model or app and score the replies. The main ones are garak, PyRIT, promptfoo, DeepTeam and Giskard. They give quick, repeatable checks of known attack patterns. They do not replace a scoped test, because they do not know your rules, your users or your data.

Which open-source AI red teaming tools exist?#

We checked these tools on their official GitHub pages on 9 November 2026. License and release data come from each repository's GitHub API record, and descriptions follow each project's own README. The order is alphabetical and is not a ranking.

Release data changes often. Check the repository for the version you install.

What does each tool do well, and where does it stop?#

These notes are our reading of each README and its docs, plus general limits that apply to this kind of tool.

garak#

The README compares it to nmap or Metasploit for LLMs. It uses static, dynamic and adaptive probes to find ways to make an LLM or dialog system fail. It reaches many model types and REST endpoints. Limit: it probes what a text interface returns. It does not know what a "bad" answer is for your product, so you must write or tune detectors for your rules.

PyRIT#

It ships a scanner, a GUI and a Python framework, so it covers both repeatable runs and custom work. Its README lists HTTP and WebSocket targets for full applications. Limit: the most useful runs need engineering time to build targets, converters and scoring that match your app.

promptfoo#

It runs from a config file and fits CI. The same tool also does plain prompt evaluation, which helps if your team already tests prompts. Limit: the README says it is now part of OpenAI, and says it stays open source under the MIT license. Check which features run locally and which use a hosted service before you send sensitive test content.

DeepTeam#

It offers more than 50 ready-made vulnerability types, with scoring by an LLM judge that runs locally. Its categories include data privacy, authorization (BOLA, BFLA, RBAC) and responsible AI. Limit: LLM judges can be wrong. A pass or fail needs a human check, especially for authorization findings that depend on your roles.

Giskard v3#

It targets agent testing with dynamic, multi-turn checks, plus RAG evaluation. Its README says v2 is no longer actively maintained. Limit: v3 is a rewrite, so older tutorials may not apply.

Inspect#

It is an evaluation framework that also serves red teaming work. It suits teams that want to build their own test suites and run them across models. Limit: you build the security tests yourself or take them from the pre-built set. Check that a pre-built test fits your app before you rely on it.

What can tools not do?#

Tools cover known patterns at scale. They miss the findings that depend on your system. The Microsoft AI Red Team makes a similar point in its paper "Lessons From Red Teaming 100 Generative AI Products": AI red teaming "is not safety benchmarking", and attacks work against end-to-end systems (arXiv 2501.07238).

Five gaps matter most:

1. Authorization. A tool does not know that customer A must not see customer B's file. A tester must set up two tenants and check.
2. Business rules. A tool does not know your refund limit. See testing customer support chatbots.
3. Indirect paths. A tool sends prompts through the chat box. It does not place instructions in your documents, tickets or tool outputs unless you build that path.
4. Chains. Real findings often join several small flaws: a retrieval leak plus a markdown image plus a weak approval step. Scans report single flaws.
5. Judging. Many tools score with another LLM. That adds error and cost. Evidence needs a person to read the exchange.

The prompt injection guide explains why a clean scan does not mean the app is safe. Nobody can promise that prompt injection is fully prevented.

How do you choose and run a tool safely?#

Before the first run:

1. Get written permission from the system owner. Test only systems you may test.
2. Use a test copy with fake data and sandboxed tools. Do not point a scanner at production.
3. Set limits on calls, spend and time. Scans can send thousands of prompts.
4. Check what leaves your network. Find out if the tool sends prompts or results to a vendor or to an LLM judge.
5. Pin versions. Write down the tool version, the probe set, the model version and the date.
6. Review every finding by reading the raw exchange. Drop false positives.
7. Repeat the run. Model output varies. Record the number of attempts and the rate.
8. Map the results to a framework only after review. See the OWASP Top 10 for LLM Applications guide.

Where do tools fit in a scoped engagement?#

Tools give breadth and testers give depth. A scan can find easy wins early, which leaves human time for authorization, tools and data paths. The methodology page shows how we record test conditions, evidence and impact. The scoping guide lists the inputs a tester needs.

If you run tools yourself, treat the output as leads to verify. If you want a scoped test of the paths that tools miss, see our LLM application security assessment and send a scoping request.