EU AI Act red teaming: what the Act says about adversarial testing

The EU AI Act (Regulation (EU) 2024/1689) never uses the words "red team". It requires "adversarial testing" from providers of the largest general-purpose AI models, and it sets resilience duties for high-risk systems. This is not legal advice. Ask a qualified lawyer how the Act applies to your product.

Which articles of the EU AI Act relate to red teaming?#

Article 55(1)(a), Article 15 and Article 9 cover adversarial testing and resilience. Article 50 and Article 26 also apply to many chatbot and agent products. The table gives each duty, who it binds and when it applies.

The "applies from" dates come from the consolidated Official Journal text and the Omnibus amendment (both linked at the end). The Chapter V dates follow Article 113 as the Omnibus leaves it.

What does Article 55 say about adversarial testing?#

Article 55(1)(a) tells providers of GPAI models with systemic risk to "perform model evaluation in accordance with standardised protocols and tools reflecting the state of the art, including conducting and documenting adversarial testing of the model with a view to identifying and mitigating systemic risks." The recitals add that this can be done through internal or independent external testing.

Article 55 binds the model provider. Under Article 51, a model is presumed to have systemic risk when its training compute passes 10^25 floating-point operations, or when the Commission designates it. If you build a product on a hosted model, Article 55 does not bind you, but it sets what your model vendor must test.

Adversarial testing of a model does not cover your application. A model can pass its provider's evaluation while your app still leaks a tenant's documents because of a retrieval bug. What is AI red teaming explains model-level and system-level testing.

What does Article 15 require for high-risk systems?#

Article 15(1) requires high-risk AI systems to reach an appropriate level of accuracy, technical resilience and cybersecurity, and to keep it through their life cycle. Paragraph 5 is the security paragraph. Systems "shall be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities."

The same paragraph names the attack types that technical solutions must address where appropriate: data poisoning, model poisoning, adversarial examples or model evasion, confidentiality attacks and model flaws. Article 15 names no test method and never says "red team" or "penetration test". The system must resist these attacks, and testing is how you build evidence that it does. A scoped test against the listed attack types gives you something concrete to put in the Article 9 risk management file.

What changed with the Digital Omnibus?#

The Digital Omnibus on AI, Regulation (EU) 2026/1744, delayed the high-risk dates. As of November 2026, it is law. The Commission proposed it on 19 November 2025. The institutions reached a political agreement on 7 May 2026. The regulation is dated 8 July 2026 and appeared in the Official Journal on 24 July 2026. Its Article 4 sets entry into force on the third day after publication, which gives 27 July 2026. The Commission's AI Act page, last updated 19 October 2026, confirms that date.

The new Article 113 sets these dates:

• Annex III high-risk systems (for example employment, education, biometrics, critical infrastructure): 2 December 2027, not 2 August 2026.
• Annex I high-risk systems (AI in regulated products such as lifts or toys): 2 August 2028, not 2 August 2027.
• Prohibited practices and AI literacy: unchanged, from 2 February 2025. New bans added by the Omnibus apply from 2 December 2026.
• GPAI duties (Chapter V): unchanged, from 2 August 2025.
• General application date: unchanged, 2 August 2026. This is when the Article 50 transparency rules and the AI Office enforcement powers start.

How should you use this when you plan a test?#

Match the test to the duty that applies to you. Common cases:

1. You provide a GPAI model with systemic risk. Article 55 asks for documented adversarial testing of the model. Plan model-level evaluation, with independent testers where you can.
2. You provide or deploy a high-risk system. Article 15 and Article 9 apply from the dates above. Test the full system: inputs, retrieval, tools and outputs. Start early so the fix and retest cycle fits before the date.
3. You ship a chatbot or agent that is not high-risk. The Act sets few security duties for these products, but customer security reviews, contracts and liability still push you to test. See testing customer support chatbots.

Write the test scope so a reviewer can map it to the article. The scoping guide lists the questions. The frameworks page explains how findings map to OWASP, MITRE ATLAS and NIST, and why no framework mapping is a certification. To start, send a scoping request.

What should you not claim?#

A red team report does not prove compliance. The Act sets duties for providers and deployers, and conformity assessment, technical documentation, post-market monitoring and incident reporting are separate duties. A test report supports part of that file and replaces none of it.

The Act does not "require pentests" for high-risk systems. It requires resilience and a risk management system. Ask your legal adviser or notified body what evidence they expect.