# AI governance: a practical program for teams that ship AI features

Source: https://airedteamingservices.com/guides/ai-governance/
Publisher: Excession Labs (airedteamingservices.com)
Last updated: 2026-11-09

AI governance explained for product teams: the minimum program (inventory, owners, risk tiers, test gates, incident path) and the evidence a security test adds.

> **Short answer:** AI governance is the set of policies, roles and controls that decide how an organization builds, buys and runs AI systems. A minimal program has an AI inventory, a named owner for each system, risk tiers, test gates before release, an incident path and review dates. Security testing adds proof that a control works. This guide is not legal advice.

AI governance is the set of policies, roles and controls that decide how an organization builds, buys, deploys and monitors AI systems. For a team that ships AI features, it comes down to six artifacts: an inventory, named owners, risk tiers, test gates, an incident path and review dates. This guide describes that minimum program and shows where security test evidence fits. It is not legal advice.

## What is AI governance?

AI governance is the management layer around AI systems. It answers who may deploy an AI feature, on what evidence, who is accountable when it fails and when someone looks at it again.

Three public documents define most of the vocabulary. The NIST AI Risk Management Framework (AI RMF 1.0, NIST AI 100-1, published 26 January 2023) is "intended for voluntary use" and organizes work into four functions: Govern, Map, Measure and Manage. ISO/IEC 42001:2023 specifies requirements for "establishing, implementing, maintaining and continually improving" an AI management system within an organization. The EU AI Act (Regulation (EU) 2024/1689) sets legal duties for providers and deployers of AI systems placed on the EU market.

They differ in force. NIST's framework is voluntary. ISO/IEC 42001 is a certifiable management-system standard for the organization, and it does not certify one AI product. The EU AI Act is binding law where it applies. [The framework guide](/guides/ai-governance-framework/) compares them in detail, and [the compliance guide](/guides/ai-compliance/) covers the laws.

Governance is also not one document. A policy PDF with no inventory behind it governs nothing, because nobody can say which systems it covers.

## Why does AI governance need test evidence?

A governance decision needs proof that a control works, and a test is the main way to get it. Without proof, a policy line such as "the support bot must not reveal other customers' data" is an intention.

Standards give a reason to produce that proof. NIST AI RMF 1.0 puts the Measure function between mapping risks and managing them, so a team measures a risk before it decides how to treat it. The EU AI Act requires high-risk systems to be resilient against attempts to alter their use, outputs or performance by exploiting vulnerabilities (Article 15(5)). It does not name a test method. NIST AI 100-2 E2025 states that current prompt injection mitigations "do not offer full protection against all attacker techniques", so a team cannot assume that a guardrail works.

What a test produces depends on how it is run. A usable test record has five parts:

1. The scope: which system version, which environment, which user roles.
2. The test cases, each tied to a risk in your register.
3. The result per case, with the request and response that show it.
4. The fix and the retest result for every finding.
5. The date, so a reviewer knows what the evidence covers.

Compare that with a vendor statement such as "we red teamed the model". That statement does not say what was tested, against which version, or what happened to the findings. Governance reviewers, customers and auditors ask for the five parts above, so ask your testers for them at scoping. See [how to scope an AI red team](/guides/how-to-scope-an-ai-red-team/).

## What should a minimal AI governance program contain?

A small team can start with six artifacts. Each one is a short document or a table in a tool you already use.

| Artifact | What it holds | Evidence that it works |
|---|---|---|
| AI inventory | Every AI system, built or bought: purpose, model and vendor, data it reads, tools it can call, users | Inventory matches what a scan of API keys, SaaS accounts and repos finds |
| System owner | One named person per system | Owner approves each release and each risk change |
| Risk tier | A class (for example low, medium, high) set by data, actions and audience | Written reason for each tier; tier drives the test gate |
| Test gate | The tests a system must pass before release, by tier | Test report with scope, results, fixes and retest |
| Incident path | Who is told, who can switch the feature off, how users are notified | A tabletop exercise or a real incident review |
| Review date | When the tier and the evidence are checked again | Calendar entry; review notes |

Start with the inventory. Teams that skip it find shadow systems later: a marketing tool with an AI assistant that reads customer lists, or a plugin that a developer connected to production data. The inventory columns that matter for security are the data the system can read, the tools it can call, and who can reach it.

> **Demonstration:** This is a demonstration. No real customer system was tested. The row below describes a fictional product, "Harbor Notes", a demo note-taking app.
> 
> | Field | Value |
> |---|---|
> | System | Harbor Notes "Ask your notes" assistant |
> | Owner | Product lead for notes |
> | Model and vendor | Hosted LLM through a vendor API |
> | Data it reads | The signed-in user's notes, through retrieval |
> | Tools it can call | Search notes; create note |
> | Audience | All signed-in customers |
> | Tier | High (customer data plus a write action) |
> | Last test | Not yet tested; gate blocks release |
## Who owns what?

Governance fails when ownership is unclear, so give each decision to one role. The table shows a split that works for a company with a few AI features. Adapt the titles to your organization.

| Decision | Owner | Input from |
|---|---|---|
| Approve the governance policy | Executive sponsor | Security, legal |
| Keep the AI inventory | Program owner | Every product team |
| Set a system's risk tier | System owner | Security, privacy |
| Approve a release | System owner | Test report, program owner |
| Accept a residual risk | Executive sponsor, in writing | System owner, security |
| Run and report tests | Security team or an outside tester | System owner |
| Run the incident path | Security lead | System owner, communications |

Two rules keep the table honest. First, the person who builds a system does not accept its residual risk alone. Second, an outside tester reports findings and evidence, and the owners decide what to fix. A tester cannot accept a risk for you.

## How do risk tiers and test gates work?

A risk tier decides how much testing a system needs before release. The tier comes from what the system can read, what it can do and who can reach it, not from how advanced the model is.

| Tier | Typical signs | Minimum gate before release |
|---|---|---|
| Low | Internal use, no sensitive data, no tools, no actions | Owner review, inventory entry, basic abuse-case checks |
| Medium | Customer-facing, reads non-sensitive data, no write actions | Prompt injection and output-handling tests, logging check |
| High | Reads customer or regulated data, calls tools, or acts for users | Full scoped test of the application, retrieval, tools and permissions; fix and retest; written risk acceptance for open items |

A system moves up a tier when it gains a data source, a tool or a new audience. A support bot that only answers from public help pages is low or medium. The same bot with access to order history and a refund tool is high, because a successful prompt injection can now cause an action.

Name the test cases in the gate. The [OWASP Top 10 for LLM applications](/guides/owasp-top-10-for-llm-applications/) gives a risk list to start from. [MITRE ATLAS](/guides/mitre-atlas/) names the attack techniques. For tools and agents, [excessive agency](/guides/excessive-agency/) is the usual cause of high-impact findings. A gate should say "tested for LLM01:2026 Prompt Injection and LLM03:2026 Excessive Agency" rather than "tested for AI risks".

A gate also needs an exit rule. Decide in advance what blocks a release, for example any open finding that exposes another user's data. Without that rule, schedule pressure sets the bar.

## Which standards and laws should you map to?

Map your program to one framework first, then add the others as customers or regulators ask. Mapping means a table that connects each of your artifacts to the clause or function it supports.

| Source | Status as of November 2026 | What it gives a program | Binding |
|---|---|---|---|
| NIST AI RMF 1.0 (AI 100-1) | Published January 2023; NIST says it is being revised under the White House AI Action Plan | A structure of four parts (Govern, Map, Measure, Manage) for risk work | Voluntary |
| NIST AI 600-1 | Generative AI Profile, July 2024 | Suggested actions, including red-teaming actions | Voluntary |
| ISO/IEC 42001:2023 | Published 18 December 2023 | A certifiable AI management system | Voluntary; certification is optional |
| EU AI Act | Regulation (EU) 2024/1689, amended by Regulation (EU) 2026/1744 | Legal duties by role and risk class | Binding where it applies |
| NIST IR 8596 | Initial preliminary draft, December 2025 | A draft Cyber AI Profile for the NIST Cybersecurity Framework | Draft, voluntary |
| OWASP AI Testing Guide | Version 1 published 26 November 2025 | Test cases for AI applications, models, infrastructure and data | Voluntary |

The EU AI Act dates moved in 2026. As of November 2026, the duties for Annex III high-risk systems apply from 2 December 2027, and the duties for Annex I systems from 2 August 2028. The duties for general-purpose AI models started on 2 August 2025. Our [EU AI Act guide](/guides/eu-ai-act-red-teaming/) lists the articles and dates. For ISO/IEC 42001, read [ISO 42001 explained](/guides/iso-42001/).

No source in this table says that a framework "requires red teaming" for ordinary AI products. Where testing appears, it appears as a way to produce evidence for a risk or resilience duty. Check each duty with your legal adviser.

## What does AI governance not cover?

Governance sets the process, and it does not make a system safe. Three limits matter.

First, a program does not stop attacks. Prompt injection has no complete fix today, and NIST says current mitigations do not give full protection. Governance decides how you design around that: least privilege, human approval for risky actions, monitoring and a way to switch a feature off.

Second, a certificate does not test a product. An ISO/IEC 42001 certificate shows that an organization runs a management system. It does not show that one chatbot resists prompt injection. Only a technical test shows that, and only for the tested version.

Third, governance covers more than security. Bias, transparency and privacy duties need their own owners and tests. This site covers the security part. The OWASP AI Testing Guide covers broader trustworthiness tests as well, and its first version lists test areas beyond security.

When you are ready to add testing to the program, start with a short [AI security risk assessment](/guides/ai-security-risk-assessment/) to set tiers, then [send a scoping request](/scoping/) for the systems in the top tier.

## Questions

### Who should own AI governance in a small company?

Name one accountable person for the program and one owner for each AI system. In a small team the program owner is often the head of engineering or security, and the system owner is the product lead. The role matters more than the title. Write both names in the inventory.

### Is AI governance only for companies that sell AI?

No. A company that uses a third-party copilot, a support chatbot or an AI feature in its own product has AI systems to inventory, own and review. Standards such as ISO/IEC 42001 apply to organizations that provide or use AI products or services.

### How often should you review an AI system?

Set the date when you classify the system. Review it again on any change that alters risk: a new model, a new data source, a new tool the system can call or a new user group. Many teams also set a fixed yearly review. The standards do not give one interval.

### Does a pentest or red team report count as governance?

It is one piece of evidence. Governance also needs policy, owners, a risk decision and monitoring. A test report shows what was tried against one version at one date. It does not show that the system is secure, and it does not replace the other artifacts.

## Sources

- [NIST AI Risk Management Framework (AI RMF 1.0, NIST AI 100-1)](https://www.nist.gov/itl/ai-risk-management-framework)
- [ISO/IEC 42001:2023 Artificial intelligence management system (IEC Webstore record)](https://webstore.iec.ch/en/publication/90574)
- [Regulation (EU) 2024/1689 (AI Act), Official Journal text](https://eur-lex.europa.eu/eli/reg/2024/1689/oj)
- [Regulation (EU) 2026/1744 (Digital Omnibus on AI)](https://eur-lex.europa.eu/eli/reg/2026/1744/oj)
- [NIST AI 100-2 E2025: Adversarial Machine Learning taxonomy](https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-2e2025.pdf)
- [NIST IR 8596 initial preliminary draft: Cyber AI Profile](https://csrc.nist.gov/pubs/ir/8596/iprd)
- [OWASP AI Testing Guide](https://owasp.org/projects/ai-testing-guide)