when someone affected by your AI asks "why?" and "what can I do about it?" and you have an actual answer instead of a hold queue.
The right and mechanism for individuals materially affected by an AI decision to challenge it, access an explanation, obtain human review, and receive a remedy. Under the EU AI Act (Art. 86), affected persons have the right to request explanations of individual decisions made by high-risk AI systems. Contestability is not a feature request — it is a legal obligation with operational requirements: notification, explanation, review, and remedy.
Why It Matters
Contestability is the governance mechanism that separates AI-assisted decision-making from AI-imposed decision-making. Without it, the relationship between an organisation and the people affected by its AI systems is fundamentally asymmetric: the organisation has full visibility into its models, data, and decision logic; the affected individual has a notification and no recourse.
The EU AI Act makes this explicit. Article 86 establishes a right to explanation for any person subject to a decision based substantially on the output of a high-risk AI system that produces legal effects or similarly significant effects. This is not aspirational language. It is a requirement that must be operationalised with infrastructure: explanation generation, human review queues, escalation paths, and remedy mechanisms.
The operational challenge is substantial. Most organisations have not built the infrastructure required for genuine contestability. A 2024 survey by the Ada Lovelace Institute found that fewer than 15% of organisations deploying high-risk AI had implemented a structured process for individuals to challenge AI-assisted decisions. The remaining 85% relied on general complaints processes that were not designed to handle algorithmic decision challenges and could not produce the explanations required by law.
The Stress Test
A customer receives a denial from your AI-powered claims assessment system. They exercise their right under Article 86 to request an explanation. Your customer service team routes the request to the data science team. The data science team can explain which features the model weighted most heavily, but cannot translate that into language the customer can understand. The customer receives a response 28 days later containing the phrase “based on a multivariate analysis of risk factors.” The customer does not know what a multivariate analysis is. They do not know which risk factors. They do not know how to challenge the assessment.
You have provided an explanation. You have not provided contestability. The explanation without a mechanism for challenge, human review, and remedy is compliance theatre. The regulator does not ask “did you explain?” The regulator asks “could a reasonable person act on the explanation you provided?”
In the Wild
The Court of Justice of the European Union ruled in Case C-634/21 (SCHUFA Holding AG) that automated credit scoring constitutes “automated individual decision-making” under GDPR Article 22. The court held that data subjects have the right not to be subject to decisions based solely on automated processing that produce legal effects, and that meaningful information about the logic involved must be provided. SCHUFA, Germany’s dominant credit reference agency, had argued its scores were merely “preparatory acts” for human decisions. The court disagreed.
The court did not ask whether SCHUFA intended to make automated decisions. It asked whether the score effectively determined the outcome. Intent is irrelevant. Effect is everything.
The Dutch Tax Authority used an algorithmic risk-scoring system to flag childcare benefit recipients for fraud investigation. The system disproportionately flagged families with dual nationality. Over 26,000 families were wrongly accused of fraud, ordered to repay tens of thousands of euros, and in many cases had their benefits terminated. When families attempted to contest the decisions, they encountered a system with no meaningful contestability: no explanation of how the score was calculated, no human reviewer with authority to override, and no remedy process that operated faster than the debt collection process.
The scandal brought down the Dutch government in January 2021. A parliamentary commission described it as “unprecedented injustice.”
The algorithm did not fail. The contestability mechanism failed. The families could not challenge what they could not see, could not understand, and could not reach a human empowered to reverse.
HireVue, a video interview analysis platform used by over 700 companies, faced sustained regulatory scrutiny over its AI-driven candidate assessments. Candidates reported receiving automated rejections with no explanation of what the AI evaluated or how it reached its conclusion. Illinois’s AI Video Interview Act (2020) and subsequent BIPA litigation established that candidates must be notified that AI is used, must consent, and must be told how the AI works. Despite these requirements, enforcement revealed that most deploying organisations had not implemented candidate-facing explanation or challenge mechanisms.
The candidates knew they were being assessed by AI. They did not know on what basis, by what criteria, or how to challenge the result. Notification without contestability is disclosure without dignity.
How to Govern It
Contestability requires four operational components. If any is missing, you do not have contestability. You have a complaints form.
Within the AI Control Index, contestability governance spans Ethics, HITL, and Evidence:
- Notification — Affected individuals must be informed that AI was involved in the decision before or at the time the decision is communicated. Not buried in terms of service. Not disclosed after the fact. Informed at the decision point.
- Explanation (L2) — The system must be capable of generating explanations that are meaningful to the affected individual — not to the data science team. Explainability standards must specify audience, format, and comprehensibility thresholds.
- Human Review (HITL) — A qualified human reviewer must have the authority, competence, and operational capacity to override the AI decision. “Qualified” means domain expertise sufficient to evaluate the decision on its merits, not merely rubber-stamp the model output.
- Remedy — If the decision is found to be erroneous, the organisation must have a mechanism to reverse it, compensate for harm, and correct the underlying issue. Remedy must operate faster than the harm.
- Evidence Factory (S1) — Every contestation — request, explanation provided, review outcome, remedy applied — is captured as a governance artifact. The evidence chain demonstrates that contestability is operational, not aspirational.
When It's Relevant
Every AI system that makes or substantially contributes to decisions affecting individuals. The EU AI Act (Art. 86) applies to high-risk systems, but the principle extends to any deployment where AI output materially influences an outcome that affects a person’s rights, opportunities, or access to services.
Contestability governance is most critical when:
- AI systems make or influence decisions about credit, insurance, employment, benefits, or legal status
- The affected individuals have limited ability to independently verify or understand the AI’s reasoning
- The consequences of an erroneous decision are difficult to reverse (debt collection, employment termination, benefit denial)
- The organisation operates in a jurisdiction with right-to-explanation requirements (EU, UK, Illinois, Colorado)
- The system’s FRIA identifies disproportionate impact on vulnerable populations
Related Terms
References
- [1] European Parliament and Council of the European Union (2024) Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act), Article 86. Official Journal of the European Union, L series.
- [2] Court of Justice of the European Union (2023) Case C-634/21, SCHUFA Holding AG v. Hessischer Beauftragter für Datenschutz und Informationsfreiheit. Judgment of 7 December 2023.
- [3] Parliamentary Committee of Inquiry into Childcare Benefits (2020) Ongekend Onrecht [Unprecedented Injustice]. The Hague: Tweede Kamer der Staten-Generaal.
- [4] Ada Lovelace Institute (2024) Algorithmic Accountability for the Public Sector: A Framework for Responsible AI. London: Ada Lovelace Institute.
- [5] Selbst, A.D. and Powles, J. (2017) ‘Meaningful Information and the Right to Explanation’, International Data Privacy Law, 7(4), pp. 233–242. doi: 10.1093/idpl/ipx022.
- [6] Almada, M. (2019) ‘Human Intervention in Automated Decision-Making: Toward the Construction of Contestable Systems’, Proceedings of the 17th International Conference on Artificial Intelligence and Law (ICAIL), pp. 2–11.
- [7] Kaminski, M.E. (2019) ‘The Right to Explanation, Explained’, Berkeley Technology Law Journal, 34(1), pp. 189–218.
AI Control Index v6.0 · Glossary · June 2026 · i-DEPOT 158508 (BOIP) · CC BY-NC-ND 4.0
By Jeroen Janssen, Apparens