As AI takes on more work, regulators and standards bodies are asking a fair question. Are people still in control? One answer is to count how often they overrule the machine.
Singapore's framework for agentic AI, published by IMDA in May 2026, gives tracking the human override rate as one way to audit human oversight. It defines that rate as how often humans reject or modify agent actions. IMDA presents the framework as guidance.
In the United States, NIST's AI Risk Management Framework lists data on how often and why humans overrule AI as worth collecting. It frames this as an open research question.
Both are sensible starting points. But an override count has a limit that leaders should understand before they rely on it.
Two teams, one number
Imagine two customer service teams in Jakarta. Each uses an AI that drafts replies to complaints. Each team overrules the AI on 20% of cases.
On paper, the teams look the same. Both show healthy human oversight.
Now look closer. Team A overrules the AI on the cases where the AI is wrong. Team B overrules it on random cases, often where the AI was right, and accepts it on many cases where it was wrong.
Team A is exercising judgment. Team B is not. The override rate cannot tell them apart.
Why overrides happen
People overrule AI for many reasons. Some are good: the AI missed a fact, or the case is unusual. Some are not about the evidence at all.
In September 2026, Harvard Business Review previewed an article by Das Narayandas of Harvard Business School. It argues that employees often override proven AI systems out of self-protection. If the AI is wrong, the employee is blamed for following it. If the employee overrules it, at least the decision is theirs.
A team under that pressure will show a high override rate. It will look like strong oversight. It may be the opposite.
The reverse is also true. A low override rate might mean the AI is very good. Or it might mean nobody is checking.
What a count needs
To know whether an override was right, you need to know the right answer. In most real work, you never find out. The customer who got the AI's reply does not tell you whether a better reply existed. The forecast you adjusted is never compared with the forecast you rejected.
Researchers handle this with a known answer. Schemmer and colleagues describe one method. A person gives their first view. Then they see the AI's advice. Then they give a final view. Because the correct answer is known, you can see whether each change was a good one. Did they move towards the AI when it was right? Did they hold when it was wrong?
That is the difference between a count and a judgment.
Override logs can show how often people overrule AI. Only a case with a known answer can show whether they were right to.
What leaders can do now
- Treat the override rate as a question, not an answer. A high rate and a low rate both need explaining.
- Ask why, not only how often. NIST is right that the reasons matter. Collect them.
- Look at outcomes by type of case. Overrides on cases where the AI is known to be weak mean something different from overrides everywhere.
- Watch for self-protection. If overriding is the safe choice for an employee, the rate will rise for reasons that have nothing to do with the evidence.
Counting overrides is a good first step. Knowing whether they were right takes a known answer.
Sources
- IMDA (20 May 2026). Model AI Governance Framework for Agentic AI, v1.5. PDF.
- NIST AI 100-1, AI Risk Management Framework, Appendix C. Link.
- Schemmer, Kuehl, Benz, Bartos and Satzger (2023). IUI '23. arXiv.
- Ignatius, A. (25 September 2026). Why Your Employees Override AI. HBR Executive Agenda, previewing an article by Das Narayandas. HBR.
