You use judgment when AI gives a recommendation, writes a report or takes action for your team. You consider the facts, the situation and what could happen.
Five everyday examples
An AI report is ready to send. You compare an important claim with its original source before signing it.
AI recommends a supplier. You ask the operations team whether the supplier can deliver when your customers need it.
An AI agent handles refunds. An AI agent is AI that can take actions, such as issuing a refund. You decide the amount it may approve and which cases must go to a person.
The AI gives useful advice. You accept it because the evidence supports it. You spend your checking time where the decision needs it.
New facts arrive after approval. You review the plan and change it when the new evidence requires a different action.
These are fictional examples. They show actions people can practise and discuss.
What StepSim makes visible
StepSim is a business simulator: an online exercise where you take a work role, review information and make decisions. It records your actions in the case. You can see the information you opened, the questions you asked, the advice you accepted or challenged and the decisions you changed.
You see your record and the consequences within the simulation. Then a trainer leads a discussion and helps the team connect those choices to its own work.
From a simple explanation to a published standard
The examples above introduce the skill. AIRS-J/1, the standard AIR APAC publishes, describes it in more detail across nine dimensions. Read the standard.
Participant certification is a separate assessment. It uses at least three missions the participant has not seen, and it needs enough evidence across the dimensions. A practice session never awards it. Compare practice and certification.
Go deeper: where the idea comes from
The rest of this guide explains the thinking behind the definition. You do not need it to start practising.
"Judgment" is a word everyone uses and few define. Leaders say they want people with good judgment. Job descriptions ask for it. Training programmes promise to build it. But ask what it looks like on a Tuesday afternoon, and the answers get vague.
With AI in every workflow, vague is no longer good enough. We need to say what judgment is, because it is now the main thing people add.
What is left when answers are cheap
Three economists, Ajay Agrawal, Joshua Gans and Avi Goldfarb, offer a useful starting point. They model AI as a tool that makes forecasts better and cheaper. They give the name "judgment" to a different task: deciding what outcomes are worth.
An AI can tell you that a new supplier is likely to deliver late twice a year. It cannot tell you how much those late deliveries matter to your business. Is a late delivery a small cost, or does it break a promise to your biggest customer? That weighing is judgment.
So as AI gets better at the first task, the second becomes more important, not less. In their model, the two work together, as long as the judgment is not too difficult.
A working definition
At StepSim we use this definition, in two parts.
Judgment is deciding what to do, and what the outcomes are worth. It is needed when the evidence is incomplete, the stakes are real, and nobody hands you the answer.
Each part matters. The evidence is incomplete: if it were complete, a rule or a machine could decide. The stakes are real: something can be lost. Nobody hands you the answer: there is no manual for this case.
What AI adds: three questions and one fixed answer
AI adds three new questions to every decision.
- When to rely on it. Some AI advice is sound. Accepting it quickly is good judgment, not laziness.
- When to check it. Some advice needs a second look, in proportion to the stakes and to how often the AI is wrong on this kind of task.
- When to overrule it. Some advice is wrong. Setting it aside, and saying why, is part of the job.
These three questions are not about trust in general. John Lee and Katrina See put it simply in 2004. Appropriate reliance depends on calibrated trust: trust that matches what the system can do. Over-trust leads to misuse. Distrust leads to disuse. Both are failures.
And one answer stays fixed. A person stays accountable. The AI can advise, draft and even act. It cannot own the outcome. Someone has to stand behind the decision.
Why this definition is useful
This definition turns judgment from a quality into actions. You cannot see "good judgment" directly. You can see what a person did.
- Did they look for evidence that could prove their plan wrong?
- Did they accept sound advice and challenge flawed advice?
- When new facts arrived, did they change their mind, or hold their ground for a good reason?
- Before they handed work to the AI, did they set limits and name an owner?
These are things people do, in order, in front of the evidence. They can be described without guessing what anyone was thinking. That makes judgment something a team can talk about, practise and improve together.
The shift for leaders
For most of the last century, organisations rewarded people for having answers. AI has made answers cheap. What is scarce now is the person who knows what an answer is worth, when to trust it and when to stand against it.
That person is not always the most senior, the fastest or the most fluent with AI. Before leaders can support that judgment, they need to be able to see it.
Sources
- Agrawal, Gans and Goldfarb (January 2018). NBER Working Paper 24243. NBER.
- Lee and See (2004). Human Factors 46(1):50–80.
