Ask an AI assistant a hard question and look at the reply. It is well organised. It gives reasons. It ends with a clear recommendation. It reads like the work of a careful senior colleague.
Now ask it a question it cannot answer well. Very often, the reply looks the same.
This is one of the most important facts about AI at work. The style of an AI answer tells you very little about its quality.
A habit that served us well
For most of our careers, style was a fair signal. A colleague who wrote a clear, structured paper had usually done the work. A messy paper often meant messy thinking. We learned to read polish as a sign of effort and care.
That habit was useful with people, because polish cost people time. A clear paper took a clear mind and a few hours.
With AI, polish costs nothing. The AI writes a clear paper on a topic it understands. It writes an equally clear paper on a topic it does not. So the signal we trusted for years has stopped working.
Most people have not adjusted
A recent survey shows how deep the old habit runs. F-Secure asked 1,500 AI users how they treat chatbot answers.
47.3% of AI users surveyed agree that if a response sounds right, they do not question it further.
Nearly half. The test they use is the one that worked with colleagues: does it sound right? With AI, that test passes almost every time, whether the answer is right or wrong.
This is how people fall behind their tools. Their skill has not gone. Their old signal has stopped working, and nobody told them.
The error you cannot see
Researchers who study automation bias describe two kinds of error. Parasuraman and Manzey review the research in a 2010 paper.
- Errors of commission. You act on the system's advice when the advice is wrong. The AI suggests approving a claim, and you approve it.
- Errors of omission. You fail to act because the system did not tell you to. The AI does not flag a risk, so you do not look for one.
The second kind is the dangerous one with AI. A complete-looking answer invites you to assume it said everything that mattered. Nothing happened, so nothing looks wrong. Yet the AI's silence was treated as an answer.
A new way to read an answer
The old question was "Does this sound right?" The new questions are different.
- What is missing? Before you check what the answer says, ask what it does not say. Which fact would you expect to see here?
- Which claim carries the weight? Pick that one claim. Open the document, the system or the person behind it.
- What is left without the tone? Remove the confident wording. What evidence remains?
- Is it too neat? Real situations are often untidy. An answer with no loose ends may have left something out.
These questions take a few minutes. On the decisions that matter, they are the work.
Sources
- F-Secure, AI chatbots: navigating distrust and usefulness in everyday life. Source: F-Secure, AI chatbot survey, April 2026, 1,500 AI users. Page.
- Parasuraman and Manzey (2010). Human Factors 52(3):381–410.
