James Glover (CEO Flint Learning Solutions) helps organizations drive measurable behavior change with AI-personalized on-the-job activities.
getty
Generative AI has made the first draft nearly free. Whether you need a summary of a 40-page policy, a reply to an unhappy client or a rewrite of a report that’s due in an hour, it now arrives in seconds, looking finished and polished. That’s a great tool with a huge upside. It’s also easy to use badly, as those quick first drafts create cleanup work for multiple people, and the polished drafts lull people into believing the work is done when it’s not.
Some people still aren’t using AI at all, which is a problem I wrote about here last year in a piece on the AI licenses companies pay for that nobody uses. That’s steadily changing. Now a growing risk sits with those who are using it because output that looks right at a glance gives them permission to stop thinking about it. They run the report and pass it on to another team. But they forgot to pass it through their brains.
It’s called “workslop,” and it’s in direct conflict with the business value of AI, which is saved time. To fight this tendency, new AI users need to practice exercising critical judgment with their AI outputs, learning when the tool will help with their specific tasks and checking what it hands back.
Automation bias has reached every desk.
The human tendency to accept what an automated system says, thus catching fewer of its mistakes, isn’t new with AI. Researchers call it automation bias. What’s new with AI is how widespread automation is.
The basic problem of automation is that when people move from doing the work to monitoring the work, they use their own expertise less. This issue used to be confined to certain industries or jobs. Think of pilots, who handed routine flying to autopilot—then sometimes missed problems it didn’t flag. Power plant operators went from running the process to watching it, and they lost the hands-on feel they needed when something went wrong. Even specialists reading mammograms caught fewer cancers when the software miscategorized the images than specialists who didn’t use the software.
The standard review of the research found automation bias in both experts and beginners and concluded that training or instructions can’t prevent it. Generative AI fits right into this weak spot.
Humans rely too much on automated processes, no matter their line of work. And AI is now in every line of work.
Unchecked work moves downstream.
The business case for AI rests on time saved, and that’s where poor judgment does its damage. A research team that included Stanford researchers, writing in Harvard Business Review, named the problem “workslop”: AI output that looks polished but lacks substance, so the effort lands on whoever receives it. They found that 41% of workers had been handed workslop, and each instance cost nearly two hours of rework.
The person who sent it finished their part faster, and the person downstream paid for it. Multiply that across a company and the expected gains shrink or vanish. Rework isn’t the only cost, though. Others include:
• Risk: An error in a document travels further the more polished it looks.
• Brand: Sloppy customer-facing copy is a bad signal to customers.
• Decisions: Consider a leader who acts on a faulty AI summary of the quarter’s numbers.
Here’s what judgment should look like in any job.
People’s tendency is to turn off their brains when the tool looks smart. Judgment means keeping a healthy skepticism.
The skepticism is warranted because the tool’s weak spots don’t announce themselves. In a study of 758 consultants at Boston Consulting Group, people using GPT-4 did markedly better work on tasks it handles well. But on a task chosen to sit just outside what the AI could do, they were 19% less likely to reach the right answer than colleagues working without it. Nothing on the surface of the answers told them which situation they were in.
Checking is how people find out. When one media company had reporters summarize long sources with AI and then compare each summary with the original, the checking caught altered quotes, swapped rankings and invented citations. The tool saved the reporters time on the first pass, and their judgment kept its mistakes out of the paper.
The habits behind that are simple, and they look much the same from one role to the next:
1. Pick the task deliberately. Use AI where it’s strong, such as a first draft or a summary, and be more careful where the answer depends on facts it can’t see.
2. Summarize, then check. Get the summary, then go back to the original to confirm the summary says what the source says.
3. Own the draft. Whoever sends it must own its accuracy and its tone, so the draft gets read as the sender’s own work.
Practice builds judgment.
A training session can show people what good judgment looks like, and that’s a useful start, but most of the benefit fades within weeks. Judgment has to be built deliberately, through practice on the work people already do. For example, decide before handing this week’s client report to AI which parts it can draft and which parts depend on numbers only the sender can confirm.
That practice has to fit the person. An HR partner checks different things than a sales engineer does, and someone who already reviews carefully needs a harder case than someone who accepts whatever comes back.
Judgment also has to be measured as a behavior rather than as usage. A manager can rate how well each team member checks and judges AI output before the practice begins and again at the end of the quarter, while each person rates themselves on the same behaviors. A record of the errors people caught shows the business the rework that never reached the next desk.
AI will keep getting faster at producing work that looks finished. The companies that get the gains will be the ones whose people bring their own judgment to it because they practiced on real work until it became how they work.
Forbes Human Resources Council is an invitation-only organization for HR executives across all industries. Do I qualify?
