The New Skill Is Knowing When to Use AI
We spend a lot of time learning how to use AI. Maybe the more important skill is knowing when to use it at all.
We keep talking about how to use AI better.
Better prompts. Better tools. Better models.
But maybe we are missing a more basic question:
When should we use AI at all?
There is an interesting study behind this question.
Researchers gave hundreds of knowledge workers different tasks. When the task was something AI was good at, the results were clear: people worked faster, completed more tasks, and often produced better work.
So far, nothing surprising.
But then came the interesting part.
On a task outside AI’s capabilities, people using AI were 19% less likely to get the right answer.
Think about that for a second.
The same technology that made people better at one task made them worse at another.
Researchers have a name for this: the jagged technological frontier.
The idea is simple. There is no clean line between what AI is good at and what it is bad at. Sometimes it handles something surprisingly difficult. Five minutes later, it can fail at something that looks obvious.
And that creates a new problem for us.
We are spending a lot of time learning how to use AI.
Maybe we should also learn when not to use it.
Another study at Procter & Gamble makes this even more interesting. In some product-development tasks, one person working with AI could perform at a level similar to a team working without it.
That is a big deal.
AI can give one person access to something that once required more people, more time, or more expertise.
But more capability does not remove the need for human judgement.
It may make judgement more important.
Knowing when AI can accelerate the work.
Knowing when its answer needs to be challenged.
And sometimes, knowing when it is better to leave AI out of the task completely.
Maybe the next generation of AI skills will not be about becoming better at talking to machines.
Maybe it will be about becoming better at deciding when machines should be part of the conversation at all.
See you in a Thoughtful Future.
References
- Dell’Acqua, F. et al. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality. Organization Science. https://pubsonline.informs.org/doi/10.1287/orsc.2025.21838
- Dell’Acqua, F. et al. The Cybernetic Teammate: A Field Experiment on Generative AI Reshaping Teamwork and Expertise. Organization Science. https://pubsonline.informs.org/doi/10.1287/orsc.2025.20702