A decade ago Julia Kirby and I described five alternative paths for humans wishing to coexist with AI. The most common, we expected, would be “stepping in,” or learning how AI systems work and perform tasks, casting a critical eye upon their actions and decisions, and trying to improve them when necessary and possible. It’s now common to refer to AI systems as “co-pilots,” but we envisioned the humans as being co-pilots to AI as well.

In the same article and a subsequent book we also described a “step up” role, which would be the boss of AI systems. Like a hedge fund manager—a very well-paid role that we viewed as an exemplar of this path—those who stepped up would oversee AI systems’ trading and portfolio construction performance and refine it as necessary. If the AI-driven hedge fund’s investments aren’t performing well, it’s up to the boss to fix the system.

But now I am thinking that “step up” is perhaps a better term for the role previously viewed as stepping in. To collaborate effectively with AI systems on the job means that most of us will have to up our games. Learning how the systems work isn’t as easy as Kirby and I envisioned it would be in 2015. We’ll have to work harder, think harder, learn faster, and exercise more critical judgment than before. It won’t necessarily make us the boss of AI, but it will perhaps grant us humans a continuing role as an AI co-pilot.

I began thinking about this when I read about an MIT and Harvard research study on the relative performance of unaided human radiologists, AI-based radiological image analysis systems, and humans and AI working collaboratively on image recognition tasks. The conventional wisdom is that the ideal situation was to have radiologists working collaboratively with AI, but it was wrong in this study. AI generally did better than unaided human radiologists, and unaided humans did better than human radiologists who reviewed AI diagnoses of the image. Radiologists working with AI also took more time to come to a conclusion than those who didn’t have an AI prediction—so much for a productivity increase with AI.

One of the authors’ possible explanations for these results is that human physicians don’t really understand how AI makes its decisions. And that’s not surprising—it’s really hard. The AI systems for radiology employ complex deep learning models, and it’s very difficult to interpret how they come to a prediction. Another study of three groups of radiologists addresses this issue—the researchers noted, “This opacity resulted in professionals experiencing increased uncertainty because AI tool results often diverged from their initial judgment without providing underlying reasoning.” In that study, many of the radiologists dealt with that opacity by refusing to adopt the AI models’ predictions. But radiologists who want to keep their jobs must somehow find a way to add value to AI technology—perhaps by sticking to the types of cases that they understand well and AI systems don’t. The number of such cases, unfortunately, is probably shrinking as AI improves.

I also thought about this issue recently when I was working with a property and casualty insurance company on their use of AI. The data science group there has developed some effective predictive models for property loss risk. These models employ risk scores based on analytical AI, and they are relatively interpretable as AI algorithms go. However, the company’s underwriters, who generally have a lot of experience in pricing risk, aren’t yet comfortable using the models, and prefer to use more categorical and intuitive risk assessment approaches.

Now, it’s hardly uncommon for employees to prefer intuitive over algorithmic approaches to decision-making, but that hasn’t stopped the algorithmic approaches from making great inroads in many financial services companies. This shift has been taking place in banking, for example, since the rise of credit scoring in the 1990s.

However, this P&C company prides itself on accepting a wide range of property values and risks, and it of course wants to avoid large losses on high-value insurance policies. Since those large policies come along less frequently, the company doesn’t have large volumes of data on the factors affecting large value risk. So there aren’t trustworthy AI models for that type of risk.

The company’s executives want the underwriters to begin using the algorithmic risk models, but also to preserve their intuitive ability to assess large risks. In other words, they need to step up to understand and exercise critical thinking about AI-based decisions for smaller risk policies, while not losing their intuitive muscles as they apply to larger risks. AI isn’t making the underwriters’ job easier, but rather more complex. As with the radiologists, there may be the possibility of segmenting the job so that some underwriters work with AI and some work with their intuition, but the company would prefer that all underwriters retain an intuitive sense of risk.

I don’t know how many knowledge workers there are in this world who are willing to learn enough about AI to step up in their jobs. One might argue that it’s hard enough to become an expert in one’s field without having to also become an expert in AI. And as AI models become larger and more complex, understanding how they make predictions or decisions is going to become more difficult.

There is an old saying that—regardless of the type of employment—that “the only people who will lose their jobs to AI will be those who refuse to use AI on the job.” I have contributed to making that statement an old chestnut—using it, for example, in this 2018 article about radiology I wrote with a leading AI expert in the field—but I am no longer sure it is accurate or fair. If you have to learn all about AI in order to use AI effectively in your job, and if AI continues to become more opaque, the number of people that AI puts out of work may be much greater than we thought.

Written by: Tom Davenport, originally published on Substack on May 30, 2025