Everyone is above average
Everyone is above average
Is AI a Leveler, King Maker, or Escalator?
Congratulations - you are now above average!
It may sound like an old, bad statistics joke, but I mean it quite literally. We now have very strong evidence that AI elevates the skills of the lowest performers across a wide range of fields to, or even far above, what was previously average performance.
This is a big deal because skill gaps among humans can be quite large. In many fields, the difference between top and bottom performers is profound, including programmers, middle managers, and radiologists, among many others. If you are able to find, train, and retain these top workers, you get tremendous benefits. A large part of schooling and work is focused on getting people to this highly skilled state.
However, people who are good at one skill may not be good at another - the top-performing salespeople are usually not also top-performing sales managers. Modern professional work consists of a wide range of activities, rather than a single specialization. For example, the job of a doctor may require many tasks, like diagnosing patients, providing treatment, offering advice, filling out expense reports, and overseeing the office staff. It is unlikely that any doctor is equally good at all of these tasks. Even the best workers have weak spots, requiring that they be part of larger organizations to ensure they can focus on what they do best.
Or they did have weak spots.
AI as Leveler
We already know one major effect of AI on the skills distribution: AI acts as a skills leveler for a huge range of professional work. If you were in the bottom half of the skill distribution for writing, idea generation, analyses, or any of a number of other professional tasks, you will likely find that, with the help of AI, you have become quite good.
The new paper that I, along with Fabrizio Dell’Acqua, Edward McFowland III, Hila Lifshitz-Assaf, Katherine Kellogg, Saran Rajendran, Lisa Krayer, François Candelon, and Karim Lakhani, just released shows this works even for the highly trained knowledge workers at Boston Consulting Group. Across a set of 18 tasks designed to test a range of business skills - from analysis to idea generation to persuasion - consultants who had previously tested in the lower half of the group increased the quality of their outputs by 43% with AI help while the top half only gained 17%. Where previously the gap between the average performances of top and bottom performers was 22%, it shrunk to a mere 4% once the consultants used GPT-4.
We aren’t alone in this type of finding; a lot of work has been done on how AI levels different kinds of writing tasks. Another study, this one from Noy and Zhang of MIT, finds that half the performance gap between good and bad writers is erased when participants use ChatGPT. The same holds true for creative writing, as Doshi and Hauser find. They measured writers on creativity and then gave some of them access to AI generated story ideas. Again, skill differences were levelled, being given 5 ideas by GPT-4 “effectively equalizes the creativity scores across less and more creative writers.”
Beyond writing, LLMs appear to help lower performing specialists become better as well. Choi and Schwarcz gave law students access to GPT-4 and found that law students near the bottom of their class using AI equalized their performance with folks at the top of the class (who actually saw a slight decline when using AI). They conclude, “This suggests that AI may have an equalizing effect on the legal profession, mitigating inequalities between elite and nonelite lawyers.” And, in a real-world experiment, Brynjolfsson, Li, and Raymond discovered that LLMs helped customer service agents in the bottom quintile catch up with higher performers, with much smaller effects for better workers.
Taken together, the results are clear: AI acts as a leveler, raising everyone to a minimum level of performance. In some ways this may not be surprising - automation has always increased the bar for productivity. It used to matter whether workers were fast or slow at digging a ditch or planting seeds, but now a machine does that work much faster, no matter how good your mere human digging or planting skills are. But just as automation reshaped manual labor, it might do the same for intellectual work. The most extreme vision of this levelling effect was something I heard when I was on a panel with a CEO who had been experimenting with GPT-4. To audible gasps he said, “Most of our employees are engineers and we have a few hundred of them… and I think in 18 months we will need 20% of them, and we can start hiring them out of high school rather than four-year colleges. Same for sales and marketing functions.”
I think this prediction for the collapse of skilled work is wrong, however. As we discussed, jobs don’t consist of just one automatable task, but rather a set of complex tasks that still require human judgement. Plus, because AI has a “jagged frontier” (it is not good at everything, but rather has uneven abilities) it is unlikely to do every task that a worker is responsible for. Improving the performance in a few areas need not lead to replacement, but instead it will free up time to do more valuable, satisfying, and productive work. And this also assumes that levelling is the only effect of AI.
AI as Escalator, AI as Kingmaker
Just because early results for AI suggest that only lower performing people benefit does not mean that this is the only possible pattern. It may be that the reason only lower performers gain from AI currently is because the current AI systems are not good enough to help top performers. Or, alternately, it might be that top performers need more training and work to get benefits from AI. If either of these conditions prove true, and they certainly seem plausible, then AI might act more as an escalator, increasing the skills for everyone, from top to bottom performers. After an adjustment period, the relative skill positions stay similar, but everyone gets more done, faster.
Alternately, it might be that some people are just really good at working with AI. They can adopt Cyborg practices better than others and have a natural (or learned) gift for working with LLM systems. For them, AI is a huge blessing that changes their place in work and society. Other people may get a small gain from these systems, but these new Kings and Queens of AI get orders of magnitude improvements. If this scenario is true, they would be the new stars of our AI age, and are sought out by every company and institution, the way other top performers are recruited today.
Change is inevitable
We don’t know the ultimate shape of the new post-AI skills distribution, but we do absolutely know that things are changing. Even with the relatively primitive tools of our current, unspecialized AI systems, it is clear that we can become much more productive, and that less-skilled workers are now at much less of a disadvantage than they used to be.
Like so much else about AIs, the implications here are not completely clear, but we have agency over what happens next. Some of the highest-paid jobs are most impacted by AI - so what do companies do in response? Hire less skilled workers and have them boosted by AI? Expect more work out of all their employees? Focus on working with employees so that they become Cyborgs? Or are they tempted to cut wages or headcount? The answers to these questions are critical, and they will be made soon by companies, with influence from government, their employees, their stakeholders, and their customers. We should be thinking right now about how we want this new world of work to look.
And it isn’t just work. If AI turns poor employees into good ones, it might do the same for other fields. I suspect that it will have a major effect on entrepreneurship, giving every entrepreneur a personalized cofounder to fill in the gaps in their skills. This suggests a potential for an expansion of entrepreneurship, especially given that GPT-4 is widely available for free around the world, helping potential founders far from the usual technology hubs. However, without guardrails, AI may also do the same for criminals, increasing the competence of the worst actors in society.
A lot is changing, quickly. For the first time, we have a broad-based tool that boosts human intellectual abilities, but we are still limited to human wisdom. We will need to draw on that to make good choices even as change continues to accelerate.
