Hi, I'm Charles Costa, researcher, strategist, and founder of Lexora Labs™. Each week I dig into AI, knowledge management, and future-of-work research so you don't have to.
EDITOR'S NOTE
Last week we looked at why nine in ten managers cannot see AI working inside their own companies, even as the productivity data says otherwise. I do not think that is a reporting problem. I think it is the instrument.
In 1998, BusinessWeek asked Steve Jobs whether Apple had researched the iMac with consumers. His answer got flattened into a slogan about visionary founders, but what he actually said was narrower and more useful.
"It's really hard to design products by focus groups. A lot of times, people don't know what they want until you show it to them."
That is a claim about a research method, not about genius. Ask people to describe the value of something they have not used and you will get a bad answer, and it will not be their fault.
Which is roughly the design of every AI pilot running right now.
———————————————————————————
THIS WEEK'S READING:
Generative AI at Work: Quarterly Journal of Economics
Researchers tracked 5,172 customer support agents at a Fortune 500 software company through an AI rollout, back when the tool still ran on GPT-3. Resolutions per hour rose 15 percent on average, and 30 percent for the least skilled and least experienced.
Why It Matters: The AI went down from time to time, technical failures at the vendor that cut off recommendations for some agents and not others, sometimes for a few minutes and sometimes for hours.
Researchers used those outages as a natural experiment, and that is the interesting part. When the system crashed a month after rollout, agents worked at their pre-AI speed. When it crashed after three months, they were still faster without it. Something had moved into the person, but not into everyone. The agents who followed the AI's suggestions early kept the gain. The ones who ignored them showed nothing, even after months of access.
So engagement is the variable, not access. But notice why we can see it at all. Contact centers already run on average handling time and chats per hour, and that instrumentation predates the AI by decades. Drop the same tool into a job nobody counts by the hour and the same variation exists, just invisibly. Hold that thought.
If nobody is teaching this, where does it come from?
Navigating the Jagged Technological Frontier: Harvard Business School
Boston Consulting Group ran two randomized experiments across 758 consultants. On tasks inside the model's competence, the ones with AI completed 12.2 percent more work, 25.1 percent faster, at over 40 percent higher quality.
Why It Matters: Researchers read 244 consultants' chat logs afterward and had to invent vocabulary for what they found. Some handed the model whole chunks of work. Others interleaved, starting a sentence and letting it finish. Nobody taught either style. They showed up anyway, along with a repertoire nobody assigned, giving the model a persona, pushing back on it, teaching by example.
Then the part I cannot stop thinking about. On one task built to sit just past the model's competence, AI users were 19 points more likely to be wrong, and the group that received prompt engineering training did worst of all, 60 percent correct against 84.5 percent for consultants using no AI. My read, and the researchers float it too, is that the training built trust, and trust is exactly what you do not want carrying you into a question the model cannot answer. Trust but verify. Treat it like a new hire who is confidently wrong, quick and fluent and missing the tribal knowledge that says the spreadsheet is not the whole story. Which is what that task was. The financials pointed one way and the insider interviews pointed the other.
Meanwhile, in the boardroom.
PwC surveyed 4,454 CEOs across 95 countries. Fifty six percent report neither higher revenues nor lower costs from AI over the past year.
Why It Matters: Be precise here, because this one gets mangled everywhere. It does not mean they got nothing. It means that when asked to attribute a financial movement to AI, they could not. A company with faster cycle times and flat revenue lands in that 56 percent, and so does the 22 percent who said AI actually increased their costs, which nobody quotes.
Look at the instrument, though. PwC asked the person furthest from the keyboard to attribute twelve months of company wide financial movement to one technology, on a three point scale where a 0.5 percent saving and a 40 percent saving land in the same box. The support agents in the first read were legible because somebody was already counting chats per hour. Almost nobody else is counted that way, and that is most of what this number is telling us.
So is the value there or not? The largest sample we have gets closest.
Gallup surveyed 22,573 US employees in May 2026. Just over half now use AI at work, and 30 percent use it at least weekly.
Why It Matters: Breadth is the finding. Among employees using AI for one or two kinds of task, 45 percent said it improved their productivity. Among those using it for seven or more, 90 percent did. On the harder "extremely positive" measure it runs from 8 percent to 44 percent. Not frequency, not hours. The number of different parts of your job you have tried it on.
Then Gallup published a companion piece the next day that complicates it. Manager support beat individual frequency, 33 percent against 9 percent where frequency managed 24 against 5. I would want to know what "support" meant, though, because there is a real difference between a manager who protects your time to go find something and one mandating you run emails through a model when you could have written them in two minutes.
The second is micromanagement in a new outfit, and it reads as support on a survey. Meanwhile the people with the most to gain usually are not resisting anything, they just do not know how to point it at something that matters.
———————————————————————————
So here is where I land, and it is not quite where I started.
Discovery is individual. Nobody hands you your use case, because the value lives in the texture of your Tuesday, and you only find it by going past the first bad answer.
But the conditions for discovery are institutional. Whether you get enough attempts depends on whether the tool fits the systems you already work in, whether your manager treats experimenting as work, and whether anyone gave you the hours.
Companies are measuring the wrong thing while quietly controlling the one variable that matters. Which makes "identify your use cases" the wrong instruction. That is the focus group Jobs was describing, run internally, with a deck at the end.
There is a further problem underneath all of this, which is that the people who find the most have the least reason to tell anyone. That is next week.
Know someone running an AI pilot right now? Forward this to them before they write the summary.
Have a take on any of this? Reply directly, I read every response.