One thing I've seen with AI is AI as used by software developers is quickly diverging from AI that's used by knowledge workers.
Knowledge workers use it primarily to "augment" their work, answer questions with AI, get it to create documents, etc. But it's usually 1 individual to 1-2 conversation threads.
Meanwhile in software development were running fleets of agents. At work we have it setup where firing off one agent request will eventually lead to like 10-20 AI calls at different parts of the process.
I make this distinctions because it changes how each group thinks about AI. For software devs it just feels like "moving up the stack". Yeah you're not writing code anymore but there is still plenty of work to do and problems to solve within your "software factory". Contrast that with knowledge workers that feel like the AI is making all of their decisions for them.
I almost think of this as a factory worker versus an artisan. The software devs are "factory workers" where we used to be on the floor but now we're managing the machines that took over the work. There are still tons of interesting problems at that stage and you still feel like you have worth. Contrast that with a knowledge worker that's more of an artisan in this analogy. For them they aren't running a factory but instead watching a robot do the work they used to do while they sit idle and watch it.
"Contrast that with a knowledge worker that's more of an artisan in this analogy. For them they aren't running a factory but instead watching a robot do the work they used to do while they sit idle and watch it".
I think your general thesis about how AI is used of agents vs. the general chat interface is true, but not the rest.
Your knowledge worker example is where people fail when using AI, and it is usually very obvious.
Knowledge workers who know what they're doing recognized very quickly that AI often goes down very bad rabbit holes, often cites incorrect sources, makes up sources, or will use a random Facebook source or Reddit post as an authority on some topic. In legal circles you have to fight it to understand what venue(s) apply and to get it to stick to the right ones, in medical research it will sometimes pull in crackpot papers from left field and intermix them with well defined research.
People in the knowledge realm caught on very quickly that you have to use AI iteratively and constantly back check what you get and correct it. Let it run on its own and it will give you a legal theory based on some weird case in Kentucky that has no bearing to your issue with a court in Vermont.
This is not that much different in programming, it just takes longer to find out.
Using AI is like using adaptive cruise control: you're still in charge of where the vehicle will ultimately end up and if you decide to go take a nap you will wake up in the hospital or end up in the morgue.
It's true though that the 'time between interventions' is lengthening steadily.
The major difference is that people routinely ship shoddy, broken products in the world of software with few consequences, beyond needing to patch it later. Try the same thing with a court filing and you’re out on your ass.
Well hang on, "just work" implies an objective standard of correct vs incorrect. No such standard exists. I'm sure that there will be legal AIs that reliably do... something... but the whole point of the legal system is to allow space for humans to argue their case. If an AI "does legal" deterministically, without human intervention, how can it be said to be "working" if its very use defeats the purpose of the system it has learned to navigate so expertly?
>This is not that much different in programming, it just takes longer to find out.
The primary difference is that it's much easier for the AI to figure out itself if it's wrong on the programming side. Knowledge workers don't have automated testing for example.
The end result is the knowledge worker needs to be more heavily involved in the process and be more mindful of mistakes which makes it a worse experience for the user and generally less productive than it is for programmers.
It is more that knowledge workers are less sophisticated at using AI. It is not that hard to check whether sources are real, using AI or other tools, for example. Something like functional testing will come to non programming tasks.
"People in the knowledge realm caught on very quickly that you have to use AI iteratively and constantly back check what you get and correct it."
If lawyers and article writers are in the knowledge realm. Then your thesis has plenty of counter-examples of people who just use AI to create stuff and don't correct it.
We have 5 star restaurants, fast food and street meat yet we still have kitchens. We can go to the super market to buy eggs, and produce yet I know people who have gardens to grow their own food and a scant few have chickens for eggs.
I don't actually use AI much at all simply because in my line of work the knowledge is industry specific and niche - an AI can't tell you how to fix a proprietary machine. I have used it a few times but to do dumb things like write some Excel script or some other piece of software I have no interest or need for learning.
I usually have a high-level agent drafting plans, then spawning sub-agents to build parts of the plan, other sub-agents to check the work of those agents. Then the PRs have different code review agents looking at the code, on top of the agents for people on my team also reviewing my code.
And I'm probably on the more amateurish end of AI development. As I learned these techniques from YouTube.
When you hear about the work Anthropic researchers are doing, they seem to be using hundreds or thousands of agents to solve problems.
They are doing that because they are running into hardware limits with single agents. If one agent made correct decisions and explored spaces efficiently, they would not need hundreds of thousands of agents. It's is the luxury of great wealth that they can produce a large number of possibly defective samples and then select the ones that are not defective.
> When you hear about the work Anthropic researchers are doing, they seem to be using hundreds or thousands of agents to solve problems.
There's a reason they're Anthropic researchers. The value proposition tanks when you actually have to foot the bill for the tokens you use.
Subagent flows can improve overall performance, but it generally also comes with disproportionately high token use. That gets worse with parallel subagents that are often doing the same things at the same time and cannot share a cache.
Yeah, I very much dispute the characterization GP made as to how software devs are working. I know some are. But I do not believe it's the majority of people in the field, not by a long shot.
I think you are onto something. I think most agents I have up is two and even that I am not super comfortable not carefully managing. Otoh, buddy has agents galore running his network and everything giving him updates in near real time. I just.. don't have that much .. I want to say faith, but that suggests unshakeable religious belief.. confidence. I don't have that kind of confidence. There is too much stuff connected now ( taxes, projects, work, fun ). I am slowly starting to separate everything even that takes some careful planning.
That's because current AI models are incapable of doing knowledge workers' jobs. At least for me as an attorney, Harvey is only actually useful when it comes to helping me sort through large amounts of data. Even then, I can't trust its judgment (it has shown to be very poor) so it's only so useful
I use the AI in the "knowledge worker" way you described. However I still feel it's augmenting my work, because I don't ask or expect it to make decisions for me. I use it to gather sources, to criticize my ideas, to generate summaries targeted to people expectations, and so on. Yes of course it's gonna influence my work with their own biases, but I don't think it is that important - my own biases are otherwise still there...
Language is a tool for abstraction, and AI is just an imperfect tool for working with language.
In the sense that "down the stack" is well-captured by existing language, AI is absolutely moving programmers up the stack. I can describe my goals using high-level abstractions, and the low-level implementation is somewhat constrained by that.
How well constrained it is depends on how well I described the problem and the quality of the model - much like how good human-written code is depends on how clearly the authors were thinking when they designed the structure.
This looks like the classic "AI can do everybody else's job but mine still needs me".
I know knowledge workers who throw what you give them into the slop machine and return the slop, but I don't need them, I can feed the slop machine myself. I still need those who have experience and taste that the machine hasn't got yet.
Knowledge workers use it primarily to "augment" their work, answer questions with AI, get it to create documents, etc. But it's usually 1 individual to 1-2 conversation threads.
Meanwhile in software development were running fleets of agents. At work we have it setup where firing off one agent request will eventually lead to like 10-20 AI calls at different parts of the process.
I make this distinctions because it changes how each group thinks about AI. For software devs it just feels like "moving up the stack". Yeah you're not writing code anymore but there is still plenty of work to do and problems to solve within your "software factory". Contrast that with knowledge workers that feel like the AI is making all of their decisions for them.
I almost think of this as a factory worker versus an artisan. The software devs are "factory workers" where we used to be on the floor but now we're managing the machines that took over the work. There are still tons of interesting problems at that stage and you still feel like you have worth. Contrast that with a knowledge worker that's more of an artisan in this analogy. For them they aren't running a factory but instead watching a robot do the work they used to do while they sit idle and watch it.