>This is not that much different in programming, it just takes longer to find out.

The primary difference is that it's much easier for the AI to figure out itself if it's wrong on the programming side. Knowledge workers don't have automated testing for example.

The end result is the knowledge worker needs to be more heavily involved in the process and be more mindful of mistakes which makes it a worse experience for the user and generally less productive than it is for programmers.

That's true. And: as long as you don't hit production mistakes are cheap and quick to fix. Not so in many other fields.

It is more that knowledge workers are less sophisticated at using AI. It is not that hard to check whether sources are real, using AI or other tools, for example. Something like functional testing will come to non programming tasks.

The major difference is that people routinely ship shoddy, broken products in the world of software with few consequences, beyond needing to patch it later. Try the same thing with a court filing and you’re out on your ass.

We're still early in AI.

All of this will sort itself out and eventually bio, legal, software will all just work.

Don't hold onto any single assumption about the world. Things are going to be weird.

Well hang on, "just work" implies an objective standard of correct vs incorrect. No such standard exists. I'm sure that there will be legal AIs that reliably do... something... but the whole point of the legal system is to allow space for humans to argue their case. If an AI "does legal" deterministically, without human intervention, how can it be said to be "working" if its very use defeats the purpose of the system it has learned to navigate so expertly?

That’s utopian thinking unburdened by reality or the need to explain how this will work. I have no interest in that.

We aren’t talking about the new version of SV spiritualism here, we’re talking about systems people actually care about.

This was true before AI too.

In practice, lawyers generally don't ever disbarred except if they lie to clients or steal from them.