> "And offloading that labor, regardless of whether it's taxing or not, is undercutting my ability to be a moral actor."

The objection in the article is specifically about offloading thinking to technology, which is something we've been doing for decades, or depending on your definition, since we first started using tools.

If offloading cognitive labor is the core problem, then what makes AI fundamentally different from the countless other technological advances that have allowed us to offload cognitive labor?

We haven't been offloading thinking. We have been offloading the calculation. Someone had to think to implement a system in the past. The system abided by those constraints, or could be checked against them as it was deterministic (vast majority). AI is non-deterministic and presents a shift to where people are using it like a magic 8-ball to think for them, even if the mechanism is probablistic calculation of word patterns.

By chopping out the first sentence of the quote, you lose the context of exactly what kind of labor he's talking about. Which is useful if you want to construct a strawman version of his argument by suggesting he might be morally opposed to four function calculators, too, but not so useful if the goal is to engage with the (presumably incomplete) version of his position that the article presents in good faith.

So let's look at that preceding sentence so that we can get a better sense of what kind of labor he was talking about:

> The process and work involved in thinking through situations, through my work, through my ethics, through the way I handle other people, is a key component in being able to develop myself as a moral person.

"thinking through situations", "my ethics", "the way I handle other people", "develop myself as a moral person". This doesn't read to me like he's taking a Butlerian Jihad-type position that humans have a moral responsibility to do their own arithmetic. We're clearly talking about higher-level, more abstract thinking and decision-making that might carry real moral weight.

And that brings us to a difference that I think he might be seeing but that you seemed to be claiming does not exist when you said, "My main issue with these people is that they think AI/LLMs are different than everything that came before them." GPT-style decoder LLMs are the first time we've ever really seen people trying to rely on computers to make abstract and open-ended decisions such as drawing up business plans, drafting memoranda, or doing deep research-type tasks on any sort of scale.

I think that you're probably aware that there is a a noticeable difference between how people are thinking about LLMs now versus how they did back when GPT 2.0 was the latest and greatest. The fact that we can construct a hypothetical continuum of possible models between GPT 2.0 and GPT 3.5 Turbo and suggest that, in this counterfactual scenario, it might not be possible to draw a hard line that cleanly separates the tech itself into two discrete categories, is fundamentally missing the point. His concern was never stated to be about the technology itself; it was about how he was being asked to use it.

Exactly. Recent LLMs are in a different category and it's disingenuous to pretend they're not. Wherever you decide to draw the line, GPT-3 and more advanced chatbots are clearly on the far side of it.

Sorites doesn't actually refute the OP. The paradox is rooted in a failure to distinguish between aggregates and integral wholes. A heap is not really a natural kind or an integral whole. It is just a convenient label to describe a vague range of collections for practical purposes, but it is not what you a "thing" in its own right. There is no intrinsic tendency of a heap to be a heap, whereas in integral wholes there are. You cannot draw a line between a heap and a non-heap, because there is no such thing as a heap in any objective sense. There is no essential difference, only an accidental one.

Computers and computer programs are much more like heaps than integral wholes. There is no essential difference between what we conventionally call "AI" and any other computer program. The manner in which we have cordoned off "AI" from "non-AI" has more to do with human purposes and perspectives rather than any objective facts on the ground.

The difference between the Sorites paradox and the continuum fallacy is explained right there in the section on the continuum fallacy that I linked to.

I realize that's embedded within a larger page on the Sorites paradox instead of being given its own page, but I'm not super interested in being that preoccupied with Wikipedia's editorial decisions.