I really enjoyed the writing style in this article.
And the bot progression from "alter user agent" to "change IP addresses" to providers having to ban whole subnets, whole ASNs, and realizing "proxy SDK monetization" is a thing mirrors threat actor progression from the time before LLMs.
It feels like this progression of increasingly drastic measures to circumvent the protections of a computer system ought to be enough to establish criminal intent and get some of the people running those crawlers into prison.
turns out the overlap between "people who can't configure their webserver to serve at wire speed" and "people who can get law enforcement to take them seriously" is the empty set
And the bot progression from "alter user agent" to "change IP addresses" to providers having to ban whole subnets, whole ASNs, and realizing "proxy SDK monetization" is a thing mirrors threat actor progression from the time before LLMs.