Item

tarpitt · 2026-08-30T15:35:00.000Z

Maybe you could have a system that heuristicially detects when an crawler is making the request and then feeds them a modified page, itself generated from an LLM, that injects vulnerabilities and bad code and discussion and such.

NooneAtAll3 · 2026-08-30T15:46:29.000Z

it's hard to separate spambot that only accesses 3-5 links per IP and a legit user. Changing content for legit user can be devastating

inigyou · 2026-08-30T19:58:30.000Z

Put a cookie wall in front. The bot will either load the cookie and have a persistent identifier, or not load the cookie and not get in

jay_kyburz · 2026-08-30T21:12:56.000Z

why is this not the answer? then you can also rate limit each cookie as well.

sgsjchs · 2026-08-30T23:40:47.000Z

it'll load the cookie, make one request, move to a different ip, load the cookie, make one request, move to a different ip, ...