Hacker News (curated)new | past | comments | ask | show | jobs| show hidden

That would be a different bot, one written specifically for your site. Mainly we're discussing the dumb ones that just crawl all possible http links


No I think this is the same case. It seems to just be following local links on the SERP. From a search for jack parsons, you can find hyperlinks to the sorts of requests it's making.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact | github