Hacker News (curated)
new
|
past
|
comments
|
ask
|
show
|
jobs
| show hidden
Symbiote
6 hours ago
|
parent
|
context
Meta is an exception. They intentionally ignore robots.txt rules.
(No second chances, I blocked their whole IP range.)
inigyou
3 hours ago
Everyone ignores robots.txt because every robots.txt blocks everyone.
nubinetwork
2 hours ago
[hidden]
I allow Google, Bing, Yahoo, Kagi, and Apple on my personal blog... who else do I need to allow? Claude, OpenAI, Alibaba, and perplexity can go to hell.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
|
github
(No second chances, I blocked their whole IP range.)