Hacker News (curated)new | past | comments | ask | show | jobs| show hidden

I made it to a third round interview at anthropic in 2024 and they had me build a web crawler as their programming test. Part way through I started on making it respect robots.txt and I could immediately tell they were no longer interested in me.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact | github