Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> As for AI scrapers impersonating googlebot I feel like yes they'd definitely start doing that, unless the risk of getting sued by google is too high?

Google releases the Googlebot IP ranges[0], so you can makes sure that it's the real Googlebot and not just someone else pretending to be one.

[0] https://developers.google.com/crawling/docs/crawlers-fetcher...



Oh good idea!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: