How did you manage search engine crawlers for the past few decades? And why are ...

		OutOfHere 9 months ago \| parent \| context \| favorite \| on: Web-scraping AI bots cause disruption for scientif... How did you manage search engine crawlers for the past few decades? And why are AI crawlers functionally different? They aren't.