Toot

Written by sudo beep on 2025-01-25 at 02:27

@worik we see this crawler behaviour heaps, readthedocs wrote details up last year https://about.readthedocs.com/blog/2024/07/ai-crawlers-abuse/

I am surprised anyone expects robots.txt would be respected tbh. That's not how extractivists work.

They will pound the earth into dust to grind out every piece of glittering goodness. They don't care about consent or sustainability.

=> More informations about this toot | View the thread | More toots from xurizaemon@toot.cafe

Mentions

=> View worik@mastodon.social profile

Tags