@aaron But this will also trap genuine search engines not using the web as LLM fodder... You could write a robots.txt that lets Googlebot through while allowing others.
https://seirdy.one/posts/2021/03/10/search-engines-with-own-indexes/
=> More informations about this toot | View the thread | More toots from midgard@framapiaf.org
=> View aaron@zadzmo.org profile
text/geminiThis content has been proxied by September (3851b).