Relying on robots.txt to scare away crawlers is like using a 'no trespassing' sign instead of a lock. No legislation will change that as long as there's competitive advantage in ignoring the file. If you're serious about preventing parsing, use CAPTCHAs.
Or, if you're serious about your information being used (and about not annoying me when I try to contribute to #rssbridge), let it go.
=> More informations about this toot | View the thread | More toots from sqrtminusone@bsd.cafe
=> View rssbridge tag This content has been proxied by September (ba2dc).Proxy Information
text/gemini