Ancestors

Toot

Written by OSNews on 2025-01-15 at 15:13

Nepenthes: a dangerous tarpit to trap LLM crawlers

If you don't want OpenAI's, Apple's, Google's, or other companies' crawlers sucking up the content on your website, there isn't much you can do. They generally don't care about the venerable robots.txt, and while people like Aaron Schwartz were legally bullied into suicide for downloading scientific articles using a guest account, whil

https://www.osnews.com/story/141545/nepenthes-a-dangerous-tarpit-to-trap-llm-crawlers/

[#]Internet

=> More informations about this toot | More toots from osnews@mstdn.social

Descendants