@Friendica Support
#fediAdmin #fediVerse #AI #KI
Text for robots.txt to disallow access for known AI crawlers:
User-Agent: GPTBot
User-Agent: ClaudeBot
User-Agent: Claude-Web
User-Agent: CCBot
User-Agent: Applebot-Extended
User-Agent: Facebookbot
User-Agent: Meta-ExternalAgent
User-Agent: diffbot
User-Agent: PerplexityBot
User-Agent: Omgili
User-Agent: Omgilibot
User-Agent: ImagesiftBot
User-Agent: Bytespider
User-Agent: Amazonbot
User-Agent: Youbot
Disallow: /AI / LLM User-Agents: Blocking Guide
Find out how to block your content from being used for AI/LLM training with robots.txt. Created by ex-Google engineer Fili.robotstxt.com