Something went wrong. Try again.
Monorepo for Tangled tangled.org
Something went wrong. Try again.
123456789101112131415161718192021222324252627282930313233# Bytespider is an aggressive scraper that ignores crawl etiquetteUser-agent: BytespiderDisallow: /# Explicitly allowed AI search & answer enginesUser-agent: GPTBotUser-agent: ChatGPT-UserUser-agent: ClaudeBotUser-agent: PerplexityBotUser-agent: Google-ExtendedUser-agent: CCBotAllow: /Disallow: /settingsDisallow: /*/settingsDisallow: /*/compareDisallow: /*/forkDisallow: /searchDisallow: /*/searchCrawl-delay: 1# Default crawler rulesUser-agent: *Allow: /Disallow: /settingsDisallow: /*/settingsDisallow: /*/compareDisallow: /*/forkDisallow: /searchDisallow: /*/searchCrawl-delay: 1Sitemap: https://tangled.org/sitemap.xml