# Note: this file is based on the robots.txt over at
#       https://seirdy.one/robots.txt

## "Social" Media

User-Agent: Facebot
Disallow: /

## Misc

User-Agent: IonCrawl
User-agent: PiplBot
Disallow: /

## Marketing or SEO

User-agent: BLEXBot
User-agent: AhrefsBot
User-agent: MJ12bot
User-agent: Barkrowler
User-agent: SemrushBot
User-agent: dotbot
User-agent: DataForSeoBot
User-agent: serpstatbot
Disallow: /

## Intellectual "Property" scanners

User-Agent: TurnitinBot
User-Agent: NPBot
User-Agent: SlySearch
User-agent: CheckMarkNetwork/1.0 (+https://www.checkmarknetwork.com/spider.html)
User-agent: BrandVerity/1.0
Disallow: /

## AI-Training data scrapers

User-Agent: ChatGPT-User
User-agent: GPTBot
User-agent: Google-Extended
User-agent: anthropic-ai
User-agent: Claude-Web
User-Agent: FacebookBot
User-Agent: AI2Bot
Disallow: /



User-agent: *
Allow: /
Sitemap: https://slatecave.net/sitemap.xml