# ================================ # Spekit robots.txt (Webflow setup) # ================================ # --- Core Crawlers --- User-agent: * Allow: / # --- AI Answer Engine Crawlers --- User-agent: GPTBot User-agent: ChatGPT-User User-agent: ClaudeBot User-agent: PerplexityBot Allow: / # --- Google / Gemini --- User-agent: Googlebot Allow: / # --- Sitemap & Host --- Sitemap: https://www.spekit.com/sitemap.xml Host: https://www.spekit.com Sitemap: https://www.spekit.com/sitemap.xml