# ════════════════════════════════════════════════════════════ # LittleDot — robots.txt # https://littledotapp.com # Last updated: 2026-05-05 # ════════════════════════════════════════════════════════════ # Default — allow all bots User-agent: * Allow: / # Block crawling of internal/legal/admin paths Disallow: /wp-admin/ Disallow: /wp-login.php Disallow: /wp-json/ Disallow: /admin/ Allow: /api/sitemap-doctors.xml Disallow: /api/ Disallow: /cart/ Disallow: /checkout/ Disallow: /my-account/ Disallow: /*?*sessionid= Disallow: /*?*sid= Disallow: /*?*utm_* Disallow: /*?*fbclid= Disallow: /*?*gclid= # Allow CSS/JS/images for proper rendering Allow: /wp-admin/admin-ajax.php Allow: /*.css$ Allow: /*.js$ Allow: /*.png$ Allow: /*.jpg$ Allow: /*.jpeg$ Allow: /*.gif$ Allow: /*.svg$ Allow: /*.webp$ # Crawl delay for less aggressive bots User-agent: AhrefsBot Crawl-delay: 10 User-agent: SemrushBot Crawl-delay: 10 User-agent: MJ12bot Crawl-delay: 10 # ════════════════════════════════════════════════════════════ # AI / LLM crawlers — explicitly allowed for AEO/GEO visibility # (Updated 2026-05-05 — flipped from default-block to explicit-allow) # ════════════════════════════════════════════════════════════ # OpenAI — ChatGPT search and training User-agent: GPTBot Allow: / Disallow: /api/ Disallow: /admin/ User-agent: ChatGPT-User Allow: / Disallow: /api/ User-agent: OAI-SearchBot Allow: / Disallow: /api/ # Anthropic — Claude User-agent: ClaudeBot Allow: / Disallow: /api/ User-agent: Claude-Web Allow: / Disallow: /api/ User-agent: anthropic-ai Allow: / Disallow: /api/ # Perplexity — AI answer engine User-agent: PerplexityBot Allow: / Disallow: /api/ User-agent: Perplexity-User Allow: / Disallow: /api/ # Google — Gemini and AI Overviews User-agent: Google-Extended Allow: / Disallow: /api/ User-agent: GoogleOther Allow: / Disallow: /api/ # Apple — Apple Intelligence User-agent: Applebot-Extended Allow: / Disallow: /api/ # Meta — Llama, Meta AI User-agent: Meta-ExternalAgent Allow: / Disallow: /api/ User-agent: FacebookBot Allow: / Disallow: /api/ # Microsoft — Copilot/Bing AI User-agent: BingPreview Allow: / Disallow: /api/ # DuckDuckGo AI User-agent: DuckAssistBot Allow: / Disallow: /api/ # CommonCrawl (used as training data by many LLMs) User-agent: CCBot Allow: / Disallow: /api/ Disallow: /admin/ # ByteDance / TikTok AI User-agent: Bytespider Allow: / Disallow: /api/ # Mistral AI User-agent: MistralAI-User Allow: / Disallow: /api/ # Diffbot (used by enterprise AI products) User-agent: Diffbot Allow: / Disallow: /api/ # Cohere User-agent: cohere-ai Allow: / Disallow: /api/ # YouBot (You.com) User-agent: YouBot Allow: / Disallow: /api/ # ════════════════════════════════════════════════════════════ # Sitemaps # ════════════════════════════════════════════════════════════ # Opis sitea za LLM crawlere: https://littledotapp.com/llms.txt # Glavni sitemap indeks (stranice, HR/EN blog, pitanja i odgovori, # kategorije i doktorski profili) — generira ga Astro iz WordPress CMS-a. Sitemap: https://littledotapp.com/sitemap-index.xml # Host (preferred canonical hostname) Host: https://littledotapp.com