How to Block cohere-ai
Complete guide to blocking cohere-ai (Cohere) from crawling your website using robots.txt, server configuration, and Switch workflows.
Should You Block cohere-ai?
cohere-ai collects data for AI model training. Blocking it prevents your content from being used in Cohere's AI products without affecting your search visibility.
This is a common and recommended action for sites that want to control how their content is used in AI training.
Blocking Methods
1robots.txt
High for cooperative crawlersAdd a Disallow rule for cohere-ai's user-agent string in your robots.txt file. This is the standard, cooperative method that well-behaved crawlers respect.
2Server-side UA filtering
HighConfigure your web server (nginx, Apache, Cloudflare) to reject requests matching cohere-ai's user-agent patterns. This blocks at the network level before your application processes the request.
3Switch Journey Workflows
Highest — granular, real-time controlCreate a custom journey in Switch that detects cohere-ai and routes it to a block action, challenge, redirect, or modified content — without touching your server configuration.
robots.txt — Block cohere-ai
Add the following to your robots.txt file (at the root of your domain) to block cohere-ai:
User-agent: cohere-ai Disallow: / User-agent: CohereBot Disallow: /
robots.txt — Allow with Restrictions
Alternatively, allow cohere-ai on most pages while blocking specific directories:
User-agent: cohere-ai Disallow: /private/ Allow: / User-agent: CohereBot Disallow: /private/ Allow: /
cohere-ai User-Agent Strings
Use these patterns to identify cohere-ai in your server logs or firewall rules:
Frequently Asked Questions
Does blocking cohere-ai affect my Google search rankings?
No. Blocking cohere-ai does not affect your Google search rankings. Only blocking Googlebot impacts Google Search visibility.
Does cohere-ai respect robots.txt?
Yes, cohere-ai respects robots.txt directives. Adding a Disallow rule for its user-agent will prevent it from crawling blocked paths.
Can I allow cohere-ai on some pages but not others?
Yes. Use robots.txt to disallow specific directories, or use Switch journey workflows for granular page-level control with conditional logic.
Go beyond robots.txt
Switch detects cohere-ai in real-time and lets you build custom journey workflows — block, challenge, redirect, or serve modified content. No server changes required.
Get Started Free