Meta Crawler
Meta's suite of crawlers for Facebook, Instagram, and WhatsApp link previews.
What is Meta Crawler?
Meta Crawler encompasses several crawler identities including facebookexternalhit, Facebot, FacebookBot, and meta-externalagent. These crawlers generate link previews when URLs are shared on Facebook, Instagram, WhatsApp, and Messenger.
When someone shares a link on any Meta platform, these crawlers fetch the page to extract Open Graph metadata, images, and descriptions. This happens billions of times daily, making Meta's crawlers among the highest-volume bots on the web.
Beyond link previews, meta-externalagent also collects data for Meta's AI training (for Llama models and Meta AI). This dual purpose means site owners need to consider both social sharing functionality and AI training implications when managing Meta crawlers.
User-Agent Strings
These are the known user-agent patterns used by Meta Crawler. Use them to identify this crawler in your server logs or configure robots.txt rules.
robots.txt example:
User-agent: facebookexternalhit Disallow: /private/ Allow: /
Detection on the Switch Network
2
Sessions detected
3
Total events
95%
Avg confidence
First detected: 2/25/2026 · Last seen: 3/5/2026
How to Manage Meta Crawler
Blocking breaks link previews on Facebook, Instagram, WhatsApp, and Messenger.
meta-externalagent also feeds AI training — manage separately if possible.
Ensure Open Graph tags are properly configured for optimal previews.
Use Switch to differentiate social preview vs. AI training visits.
Start managing Meta Crawler today
Switch detects, tracks, and lets you build custom journeys for Meta Crawler and 35+ other AI agents and crawlers. Set up in five minutes.
Get Started Free