AI crawler
Yandex — AI crawler guide
What Yandex (Yandex) does, how to configure robots.txt, and how BatSignal measures access.
Operator and category
| Field | Value |
|---|---|
| User-agent | Yandex |
| Operator | Yandex |
| Category | search |
Purpose
Yandex search crawler important for certain geographic markets.
robots.txt guidance
Allow when Yandex is a meaningful market; geo-specific bot policy can differ from US-centric AI bots.
Measurement
Include in robots policy reviews for international sites.
Also read robots.txt for AI crawlers and AI crawl access.
Related bots
- GPTBot — training
- OAI-SearchBot — search
- ChatGPT-User — agent
- ClaudeBot — training
- Claude-Web — search
- anthropic-ai — training
Related
FAQ
What is Yandex?
Yandex is operated by Yandex. Category: search. Yandex search crawler important for certain geographic markets.
Should I allow Yandex in robots.txt?
Allow when Yandex is a meaningful market; geo-specific bot policy can differ from US-centric AI bots.
How does BatSignal treat Yandex?
Include in robots policy reviews for international sites.