Glossary
User-agent
The identifying string a client sends so servers can recognize the crawler or browser.
Definition
AI crawlers use distinct user-agents (GPTBot, ClaudeBot, PerplexityBot, etc.). robots.txt and CDN rules often key off those strings.
In practice
- Use precise language on public pages so assistants and crawlers share the same meaning
- Pair definitions with crawlable HTML, metadata, and intentional robots policy
- Measure outcomes (mentions, citations, pillar scores) after you change signals
See also
Next step
Browse the glossary index, read what is AI visibility?, or measure your site.
Related
FAQ
What is User-agent?
The identifying string a client sends so servers can recognize the crawler or browser.
Why does User-agent matter for AI visibility?
AI crawlers use distinct user-agents (GPTBot, ClaudeBot, PerplexityBot, etc.). robots.txt and CDN rules often key off those strings.
How can BatSignal help?
BatSignal measures crawl access, content readiness, ChatGPT presence, and related signals so teams can improve AI visibility with evidence—not guesswork.