About
Google-Extended is not a separate crawler: it is a robots.txt control token. Disallowing it tells Google not to use your content for Gemini training and grounding, while normal Google crawling and indexing continue via Googlebot.
Gemini training and grounding
Details
- Type
- AI training control
- Operator
- Access
- Direct — crawls on the operator’s own schedule
- Traffic rank