robots.txt Gets a Shared AI Training Opt-Out: Google, Microsoft and OpenAI Sign On, Rollout Still Pending

robots.txt is getting a single, shared AI training opt-out signal. On 30 October 2026, OpenAI, Google and Microsoft announced on the same day that they would support the shared signal __usage_ai-training. It comes from an Internet-Draft of the Internet Engineering Task Force (IETF) AI Preferences working group, drafted by Kevin Kelley of Anthropic. A website need only write two lines in its robots.txt — User-Agent: __usage_ai-training plus Disallow: /, optionally scoped to selected paths — to tell every participating provider at once that its content should not be used for training, instead of maintaining a separate rule for each crawler.

The signal's limits matter just as much. It covers training only: it does not block crawling or search indexing, and it does not keep AI answers from quoting page content. Google added that a site using the new signal without mentioning Google-Extended will be treated as also blocking Google-Extended, while Google-Extended itself remains. Microsoft goes further: alongside the shared training preference, it adds a page-level training preference and a separate AI Grounding preference, the latter deciding whether content may be used to generate AI answers and so separating the two uses of training and quoting in answers.

For site owners, this one configuration is not yet enough on its own. The draft remains voluntary, is not formally endorsed by the IETF, and providers' timelines differ; until each provider actually supports it during 2027, existing crawler controls must stay in place. The real change lies in the coordination itself: where opting out of training once meant negotiating rules provider by provider, four major model providers now stand behind the same signal for the first time. Whether those two lines deliver on their promise can only be judged once the rollout timetables are met.