OpenAI, Google and Microsoft backed a shared way for website owners to refuse use of their content for generative AI training on October 30. The proposal could let publishers, independent creators and small businesses express that preference once in a site's robots.txt file, instead of relying entirely on separate company controls.
The announcements describe a transition that will unfold over months. Website owners still need their existing controls while providers implement the new preference.
What the shared signal does
The Internet-Draft defines a special policy token, __usage_ai-training, for robots.txt. A site can express a refusal for the whole site or selected paths. The draft's whole-site refusal uses these two lines:
User-Agent: __usage_ai-training
Disallow: /This expresses a preference about how content is used. It does not itself block a crawler from requesting pages. The draft explicitly keeps usage preferences separate from rules governing crawl access. Its status is an individual Internet-Draft, a work in progress, rather than a completed IETF standard.
Training preferences, crawler access and eligibility to appear in search or AI answers can have different effects and require different settings.
Different rollout dates
| Provider | Announced timing | What website owners should know |
|---|---|---|
| OpenAI | Coming months | Existing crawler controls continue to work. More implementation details will follow. |
| Aiming for 2027 | The shared refusal interacts with Google-Extended. | |
| Microsoft | February 1, 2027, as content is crawled | New training and separate grounding preferences begin taking effect. |
OpenAI says the voluntary signal will supplement its existing controls. It has not given a specific launch date.
Google says that, when a site expresses the shared training refusal without explicitly specifying Google-Extended, it will interpret that refusal as disallowing Google-Extended. Company-specific preferences will continue to work. Explicitly allowing Google-Extended keeps content eligible for grounding in Gemini Apps, where web content can help inform answers. Eligibility does not establish that a site will receive traffic.
Microsoft's deadline for older controls
Microsoft's plan separates training from grounding, which means using web content to inform generated answers. Neither preference changes eligibility for traditional Bing search results, subject to normal crawl and indexing requirements.
Sites currently using NoArchive or NoCache face a March 1, 2027 deadline. Microsoft will stop honoring those older tags for training and grounding after that date. To maintain training restrictions, publishers must publish the new training preference.
Grounding requires separate page preferences. Eligible sites also need to record a temporary grounding choice in Bing Webmaster Tools to bridge the wait for recrawling. That transition option opens December 15, 2026, for sites already using the older tags on that date. Temporary protection lasts until the page is recrawled or September 30, 2027, whichever comes first. Microsoft's announcement includes the migration steps and configuration examples.
