How to tell AI crawlers whether your pages may be used for search, answers or training.
Opening paragraph: the direct answer to the title in two or three sentences, so a reader, or an AI answer, gets it straight away.
A second paragraph that says why it matters to the business: what a buyer or an agent misses when this goes wrong.
First section heading, phrased as the reader's question
A short paragraph that explains the point in plain words, with one concrete example from a real site.
- A specific step the reader can take
- A second step, with the tool or file it involves
- A third step, and how to check it worked
Second section heading
Where a technical detail helps, it goes in a short code block like the one below.
User-agent: OAI-SearchBot
Allow: /What to do next
A closing paragraph that points to the next step: check your own site, or talk to an engineer.