Discoverability
robots.txtCore
The first file any crawler or agent reads. It says what may be fetched and where the sitemap is.
XML sitemapCore
Lists every page you want found, so an agent does not have to guess your URLs.
llms.txtCore
A short Markdown guide for language models: what the site is and where the important pages are.
Link response headersAdvanced
HTTP Link headers on the homepage point agents straight at your machine-readable resources.
DNS-AID recordsAdvanced
Lets agents find your agent endpoints through DNS, before fetching anything from your site.