A website is agent ready when AI agents can find it, read it without guessing, know what they are allowed to do with it, and use what it offers. In practice that comes down to 22 checks in six groups, most of them small files and headers. This checklist explains each group, which checks every site needs, and which ones only apply in specific cases.
It is the same list our free agent-readiness scan runs, so you can read along and then test your own site.
How the checks are weighted
Not every check applies to every site, and a checklist that pretends otherwise sends people chasing the wrong fixes. We sort the checks into four tiers.
| Tier | Who needs it | Weight in the score |
|---|---|---|
| Core | Every public website | Counts double |
| Advanced | Sites that want agents to use what they offer | Counts once |
| Protected APIs | Only sites where agents must log in to an API | Listed, not scored |
| Payments | Only sites where agents can buy directly | Listed, not scored |
Core checks are the foundation, so they count double. Advanced checks count once. The authentication and payment checks are listed so you know they exist, but they are never scored, because a site with no protected API or no online checkout should not lose points for leaving them out.
Discoverability: can agents find you?
- robots.txt (core). The first file any crawler or agent reads. It says what may be fetched and where your sitemap is. The format is standardized in RFC 9309.
- XML sitemap (core). Lists every page you want found, so an agent does not have to guess your URLs.
- llms.txt (core). A short Markdown guide for language models: what the site is and where the important pages are. See llmstxt.org.
- Link response headers (advanced). HTTP Link headers on the homepage, defined in RFC 8288, point agents straight at your machine-readable resources. Our post on API catalogs and Link headers explains both.
- DNS-AID records (advanced). A draft proposal for publishing agent endpoints in DNS, so agents can find them before fetching anything from your site.
Content: can agents read you cheaply?
- Markdown on request (core). When a request sends
Accept: text/markdown, the site returns a Markdown version of the page instead of HTML. - Markdown alternate link (core). A
rel="alternate"link in the page head tells agents where the Markdown copy lives. Both are covered in Markdown for agents. - Structured data (core). JSON-LD that states facts about the page and the business. See the JSON-LD types that matter.
Bot rules: do agents know your preferences?
- AI crawler rules (core). Naming AI crawlers in robots.txt makes your policy explicit instead of leaving it to a wildcard rule written before AI crawlers existed.
- Content Signals (core). A line in robots.txt stating whether your content may be used for search, for answering questions and for model training. See Content Signals explained.
Agent discovery and APIs: can agents use what you offer?
- API catalog (advanced). A well-known file, defined in RFC 9727, listing the APIs you offer and where their descriptions are.
- Agent Skills index (advanced). Ready-made instructions that teach an agent how to work with your site.
- AI catalog (advanced). A manifest of your agent-facing resources for registries and agents to index. Both are covered in agent skills and the AI catalog.
- MCP server card (advanced). If you run an MCP server, the card lets agents find it. If you do not, this check does not apply to you. See MCP servers for business.
- A2A agent card (advanced). If you run an agent that other agents can talk to, its card describes it. Again, only relevant if you have one.
- WebMCP tools (advanced). Your pages register their key actions as tools for in-browser agents. See WebMCP explained.
Authentication and payments: only if they apply
Three checks cover authentication: OAuth or OpenID discovery metadata, protected resource metadata and an auth.md file. They matter when agents need to sign in to an API you run, and not otherwise. Three more cover agent payments: UCP, ACP and machine payment declarations for sites that let agents buy directly or pay per API call.
If neither applies to you, skip them. Publishing metadata for an API or checkout you do not have would mislead agents, which is worse than leaving it out.
Keep every file true
Agent-facing files are promises. An llms.txt that lists pages you have removed, a sitemap full of redirects, an API catalog that points at a dead description or structured data that disagrees with the visible page all teach agents that your site is unreliable. Generate these files from the same source as your pages where you can, so they change when the site changes, and re-run a scan after every release that touches templates, navigation or robots.txt.
What to fix first
Start with the core checks. They are small, they help search engines as well as agents, and none of them depends on having an API.
A realistic order for most sites:
- Make robots.txt valid and add a Sitemap line. Remove any directive that is not part of the standard, because some validators reject the whole file over one unknown line.
- Publish the sitemap and llms.txt.
- Add JSON-LD to your templates.
- Decide your AI crawler policy, then write it into robots.txt with named groups and a Content-Signal line.
- Serve Markdown copies of your pages and link them from each page head.
- Then move to the advanced checks that fit your business: an API catalog if you have APIs, skills and an AI catalog if you want agents to use your site on a person's behalf, WebMCP if your pages offer actions.
Next step
Run the free agent-readiness scan on your site. It shows your score and which of the 22 checks pass, and a free account shows how to fix each gap. If you would like the work done for you, start a project.