Agentic browsing, AEO and SEO, explained
Plain guides to how AI agents and answer engines find, read and act on websites, and what to change on yours. No invented numbers, only primary sources.
robots.txt for AI crawlers: GPTBot, ClaudeBot and the rest
AI providers run separate crawlers for training, for search and for fetching a page a user asked about. robots.txt lets you treat each one differently. Here is who is who and how to write the rules.Read the postllms.txt explained: what it is and how to write one
llms.txt is a short Markdown file at your site root that tells language models what your site is and where the important pages are. It is a proposal, not a standard, and it is cheap to publish.Read the postWhat is answer engine optimization (AEO)?
Answer engine optimization is the work of making your content easy for AI assistants to find, understand and quote. Here is how answer engines pick passages and what you control.Read the postWhat is agentic browsing? A plain guide for site owners
AI agents now open web pages, read them and take actions for the people who send them. Here is what agentic browsing is, how it differs from a person browsing, and the changes that help a site work well for both.Read the postWebMCP explained: exposing your site's actions to AI agents
Instead of making an AI agent click through your interface, WebMCP lets the page hand it named tools with clear inputs. Here is what the draft standard does, how it relates to MCP servers, and what to watch for when you add it.Read the postThe agent readiness checklist: 22 checks for an AI-ready website
Twenty-two checks in six groups, from robots.txt and llms.txt to API catalogs, agent skills and WebMCP. What each one tells an AI agent, which ones every site needs, and which only matter if you run APIs or sell online.Read the postMeasuring visibility in AI answers: what you can track
There is no search console for AI answers. You can still track three honest signals: referral visits from assistants, AI crawler activity in your logs, and a fixed panel of test questions.Read the postMarkdown for agents: serving Markdown with content negotiation
Agents read Markdown far more cheaply than HTML. Serve a Markdown copy when a request asks for text/markdown, keep HTML the default, and link the copy from every page.Read the postMCP servers for business: when you need one and how to keep it safe
An MCP server exposes your systems to AI agents through one standard interface. It is worth building when agents need to act inside your business, not just read about it.Read the postLighthouse's agentic browsing audit: what it checks and how to pass
Recent Lighthouse versions add an Agentic Browsing category that checks how well AI agents can read and use a page. Here is what it looks at and how to fix what it finds.Read the postJSON-LD schema that matters: the types worth adding and how to link them
A short list of schema.org types covers most business sites. Link them into one graph with @id, keep them true to the visible page, and validate before you ship.Read the postHow to write content AI assistants will quote
Assistants quote passages, not pages. Write each section so it answers one question on its own, in plain words, with the answer first.Read the postHow to make an ecommerce store agent-ready
Shopping agents compare products, read policies and fill carts for people. Stores that give them clean data and a checkout they can complete, with the shopper confirming, get chosen.Read the postHow AI agents read a web page: HTML, accessibility tree and Markdown
An agent never sees your page the way a person does. It reads the HTML, the accessibility tree, a screenshot or a clean Markdown copy. Here is what each channel sees, what breaks it, and how to make your pages readable in all of them.Read the postForms and checkouts AI agents can complete
An agent fills a form the way a screen reader user does: by name, label and role. Build for that, keep steps predictable, and put a person in front of every payment.Read the postFAQ and structured data for answer engines
Structured data states the facts on a page in a form machines do not have to guess. Here is which schema.org types matter for answers, and the one rule you cannot break.Read the postContent Signals explained: search, ai-input and ai-train in robots.txt
Content Signals add one line to robots.txt that says how your content may be used once it is fetched: for search, as input to AI answers, or for model training.Read the postAgent skills and the ai-catalog: telling agents how to use your site
Agent skills are short instruction files that teach an agent to do one job with your site. A discovery index lists them, and the draft ai-catalog lets registries find them.Read the postAPI catalogs and Link headers: how agents find your APIs
Two small standards tell agents where your APIs are: an HTTP Link header on the homepage, and a well-known api-catalog file that lists each API and its description.Read the postAEO vs SEO vs GEO: what the labels mean
SEO, AEO and GEO are overlapping industry labels, not separate disciplines. A plain comparison of what each one targets and the work they share.Read the postSee how your own site scores
The free agent-readiness scan checks the files, headers and endpoints this blog writes about, and tells you what to fix first.