Agent readiness, for engineering leaders

How is the scoring actually built?

The scorer is a pure function: score(pages, config, rubric_version) → dimension_scores. No network calls, no model inference, no randomness. It is unit-tested against fixture sites, so an unchanged site returns a byte-identical result.

Anything requiring judgement — whether a section genuinely answers its heading, whether entity descriptions are semantically consistent — runs separately, at temperature zero with a pinned model version, and produces flags with evidence rather than points. Fifteen of the fifty-five checks work this way and none of them touch the number.

The reason is narrow and worth stating: if a model can move the score, a month-on-month improvement is not a measurement. The determinism exists to make the before-and-after meaningful, not for its own sake.

What can you verify yourself?

Every Tier 1 check maps to something reproducible. The report names the command.

$ curl -sI -H "Accept: text/markdown" https://yoursite.com/ | grep -i "content-type\|vary"
$ curl -s https://yoursite.com/pricing | grep -c "<h1"
$ curl -s https://yoursite.com/robots.txt | grep -iA2 gptbot
$ curl -sI https://yoursite.com/llms.txt | head -1
$ curl -s https://yoursite.com/ | grep -o 'rel="alternate"[^>]*'

If our result and your result disagree, ours is wrong and we want the report. Every lost point stores the URL and the failing test for exactly this reason.

What we do not claim

• That the score predicts citation rates. Not empirically validated. The weights are informed judgement and we say so in the rubric.
• That llms.txt works. Contested, and Google’s own advocate is the loudest sceptic. It is one check of fifty-five and weighted accordingly.
• That Tier 2 output is stable forever. Providers update models. We pin versions, record the string with every flag, and note upgrades in the report rather than letting analysis shift silently.
• That we monitor everything AI says about you. Nobody can. Google AI Overviews has no API. We sample a defined query set and describe it as sampling.

If a competitor will not give you an equivalent list, that is informative.

How does it fit into CI?

The audit is a containerised service with a REST API and an MCP server. The usual pattern is a run against your preview URL on deploy, with the score and per-dimension deltas posted back to the pull request, failing the build on a regression in dimensions you have chosen to gate on.

Your coding agent can also consume the MCP server directly: run_audit, then get_remediation for each failure, writing the generated files into the repo for you to review as a normal diff.

Where this is contested

The standards are not ratified. WebMCP is an origin trial, agents.json is early, and some of what we audit will not survive. Your rubric version is pinned precisely so that our changing opinion does not silently change your trend line.

The second honest risk: Google may ship a proper 0–100 agentic score and commoditise the scoring layer. We think that is reasonably likely. It would not remove the need for remediation or longitudinal comparison, but it is the right question to ask us.