Sites in the Woods

How we score a site.

The score is what a site earns over what it could: a check that passes earns its weight, one that only partly passes earns half, and a missing one nothing. Checks that don't apply to a site, and checks we couldn't run this time, don't count either way. Newer standards count at half weight.

The checks

CheckStandardTierWeight
robots.txt robotsRFC 9309established1
Googlebot may read the homepage and llms.txt robots.googlebotRFC 9309established2
Bingbot may read the homepage and llms.txt robots.bingbotRFC 9309established2
OAI-SearchBot may read the homepage and llms.txt robots.oai_searchbotRFC 9309established2
Claude-SearchBot may read the homepage and llms.txt robots.claude_searchbotRFC 9309established2
PerplexityBot may read the homepage and llms.txt robots.perplexitybotRFC 9309established2
The homepage may be indexed crawlers.noindexrobots meta tagestablished3
A description share.descriptionHTML meta descriptionestablished3
A share card title share.og_titleOpen Graphestablished1
A share card image share.og_imageOpen Graphestablished2
Structured data that parses share.structured_dataschema.org JSON-LDestablished2
llms.txt agents.llms_txtllmstxt.orgcommon3
security.txt agents.security_txtRFC 9116established2
An OpenAPI description (sites with an API) agents.openapiOpenAPIcommon3
An MCP manifest (sites with an API or MCP server) agents.mcp_manifestMCPcommon3
An agent card (sites with an agent) agents.agent_cardA2Aemerging1

A site that publishes an OpenAPI description, an MCP manifest or an agent card is scored as a site with an API: then a missing OpenAPI description or MCP manifest counts against it. Otherwise they don't count.

Grades

A homepage that asks not to be indexed, or a robots.txt that stops every search and AI crawler we check, holds the grade at D (at most 59), whatever else passes.

Why scores differ between tools

Other scanners check other things, weigh them differently and decide differently what applies: on the same day, two tools running the same engine gave one site 88 and 96. Ours is a look at the homepage and its public files, not a security test or a certification. The weights are also data.

Changes

Last updated .