Watch: Agent Readiness evaluator
Recurring monitoring of a domain's agent-readiness posture using Verify's readiness methodology, with the evaluator implementation and the scoring methodology tracked as two separate, independent versions.
Product model
The same readiness methodology is available through the public AI Agent Readiness Score Actor on Apify and through POST /v1/audits/readiness -- both are one-off execution surfaces for the same underlying methodology, run on demand, with no persistence between runs. Sentinel Watch is the recurring surface: it adds persistent baselines, scheduled execution (evaluator: "agent_readiness" on the wire), comparison against the prior snapshot, signals, policies, and notification history. Verify remains the canonical readiness backend across all three surfaces -- the Apify Actor is not itself the thing providing Watch's scheduling or persistence.
Watch never computes an independent score
Watch does not re-implement or approximate readiness scoring itself. Each run's readiness result comes from the pinned AI Agent Readiness Score evaluator, and every snapshot records both the evaluator implementation version and the methodology version used to produce that result, so a score change can be traced to its actual source rather than attributed to drift inside Watch's own comparison logic.
evaluatorVersion, methodologyVersion, and verifyVersion are three separate identifiers
Do not conflate these -- each changes independently of the other two, and neither a methodology change nor an evaluator-build change is guaranteed to also change the other.
- evaluatorVersion (top-level field on every snapshot, GET /v1/watch/{watch_id}/snapshots) identifies the evaluator implementation used for the run, including the pinned AI Agent Readiness Score Actor build. It changes only when that pinned build itself is updated.
- normalizedState.methodologyVersion (nested inside the same snapshot's normalizedState, for example "readiness-1.0") identifies the readiness scoring methodology used to compute the result. A methodology update (for example readiness-1.0 to readiness-1.1) is not guaranteed to change evaluatorVersion, and an evaluator-build update is not guaranteed to change the methodology it implements.
- rawReference.verifyVersion (nested inside both the snapshot's rawReference and the run's actorReference) identifies the deployed Verify application version that orchestrated the evaluation.
Attributing a score change
Readiness snapshots record the methodology version used for scoring, independent of the pinned evaluator implementation. If Verify's readiness methodology changes, the methodology version on new snapshots changes to reflect that -- a separate source of score movement from a genuine change on the monitored site, and one that is not necessarily accompanied by an evaluatorVersion change. Historical snapshots preserve the methodology version that was actually in effect when they were scored, so a score movement between two snapshots can be checked against both dimensions -- did the target change, did the methodology version change, or both -- before concluding which one caused it.
Representative signal types
These are the current signal_type values this evaluator's comparison can emit:
- READINESS_SCORE_DECREASED / READINESS_SCORE_INCREASED
- READINESS_FINDING_ADDED / READINESS_FINDING_RESOLVED
- CRAWLER_POLICY_CHANGED
- API_DISCOVERABILITY_CHANGED
Where each version field lives (illustrative snapshot excerpt)
{
"evaluator": "agent_readiness",
"evaluatorVersion": "0.4",
"normalizedState": {
"methodologyVersion": "readiness-1.0",
"overallScore": 71.5
},
"rawReference": {
"verifyVersion": "1.0.794",
"verifyBuildSha": "03f3306e..."
}
}Alternate execution surfaces
The same readiness methodology also runs as the public AI Agent Readiness Score Actor on Apify, and as the stateless POST /v1/audits/readiness endpoint for one-off checks. Both return the same scored, per-category, findings-bearing result shape -- but neither one persists a baseline, schedules itself, or emits a signal on change; that is what subscribing the same domain to a Sentinel Watch instead adds. Verify remains the canonical readiness backend for all three.