# relay-eval

relay-eval compares what multiple AT Protocol relays see during the same
measurement window. It is mainly for answering: "did this relay emit the same
repo activity as the rest of the observed relay set?"

Site:
- https://relay-eval.waow.tech/
- Source: https://tangled.sh/@zzstoatzz.io/relay/tree/main/relay-eval
- Human API guide: https://relay-eval.waow.tech/diffs-api

Important vocabulary:
- repo: an AT Protocol repository identified by a DID.
- relay: a service that aggregates repository synchronization streams.
- stream gap: a repo was observed on at least one relay during a run, but not
  on the relay being inspected.
- missing repo: a row in that stream-gap set. This is a stream-comparison fact,
  not by itself proof that the account is currently live.
- checked row: relay-eval spent resolver/classifier effort on the row.
- unchecked row: the row was outside the resolver/classifier budget.
- next_cursor: an opaque resume token. Pass it back as `cursor` to fetch the
  next page. It is not a numeric page number.

Core JSON APIs:

1. Latest run summary
GET https://relay-eval.waow.tech/api/latest

Returns the current dashboard snapshot: run metadata, per-relay coverage stats,
small diff samples, and aggregate diff counts. Diff samples are intentionally
small; use the Diffs API for the full list.

2. Diffs API: fetch missing repos
GET https://relay-eval.waow.tech/api/latest/diffs?relay=<host>&limit=5000
GET https://relay-eval.waow.tech/api/runs/<run_id>/diffs?relay=<host>&limit=5000

Use `/api/latest/diffs` for convenience. Use `/api/runs/<run_id>/diffs` when
exporting or paging so every response comes from the same measurement run.

Common parameters:
- relay=<host>: relay being inspected, for example relay.xero.systems.
- limit=<n>: rows per response. Default 1000, maximum 5000.
- cursor=<next_cursor>: resume token from the previous response.
- lane=live: checked rows that look like likely live stream gaps.
- lane=unchecked: rows not checked because the classifier budget was not spent.
- lane=review: resolver failures, malformed DIDs, unsupported methods, invalid
  DID documents, or legacy unresolved rows.
- lane=inactive: rows with inactive/no-PDS evidence.
- lane=checked: all rows except `classification_not_attempted`.
- classification=<label>: exact stored classifier label.
- q=<substring>: DID substring search.
- pds_host=<host>: filter by resolved PDS host, with or without https://.

Example: pull the full broad missing-repo set for a relay

relay='relay.xero.systems'
run_id="$(curl -fsS "https://relay-eval.waow.tech/api/latest/diffs/summary?relay=${relay}" | jq -r '.run_id')"
cursor=''

while true; do
  url="https://relay-eval.waow.tech/api/runs/${run_id}/diffs?relay=${relay}&limit=5000"
  [ -n "$cursor" ] && url="${url}&cursor=${cursor}"
  page="$(curl -fsS "$url")"
  echo "$page" | jq -r '.diffs[] | [.did, .classification, (.pds // "")] | @tsv'
  cursor="$(echo "$page" | jq -r '.next_cursor // empty')"
  [ -z "$cursor" ] && break
done

Example: likely live stream gaps only
GET https://relay-eval.waow.tech/api/latest/diffs?relay=relay.xero.systems&lane=live&limit=5000

3. Summary API: counts and facets for the Diffs API
GET https://relay-eval.waow.tech/api/latest/diffs/summary?relay=<host>
GET https://relay-eval.waow.tech/api/runs/<run_id>/diffs/summary?relay=<host>

Use this before paging. It returns:
- run_id: use this with `/api/runs/<run_id>/diffs` for a stable export.
- total: total rows matching the filters.
- checked: rows where relay-eval attempted classification.
- unchecked: rows still outside the classifier budget.
- estimate.live_gap_lower_bound: checked lower bound for likely live gaps.
- estimate.live_gap_display: lower bound formatted with `+` when unchecked rows
  mean more live gaps may exist.
- facets.relays, facets.classifications, facets.pds: grouped counts for the
  current filter set.

4. Trend API: coverage over recent runs
GET https://relay-eval.waow.tech/api/trend?limit=500
GET https://relay-eval.waow.tech/api/trend?since=<unix-or-iso>&until=<unix-or-iso>&relays=<csv>

Returns per-relay coverage points. `since` and `until` may be Unix seconds,
Unix milliseconds, or accepted ISO-ish timestamps. `relays` is optional CSV.

Chart route using the same range idea:
https://relay-eval.waow.tech/chart/<since>/<until>/<relay_csv>

5. Relay history and live-rate APIs
GET https://relay-eval.waow.tech/api/relays
GET https://relay-eval.waow.tech/api/relays/history?name=<host>&limit=288
GET https://relay-eval.waow.tech/api/relays/live

`/api/relays/live` is used by the sonification page and reports current rolling
events/sec per relay.

Current classification labels:
- active_missing: DID resolved and currently has PDS evidence; likely live
  stream gap.
- no_pds_endpoint: DID resolved but no usable PDS endpoint was found.
- malformed_did: invalid DID syntax.
- unsupported_did_method: relay-eval cannot resolve this DID method.
- did_resolution_failed: resolver/network failure.
- invalid_did_document: DID document could not be interpreted safely.
- classification_not_attempted: classifier budget was exhausted before this row.
- coverage_gap, unresolvable, deactivated: legacy v1 labels; preserve for
  historical queries, but prefer v2 labels for new analysis.

Interpretation rules:
- Use `lane=live` for the strongest lower-bound signal of relay coverage debt.
- Use the broad Diffs API without `lane` when the user asks for the full missing
  repo set.
- Do not describe every missing repo as "live"; unchecked and review rows need
  additional interpretation.
- Prefer a stable `run_id` for exports, reports, and multi-page pulls.