AgentGlass · methodology

How the numbers are computed

Every public score AgentGlass publishes is reproducible from public inputs. The address (sha256:<hex>) lets you pin the exact scan; the formulas below let you recompute the score from the findings list. If you find a discrepancy, the audit feed is the source of truth.

AX score · 0–100

The Agent eXperience score is the headline number on every report. It estimates the probability that a typical browser agent (Claude, OpenAI, Gemini) can complete the canonical task on this URL.

AX = 100 − sum(severity_weight(f) for f in kept_findings)
              + recovery_bonus(verified_patches)

severity_weight = { critical: 18,
                    high:     12,
                    medium:    6,
                    low:       2 }

recovery_bonus  = min(35,
                      sum(0.65 × severity_weight(f)
                          for f in verified_patches.fixes))

Why these weights? Empirical fit against the AX-Bench v0 ground-truth set. The 0.65 recovery factor reflects that a verified patch usually but not always lifts completion all the way back to the no-finding baseline — some collateral friction persists.

Hallucination rate · verifier accuracy

The percentage of Stage-2 LLM-generated findings that the adversarial verifier rejects as false-positives. The killer number on the public verification page — it quantifies what the verifier actually does.

hallucination_rate = stage2_vetoed / stage2_total

where stage2_total  = count of stage-2 findings BEFORE verifier
      stage2_vetoed = count with verifier_decision = 'false-positive'

Reported as: "{hallucination_rate:.0%} on first pass → 0% after verifier"

Why zero after the verifier? By construction — the verifier-rejected findings are filtered out of the final report. The final report's hallucination rate is 0% by definition. The pre-filter number is the interesting measurement.

Recipe confidence · cache-hit threshold

AgentGlass caches verified patches in patch_recipes and serves them from cache (skipping a fresh LLM call) when the recipe's confidence clears a threshold. The formula:

confidence = 0.35 × deterministic_signal
           + 0.35 × customer_merge_signal
           + 0.15 × no_regression_signal
           + 0.15 × cross_site_match_signal

where
  deterministic_signal   = success_count / max(1, success_count + failure_count)
  customer_merge_signal  = customer_merge_count / max(1, customer_merge_count + customer_reject_count)
  no_regression_signal   = no_regression_count / max(1, no_regression_count + regression_count)
  cross_site_match_signal = log10(1 + cross_site_match_count) / 3.0   # capped at 1.0

cache-hit threshold = 0.62

Why a weighted blend? Each signal alone is gameable. Deterministic match alone misses semantic regressions. Customer merge alone over-rewards rubber-stamped PRs. The blend keeps the cache honest — a recipe has to earn its place against four independent signals.

Scan cache · cents saved

When a scan request's content-address matches a recent completed scan (within 24h freshness window), the existing result is returned and the agent budget is skipped.

cents_saved = cents_per_agent × |live_agents|

cents_per_agent           = 20  (conservative average)
cache_freshness_seconds   = 86400
scans.cache_hits_count    = monotonic counter on the original scan
scans.agent_cents_saved   = sum of cents_saved over all hits

Who gets the savings? Today: the caller (the org running the duplicate scan). The original contributor's account accumulates cache_hits_count + agent_cents_saved as a royalty-eligible metric. Payout rails (A4 NFT or fiat) are on the roadmap, not live yet.

How to verify any score yourself

  1. Find the scan's content-address: sha256:<hex> (visible on the public verification page).
  2. Open /v/<scan_address> — the canonical-form bytes are surfaced there, base64-encoded.
  3. Paste the bytes into any SHA-256 tool. You should reproduce the address exactly.
  4. Plug the findings list into the formulas above. You should recompute the AX score to within 1 point.

Discrepancy? File a public issue against the published audit feed. The audit feed is the source of truth — we'll match it.