/pulse deep — The Judge That Tells You What It Cost

Eight posts built the gym on deterministic signals — free, offline, no model calls. This one is different: /pulse deep runs an LLM judge over the session through Hermes’s own model connection, then tells you exactly what it spent. One extra model call, billed to your active model, cost printed on the card. No key setup, no hidden spend, no silent fallback.

Why a slash command, not a flag

The CLI has --deep (stdlib urllib, PULSE_API_KEY chain, gpt-4o-mini default) for the portable lab. Inside Hermes that design is wrong: the host already owns routing, auth, fallback, and credentials. So /pulse deep reuses the live connection via the plugin ctx.llm lane — whatever model you’re talking to judges the session, and the plugin never sees a key.

What it looks like

/pulse deep

Real output from a seeded 5-turn session (test double standing in for the live lane — including the ~$0.0020 figure, which the double supplies):

── Pulse ──────────────────────────────────────
  5 turns  ~62 tok  0 tools  chat  [GREEN] (calibrating)
  Model: muse-spark
── Signals ────────────────────────────────────
  [ok] agt: Judge: yes — goal_completion
         solved
── Coaching ──────────────────────────────────
  Keep doing what you're doing
── Feedback ───────────────────────────────────
  Was this accurate? Reply with /pulse useful or /pulse not-useful
  Did this solve your problem? Reply with /pulse yes or /pulse no
───────────────────────────────────────────────
── Judge ──────────────────────────────────────
  Model: meta/muse-spark-1.3 (openrouter)
  Verdicts: goal_completion
  Cost: 240 tokens (200 in / 40 out), ~$0.0020, 0s — billed to your active model
  Provisional until agreement-gated (kappa>=0.6, n>=50).
───────────────────────────────────────────────

The deterministic card comes first, unchanged. The Judge section names the model, the verdicts, the tokens in/out, the dollar estimate when the host reports one, and the elapsed seconds. Then the honesty label: provisional until the agreement gate passes.

The cost contract

Dollars come from Hermes alone: when the harness reports cost_usd on the judge call, the card shows ~$X (cost from Hermes). When it doesn’t — and today it typically doesn’t — the card says so out loud (“Hermes reported no dollar cost; tokens are enough”) and the tokens stand on their own. No local price table, nothing to go stale, nothing estimated.

Three rules, all visible:

1. Opt-in, never default. /pulse stays free and offline. deep is a separate word you type on purpose. The command description and args hint both say it costs tokens before you run it.
2. Actuals on the card. Not “about $0.02–0.05” — the real tokens, the real model, the real seconds, every run. Estimates live in docs; actuals live in output.
3. Failure is loud. Judge timeouts, trust errors, unparseable verdicts return an error string plus the deterministic card — which still stands. Never a silent fallback that pretends the judge ran.

The four verdicts are goal_completion, context_retention, correction_quality (user-targeted — it grades your correction specificity), and hallucination. One combined temperature-0 JSON-mode call; results persist to state.db with run_mode='deep' so trends can separate judged sessions from deterministic ones.

What it doesn’t claim

Same gate as the CLI judge: no “Pulse scores, but ML” until kappa ≥ 0.6 at n ≥ 50 (pulse agreement). The verdicts are labeled provisional on the card itself. First real-key run, published as measured (2026-09-05, openai/gpt-4o-mini via OpenRouter, local 10-trace corpus): the judge returned zero verdicts on all 10 traces — thin textless TIMELINEs give it nothing to grade — so kappa=0.0/agree=1.0 on both comparable pairs, gate FAIL (pending) exactly as designed. A judge without an agreement story is a vibe machine with a token bill; this one has the story and its first honest datapoint, and the gate wants n ≥ 50 rich sessions before any hosted-judge claim.

That closes the loop the coverage analysis left open: B2 shipped as a command you can run tonight, priced on the receipt, gated before it brags. The gym is complete — deterministic downstairs, judge upstairs, both with the bill attached.

Written from the workshop — the judge shows its receipt, or it doesn’t judge.