Onboard From Excellence — The Leaderboard as a Curriculum

Part 3 handled the worst night. Part 4 is the opposite: your best sessions are a curriculum and you didn’t know it. pulse leaderboard --task <type> --top 5 prints the traces new hires should read first — top-scored, evidence attached, anonymized. No new detectors, no new pipeline. A curation habit plus one query.

The query

uv run pulse leaderboard --corpus corpus --task chat --top 5

== chat (10 traces) ==
  BEST:
    100  008e785bc8ee  meta/muse-spark-1.3
    100  97d020c11e8b  meta/muse-spark-1.3
    100  777790ac1029  meta/muse-spark-1.3
    100  83afc3e871d1  meta/muse-spark-1.3
    100  fbd2f44a67ac  meta/muse-spark-1.3
  WORST:
     90  6a71658a24e2  meta/muse-spark-1.3 latency_regression
    100  008e785bc8ee  meta/muse-spark-1.3
    100  97d020c11e8b  meta/muse-spark-1.3
    100  777790ac1029  meta/muse-spark-1.3
    100  83afc3e871d1  meta/muse-spark-1.3

--task filters to one task type (per-type top-5 for the onboarding doc); --top sets the bucket size. IDs are sha256-anonymized — safe to paste into the wiki. The worst slot names its signal, so the “what not to do” example comes free with the “what to do” list.

The recipe

1. Replay the top-5 per task type with --show-state and read them like code review.
2. Ask what the best sessions have in common: specific first prompts? short correction loops? tools before second-guessing?
3. Turn the patterns into three onboarding bullets. Real examples beat style guides — “here’s a 100-point session, read the first five turns” teaches faster than “be specific”.

The honest bit

This corpus is thin — 9×100 plus one 90, textless TIMELINEs, one model. “Read the best” still works (the mechanism is real: rank → read → imitate), but the curriculum gets interesting at 50+ sessions across models and task types, where BEST actually discriminates. Start the habit now; the corpus catches up.

And the flip side belongs in onboarding too: the WORST slot with its named signal is the cheapest “don’t do this” example you’ll ever get. One command prints both.

Next post: From Traces to Training Data — the exporter, the review mode, and why not every correction is a clean pair.

Written from the workshop — read the best, imitate the best.