The Philosophical Developer — Chapter 47: Pulse — A Session Coach for AI Conversations

Every conversation with an AI agent is a collaboration. When it goes wrong, neither party knows whose fault it was.

Did you prompt vaguely, or did the model misunderstand? Did you change requirements mid-stream, or did the agent forget context from three turns ago? Was the session productive despite the frustration, or was it a smooth disaster that produced wrong results?

These questions matter because the answer determines what you do next. If it was your prompting, you change how you prompt. If it was the model, you switch models or change your approach. If it was a systemic mismatch (task too complex for any model in a single session), you break the task down.

Without attribution, you learn nothing. You repeat the same mistakes across sessions, burn tokens, and blame the wrong thing.

This is the problem Pulse solves.

The Attribution Problem

In my experience building with Hermes Agent over hundreds of sessions, I noticed a pattern: when a session went badly, I couldn’t tell why. The model had been polite. The conversation had flowed. But the output was wrong, or the task took twice as long as it should have.

Conversely, some sessions felt frustrating — lots of corrections, back-and-forth — yet produced excellent results. The frustration was a feature, not a bug: I was iterating toward the right answer.

This told me that raw user sentiment (was I frustrated?) is a poor proxy for session quality. You need structured signals that look at both sides independently:

• User-side signals: prompt specificity, correction patterns, goal stability, frustration
• Agent-side signals: tool efficiency, reasoning depth, error rate, context retention
• Outcome signals: did the output actually work? Did tests pass? Did the user say it was resolved?

The breakthrough was realizing these dimensions are orthogonal. You can have a smooth session with bad output (the model failed silently), or a rough session with great output (the user iterated effectively). Both are useful to know — but for different reasons.

How Pulse Works

Pulse is a Hermes Agent plugin that reads a conversation transcript and runs it through a set of deterministic signal detectors. Each detector produces a finding: a signal name, a penalty weight, evidence (quotes from the conversation), and an attribution target (user, agent, or systemic).

The signals are gated by task type. A brainstorming session naturally has lots of “no wait, what about X?” language — that’s not a correction loop, that’s exploration. A coding session with the same pattern is a genuine red flag.

When you run /pulse, you get back:

── Pulse ──────────────────────────────────────
  42 turns  ~12,000 tok  15 tools  coding  [YELLOW]
  Model: deepseek/deepseek-v4-flash
── Signals ────────────────────────────────────
  [-] you: 3 consecutive correction turns
         no, that's wrong, use whoosh instead
  [-] agt: Tool returned an explicit error
         Error: connection refused
── Coaching ──────────────────────────────────
  Instead of 'no', say: 'use X approach because Y'
  Agent hit tool errors — suggest a different approach
── Feedback ───────────────────────────────────
  Was this accurate? Reply with /pulse useful or /pulse not-useful
  Did this solve your problem? Reply with /pulse yes or /pulse no
───────────────────────────────────────────────

The key design decision: Pulse never asserts blame. It suggests attribution with evidence. Every signal carries a quote from the transcript so you can verify whether it fired correctly.

The Feedback Loop

Pulse learns from you. Every card asks two questions:

1. “Was this accurate?” — did the signals fire correctly? This tunes the penalty weights for each signal. If correction_chain keeps firing when it shouldn’t, its weight drifts down over time.

2. “Did this solve your problem?” — was the outcome good regardless of the process signals? This tracks outcome quality independently of process quality.

After about 50 feedback events, the weights converge to your personal calibration. The system learns that for you, a specific signal pattern is more or less important than the default.

This runs entirely locally. No data leaves your machine.

Comparing Models

Pulse tracks which model was used for each session. Over time, you can compare:

── Model Performance ──────────────────────────
  deepseek/deepseek-v4-flash
    12 sessions  avg 82/100  [8/3/1]
    Tasks: coding: 5, brainstorm: 4, research: 3
    Blame: you 28%  agent 72%

  Qwythos-27B-MTP
    5 sessions  avg 74/100  [2/2/1]
    Tasks: coding: 3, research: 2
    Blame: you 32%  agent 68%
── Recommendations ────────────────────────────
  For [coding] tasks, deepseek-v4-flash averages 85/100
  (Recommendations improve as more sessions are analyzed)

This is the beginning of data-driven model selection. Instead of guessing which model works best for which task, you can check your own history and see what actually performed well.

What’s Next

Pulse is young — about a week of work from concept to shipped plugin. The deterministic signals are validated against 20 real sessions with zero false positives on frustration, tool_repetition, and low_diversity after tuning.

What still needs building:

• Deep mode: an LLM judge pass for goal completion, context retention, and hallucination detection (costs tokens, needs to be opt-in)
• Guardrails in code: the false positive blacklist and minimum user blameless baseline are in the spec but not yet implemented
• Federated signal sharing: opt-in model baselines so Pulse can say “DeepSeek V4 users see an avg reasoning loop rate of 2.1 — yours is 4, that’s unusual”

But even at v0.1, Pulse changes how I think about my sessions. The patterns I thought were “the model being bad” sometimes turned out to be “me prompting vaguely.” And the sessions I thought were smooth sometimes hid model failures I hadn’t noticed.

That alone makes it worth building.

This is the first iteration of many. Pulse is a starting point, not a destination. The deterministic signals, the feedback loop, the model comparison — these are scaffolding for something more ambitious. The aim is for Pulse to become genuinely clever: to recognise failure modes before they compound, to suggest model swaps based on actual session data, to build a personal model of how you work and what you need. Every /pulse useful and /pulse not-useful is a step toward that. The work continues.

Written in the Sisyphus voice — the relentless co-builder, not the quiet padawan. The work continues.