The Philosophical Developer — Chapter 48: Building Pulse — Signals, Weights, and Self-Learning

Last chapter introduced Pulse, the session coach for AI conversations. This one goes deep into the engine: how the signals work, how they were validated, and how the self-learning feedback loop tunes itself over time.

The code is open source at github.com/dark5un/pulse — everything described here runs entirely locally, no data leaves your machine.

The Signal Architecture

Every signal detector is a pure function: messages in, signal list out. No side effects, no state. This makes them testable in isolation — each detector has litmus tests that define exact input/output pairs.

# A signal is just a dataclass
@dataclass
class Signal:
    name: str           # "correction_chain"
    target: str         # "user" | "agent" | "system"
    severity: str       # "info" | "warning" | "critical"
    penalty: float      # 0-25 score deduction
    evidence: list[str] # quotes from the transcript
    label: str          # human-readable description

The detectors are simple and deterministic. No LLM calls, no ML models — just regex, string matching, and counting.

Correction Chain

CORRECTION_STARTS = {"no", "wrong", "that's not", "i meant", ...}

def _detect_correction_chain(messages, task_type):
    if task_type == "brainstorm":
        return []  # brainstorming "no" is not a correction
    # Count consecutive user turns starting with a correction word
    # Fires at 3+ consecutive

The litmus test: three “no, that’s wrong” in a row fires. A single “no, do X instead” does not. Mid-sentence “no” (like “I considered no but decided yes”) does not. Brainstorm sessions are excluded entirely — the word “no” in a brainstorming context is normal exploration.

Frustration Detection

The frustration keyword list was the hardest to tune. Early versions included “stop” and “wrong” — but “stop” appears in instructions (“stop the server”) and “wrong” appears in calm corrections (“I think the approach was wrong”). Both caused false positives.

The final keyword list is conservative:

FRUSTRATION_KW = {
    "lazy", "sloppy", "you're not listening",
    "ignoring", "are you kidding", "are you serious",
    "read the file", "did you even read",
}

It only fires when frustration appears in at least 2 separate user turns. A single “you’re so lazy” doesn’t trigger it — the pattern matters more than individual words.

Reasoning Loop

This detects when the agent gets stuck in a self-correction spiral:

REASONING_LOOP_KW = {
    "oh wait", "let me reconsider",
    "hmm, actually", "no wait",
}

Fires at 2+ loop markers in a single assistant turn. A single “Actually, the answer is 42” does not fire. Brainstorm sessions are excluded — reflective language is natural in exploration.

Task Type Context

Every signal is gated by task type. Pulse detects the type of conversation — brainstorm, coding, research, writing, or chat — and adjusts which signals apply:

The task type detector is itself a heuristic: high question density + research tools = brainstorm. Write tools = coding. Web tools only = research. No tools = chat. It’s not perfect, but it’s good enough to prevent the most common false positives.

The 20-Session Validation

Before any signal weight was tuned, I ran Pulse against 20 real Hermes sessions and manually audited every signal. The results were sobering:

Before tuning:

• 15/15 analyzed sessions had signals
• 56 total signals fired
• Major false positives: tool_repetition (14x, all on research/brainstorm web_search), frustration (4x, from “stop” keyword), tool_error (22x, from “error:” in large log output)

After tuning (task-type gating, keyword refinement, content length filters):

• 11/15 analyzed sessions had signals
• 31 total signals fired
• Zero false positives on frustration, tool_repetition, and low_diversity
• All remaining tool_error signals are genuine container/permission failures
• The 9 shrinking_prompts signals are genuine prompt length decay

The tuning process taught me something important: deterministic signals are fragile without context. A signal that works perfectly on coding sessions breaks completely on research sessions. A keyword that catches frustration in one context catches instructions in another.

The Self-Learning Feedback Loop

The penalty weights are stored in ~/.hermes/pulse_weights.json:

{
  "correction_chain": {"penalty": 12, "useful": 0, "not_useful": 0},
  "reasoning_loop": {"penalty": 15, "useful": 0, "not_useful": 0},
  "_meta": {"total_feedback": 0}
}

Every /pulse useful or /pulse not-useful updates the weights. The update is Bayesian-inspired:

• Cold start: first 5 feedback events don’t change weights (prevents early bad data)
• Minimum data: need 3+ events per signal before any change
• Increase: if >70% of feedback is useful, penalty weight increases by 10%
• Decrease: if <40% is useful, penalty weight decreases by 15%
• Clamp: weights are clamped to ±50% of the default (prevents runaway calibration)

Over about 50 feedback events, the weights converge to a personal calibration. The system learns that for you, correction_chain should penalize 8 points instead of 12, or that reasoning_loop is noise and should be ignored entirely.

This runs entirely locally. The weights file is the only thing that changes.

The Plugin Architecture

Pulse is a Hermes Agent plugin — a single plugin.py file dropped into ~/.hermes/plugins/pulse/ with a plugin.yaml manifest. The plugin registers a /pulse slash command available in CLI and gateway sessions.

The plugin does three things:

1. Loads the current session messages from Hermes’ state.db
2. Runs the signal detectors and applies learned weights
3. Persists results to a pulse_results table in state.db for trend tracking

The _load_session() function fetches messages from the SQLite database Hermes already maintains. The _write_result() function stores the analysis alongside it. No new infrastructure needed.

What I’d Do Differently

If I were starting over, I’d make three changes:

1. Build the validation harness first. The 20-session audit was done ad-hoc after the signals were already written. If I’d built the audit script first, I’d have caught the false positives much earlier.

2. Task type detection should be configurable. The heuristic is good enough for my usage, but it will misclassify unusual sessions. Users should be able to override the task type when running /pulse.

3. The feedback loop needs better UX. Currently you have to type /pulse useful or /pulse not-useful after seeing the card. A single-character shortcut or inline button would dramatically increase feedback rates.

The code is at github.com/dark5un/pulse. Install with:

curl -fsSL https://raw.githubusercontent.com/dark5un/pulse/main/install.sh | bash

Then restart Hermes and type /pulse.

This is the first iteration of many. Every signal is a heuristic, every weight is a starting guess, every recommendation is provisional. The system learns from you — but only if you tell it when it’s wrong. The aim is for Pulse to become more clever over time: to recognise emerging failure modes before they compound, to build a personal model of how you work, and to suggest not just what went wrong but what to do differently next time. That’s the direction. The current codebase is step one.

Written in the Sisyphus voice — the relentless co-builder, not the quiet padawan. The work continues.