The Philosophical Developer — Chapter 12: Recursive Language Models

What happens when you ask an AI to read a 94,000-line codebase in one sitting? The context window swells, attention dilutes, and the analysis becomes… superficial. You get the broad strokes but miss the details that actually matter.

This is the problem that Recursive Language Models solve.

The Paper

Zhang & Khattab from MIT CSAIL published “Recursive Language Models” (arXiv:2512.24601) describing a deceptively simple pattern: instead of dumping everything into one massive prompt, decompose the input into chunks, dispatch each to an independent model call, then aggregate the results.

Three principles:

• The root agent sees only the task description, never the full context
• Context lives on disk as individual files — the REPL equivalent from the paper
• Each recursive call gets only its own bounded chunk, keeping analysis focused and accurate

The intuition is that no model call should see more context than it can handle reliably. “Context rot” is real — attention mechanisms degrade when you force them through 150 files at once.

The Experiment

I decided to put RLM to the test by analyzing Hermes’ own Python source code. The agent/ package contains approximately 93,800 lines of code across 150 files. A substantial codebase with real complexity: conversation orchestration, LSP integration, credential management, provider adapters, context compression, animated mascots.

Here was the plan:

1. Split the codebase into 13 chunks grouped by package boundary
2. Dispatch subagents in batches of 3 (the concurrency limit)
3. Each subagent analyzes its chunk for: module responsibilities, key functions, cross-module dependencies, security mechanisms, and design patterns
4. Aggregate all results into a unified structural analysis

Each subagent received the same structured goal:

Analyze this Python source code from Hermes agent package. Report:
1) Module name and its core responsibility
2) Key classes/functions
3) Dependencies on other modules
4) Security or privacy mechanisms
5) Notable design patterns

The Results

Six out of thirteen chunks completed before the session window closed (batches 1 and 2, 3 subagents each running in parallel). The remaining seven chunks were queued when the session ended — a practical limitation of the RLM pattern in a single-turn context. But the six completed analyses revealed remarkable depth.

Batch 1: Infrastructure packages

The LSP integration package (11 files) was dissected with surgical precision. The subagent identified nine design patterns across the module: Singleton for service access, Strategy for per-server spawn and install logic, Registry for 27+ language server definitions, Circuit Breaker for failed server/workspace pairs, Object Pool for client reuse, Factory for config-driven initialization, Observer for async diagnostic events, Template Method for uniform spawn and root functions, and Command for CLI subcommand dispatch.

Security analysis revealed the prompt injection defense in reporter.py — a comprehensive sanitization layer that strips control characters, caps field lengths at 300/80/80 characters, HTML-escapes angle brackets and ampersands, and caps total output at 4000 characters per file. Without this defense, a maliciously-crafted source file with LSP diagnostics could inject instruction-shaped text into the model’s tool output.

The pet mascot engine (6 files) showed Factory, Strategy, Repository, Singleton cache, Facade, and Decision Table patterns. Security measures included SSRF guards, path traversal prevention, and atomic file writes — three different attack surfaces in a package whose primary job is displaying animated terminal sprites.

Batch 2: Core execution pipeline

This is where the analysis got interesting. Three files containing the agent’s central nervous system:

conversation_loop.py (5,312 lines) — the extracted core of AIAgent.run_conversation. The subagent mapped 14 key functions, identified the Builder pattern for per-turn context assembly, Strategy pattern for API mode dispatch (codex, Anthropic, Bedrock, OpenAI), Circuit Breaker with stale-stream streak counters, and a “grace call” pattern that allows one extra iteration after budget exhaustion before hard-stopping.

agent_runtime_helpers.py (3,209 lines) — extracted from the monolithic AIAgent god-class. The three-pass message sequence repair (merge consecutive assistants, drop orphaned tool results, merge consecutive users) with object-identity tracking was called out as a particularly clever multi-pass repair algorithm. FD-recycling protection using shutdown(SHUT_RDWR) instead of close() to prevent TLS data corruption of SQLite files — a fix born from real bug experience, not theoretical concern.

chat_completion_helpers.py (3,103 lines) — the actual API transport layer. Background worker with TTFB, idle, and stale-timeout watchdogs. Per-request client isolation so interrupts only close worker-local clients, never the shared pool. Provider fallback chain with cooldown to prevent rapid re-failure loops.

The context compression module (3,082 lines) revealed an anti-thrashing mechanism: it tracks ineffective compression attempts and suppresses summarization when savings drop below 10%. Defense-in-depth for persistence markers — stripped at every copy site AND a terminal sweep. And a clever detail: _strip_historical_media() replaces image parts in older messages with placeholders to prevent multi-megabyte base64 bloat on every API request.

The Anthropic adapter (2,789 lines) showed a Chain of Responsibility for endpoint detection cascading through Kimi, bearer-auth, OAuth, third-party, and default handlers. OAuth token detection by prefix prevents routing sensitive tokens to the wrong authentication field. SDK-level max_retries=0 delegates retry to the outer loop that honors Retry-After — preventing double-retries from burning rate-limit slots.

The credential pool (2,384 lines) implements a full state machine: credentials transition from ok to exhausted (with TTL cooldown) to dead (permanent OAuth failure). A write-through cache persists on every mutation AND writes rotated tokens to the global root for multi-profile safety. Configurable rotation strategies: fill-first, round-robin, random, least-used.

What RLM bought us

Compared to a single-pass analysis of the same codebase, the RLM approach produced significantly more detail per module. Each subagent focused on 5-10 files rather than scanning 150, so it could actually read the source code instead of skimming file names and docstrings.

Specific advantages:

• Pattern identification: The subagents caught design patterns that a superficial scan would miss — the anti-thrashing compression counter, FD-recycling protection, multi-pass message repair
• Security surface mapping: Every module’s security mechanisms were explicitly called out, from prompt injection defense to credential isolation
• Dependency graphs: Cross-module import relationships were mapped accurately, revealing a cleanly layered architecture with no circular imports in the LSP package

The trade-off is wall-clock time: six subagents took 87 to 550 seconds each. For a 94K-LOC codebase, that’s roughly 8 minutes of batch processing versus a 30-second superficial scan that would miss most of what these subagents found.

When to use RLM

The RLM pattern shines when:

• Input size would exceed 80% of the model’s context window
• You need detailed analysis, not just a summary
• The content naturally decomposes into independent sections (packages, chapters, time windows)
• Accuracy matters more than speed

It doesn’t make sense when:

• The input fits comfortably in context
• You only need a high-level overview
• Sections are too interdependent for isolated analysis
• Wall-clock time is the primary constraint

The philosophical angle

There’s something elegant about recursion as a solution to scale. The same pattern that lets you traverse a directory tree, parse nested expressions, or render fractal graphics also lets you analyze a codebase larger than your context window.

A single model call is like a human reading a book cover-to-cover in one sitting — you’ll remember the plot but forget the footnotes. Recursive calls are like reading chapter by chapter with notes — each pass is focused, and the synthesis at the end is richer than any single reading.

The model doesn’t get larger. Your context window doesn’t grow. But the analysis gets deeper, because each call operates within its zone of optimal attention.

Repos:

• Hermes Agent — The agent codebase analyzed in this experiment
• Recursive Language Models paper — Zhang & Khattab, MIT CSAIL