The Philosophical Developer — Chapter 38: Spellcast, Phase 2 Complete

Chapter 37 was the first checkpoint. A working PTY wrapper that captured audio, ran it through Whisper on the RTX 5090, and injected text into a shell. It demonstrated the idea. But a dictation tool that cannot handle code spelling, continuous listening, Vim navigation, and multiple GPUs is not a tool I can use every day.

I have been using macOS and iPhone dictation for years. It works well for prose. You dictate an email, it types. You dictate a message, it sends. But try coding with it and it falls apart. Identifiers like fooBar and foo_bar are indistinguishable to a speech engine that only knows words. Operators like -> and => require spelling out “dash greater than” which defeats the purpose. Language switching is a multi-second modal delay that breaks flow. And if you mispronounce or cannot remember the exact word you are reaching for, there is no way to explain the concept and have the system find it. You type it or you fail.

I wanted an operating system with a natural input point. A system where speech is a first-class input channel, not a post-processed afterthought. That vision is what became Chaossynergy. Spellcast is the dictation component of that vision, and Phase 2 is the step that turns it from a prototype into something I can begin to use for real work.

So I sat down with my Padawan, OrsonRius, and we went through Phase 2 in the recommended order. Thirteen sub-phases, each test-first, each committed before the next began. The result is at github.com/dark5un/spellcast.

Phase 2A: Tree-sitter and code spelling

The heuristic tokenizer from Phase 1 split text into tokens using regex patterns. It worked for simple cases but fell apart on nested languages like Markdown with embedded code blocks. I wanted a tokenizer that understood the language it was parsing.

OrsonRius integrated Tree-sitter with seventeen language grammars. Rust, Python, Go, JavaScript, TypeScript, C, C++, Java, Bash, Markdown, JSON, TOML, YAML, HTML, CSS, SQL. Tree-sitter is always compiled in, no feature flag needed. Language detection works by file extension, shebang line, or syntax pattern matching. A Markdown file with a fenced Python block uses the Python grammar for the code section and prose tokenization for the surrounding text.

Six code spelling modes emerged from the same phase. I say “snake foo bar baz” and it becomes foo_bar_baz. “camel foo bar baz” becomes fooBarBaz. Pascal, kebab, screaming snake, and a NATO spelling alphabet that turns “alpha bravo charlie” into abc. The modes are sticky or one-shot, configurable in the TOML config.

The symbol dictation module was a surprise win. Forty spoken commands for symbols and operators, context-dependent. “Arrow” in JavaScript gives =>, in C gives ->. “Dot dot” in Rust gives ..=, in JavaScript gives .... You can say “open bracket” and get [ in code, < in HTML. All user-overridable in config.

Phase 2B: Continuous listening

Push-to-talk works for demos. It is not how I want to interact with a computer. I want to speak naturally and have the system figure out when I am talking and when I am thinking.

Silero VAD via its Rust bindings provides neural voice activity detection. The audio stream flows through a ring buffer; when speech is detected, segments are sent to the ASR engine. If I interrupt myself, the barge-in buffer captures the new audio while the previous segment is still being transcribed. When the ASR finishes, the buffered audio is dispatched immediately.

The VAD is always compiled in. No feature flag needed.

Phase 2C: Vim-style navigation

The MVP had h and l for previous and next token. That is table stakes. Phase 2C added the full Vim navigation vocabulary: w and b for word-forward and word-backward (skipping punctuation), H and L for previous and next line, curly braces for paragraph and code block boundaries, gg and G for first and last token, 0 and dollar for line boundaries.

Count prefixes work. 3h moves three tokens left. 2} moves two paragraphs forward.

A fuzzy token search lets me press slash and speak a token name. All matching tokens are highlighted. N and n cycle through them. Matching is phoneme-similarity aware, so saying “receive” finds “receive” even when the ASR produced “recieve”.

Visual mode with character-wise and line-wise selection. Cut, copy, paste on selected ranges. The whole thing is cleanly separated into its own module with twelve tests.

Phase 2H: Emoticons and macros

Twenty-four built-in emoticons with context filtering. In prose mode, “happy face” gives 😊, “shrug” gives ¯\(ツ)/¯, “flip table” gives (╯°□°)╯︵┻━┻. In code mode, “todo” gives TODO:, “fix me” gives FIXME:.

User-defined macros with variable interpolation. I can define a license header macro and have it expand $DATE, $TIME, and $FILE automatically. Cursor positioning markers let the macro place the cursor where I want to start typing after expansion. A CLI (spellcast macro add, spellcast macro list, spellcast macro edit) manages the macro database.

Phase 2I: Multi-GPU

The RTX 5090 handles ASR. The RTX 4070 Ti handles the LLM explainer. Both run simultaneously without contending for GPU memory. The multi-GPU manager auto-detects devices via nvidia-smi and assigns workloads based on the configuration. Blackwell SM 12.0 detection is built in for future RTX 5090-specific optimizations.

Status bar shows ASR: RTX 5090 | LLM: RTX 4070 Ti. If one GPU fails, both workloads consolidate onto the remaining GPU.

Phase 2J: In-terminal highlighting

Instead of showing the current token in the status line, the terminal body itself highlights it using ANSI reverse video escape sequences. The highlight engine saves and restores cursor position so the injection is non-intrusive. Predictions appear inline below the current line using ANSI cursor positioning.

Fallback to the status line when the token has scrolled off screen.

Phase 2F: Prediction engine v2

The Phase 1 predictor used raw Double Metaphone distance. Phase 2F replaced it with a phoneme-level weighted edit distance that understands that vowels are more likely to be misheard than consonants, and that p and b are more confusable than p and t.

A context language model maintains bigram and unigram frequency tables. Predictions are re-ranked by context: if the previous token is “import”, module names rank higher than random English words. The display mode adapts to ASR confidence: hidden above 90%, dimmed between 70 and 90%, prominent below 70%.

The system learns from corrections. Every accepted prediction feeds back into the language model.

Phase 2G: Explain feature v2

The explainer maintains a conversation window. I say “the connection pool” and it returns ConnectionPool. I then say “same thing but async” and it returns AsyncConnectionPool, because the LLM prompt includes the previous explanation.

Domain-specific explanation packs ship built-in. Python standard library: “the function that reads a file” returns open(). Rust standard library: same explanation returns fs::read_to_string(). SQL, JavaScript, and more. Community-maintained packs can be loaded from TOML files.

The heuristic fallback converts natural language descriptions to PascalCase using a set of common code patterns. “The user repository class” becomes UserRepository.

Phase 2K, 2L, 2M: Packaging, accessibility, plugins

Fedora RPM spec, Flatpak manifest, AppStream metadata, Arch Linux PKGBUILD. A model management script (scripts/download-models.sh) with checksum verification and version tracking.

Audio feedback tones announce mode transitions. Screen reader events fire via speech-dispatcher for visually impaired users. An interactive onboarding wizard walks through microphone testing, GPU detection, model download, dictation test, key bindings, and kill switch verification.

The plugin system exposes a SpellcastPlugin trait with hooks for token commitment, navigation, explanation, and custom predictions. Two built-in plugins ship: a medical dictionary that knows ECG, EEG, and MRI, and a code symbols plugin that suggests context-appropriate alternatives for “lambda” and “arrow”.

The numbers

What Phase 2D means

The one sub-phase I have not started is Phase 2D: the client-daemon split. This is a deliberate decision. Phase 2D refactors Spellcast from a monolithic PTY wrapper into a background daemon with multiple frontends — terminal client, IBus IME engine, AT-SPI accessibility overlay on Linux, and Input Method Kit on macOS. It is the bridge between a terminal experiment and a system-wide input method.

I am taking a checkpoint before that refactor because Phase 2D is an architecture decision, not a feature addition. The daemon owns audio, ASR, LLM, memory, and the token stream. Frontends connect via a Unix domain socket IPC protocol. The terminal client becomes one frontend among many.

This is the shape of the Chaossynergy input layer. The uinput injector spike that follows will register Spellcast as a kernel-level virtual keyboard device, replacing standard input at the OS level. Whether that becomes a herdr plugin or a standalone herdr replacement is still open research. The uinput spike will settle the architecture.

Repos:

• github.com/dark5un/spellcast
• github.com/dark5un/Chaossynergy

Here I geek out with my young Padawan, OrsonRius.