The Philosophical Developer — Chapter 06: The Pipeline on My Own Machine

The whole point of this series is local inference for development. Data ownership matters. When the model runs on your machine, the code never leaves. No API calls, no token billing, no third party seeing what you are working on. I want development to work on my own machine. The cloud is just a tool to move faster while we figure things out.

DeepSeek V4 Flash is the reason this is possible. I use their cloud service and I am really happy with it. It is insanely cheap and does the job I need. Full credit where it is due.

After establishing the methodology on cloud infrastructure, I needed to know one thing. Does the pipeline depend on the cloud, or does it work anywhere? I set up DwarfStar — antirez’s local inference engine — on my M2 Max Mac with 96GB of RAM and tested it.

The results exceeded expectations.

The Setup

I have an M2 Max Mac with 96GB of RAM. DwarfStar runs on Metal, Apple’s GPU framework. The model itself is 35GB. With SSD streaming, the routed experts live on disk and load on demand. Without that flag, the model would not even initialize. Memory is always the constraint, even with 96GB.

./ds4-server ./gguf/model.gguf --ctx 200000 --ssd-streaming

The Task

I gave my padawan the same task it had done on the cloud: implement the complete-task feature in a task tracker CLI from scratch. Three TDD cycles. RED first. Per-cycle commits. Squash merge. Trace tag. The full pipeline.

The Results

The local model produced three clean cycles. No bundled commits. It followed the RED-first discipline strictly. When a test passed immediately because the behavior was already present from a prior cycle, it documented that honestly as a confirm-only cycle instead of forcing a fake RED-GREEN loop.

The trace tag on GitHub shows exactly what happened:

trace/complete-task:
  Cycle 3: done subcommand wires mark_done to CLI
  Cycle 2: error handling for out-of-bounds
  Cycle 1: mark task complete by index

Seven tests passed. Zero failures.

The Numbers

• GPU utilization: 93%
• GPU temperature: 59 degrees Celsius
• GPU power: 14.3 watts
• Decoding speed: 8-11 tokens per second
• Session time: 21 minutes
• RAM used: 75GB out of 96GB

What This Means

The pipeline is not tied to the cloud. It is not tied to any model. It is a process, and processes travel where they are needed.

Local inference is not about replacing the cloud. It is about having a choice. When the model runs on your machine, the code never leaves. No API calls, no token billing, no third party seeing what you are working on.

The discipline of writing tests first, committing per cycle, and tagging the trace works the same way regardless of where the model runs. The LSP guardrail catches bad code before it reaches the test runner. The trace tags make the reasoning auditable. All of that works on a laptop.

Running DeepSeek V4 Flash on a laptop at 14 watts is not a party trick. It is the direction I want development to go.

Repos:

• task-tracker — trace/complete-task tag on GitHub

Also on LinkedIn.