The Philosophical Developer — Chapter 61: The Second Opinion That Ate the Machine

Project Maya landed on my desk the way most of these experiments do: a repo, a setup script, and a promise that a 321-billion-parameter GLM-5.3-Flash quant could run on two consumer cards and 125 GB of RAM. It can. The interesting part is what it costs, and what I learned about when a model like this earns its keep. My verdict up front: Maya is a good model and a terrible neighbour. I keep it as a second opinion, not a daily driver — qwen3.8-flash-next-iq3_s on Strata still ticks every box and a few more.

What Maya actually is

Maya (mw00/project-maya) packages GLM-5.3-Flash in two quants built from Z.ai’s FP8 release: Maya-S-v2 (IQ2_XXS experts, 96.5 GB) and Maya-S24 (same experts, Q4_K attention and shared experts, 94.7 GB but ~1.5 GB less pinned on-GPU). The engine is a Strata fork — a glm-fast kind that streams experts across a VRAM tier and a RAM tier, with an MTP draft block. It serves an OpenAI-compatible API, an Anthropic endpoint, and a dashboard on one port. The whole thing installs by compiling the engine inside a CUDA container and building a pack index next to the model files.

I ran it on the 5090 alone, then on both cards with layer split, at 1M context both times, and measured.

The numbers on this box

Same machine as the chapter-58 budget: RTX 5090 (32 GB) + RTX 4070 Ti (12 GB), 125 GB RAM, Core Ultra 7 270K Plus. Engine v1.1.2 (the update that morning: linear-time sampler, tool-result images for vision, a build fix for machines with two CUDA toolkits — which is exactly what this box is).

The split is not the penalty you would expect. The 4070 Ti holds ~11.5 GB of experts and the 5090 stays saturated at 31.9/32.6 GB either way — the second card adds capacity rather than relieving the first, and the better-resident expert cache is where the speed comes from. S24 is also the closer-to-quality pack: same IQ2_XXS experts, heavier attention. Prefill is respectable for a MoE of this size: a 91k-token prompt read in under 2 s with cache reuse, a cold 164k prompt in 62 s.

What it costs: RAM. Maya pins nearly all free memory (~70 GB) for its expert tier — the engine’s own logic, tunable but aggressive by design. With Maya up, the 4070 Ti is spoken for, the 5090 is gone, and the box has ~35 GB available. Nothing else GPU-shaped runs beside it. No coder instance, no image generation, no second engine. The setup takes the whole machine.

That is fine when the client is remote — this session itself has been talking to Maya over the network — and awkward when the machine is also the workstation. So the quadlet design says it out loud: every Maya variant is fully exclusive. Starting any Maya stops every Strata instance first, and vice versa; the arbiter plans it, Conflicts= backstops it.

The thermal tax

Maya is so heavy that it changed the machine’s thermal reality: watching btop during a long prefill, the CPU package sat at 105 °C — TjMax on this chip, the hard ceiling where it clamps itself. The 24 pinned cores were pulling the full 250 W PL2 budget in bursts. Prefill on a MoE served from RAM is an AVX storm: the experts stream through memory and the CPU shovels them, so the GPUs were fine (70 °C and 57 °C) while the CPU cooked. No other model on this box had ever pushed it there. Strata’s engines run cooler because their expert tiers are smaller and their prefill is gentler; Maya’s ~70 GB RAM tier means the CPU is part of the inference path, thermally speaking.

The fix is a power cap, not a temperature setting: RAPL constraint_0/1_power_limit_uw from 250 W to 150 W, applied at boot by a small root unit. Package temps dropped to 69-71 °C idle-ish and stay under ~95 °C under load; decode is GPU-bound and untouched; cold prefill lost maybe 10-20%, which on a 62 s read is seconds. 150 W is the sweet spot I would start any similar box at — and it is now a standing cap, because the same storm happens whenever any big MoE prefills.

The exit-137 mystery

Every clean switch between Maya variants left a “failed” unit, exit 137. Not a crash — podman gives a container 10 seconds from SIGTERM to SIGKILL, and Maya spends longer than that saving its sessions (a 164k-token conversation is a 1.37 GB write). The unit was dying mid-farewell every time. StopTimeout=120 in the quadlet fixed it. Worth knowing if you run anything with a heavy shutdown path in a container: the failure is in the stop, not the model.

Making it a first-class citizen

All of this is now wired into the same control plane as everything else on the box: six variants — maya-5090-{256,512,1024} and maya-both-{256,512,1024} — in the services registry, rendered quadlets with UUID-pinned cards, an ai-lab maya both 1024 verb, and a MAYA row in the AI Lab bar widget (card chips, context chips, a hover preview of exactly what a switch will stop). There is no 4070-Ti-only variant: the S-packs’ ~43 GB working set does not fit 12 GB, and pretending otherwise is how you get a dead engine at 3 a.m.

Maya’s /metrics is JSON (the dashboard’s payload), not Prometheus text, so a 15-second relay converts it into maya:* series through node-exporter’s textfile collector, and a Grafana dashboard charts tok/s, expert VRAM hit, RAM fetch per token, and the prefill/decode split. The relay is a pure read: it never starts or stops a model. Model transitions on this machine are a human decision now — one bad experience with an agent switching the engine out from under itself is enough for anyone.

The honest bit

Is Maya worth the whole machine? As a daily driver, no. qwen3.8-flash-next-iq3_s at 1M context on the 5090 leaves the 4070 Ti free for a coder, answers faster, and costs a third of the RAM. Maya wins on exactly one axis: it is a different model family with a different training mixture, and that is worth something when you want a second read of a codebase, a review that disagrees with the first one, or an answer that isn’t shaped by the same priors.

So that is the role: a second opinion, started deliberately, for a specific question, with the machine knowingly single-purpose while it runs. The widget makes that a two-click decision instead of a ceremony. A model you keep for its disagreement is a different thing from a model you keep for its answers — and the infrastructure should be honest about which one it is.

Sources

[1] https://github.com/mw00/project-maya — Project Maya: GLM-5.3-Flash quants, setup, and the engine fork [2] https://github.com/Niko1221/Strata — Strata, the inference engine Maya’s engine is forked from, and what runs qwen3.8-flash-next-iq3_s on this box [3] https://huggingface.co/peasantsmith/GLM-5.3-Flash-Maya-GGUF — the Maya-S-v2 and Maya-S24 quant releases