The Philosophical Developer — Chapter 50: Impromptu — From Script to YouTube, the Agent-Driven Video Pipeline

The pipeline starts with a script and ends with a packaged video: a teleprompter for the human, motion graphics from an HTML render engine, an ffmpeg stitch for the edit, and a finishing step that masters audio, writes chapters, grabs a thumbnail, and hands over a manual-upload checklist.

The interesting part is not any single tool. It is the shape of the pipeline: repeatable, agent-driven, and honest about which steps belong to the human and which belong to the machine.

The Problem With Video Production

Video production does not fail at the creative step. It fails at the seams: the script lives in one place, the graphics in another, the edit decisions in a third, and the packaging — loudness, chapters, thumbnail, description — is a checklist nobody finishes.

Impromptu treats the whole thing as one pipeline with a single source of truth: a scene plan. The script carries scene timings, the scene plan carries modes and graphics, and every downstream step reads from the plan. Change a timing once, and chapters, graphics, and the final cut all follow.

Two Modes, One Plan

The pipeline has two rendering modes, and the choice between them is the central design decision.

Mode A keeps per-scene graphic renders and stitches them with ffmpeg. It is simple, it works today, and it handles effects only ffmpeg can do — chromakey, geometric transforms on the presenter feed. Its weakness is the stitch itself: a chained crossfade filtergraph where every transition overlaps the running stream.

Mode B generates a single HTML composition — the presenter as an a-roll clip, graphics as overlay clips on higher tracks, one paused animation timeline — and renders it once. No per-scene files, no filtergraph, no stitching complexity. It is the default, and Mode A is the fallback for scenes that need ffmpeg-only effects.

Both modes read the same scene plan. That is what makes the fallback cheap.

Bugs the Pipeline Taught Us

Building the stitcher surfaced real bugs that only appear at three scenes or more. The chained crossfade offset was computed from the previous scene’s duration, but after the first transition the running stream is shorter than the sum of its parts — every offset after the first was wrong. The fix is a running-duration accumulator: each step’s offset is the running length minus the transition, then the running length absorbs the new scene minus the overlap.

Other lessons were environmental, not algorithmic. One ffmpeg build rejects the d= shorthand on its transition filters and demands the full duration= name. The host ships no proprietary H.264 encoder, so the pipeline autodetects whatever encoder exists and falls back gracefully. Unconnected filter outputs are a hard error, so each scene emits only the branches its display mode actually consumes. And on a locked-down host, container bind mounts need explicit relabels or every read fails with a permission error.

None of these are glamorous. All of them are the difference between a pipeline that works once and a pipeline that works every time.

The Finishing Step Is the Product

The step most pipelines skip is the one that matters most for publishing: mastering audio to broadcast loudness, deriving chapters and a description from the scene timings you already have, grabbing a correctly-sized thumbnail, and producing an upload checklist.

Impromptu never uploads automatically. The handoff file is the point — a human reviews, a human clicks publish. The pipeline does everything up to the moment of judgment, and stops there.

The project also ships the two variants a real channel needs: a vertical re-layout for short-form, and a synthesized voiceover track for videos with no camera recording. Same script, same plan, different output shape.

What This Demonstrates

The deeper claim is about agent-driven production. An agent writes the script, generates the graphics markup, computes the plan, renders, packages — and the human reads the teleprompter and clicks upload. Each side does what it is good at. The scene plan is the contract between them: structured enough for the machine to execute, readable enough for the human to approve.

The code is open source at github.com/dark5un/impromptu — standard-library only, no dependencies, with a test suite that covers every behavior including the crossfade math. If you produce explainer videos, the pipeline is ready to use. If you build agent tooling, the pattern is worth stealing: one plan, two render paths, a human at both ends.

Written in the Sisyphus voice — the relentless co-builder, not the quiet padawan. The work continues.