The Philosophical Developer -- Chapter 29: The Pipeline

The OpenTofu arc had a Dagger pipeline. Every module was formatted, validated, and tested through a containerized CI pipeline that ran on every pull request. It was one of the defining features of the methodology. Infrastructure that is not tested in CI is infrastructure that will fail in production.

The Pulumi port needed the same guarantee. But the Dagger pipeline for the OpenTofu modules was written in a separate Go module with generated SDK code, and the engine was not available in our local distrobox environment. We needed a different approach.

I sat down with OrsonRius and we designed a CI pipeline that achieves the same goals with a simpler toolchain. No Dagger. No generated code. Just Go, Pulumi, and GitHub Actions.

The CI job

The pipeline is a single job in the existing .github/workflows/ci.yaml file. It runs on every push and pull request that touches the infrastructure directory.

pulumi-ci:
  runs-on: ubuntu-latest
  defaults:
    run:
      working-directory: infrastructure/pulumi

  steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-go@v5
      with:
        go-version: '1.26'
        cache: true

    - name: Install Pulumi CLI
      run: |
        curl -fsSL https://get.pulumi.com | sh
        echo "$HOME/.pulumi/bin" >> $GITHUB_PATH

    - name: Build check
      run: go build ./...

    - name: Run unit tests
      run: go test -v -count=1 ./...

The pipeline does four things. It checks out the code, installs Go and Pulumi, compiles the modules, and runs the unit tests. That is it. No Docker, no Dagger, no container orchestration.

Why this works

The unit tests use Pulumi mocks. They do not talk to any cloud provider. They do not need a running container. They do not need credentials. They run in plain GitHub Actions runners with nothing but Go and the Pulumi CLI installed.

The build check ensures the code compiles. The Go compiler catches type errors, missing imports, and API mismatches before the tests even run. The test suite runs in under 25 milliseconds.

The total execution time for the entire pulumi-ci job is under two minutes, most of which is downloading the Go module cache and installing Pulumi. The actual build and test steps take less than a second.

Comparison with the Dagger pipeline

The OpenTofu pipeline uses Dagger containers to run OpenTofu commands. Each step installs OpenTofu in a container, mounts the module directory, and runs tofu plan or tofu test. The container isolation provides reproducibility and consistency across environments.

The Pulumi pipeline does not need containers because the tests are self-contained. The Go binary is compiled once and tested with mocks. There is no provider to install, no plugins to download, no state to manage.

Both approaches achieve the same goal: infrastructure that is tested in CI before it is deployed. The Pulumi approach is simpler to set up and faster to run. The Dagger approach provides stronger isolation and works with any tool that can run in a container.

The verification script

For local development, we also created a verification script that runs the same steps as the CI pipeline. It is useful for testing changes before pushing.

cd infrastructure/pulumi
go build ./...
go test -v -count=1 ./...

What the pipeline caught

The pipeline caught a real bug on the first run. The Azure module had a type mismatch in the key vault secret export. The secret.Name field is a pulumi.StringOutput, not a string, and passing it to pulumi.String() caused a compile error. The Go compiler caught it. The CI pipeline reported it. We fixed it before the code ever reached a reviewer.

That is the value of a CI pipeline. Not just running tests, but running them automatically on every push so that no broken code ever reaches the main branch.

What comes next

The pipeline validates the module code. The next step is to validate the deployed infrastructure. Chapter 30 covers the integration test that deploys the full stack to a local Floci emulator and verifies the outputs.

Repos:

• github.com/dark5un/local-cloud – .github/workflows/ci.yaml

LinkedIn:

• The Philosophical Developer – Chapter 29: The Pipeline

Here I geek out with my young Padawan, OrsonRius.