Dagger on Backend Engineering Strategy Tools

Image Tooling

Thu, 18 Jun 2026 00:00:00 +0000

Versioned, multi-arch Docker images for Kubernetes workflows — built with Dagger, published to Docker Hub, triggered by a version tag.

The motivation is in Shared Tooling Images: one image, consistent versions, three contexts — CI, local, colleagues.

Images

GitHub repo	Docker Hub	Contents
`image-tooling`	`best-tools/tooling-k8s`	kubectl, helm, kustomize, argocd CLI, k9s, jq, yq
`image-tooling`	`best-tools/tooling-k8s-aws`	`tooling-k8s` + AWS CLI
`image-tooling`	`best-tools/tooling-k8s-openstack`	`tooling-k8s` + OpenStack CLI
`image-buildx`	`best-tools/buildx`	CI builder — Docker buildx, AWS CLI, Dagger CLI
`image-pandoc`	`best-tools/pandoc`	PDF generation — pandoc + TeX Live

All images publish as multi-arch manifests: linux/amd64 + linux/arm64.

Quick start

Interactive shell with kubeconfig mounted:

docker run -it --rm \
 -v ~/.kube:/mnt/kube:ro \
 -v $(pwd):/work \
 -w /work \
 docker.io/best-tools/tooling-k8s:latest

The image entry point symlinks /mnt/kube → /root/.kube on startup, so kubectl picks it up immediately.

Shell alias for daily use:

alias k8s='docker run -it --rm \
 -v ~/.kube:/mnt/kube:ro \
 -v $(pwd):/work -w /work \
 docker.io/best-tools/tooling-k8s:latest'

k8s helm lint .
k8s kubectl get pods -n argocd

In CI (GitHub Actions):

- name: Lint chart
 run: docker run --rm -v ${{ github.workspace }}:/work -w /work docker.io/best-tools/tooling-k8s:latest helm lint .

Or reference the image directly as the job container — no install step needed.

Setup (contributors / maintainers)

Credentials are set once as GitHub org-level secrets and inherited by all image-* repos automatically.

Secret	Where to get it
`DOCKERHUB_TOKEN`	hub.docker.com → Account → Security → Access Tokens (Read, Write, Delete)
`DAGGER_CLOUD_TOKEN`	cloud.dagger.io → Organisation → Tokens

Path: github.com/Backend-Engineering-Strategy-Tools → Settings → Secrets and variables → Actions → New organisation secret.

Releasing

git tag -a v1.0.0 -m "Release v1.0.0"
git push origin v1.0.0

The GitHub Actions workflow triggers on v*.*.* tags, calls dagger call publish-multi-arch, and pushes both best-tools/<image>:v1.0.0 and best-tools/<image>:latest to Docker Hub. Pipeline trace at cloud.dagger.io.

These Are Not the Pipelines You Are Looking For

Thu, 04 Jun 2026 00:00:00 +0000

There are a lot of CI/CD platforms. GitHub Actions, Tekton, Jenkins, Jenkins X, Harness, Bitbucket Pipelines, GitLab CI, CircleCI, Argo Workflows. The choice between them is a perennial argument — and mostly the wrong one.

The mistake is putting your build logic inside the pipeline. Once you do that, the pipeline owns your build. Reproducing a failure locally means setting up the CI environment. Switching platforms means rewriting all the steps. Testing the pipeline means pushing a commit and waiting. The platform becomes load-bearing.

The fix is straightforward: keep all logic in Make or Dagger. The pipeline calls one command. make build. make test. dagger call publish. That’s it.

The pipeline shrinks to thin orchestration: trigger on a git event, check out the code, call the command, report the result. It does not know what the build does. It does not care. You can swap GitHub Actions for Tekton or Tekton for Argo Workflows and the only thing that changes is the YAML wrapper around the same command.

More importantly: you can run the same command on your laptop. No CI environment to replicate, no “it works locally but fails in CI” gap to debug. The build is the build, wherever it runs.

This matters more the longer a project lives. Build tools consolidate and diverge. Platforms get acquired, deprecated, or repriced. Jenkins pipelines from five years ago are maintained by people who weren’t there when they were written and can’t run them outside of Jenkins. The projects that survive these transitions cleanly are the ones where the build logic was never in the pipeline to begin with — it was in a Makefile or a build tool that could run anywhere.

Gradle, Maven, Make — these predate CI/CD platforms by decades and will outlast the current generation of them. Put your logic there. The pipeline is a trigger and a reporter. Keep it that way.

The pattern in practice:

build:
 docker build -t myimage:$(VERSION) .

test:
 go test ./...

publish:
 docker push myimage:$(VERSION)

# GitHub Actions — the entire pipeline
steps:
 - uses: actions/checkout@v4
 - run: make build
 - run: make test
 - run: make publish

# Tekton — same command, different wrapper
steps:
 - name: build-test-publish
 image: golang:1.22
 script: |
 make build
 make test
 make publish

Same logic. One command. Any pipeline.

Dagger

Tue, 02 Jun 2026 00:00:00 +0000

Your build script works on your laptop. It breaks in CI because a tool version differs. It breaks for a colleague because they’re on a different OS. You’re writing apt-get install steps in YAML and maintaining a separate script for local builds.

Dagger solves this the same way Docker solved “works on my machine” for applications: run everything in containers. Every pipeline step executes inside a defined container image — identically on your laptop, in GitHub Actions, or inside a Kubernetes pod. The Dagger Engine is a containerised daemon your pipeline calls into. Where it runs is an operational detail; what it does is defined once in code.

The other thing Dagger gives you that YAML-based CI systems don’t: your pipeline is real code. Write it in Go (or TypeScript or Python), build a library of reusable functions, share it across projects, and test it the same way you’d test any other code.

Module

A Dagger module is a Go package that exposes pipeline functions. Initialise one in your repo:

dagger init --sdk go --name my-pipeline
dagger develop

dagger init creates the scaffolding. dagger develop generates the Go bindings and drops you into a module ready to edit. A minimal function:

package main

import (
 "context"
 "dagger/my-pipeline/internal/dagger"
)

type MyPipeline struct{}

func (m *MyPipeline) Build(ctx context.Context, src *dagger.Directory) *dagger.Container {
 return dag.Container().
 From("golang:1.22").
 WithDirectory("/src", src).
 WithWorkdir("/src").
 WithExec([]string{"go", "build", "./..."})
}

Call it from anywhere — local terminal, CI step, Argo workflow:

dagger call build --src .

Library pattern

Because a module is just Go code, you build it like any other Go library: shared types, helper functions, unit tests. The pattern that works well in practice is a single internal module that codifies your organisation’s conventions — base images, registry credentials, standard lint and test steps — and each project imports it.

A fix to the shared module propagates everywhere. You’re not updating 40 YAML files.

Testing pipeline logic with go test works normally. A test that calls dagger call exercises the real container execution, not a mock. This is the thing YAML pipelines can’t give you: a fast feedback loop on the pipeline logic itself before it hits CI.

Multi-arch builds

Dagger has first-class support for multi-architecture container builds. Pass the platform as an argument:

func (m *MyPipeline) BuildMultiArch(ctx context.Context, src *dagger.Directory) []*dagger.Container {
 platforms := []dagger.Platform{"linux/amd64", "linux/arm64"}
 containers := make([]*dagger.Container, len(platforms))
 for i, platform := range platforms {
 containers[i] = dag.Container(dagger.ContainerOpts{Platform: platform}).
 From("golang:1.22").
 WithDirectory("/src", src).
 WithWorkdir("/src").
 WithExec([]string{"go", "build", "./..."})
 }
 return containers
}

Same function, multiple targets, no extra tooling.

Running in Kubernetes with Argo Workflows

Dagger Engine runs as a container. In Kubernetes, you run it as a sidecar or dedicated pod and point dagger call at it via the _EXPERIMENTAL_DAGGER_RUNNER_HOST environment variable. Each Argo Workflows step calls dagger call <function> — Argo handles DAG orchestration, retries, and observability; Dagger handles the build execution inside containers.

The split is clean: Argo owns workflow-level concerns, Dagger owns build reproducibility.

Key commands

Command	What it does
`dagger init --sdk go`	Initialise a new module with the Go SDK
`dagger develop`	Generate bindings, enter the development loop
`dagger functions`	List available functions in the current module
`dagger call <function> [args]`	Invoke a pipeline function
`dagger run <command>`	Run an arbitrary command with Dagger Engine available

Dagger Cloud

Dagger Cloud is a hosted observability layer for Dagger pipelines. Free tier available.

Every pipeline run — local or CI — produces a trace: a visualisation of the container graph with per-step timing, cache hit/miss, and log output. The trace is linked from the terminal output and browsable at cloud.dagger.io.

Enable it by setting one environment variable:

export DAGGER_CLOUD_TOKEN=<token>
dagger call build --src .
# → Run URL: https://app.dagger.cloud/runs/<id>

In GitHub Actions, add DAGGER_CLOUD_TOKEN as an organisation-level secret (see GitHub Actions). The dagger/dagger-action picks it up automatically — no workflow change needed beyond the env var being present.