Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

🏠 Back to Blog

Argo Workflows Labs

Hands-on labs for Argo Workflows (CAPA). All yaml lives in yaml/. Run everything from this directory.

Argo Workflows = container-native workflow engine for kubernetes. Each step runs as a pod. Workflows are a CRD (Workflow), reconciled by the workflow controller.

Lab 0: Setup

Using kind + the quick-start manifest (includes minio as the artifact repo, needed for lab 4).

kind create cluster
kubectl create namespace argo
kubectl apply -n argo -f https://github.com/argoproj/argo-workflows/releases/download/v3.5.10/quick-start-minimal.yaml
kubectl -n argo wait --for=condition=ready pod --all --timeout=180s
  • Check https://github.com/argoproj/argo-workflows/releases for the latest version, 3.5.10 is just what I pinned
  • Install the argo CLI from the same releases page

UI:

kubectl -n argo port-forward svc/argo-server 2746:2746
  • Browse to https://localhost:2746 (self signed cert, quick-start runs the server with --auth-mode=server so no login)

Verify:

  • kubectl get pods -n argo shows workflow-controller, argo-server, minio, postgres
  • argo version works
  • UI loads

Cleanup between labs: argo delete --all -n argo

Lab 1: Hello World

Goal: submit a single container workflow and read its logs.

argo submit -n argo --watch yaml/01-hello.yaml
argo list -n argo
argo logs -n argo @latest
  • generateName gives each run a unique name (hello-xxxxx)
  • entrypoint picks which template runs first
  • @latest = most recent workflow

Try:

  • Find the pod with kubectl get pods -n argo. Why does it have 2 containers (main and wait)?
  • View the workflow in the UI
  • kubectl get wf -n argo and kubectl get wf -n argo -o yaml (look at status.nodes)
  • Change the image/args and resubmit

Lab 2: Steps and Parameters

Goal: sequential vs. parallel steps, passing parameters.

argo submit -n argo --watch yaml/02-steps-params.yaml
argo submit -n argo --watch yaml/02-steps-params.yaml -p message=goodbye
  • Outer list = sequential. Inner list = parallel
  • workflow.parameters.x are global, inputs.parameters.x are per template

Try:

  • Confirm parallel-a and parallel-b start at the same time
  • Override the global param with -p
  • Add a 4th group that depends on nothing else

Lab 3: DAGs

Goal: express dependencies as a graph instead of ordered steps.

argo submit -n argo --watch yaml/03-dag.yaml
  • Diamond: A > (B, C) > D
  • DAG tasks start as soon as their dependencies are done. Steps groups wait for the entire previous group

Try:

  • Add sleep 10 to B only. Does C finish first? Does D wait?
  • Add task E that depends on C only. Does it wait for B?
  • Make C fail (exit 1). What happens to D? (look at failFast)
  • Look at the graph in the UI

Lab 4: Outputs and Artifacts

Goal: pass data between steps.

argo submit -n argo --watch yaml/04-outputs-artifacts.yaml
argo logs -n argo @latest
  • Output parameter: small value read from a file (valueFrom.path). Stored in the workflow status
  • Artifact: a file/dir, stored in the artifact repo (minio here). For larger data
  • Artifacts need an artifact repository configured, that is why we used quick-start

Try:

  • Download the artifact from the UI
  • Find the artifact in minio (kubectl -n argo port-forward svc/minio 9001:9001, login admin / password)
  • What happens to the workflow if you remove the artifact repo config? (kubectl get cm -n argo)

Lab 5: Loops and Conditionals

Goal: dynamic fan-out and branching.

argo submit -n argo --watch yaml/05-loops-conditionals.yaml
  • withItems creates one pod per item
  • when evaluates an expression, skipped steps show as Skipped
  • script templates put stdout in outputs.result

Try:

  • Run it a few times. Does the branch change?
  • Use withParam with a JSON list from a previous step instead of withItems
  • Limit parallelism with spec.parallelism: 1

Lab 6: WorkflowTemplates

Goal: reuse templates across workflows.

kubectl apply -f yaml/06-workflow-template.yaml
argo template list -n argo
argo submit -n argo --watch yaml/06-use-template.yaml
  • WorkflowTemplate = namespaced and reusable. ClusterWorkflowTemplate = cluster wide
  • templateRef calls a template from another WorkflowTemplate
  • 06-workflow-template.yaml is the template, 06-use-template.yaml is a workflow that calls it

Try:

  • Add spec.entrypoint: greet to the template, then run it directly: argo submit -n argo --watch --from workflowtemplate/greeter -p name=bob
  • Convert it to a ClusterWorkflowTemplate (templateRef.clusterScope: true)

Lab 7: CronWorkflows

Goal: run workflows on a schedule.

kubectl apply -f yaml/07-cron.yaml
argo cron list -n argo
# wait a couple minutes
argo list -n argo
  • Same cron syntax as k8s CronJob
  • concurrencyPolicy: Allow / Forbid / Replace

Try:

  • argo cron suspend every-minute -n argo then resume
  • Set concurrencyPolicy: Forbid and make the job sleep 90s. What happens?
  • Clean up: argo cron delete every-minute -n argo

Lab 8: Retries and Exit Handlers

Goal: handle failure.

argo submit -n argo --watch yaml/08-retry-exit.yaml
argo get -n argo @latest
  • retryStrategy: limit, policy (Always, OnFailure, OnError), backoff
  • onExit template always runs, workflow.status is available in it

Try:

  • Lower the retry limit to 1 until you get a failed workflow. Does the exit handler still run?
  • Add activeDeadlineSeconds to the template and a long sleep
  • argo retry @latest -n argo on a failed workflow

Lab 9: Template Types

Goal: use each of the 7 template definition types. Files are in yaml/templates/.

Templates are either definitions (do work) or invocators (call other templates: steps and dag, labs 2/3).

kubectl apply -f yaml/templates/00-rbac.yaml   # needed for resource + http
for f in yaml/templates/0[1-7]*.yaml; do argo submit -n argo --watch $f; done
TypeWhat it doesFile
containerrun one container01-container.yaml
scriptinline source, stdout in outputs.result02-script.yaml
resourcecreate/patch/delete a k8s resource, no kubectl image03-resource.yaml
suspendpause until resumed or a duration passes04-suspend.yaml
containerSetmultiple containers in one pod, with dependencies05-container-set.yaml
httpHTTP request from the agent pod, no pod per step06-http.yaml
datasource artifacts + transform with an expression07-data.yaml
  • suspend (no duration): argo resume @latest -n argo
  • containerSet shares volumes, so no artifacts needed between containers. All containers in the set run in one pod
  • resource and http need RBAC for the SA (00-rbac.yaml), otherwise they fail with forbidden
  • (plugin is an 8th, needs an executor plugin installed, skipped)

Try:

  • kubectl get cm -n argo after the resource lab. Change action to delete and clean up
  • Count the pods for 05-container-set vs. the same thing as 3 separate steps
  • Make the http successCondition fail (== 404) and look at the result
  • Add an outputs to the http template and use response.body

Challenges

No solutions. Build these yourself:

  • CI-like pipeline: clone > build > test > report, as a DAG, with the test step fanning out over 3 versions
  • Use a shared volume (volumeClaimTemplates) instead of artifacts to pass files between steps
  • Use suspend to build a manual approval gate, resume with argo resume
  • Use a resource template to create a k8s ConfigMap from a workflow
  • Create an ArgoCD Application that deploys a WorkflowTemplate from git (ties back to the earlier notes)

Cheatsheet

argo submit -n argo FILE [--watch] [-p k=v] [--from workflowtemplate/NAME]
argo list -n argo
argo get -n argo NAME|@latest
argo logs -n argo NAME|@latest [-f]
argo watch -n argo NAME|@latest
argo retry|stop|terminate|resubmit -n argo NAME
argo suspend|resume -n argo NAME
argo delete -n argo NAME | --all
argo template list|create|delete
argo cron list|create|suspend|resume|delete
argo lint FILE