Master the Art of Submitting an Argo-Workflow Using Cron Workflow

Overview

Argo Workflows is a powerful open-source container-native workflow engine designed for orchestrating parallel jobs on Kubernetes. It’s a fantastic tool for automating complex job workflows and has carved a niche in the DevOps community for its flexibility and scalability.

But did you know you can submit one workflow from within another? This advanced technique allows you to create modular and reusable workflow components, simplifying complex operations and reducing redundancy.

Prerequisites

Before diving into the procedure, ensure you have the following prerequisites:

1. A Kubernetes Cluster: Make sure you have access to a Kubernetes cluster. This could be a local setup using Minikube or a managed cluster on a cloud provider like GKE, EKS, or AKS.
2. Argo Workflows Installed: You need to have Argo Workflows installed and configured in your Kubernetes cluster. Installation instructions can be found in the official Argo Workflows documentation.
3. Basic Understanding of YAML: Workflows in Argo are defined using YAML, so familiarity with YAML is essential.
4. Kubectl Command-line Tool: Ensure you have kubectl installed and configured to communicate with your Kubernetes cluster.

Procedure

Step 1: Create the Primary Workflow

Below is a cron workflow example that runs a secondary workflow.

apiVersion: argoproj.io/v1alpha1
kind: CronWorkflow
metadata:
  name: cron-workflow-a
spec:
  schedule: "*/15 * * * *"
  workflowSpec:
    entrypoint: main
    templates:
    - name: main
      steps:
      - - name: task-a
          template: task-a
      - - name: submit-b
          template: submit-b
          when: "{{steps.task-a.status}} == Succeeded"

    - name: task-a
      container:
        image: alpine:latest
        command: [echo]
        args: ["Hello from Cron Workflow A!"]

    - name: submit-b
      resource:
        action: create
        manifest: |
          apiVersion: argoproj.io/v1alpha1
          kind: Workflow
          metadata:
            generateName: workflow-b-
          spec:
            workflowTemplateRef:
              name: workflow-template-b

Step 2: Define the template for Secondary Workflow

Here u will be creating a workflow template for your secondary workflow:

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: workflow-template-b
spec:
  entrypoint: main
  templates:
  - name: main
    steps:
    - - name: print-message
        template: print-message

  - name: print-message
    container:
      image: alpine:latest
      command: [echo]
      args: ["Hello from Workflow B!"]

Also define the same(workflow template) for primary workflow as shown like below.

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: workflow-template-A
spec:
  entrypoint: main
  templates:
  - name: main
    steps:
    - - name: print-message
        template: print-message

  - name: task-a
    container:
      image: alpine:latest
      command: [echo]
      args: ["Hello from Workflow A!"]

Upload the above yaml files to argo workflow. Next submit the primary cron-workflow

Step 3: Submit the Primary Workflow

Submit the primary workflow using the argo command-line tool:

argo submit -n argo primary-workflow.yaml

This command uploads the primary workflow to the Argo Controller, which then executes the steps defined, including submitting the secondary “hello-world” workflow.

Step 4: Monitor Workflows

To monitor the progress of both workflows, use the Argo UI or the command line:

argo list -n default
argo get -n default <primary-workflow-name>
argo get -n default <secondary-workflow-name>

Conclusion :-

Submitting an Argo workflow using another workflow exemplifies the power and flexibility of Argo Workflows. By mastering this technique, you can create more dynamic and modular workflows, paving the way for efficient job orchestration and automation.

Experiment with different workflow configurations and explore how this feature can simplify your scheduling and execution processes. Happy orchestrating!