Submitting a Workflow Template in Argo-Workflow Using a Lambda Function

Introduction :-

Welcome to the world of automation and efficient workflow management! Argo-Workflows offers a powerful way to manage complex workflows on Kubernetes. If you’re looking to streamline the process of submitting workflow templates, you’re in the right place. This blog post will guide you through submitting a workflow template in Argo-Workflow using an AWS Lambda function. Let’s dive in!

Pre-requisites :-

Before we jump into the procedure, let’s cover the essentials you’ll need:

• Kubernetes Cluster: A running Kubernetes cluster where you have admin access. If you don’t already have one, you can easily set up a local cluster using Minikube or use a managed Kubernetes service like EKS (Amazon Elastic Kubernetes Service).
• Argo-Workflows Installed: Argo-Workflows should be installed in your Kubernetes cluster. Follow the official Argo-Workflows installation guide if you haven’t set it up yet.
• AWS Account: An active AWS account with IAM permissions set up to create and manage Lambda functions and other dependent services.
• AWS CLI & Kubectl Installed: Ensure you have both AWS CLI and kubectl installed and configured on your local machine.

Procedure :-

Here’s a step-by-step guide to submitting a workflow template in Argo-Workflow using an AWS Lambda function:

Step 1: Create the Workflow Template

First, you need to define your workflow template in YAML format. Here’s a simple example template named hello-world.yaml:

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: hello-world-template
spec:
  entrypoint: whalesay
  templates:
  - name: whalesay
    container:
      image: docker/whalesay
      command: ["cowsay"]
      args: ["hello world"]

Save this YAML file for later use.

Step 2: Deploy the Workflow Template

Use the following command to deploy the workflow template to your Kubernetes cluster:

kubectl apply -f hello-world.yaml

Verify the template is successfully submitted by running:

kubectl get workflowtemplates

You should see hello-world-template listed in the output.

Step 3: Setup an AWS Lambda Function

Log into your AWS Management Console and navigate to the Lambda service. Create a new Lambda function with the following basic configuration:

• Function Name: SubmitArgoWorkflow
• Runtime: Python 3.x
• Role: Create a new role with basic Lambda permissions

Step 4: Write Your Lambda Function

Replace the auto-generated code in Lambda with the following Python code. This script will trigger the workflow template you deployed earlier:

import boto3
import os
def lambda_handler(event, context):
    # Set up boto3 Kubernetes client
    eks_client = boto3.client('eks')
    kube_config = eks_client.generate_kubeconfig(
        name='your-cluster-name'
    )
    
    configuration = Configuration()
    configuration.host = kube_config['cluster']['server']
    configuration.verify_ssl = False
    
    client.Configuration.set_default(configuration)
    v1 = client.Apiv1Api()
    
    # Define the workflow parameters
    workflow = {
        'apiVersion': 'argoproj.io/v1alpha1',
        'kind': 'Workflow',
        'metadata': {
            'generateName': 'hello-world-'
        },
        'spec': {
            'entrypoint': 'whalesay',
            'templates': [{
                'name': 'whalesay',
                'container': {
                    'image': 'docker/whalesay',
                    'command': ['cowsay'],
                    'args': ['hello world']
                }
            }]
        }
    }
    
    # Create the workflow in the Kubernetes cluster
    v1.create_namespaced_custom_object(
        group="argoproj.io",
        version="v1alpha1",
        namespace="default",
        plural="workflows",
        body=workflow
    )
    
    return {
        'statusCode': 200,
        'body': 'Workflow submitted successfully!'
    }

Don’t forget to update 'your-cluster-name' with your actual EKS cluster name.

Step 5: Test the Lambda Function

After writing your Lambda function, it’s time to test it. Use the AWS Lambda console to create a test event, and run your function. You should see Workflow submitted successfully! in the output if everything was configured correctly.

Step 6: Verify Workflow Submission

Go back to your Kubernetes cluster and ensure the workflow has been submitted correctly. Use the following command to check:

kubectl get workflows

Conclusion :-

Congratulations! You’ve successfully submitted a workflow template in Argo-Workflow using an AWS Lambda function. This automation can significantly streamline your workflow management, making it easier to trigger complex workflows on Kubernetes with minimal manual effort.