Kubernetes can analytical jobs be chained together in a workflow?

I have used initContainers under the PodSpec in the past to solve problems like this: https://kubernetes.io/docs/concepts/workloads/pods/init-containers/

apiVersion: v1kind: Podmetadata:  name: myapp-pod  labels:    app: myappspec:  containers:  - name: myapp-container    image: busybox    command: ['sh', '-c', 'echo The app is running! && sleep 3600']  initContainers:  - name: init-myservice    image: busybox    command: ['sh', '-c', 'until nslookup myservice; do echo waiting for myservice; sleep 2; done;']  - name: init-mydb    image: busybox    command: ['sh', '-c', 'until nslookup mydb; do echo waiting for mydb; sleep 2; done;']

Take a look here for the chaining of containers using the "depends" keyword is also an option:

https://github.com/kubernetes/kubernetes/issues/1996

Overall, no. Check out things like Airflow for this. Job objects give you a pretty simple way to run a container until it completes, that's about it. The parallelism is in that you can run multiple copies, it's not a full workflow management system :)

It is not possible to manage job workflows with Kubernetes core API objects.

• Argo Workflow looks like an interesting tool to manage workflow inside Kubernetes: https://argoproj.github.io/projects/argo.
  • It looks like it can handle Kubernetes jobs workflow: https://argoproj.github.io/argo/examples/#kubernetes-resources
  • It is in the CNCF incubator: https://www.cncf.io/blog/2020/04/07/toc-welcomes-argo-into-the-cncf-incubator/

Other alternatives include:

• https://www.nextflow.io/
• https://www.pachyderm.com/
• Airflow: https://airflow.apache.org/docs/apache-airflow/stable/kubernetes.html

This document might also help: https://www.preprints.org/manuscript/202001.0378/v1/download