Compilation of Elyra-Pipelines to Tekton based Kubeflow fails Compilation of Elyra-Pipelines to Tekton based Kubeflow fails kubernetes kubernetes

Compilation of Elyra-Pipelines to Tekton based Kubeflow fails


There are different aspects to consider here:

There is a need to perform a custom deployment to enable Kubeflow Pipelines to integrate/use a Tekton environment. As you mentioned the steps are described here.

As for Elyra support, Elyra recent releases starting with Elyra 2.x has incorporated support for KFP using Tekton and further documentation is available in the Elyra user guide

If you are still using Elyra 1.x or lower, the below still applies:

Now, related to compiling and executing the Kubeflow Pipeline on a Tekton environment, Elyra is currently using the KFP SDK python package and the regular compiler to compile and generate the default ARGO YAML (see code here). In order to support Tekton, we will need to use a different python package (kfp-tekton==0.3.0) and there is also a need for a different code path during compilation (see example).

Another aspect is that it seems that there are some incompatibilities between the two pipelines as described in the migration path.

Regarding support, the Elyra project would welcome contributions.


As of now the Tekton compiler is in a separate package. You can install it with pip install kfp-tekton==0.3.0 for kubeflow 1.2 . Here is the user guide

Currently, Elyra doesn't support compiling for kfp-tekton, only kfp-argo

There is an open Issue on that with the Elyra team