How to reference sys.stdout created outside of a DAG to be used inside of a DAG with withParam?
A bit of background about template outputs
Argo Workflows supports a number of different types of templates.
Each type of template supports different types of reference within the template.
Within a steps
template, you may access the output parameters of other steps with steps.step-name.outputs.parameters.param-name
(for named parameters) or steps.step-name.outputs.result
(for the stdout of a script
or container
template).
Example (see full Workflow):
- name: output-parameter steps: - - name: generate-parameter template: whalesay - - name: consume-parameter template: print-message arguments: parameters: - name: message value: "{{steps.generate-parameter.outputs.parameters.hello-param}}"
Within a dag
template, you may access the output of various tasks using a similar syntax, just using tasks.
instead of steps.
.
Example (see full Workflow):
- name: main dag: tasks: - name: flip-coin template: flip-coin - name: heads depends: flip-coin template: heads when: "{{tasks.flip-coin.outputs.result}} == heads" - name: tails depends: flip-coin template: tails when: "{{tasks.flip-coin.outputs.result}} == tails"
Within a container
or script
template, you may access only the inputs of that template*. You may not directly access the outputs of steps or tasks from steps or tasks templates from a container or script template.
Referencing a step output from a DAG
As mentioned above, a DAG template cannot directly reference step outputs from a steps
template. But a step within a steps
template can pass a step output to a DAG template.
In your example, it would look something like this:
templates: - name: the-entrypoint steps: - - name: step01 template: first-step - - name: step02 template: second-step - - name: step03 template: third-step - - name: step04-the-dag-step template: fourth-step arguments: parameters: - name: some-param value: "{{steps.step03.outputs.result}}" - name: fourth-step inputs: parameters: - name: some-param dag: tasks: # use the input parameter in the fourth-step template with "{{inputs.parameters.some-param}}"
tl;dr
steps.
and tasks.
variables are meant to be referenced within a single steps- or tasks-template, but they can be explicitly passed between templates. If you need to use the output of a step in a DAG, directly pass that output as an argument where the DAG is invoked.
In your case, the DAG template is invoked as the last of four steps, so that is where you will pass the argument.
* Okay, you also have access to various other variables from within a script
or container
template, but you don't have access to variables that are scoped as internal variables within another template.