Reuse tasks in airflow
- Just like
object
is an instance of aclass
, an Airflow task is an instance of anOperator
(strictly speaking,BaseOperator
) - So write a "re-usable" (aka generic) operator and use it 100s of times across your pipeline(s) simply by passing different params (particularly
task_id
)