Add custom volumeMount into Airflow worker pods (with k8s Executor) Add custom volumeMount into Airflow worker pods (with k8s Executor) kubernetes kubernetes

Add custom volumeMount into Airflow worker pods (with k8s Executor)


Sorry i'd to answer my own question. Maybe will help someone. Inside the dag file, where I define the task, I simply had to add the executor_config as follows;

        IngestionStatusSensor(            task_id=...,            executor_config={"KubernetesExecutor": {                                  "image": "airflow:runner2",                                  "volume_mounts": [                                      {                                          "name": "airflow-dags",                                          "mountPath": "/usr/local/airflow/libs",                                          "subPath": "airflow/development/libs"                                      },                                      {                                          "name": "airflow-dags",                                          "mountPath": "/usr/local/airflow/plugins",                                          "subPath": "airflow/development/plugins"                                      }],                                  }                             },            dag=dag,            ingestion_feed=table,            poke_interval=60 * 30,            offset=0,            start_date=start_date        )

where airflow-dags volume is already defined by pod creator that claims the PVC defined in configs under kubernetes section of airflow.conf e.g. dags_volume_claim = airflow-pvc.