Airflow: how to get pip packages installed via their docker-compose.yml?

There is a pretty detailed guide on how to achieve what you are looking for on the Airflow docs here. Depending on your requirements, this may be as easy as extending the original image using a From directive while creating a new Dockerfile, or you may need to customize the image to suit your needs.

If you go with the Extending the image approach your new Dockerfile will be something like this:

FROM apache/airflow:2.0.1USER rootRUN apt-get update \  && apt-get install -y --no-install-recommends \         build-essential my-awesome-apt-dependency-to-add \  && apt-get autoremove -yqq --purge \  && apt-get clean \  && rm -rf /var/lib/apt/lists/*USER airflowRUN pip install --no-cache-dir --user my-awesome-pip-dependency-to-add

Then you could just add something like these to the docker-compose file:

...version: "3"x-airflow-common: &airflow-common  build: . # this is optional  image: ${AIRFLOW_IMAGE_NAME:-the_name_of_your_extended_image  ......

Finally, build your image and turn everything back on using compose. Try the docs for details or a full explanation. Hope that works for you!