Integrating Python Poetry with Docker
There are several things to keep in mind when using poetry
together with docker
.
Installation
Official way to install poetry
is via:
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
This way allows poetry
and its dependencies to be isolated from your dependencies. But, in my point of view, it is not a very good thing for two reasons:
poetry
version might get an update and it will break your build. In this case you can specifyPOETRY_VERSION
environment variable. Installer will respect it- I do not like the idea to pipe things from the internet into my containers without any protection from possible file modifications
So, I use pip install 'poetry==$POETRY_VERSION'
. As you can see, I still recommend to pin your version.
Also, pin this version in your pyproject.toml
as well:
[build-system]# Should be the same as `$POETRY_VERSION`:requires = ["poetry>=1.0"]build-backend = "poetry.masonry.api"
It will protect you from version mismatch between your local and docker
environments.
Caching dependencies
We want to cache our requirements and only reinstall them when pyproject.toml
or poetry.lock
files change. Otherwise builds will be slow. To achieve working cache layer we should put:
COPY poetry.lock pyproject.toml /code/
After the poetry
is installed, but before any other files are added.
Virtualenv
The next thing to keep in mind is virtualenv
creation. We do not need it in docker
. It is already isolated. So, we use poetry config virtualenvs.create false
setting to turn it off.
Development vs Production
If you use the same Dockerfile
for both development and production as I do, you will need to install different sets of dependencies based on some environment variable:
poetry install $(test "$YOUR_ENV" == production && echo "--no-dev")
This way $YOUR_ENV
will control which dependencies set will be installed: all (default) or production only with --no-dev
flag.
You may also want to add some more options for better experience:
--no-interaction
not to ask any interactive questions--no-ansi
flag to make your output more log friendly
Result
You will end up with something similar to:
FROM python:3.6.6-alpine3.7ARG YOUR_ENVENV YOUR_ENV=${YOUR_ENV} \ PYTHONFAULTHANDLER=1 \ PYTHONUNBUFFERED=1 \ PYTHONHASHSEED=random \ PIP_NO_CACHE_DIR=off \ PIP_DISABLE_PIP_VERSION_CHECK=on \ PIP_DEFAULT_TIMEOUT=100 \ POETRY_VERSION=1.0.0# System deps:RUN pip install "poetry==$POETRY_VERSION"# Copy only requirements to cache them in docker layerWORKDIR /codeCOPY poetry.lock pyproject.toml /code/# Project initialization:RUN poetry config virtualenvs.create false \ && poetry install $(test "$YOUR_ENV" == production && echo "--no-dev") --no-interaction --no-ansi# Creating folders, and files for a project:COPY . /code
You can find a fully working real-life example here: wemake-django-template
Update on 2019-12-17
- Update
poetry
to 1.0
Multi-stage Docker build with Poetry and venv
Do not disable virtualenv creation. Virtualenvs serve a purpose in Docker builds, because they provide an elegant way to leverage multi-stage builds. In a nutshell, your build stage installs everything into the virtualenv, and the final stage just copies the virtualenv over into a small image.
Use poetry export
and install your pinned requirements first, before copying your code. This will allow you to use the Docker build cache, and never reinstall dependencies just because you changed a line in your code.
Do not use poetry install
to install your code, because it will perform an editable install. Instead, use poetry build
to build a wheel, and then pip-install that into your virtualenv. (Thanks to PEP 517, this whole process could also be performed with a simple pip install .
, but due to build isolation you would end up installing another copy of Poetry.)
Here's an example Dockerfile installing a Flask app into an Alpine image, with a dependency on Postgres. This example uses an entrypoint script to activate the virtualenv. But generally, you should be fine without an entrypoint script because you can simply reference the Python binary at /venv/bin/python
in your CMD
instruction.
Dockerfile
FROM python:3.7.6-alpine3.11 as baseENV PYTHONFAULTHANDLER=1 \ PYTHONHASHSEED=random \ PYTHONUNBUFFERED=1WORKDIR /appFROM base as builderENV PIP_DEFAULT_TIMEOUT=100 \ PIP_DISABLE_PIP_VERSION_CHECK=1 \ PIP_NO_CACHE_DIR=1 \ POETRY_VERSION=1.0.5RUN apk add --no-cache gcc libffi-dev musl-dev postgresql-devRUN pip install "poetry==$POETRY_VERSION"RUN python -m venv /venvCOPY pyproject.toml poetry.lock ./RUN poetry export -f requirements.txt | /venv/bin/pip install -r /dev/stdinCOPY . .RUN poetry build && /venv/bin/pip install dist/*.whlFROM base as finalRUN apk add --no-cache libffi libpqCOPY --from=builder /venv /venvCOPY docker-entrypoint.sh wsgi.py ./CMD ["./docker-entrypoint.sh"]
docker-entrypoint.sh
#!/bin/shset -e. /venv/bin/activatewhile ! flask db upgradedo echo "Retry..." sleep 1doneexec gunicorn --bind 0.0.0.0:5000 --forwarded-allow-ips='*' wsgi:app
wsgi.py
import your_appapp = your_app.create_app()
That's minimal configuration that works for me:
FROM python:3.7ENV PIP_DISABLE_PIP_VERSION_CHECK=onRUN pip install poetryWORKDIR /appCOPY poetry.lock pyproject.toml /app/RUN poetry config virtualenvs.create falseRUN poetry install --no-interactionCOPY . /app
Note that it is not as safe as @sobolevn's configuration.
As a trivia I'll add that if editable installs will be possible for pyproject.toml
projects, a line or two could be deleted:
FROM python:3.7ENV PIP_DISABLE_PIP_VERSION_CHECK=onWORKDIR /appCOPY poetry.lock pyproject.toml /app/RUN pip install -e .COPY . /app