Celery and Flask in same docker-compose

Solved my problem. I eventually figured out I could get a command prompt on the Docker image:

docker build -t <image name> .docker run -it <image name> /bin/bash

Then trying to run celery within the container revealed the problem:

root@4a6edc5d7372:/usr/src/app# celery -A app.celery worker --loglevel=infoRunning a worker with superuser privileges when theworker accepts messages serialized with pickle is a very bad idea!If you really want to continue then you have to set the C_FORCE_ROOTenvironment variable (but please think about this before you do).User information: uid=0 euid=0 gid=0 egid=0

Docker usually runs as root, and Celery doesn't like running as root for security reasons (I believe you can get code execution with pickle deserialization). The safer solution was to set the celery container to run as nobody. Working docker-compose.yml:

flask:    build: .    command: "python3 app.py"    ports:        - '5000:5000'    links:        - redis        - celery    volumes:        - .:/usr/src/app:rocelery:    build: .    command: "celery -A app.celery worker --loglevel=info"    user: nobody    links:        - redis    volumes:        - .:/usr/src/app:roredis:    image: redis    ports:        - '6379:6379'

if you dont want to deal with docker-compose and just want to use docker only this may be useful for you. First you need to create a "supervisord" file like below.

[supervisord]nodaemon=true[program:python]stdout_logfile=/dev/stdoutstdout_logfile_maxbytes=0stderr_logfile=/dev/stderrstderr_logfile_maxbytes=0command=python flask_api.py[program:celeryworker]stdout_logfile=/dev/stdoutstdout_logfile_maxbytes=0stderr_logfile=/dev/stderrstderr_logfile_maxbytes=0command=celery -A flask_api worker -c 2

Then your docker file should look like this. Obviously this is just a template you can change it as you want.

FROM ubuntu:16.04RUN apt-get update -yRUN apt-get install -y python-pip python-dev build-essential libpq-dev git postgresql-9.5 postgresql-contribRUN apt-get install -y supervisorRUN pip install --upgrade pipCOPY . /appWORKDIR /appRUN pip install -r requirements.txtWORKDIR /appCMD ["supervisord"]

In this way, you can run the flask application and the celery worker in the same docker container.