couldn't start Celery with docker-compose couldn't start Celery with docker-compose flask flask

couldn't start Celery with docker-compose


First of all the celery image is deprecated in favour of standard python image more info here.

WORKDIR sets the working directory for all the command after it is defined in the Dockerfile, which means the command which you are try to run will run from that directory. Docker image for celery sets the working directory to /home/user.

Since your code is mounted on /celery_smaple and the working directory is /home/user, Celery is not able to find your python module.

One alternative is to cd into the mounted directory and execute the command:

celery:    image: celery:3.1.25    command: "cd /celery_sample && celery worker -A my_celery -l INFO"    volumes:      - .:/celery_sample    networks:      - webnet

notice the command

And another one is to create your own image with WORKDIR set to /celery_sample eg:

FROM python:3.5RUN pip install celery==3.1.25WORKDIR /celery_sample

after building you own image you can use the compose file by changing the image of celery service

Edit

You need to link the services to one another in order to communicate:

version: "3"services:  web:    build:      context: .      dockerfile: Dockerfile    command: "python my_celery.py"    ports:      - "8000:8000"    networks:      - webnet    volumes:      - .:/celery_sample    links:      - redis  redis:    image: redis    networks:      - webnet  celery:    image: celery:3.1.25    command: "celery worker -A my_celery -l INFO"    volumes:      - .:/home/user    networks:      - webnet    links:      - redisnetworks:  webnet:

and your configuration file should be:

## Broker settings.BROKER_URL = 'redis://redis:6379/0'    ## Using the database to store task state and results.CELERY_RESULT_BACKEND = 'redis://redis:6379/0'

once you have linked the services in compose file you can access the service by using the service name as the hostname.


The simplest change you can make is to map the directory for celery image correctly in yaml

  celery:    image: celery:3.1.25    command: "celery worker -A my_celery -l INFO"    volumes:      - .:/home/user/    networks:      - webnet

Also as @vedarthk pointed, you should be using the official Python image for this instead of celery image