KEYSTORE.JKS exists FAILED - exited with code 1 #662 - Confluent kafka
Below are the steps that one can use to start the kafka docker-compose with the SSL support (@Senthil already provided some guidance in his comments)
in the docker-compose directory there is a so called secrets directory which contains the shell script for generating the keystore, truststore and ssl passwords. Go into the root of the docker-compose for kafka and run this script that will generate the needed files (eg:
./secrets/create-certs)copy all the generated files into the secrets directory
mount the volume of the secrets directory from the host machine to the dockerized one. put the following on the docker-compose file at the volumes sections
volumes: - ./secrets/:/etc/kafka/secrets
Run with docker-compose up
FWIW, here is what I used to resolve this and what issues I have run into with it. Here is part of my docker compose file. If you were to open the file kafka_Secret.txt, you would see only P@ssword in it. A problem I want into is that - ./kafka/secrets:/etc/kafka/secrets was set up as a volume instead of a bind mount. I confirmed this by running container inspect . (Get the container name by running docker container ls). It showed a volume mount instead of a bind mount. To fix it, I deleted the volumes from my docker to start over. The volume that hung around kept attaching to my kafka container even if I recreated the container.
zookeeper:image: zookeeper:3.4.9hostname: zookeeperports: - '2181:2181'environment: ZOO_MY_ID: 1 ZOO_PORT: 2181 ZOO_SERVERS: server.1=zookeeper:2888:3888 ZOO_LOG4J_PROP: "${KAFKA_LOG_LEVEL},CONSOLE"networks: - ms_networkvolumes: - ./kafka/zookeeper/data:/data - ./kafka/zookeeper/datalog:/datalogkafka:image: confluentinc/cp-kafka:5.5.0hostname: kafkaports: - '19092:19092'environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181' KAFKA_ADVERTISED_LISTENERS: SSL://kafka:19092 KAFKA_SSL_KEYSTORE_FILENAME: keystore.jks KAFKA_SSL_KEYSTORE_CREDENTIALS: kafka_secret.txt KAFKA_SSL_KEY_CREDENTIALS: kafka_secret.txt KAFKA_SSL_TRUSTSTORE_FILENAME: truststore.jks KAFKA_SSL_TRUSTSTORE_CREDENTIALS: kafka_secret.txt KAFKA_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: " " KAFKA_SSL_CLIENT_AUTH: requested KAFKA_SECURITY_INTER_BROKER_PROTOCOL: SSL KAFKA_LOG4J_LOGGERS: 'org.apache.zookeeper=${KAFKA_LOG_LEVEL},org.apache.kafka=${KAFKA_LOG_LEVEL},kafka=${KAFKA_LOG_LEVEL},kafka.cluster=${KAFKA_LOG_LEVEL},kafka.controller=${KAFKA_LOG_LEVEL},kafka.coordinator=${KAFKA_LOG_LEVEL},kafka.log=${KAFKA_LOG_LEVEL},kafka.server=${KAFKA_LOG_LEVEL},kafka.zookeeper=${KAFKA_LOG_LEVEL},state.change.logger=${KAFKA_LOG_LEVEL},kafka.producer.async.DefaultEventHandler=${KAFKA_LOG_LEVEL},kafka.authorizer.logger=${KAFKA_LOG_LEVEL},kafka.log.LogCleaner=${KAFKA_LOG_LEVEL},kafka.request.logger=${KAFKA_LOG_LEVEL}' KAFKA_LOG4J_ROOT_LOGLEVEL: ${KAFKA_LOG_LEVEL} KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1volumes: - ./kafka/secrets:/etc/kafka/secrets - ./kafka/data:/var/lib/kafka/datadepends_on: - zookeepernetworks: - ms_network
These steps worked for me in Windows:
1 - Generate keys using Windows WSL:
cd $(pwd)/examples/kafka-cluster-ssl/secrets./create-certs.sh(Type yes for all "Trust this certificate? [no]:" prompts.)
2 - Set the environment variable KAFKA_SSL_SECRETS_DIR using PowerShell:
$env:KAFKA_SSL_SECRETS_DIR= "xxxx\cp-docker-images\examples\kafka-cluster-ssl\secrets"
3 - Use the environment variable to run kafka-ssl cluster node:
docker run -d --net=host --name=kafka-ssl-1 -e KAFKA_ZOOKEEPER_CONNECT=localhost:22181,localhost:32181,localhost:42181 -e KAFKA_ADVERTISED_LISTENERS=SSL://localhost:29092 -e KAFKA_SSL_KEYSTORE_FILENAME=kafka.broker1.keystore.jks -e KAFKA_SSL_KEYSTORE_CREDENTIALS=broker1_keystore_creds -e KAFKA_SSL_KEY_CREDENTIALS=broker1_sslkey_creds -e KAFKA_SSL_TRUSTSTORE_FILENAME=kafka.broker1.truststore.jks -e KAFKA_SSL_TRUSTSTORE_CREDENTIALS=broker1_truststore_creds -e KAFKA_SECURITY_INTER_BROKER_PROTOCOL=SSL -v ${env:KAFKA_SSL_SECRETS_DIR}:/etc/kafka/secrets confluentinc/cp-kafka:5.0.0