What host and port to use for http plugin for a Logstash container running in K8s pod?
As per your information, you are using helm to deploy and the version of the docker image from elastic on the stable chart is the 7.1.1
If you check the Dockerfile for that version
They create a user logstash here
And during the Dockerfile definition switch to root but returns at the end to that user, because of that you cannot use port 80
Try to use a different port like 8080
and adjust all other services that need to connect to logstash to use the new port or map that port on the service to use on the services level the 80
port but with the target pointing to the 8080
I ended up switching to TCP to avoid HTTP headers in my messages.
In my Logstash Helm configuration (https://github.com/helm/charts/tree/master/stable/logstash), I set up the service as such:
service: type: ClusterIP annotations: {} ports: tcp-data: port: 1514 targetPort: tcp-data protocol: TCP tcp-event: port: 1515 targetPort: tcp-event protocol: TCPports: - name: tcp-data containerPort: 1514 protocol: TCP - name: tcp-event containerPort: 1515 protocol: TCP
And the tcp plugin:
inputs: data: |- input { tcp { port => 1514 type => json } } event: |- input { tcp { port => 1515 type => json } }
And then on the Python side, I was able to use the socket library to send messages to Logstash using the Logstash clusterIP service's IP and port 1514 or 1515.