How do I pipe live pcap records from a Kubernetes pod to Wireshark running locally?
I haven't used k8s a lot, but the docker run gets the entire clean stdout, while I get the impression that k attach doesn't.
I don't think that kubectl has an equivalent of docker run, that gives you clean stdout, but you might be able to do something with kubectl exec.
A possible test would be to redirect the output to a file, and see if it's valid output for the command you're running, and that there's nothing unexpected there.
I highly suggest you to read Using sidecars to analyze and debug network traffic in OpenShift and Kubernetes pods.
This article explains why you cant read traffic data directly from a pod and gives you an alternative on how to do it using a sidecar.
In short words, the containers most likely run on an internal container platform network that is not directly accessible by your machine.
A sidecar container is a container that is running in the same pod as the actual service/application and is able to provide additional functionality to the service/application.
TCPdump effectively in Kubernetes is a bit tricky and requires you to create a side car to your pod. What you are facing is actually the expected behavior.
run good old stuff like TCPdump or ngrep would not yield much interesting information, because you link directly to the bridge network or overlay in a default scenario.
The good news is, that you can link your TCPdump container to the host network or even better, to the container network stack. Source: How to TCPdump effectively in Docker
The thing is that you have two entry points, one is for nodeIP:NodePort the second is ClusterIP:Port. Both are pointing to the same set of randomization rules for endpoints set on kubernetes iptables.
As soon as it can happen on any node it's hard to configure tcpdump to catch all interesting traffic in just one point.
The best tool I know for such kind of analysis is Istio, but it works mostly for HTTP traffic.
Considering this, the best solution is to use a tcpdumper sidecar for each pod behind the service.
Let's go trough an example on how to achieve this
apiVersion: apps/v1kind: Deploymentmetadata: labels: app: web name: web-appspec: replicas: 2 selector: matchLabels: app: web template: metadata: labels: app: web spec: containers: - name: web-app image: nginx imagePullPolicy: Always ports: - containerPort: 80 protocol: TCP - name: tcpdumper image: docker.io/dockersec/tcpdump restartPolicy: Always---apiVersion: v1kind: Servicemetadata: name: web-svc namespace: defaultspec: ports: - nodePort: 30002 port: 80 protocol: TCP targetPort: 80 selector: app: web type: NodePortOn this manifest we can notice tree important things. We have a nginx container and one tcpdumper container as a side car and we have a service defined as NodePort.
To access our sidecar, you have to run the following command:
$ kubectl attach -it web-app-db7f7c59-d4xm6 -c tcpdumperExample:
$ kubectl get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEkubernetes ClusterIP 10.96.0.1 <none> 443/TCP 13dweb-svc NodePort 10.108.142.180 <none> 80:30002/TCP 9d$ curl localhost:30002<!DOCTYPE html><html><head><title>Welcome to nginx!</title><style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed andworking. Further configuration is required.</p><p>For online documentation and support please refer to<a href="http://nginx.org/">nginx.org</a>.<br/>Commercial support is available at<a href="http://nginx.com/">nginx.com</a>.</p><p><em>Thank you for using nginx.</em></p></body></html>$ kubectl attach -it web-app-db7f7c59-d4xm6 -c tcpdumperUnable to use a TTY - container tcpdumper did not allocate oneIf you don't see a command prompt, try pressing enter.> web-app-db7f7c59-d4xm6.80: Flags [P.], seq 1:78, ack 1, win 222, options [nop,nop,TS val 300957902 ecr 300958061], length 77: HTTP: GET / HTTP/1.112:03:16.884512 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.1336: Flags [.], ack 78, win 217, options [nop,nop,TS val 300958061 ecr 300957902], length 012:03:16.884651 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.1336: Flags [P.], seq 1:240, ack 78, win 217, options [nop,nop,TS val 300958061 ecr 300957902], length 239: HTTP: HTTP/1.1 200 OK12:03:16.884705 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.1336: Flags [P.], seq 240:852, ack 78, win 217, options [nop,nop,TS val 300958061 ecr 300957902], length 612: HTTP12:03:16.884743 IP 192.168.250.64.1336 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 240, win 231, options [nop,nop,TS val 300957902 ecr 300958061], length 012:03:16.884785 IP 192.168.250.64.1336 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 852, win 240, options [nop,nop,TS val 300957902 ecr 300958061], length 012:03:16.889312 IP 192.168.250.64.1336 > web-app-db7f7c59-d4xm6.80: Flags [F.], seq 78, ack 852, win 240, options [nop,nop,TS val 300957903 ecr 300958061], length 012:03:16.889351 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.1336: Flags [F.], seq 852, ack 79, win 217, options [nop,nop,TS val 300958062 ecr 300957903], length 012:03:16.889535 IP 192.168.250.64.1336 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 853, win 240, options [nop,nop,TS val 300957903 ecr 300958062], length 012:08:10.336319 IP6 fe80::ecee:eeff:feee:eeee > ff02::2: ICMP6, router solicitation, length 1612:15:47.717966 IP 192.168.250.64.2856 > web-app-db7f7c59-d4xm6.80: Flags [S], seq 3314747302, win 28400, options [mss 1420,sackOK,TS val 301145611 ecr 0,nop,wscale 7], length 012:15:47.717993 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.2856: Flags [S.], seq 2539474977, ack 3314747303, win 27760, options [mss 1400,sackOK,TS val 301145769 ecr 301145611,nop,wscale 7], length 012:15:47.718162 IP 192.168.250.64.2856 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 1, win 222, options [nop,nop,TS val 301145611 ecr 301145769], length 012:15:47.718164 IP 192.168.250.64.2856 > web-app-db7f7c59-d4xm6.80: Flags [P.], seq 1:78, ack 1, win 222, options [nop,nop,TS val 301145611 ecr 301145769], length 77: HTTP: GET / HTTP/1.112:15:47.718191 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.2856: Flags [.], ack 78, win 217, options [nop,nop,TS val 301145769 ecr 301145611], length 012:15:47.718339 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.2856: Flags [P.], seq 1:240, ack 78, win 217, options [nop,nop,TS val 301145769 ecr 301145611], length 239: HTTP: HTTP/1.1 200 OK12:15:47.718403 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.2856: Flags [P.], seq 240:852, ack 78, win 217, options [nop,nop,TS val 301145769 ecr 301145611], length 612: HTTP12:15:47.718451 IP 192.168.250.64.2856 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 240, win 231, options [nop,nop,TS val 301145611 ecr 301145769], length 012:15:47.718489 IP 192.168.250.64.2856 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 852, win 240, options [nop,nop,TS val 301145611 ecr 301145769], length 012:15:47.723049 IP 192.168.250.64.2856 > web-app-db7f7c59-d4xm6.80: Flags [F.], seq 78, ack 852, win 240, options [nop,nop,TS val 301145612 ecr 301145769], length 012:15:47.723093 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.2856: Flags [F.], seq 852, ack 79, win 217, options [nop,nop,TS val 301145770 ecr 301145612], length 012:15:47.723243 IP 192.168.250.64.2856 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 853, win 240, options [nop,nop,TS val 301145612 ecr 301145770], length 012:15:50.493995 IP 192.168.250.64.31340 > web-app-db7f7c59-d4xm6.80: Flags [S], seq 124258064, win 28400, options [mss 1420,sackOK,TS val 301146305 ecr 0,nop,wscale 7], length 012:15:50.494022 IP web-app-db7f7c59-d4xm6.80 > 192.168.250.64.31340: Flags [S.], seq 3544403648, ack 124258065, win 27760, options [mss 1400,sackOK,TS val 301146463 ecr 301146305,nop,wscale 7], length 012:15:50.494189 IP 192.168.250.64.31340 > web-app-db7f7c59-d4xm6.80: Flags [.], ack 1, win 222, options You can also take a look at ksniff tool, a kubectl plugin that utilize tcpdump and Wireshark to start a remote capture on any pod in your Kubernetes cluster.