jvm heap usage history in a killed Kubernetes pod

You can install addons or external tools like Prometheus or metrics-server.

Prometheus is an open-source systems monitoring and alerting toolkit originally built at SoundCloud.

You can define queries:For CPU percentage

avg((sum (rate (container_cpu_usage_seconds_total {container_name!="" ,pod="<Pod name>" } [5m])) by (namespace , pod, container ) / on (container , pod , namespace) ((kube_pod_container_resource_limits_cpu_cores >0)*300))*100)

For Memory percentage

avg((avg (container_memory_working_set_bytes{pod="<pod name>"}) by (container_name , pod ))/ on (container_name , pod)(avg (container_spec_memory_limit_bytes>0 ) by (container_name, pod))*100)

Take a look: prometheus-pod-memory-usage.

You can visualize such metrics using Grafana - take a look how to set it up with Prometheus - grafana-prometheus-setup.

Metrics-server is a scalable, efficient source of container resource metrics for Kubernetes built-in autoscaling pipelines.

Metrics Server collects resource metrics from Kubelets and exposes them in Kubernetes apiserver through Metrics API for use by Horizontal Pod Autoscaler and Vertical Pod Autoscaler. Metrics API can also be accessed by kubectl top, making it easier to debug autoscaling pipelines.

You can execute:

$ kubectl top pod <your-pod-name> --namespace=your-namespace --containers

The following command will give you both the CPU usage as well as the memory usage for a given pod and its containers.

See how to firstly install metrics-server: metrics-server-installtion.

Otherwise if you want to check cpu/memory usage without installing any third party tool then you can get memory and cpu usage of pod from cgroup.

1. Go to shell of running container kubectl exec pod_name -- /bin/bash
2. Go to cd /sys/fs/cgroup/cpu for cpu usage run cat cpuacct.usage
3. Go to cd /sys/fs/cgroup/memory for memory usage run cat memory.usage_in_bytes

Remember that memory usage is in bytes.

Take a look: memory-usage-kubernetes.