Why can't I scale horizontally a simple HTTP/2 service on Kubernetes?
If I understood correctly, you did run the load testing on same cluster/nodes as your pods, this will definitely have an impact on the overall result, I would recommend you split the client from the server on separate nodes so that one does not affect each other.
For the values you reported, is clearly visible that the workers are consuming more CPU that the nginx servers.
You should check either:
- The Host CPU utilization, it might be under high pressure with context switches because the amount threads is much higher than the number of CPU available.
- A network bottleneck, maybe you could try add more nodes or increase the worker capacity(SKU) and split client from servers.
- The clients does not have enough capacity to generate the load, you increase the threads but the raw limits are the same
You should also test individual server capacity to validate the limit of each server, so you have a parameter to compare if the results are in line with the expected values.