How to benchmark apache/nginx setup

Better solution? Siege.

More accurate benchmarking tool than ab

A few things:

1. Don't use ab. It's single-threaded and you will probably end up benchmarking ab rather than your HTTP server.

2. Don't run whatever stress tool you use on the same system as the server. The HTTP server will be competing with the tool for CPU and other resources. Plus localhost's idealized version of a network doesn't tell the whole picture (see point 4).

3. Pay attention to memory and CPU utilization during the tests. So many people never consider this factor. Even if both setups perform equally, if one uses a fraction of the RAM/CPU, then you have a winner.

4. RPS isn't the only meaningful metric. Things like slow clients (3G smartphones, congested networks, slow PC's) can have a distinctly negative impact on threaded servers. The idealized lab setup (localhost or isolated switch) won't reflect this.

5. Your FCGI script will be the bottleneck for either server. I'd suggest using a tool that can pull multiple resources (ideally an entire page, including static content) so you can get a complete picture of page load times.

You might consider using one of the "cloud-based" testing tools, such as browsermob.com or loadimpact.com.

one word: ab