Nginx High volume traffic load balancing

Here are some good references:

http://dak1n1.com/blog/12-nginx-performance-tuning

Server fault:https://serverfault.com/questions/221292/tips-for-maximizing-nginx-requests-sec

A very well documented config from the dak1n1 link:

# This number should be, at maximum, the number of CPU cores on your system. # (since nginx doesn't benefit from more than one worker per CPU.)worker_processes 24;# Number of file descriptors used for Nginx. This is set in the OS with 'ulimit -n 200000'# or using /etc/security/limits.confworker_rlimit_nofile 200000;# only log critical errorserror_log /var/log/nginx/error.log crit# Determines how many clients will be served by each worker process.# (Max clients = worker_connections * worker_processes)# "Max clients" is also limited by the number of socket connections available on the system (~64k)worker_connections 4000;# essential for linux, optmized to serve many clients with each threaduse epoll;# Accept as many connections as possible, after nginx gets notification about a new connection.# May flood worker_connections, if that option is set too low.multi_accept on;# Caches information about open FDs, freqently accessed files.# Changing this setting, in my environment, brought performance up from 560k req/sec, to 904k req/sec.# I recommend using some varient of these options, though not the specific values listed below.open_file_cache max=200000 inactive=20s; open_file_cache_valid 30s; open_file_cache_min_uses 2;open_file_cache_errors on;# Buffer log writes to speed up IO, or disable them altogether#access_log /var/log/nginx/access.log main buffer=16k;access_log off;# Sendfile copies data between one FD and other from within the kernel. # More efficient than read() + write(), since the requires transferring data to and from the user space.sendfile on; # Tcp_nopush causes nginx to attempt to send its HTTP response head in one packet, # instead of using partial frames. This is useful for prepending headers before calling sendfile, # or for throughput optimization.tcp_nopush on;# don't buffer data-sends (disable Nagle algorithm). Good for sending frequent small bursts of data in real time.tcp_nodelay on; # Timeout for keep-alive connections. Server will close connections after this time.keepalive_timeout 30;# Number of requests a client can make over the keep-alive connection. This is set high for testing.keepalive_requests 100000;# allow the server to close the connection after a client stops responding. Frees up socket-associated memory.reset_timedout_connection on;# send the client a "request timed out" if the body is not loaded by this time. Default 60.client_body_timeout 10;# If the client stops reading data, free up the stale client connection after this much time. Default 60.send_timeout 2;# Compression. Reduces the amount of data that needs to be transferred over the networkgzip on;gzip_min_length 10240;gzip_proxied expired no-cache no-store private auth;gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml;gzip_disable "MSIE [1-6]\.";

Also more info on linux system tuning for sysctl.conf:

# Increase system IP port limits to allow for more connectionsnet.ipv4.ip_local_port_range = 2000 65000net.ipv4.tcp_window_scaling = 1# number of packets to keep in backlog before the kernel starts dropping them net.ipv4.tcp_max_syn_backlog = 3240000# increase socket listen backlognet.core.somaxconn = 3240000net.ipv4.tcp_max_tw_buckets = 1440000# Increase TCP buffer sizesnet.core.rmem_default = 8388608net.core.rmem_max = 16777216net.core.wmem_max = 16777216net.ipv4.tcp_rmem = 4096 87380 16777216net.ipv4.tcp_wmem = 4096 65536 16777216net.ipv4.tcp_congestion_control = cubic

nginx should definately be able to handle more then 1000 req/s (I get about 2800 req/s in nginx when playing around on my cheap laptop with jmeter using one and a halve of the 2 cores)

You're using epoll which is the optimal option on current linux kernel as I understand it.

You've turned acces_log off, so you'r disc IO shouldn't be a bottleneck either (note: you could also set the access_log to buffered mode with a large buffer where it only writes after each x kb, which avoids the disk io being constantly hammered, but keeps the logs for analysis)

My understanding is that in order to maximize nginx performance you normally set the number of worker_processes equal to the number of core/cpu's, and then up the number of worker_connections to allow more concurrent connections (along with the number of open files os limit). Yet in the data you posted above you have a quadcore cpu with 10 worker processes with 10k connections allowed each. Consequently on the nginx side I'd try something like:

worker_processes 4;worker_rlimit_nofile 999999;events {  worker_connections 32768;  use epoll;  multi_accept on;}

On the kernel side I'd tune tcp read and write buffers differently, you want a small minimum, small default and large max.

You've upped the ephemeral port range already.

I'd up the number open files limit more, as you'll have lots of open sockets.

Which gives the following lines to add/change in your /etc/sysctl.conf

net.ipv4.tcp_rmem = 4096 4096 25165824                                net.ipv4.tcp_wmem = 4096 4096 25165824fs.file-max=999999

Hope that helps.

I found that using the least connected algorithm was problematic. I switched to

hash $remote_addr consistent;

and found the service much quicker.