elastic beanstalk weird nginx configuration

You are forgetting to look at one part of that nginx config:

upstream nodejs {    server 127.0.0.1:8081;    keepalive 256;}

That part is telling nginx to make a group of servers called nodejs as you can read about here.

8081 is the port that NodeJS is running on (if you use the sample application for instance).

You can verify this by looking at the Elastic Beanstalk logs:

-------------------------------------/var/log/nodejs/nodejs.log-------------------------------------Server running at http://127.0.0.1:8081/

Then if we continue in the nginx.conf file we can see what you already posted:

server {    listen 8080;    location / {        proxy_pass  http://nodejs;        proxy_set_header   Connection "";        proxy_http_version 1.1;        proxy_set_header        Host            $host;        proxy_set_header        X-Real-IP       $remote_addr;        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;    }}

This tells nginx to use the proxy pass module to pass all from port 8080 to our upstream group nodejs which is running on port 8081. This means that port 8081 is just for accessing it locally but port 8080 is what let's outside entities talk to the nginx which then passes stuff onto nodejs.

Some of the reasoning for not exposing NodeJS directly can be found in this StackOverflow answer.

Port 8080 is used because it is the HTTP alternate port that is "commonly used for Web proxy and caching server, or for running a Web server as a non-root user."

That explains the ports. Now the issue of ELB and how things are talking to each other.

Since the security group is only allowing access on port 80, there is an iptables rule that is setup to forward port 80 to port 8080. This allows non-root to bind to port 8080 because lower port numbers require root privileges.

You can verify this by running the following:

[ec2-user@ip-xxx-xx-xx-x ~]$ sudo iptables -t nat -LChain PREROUTING (policy ACCEPT)target     prot opt source               destination         REDIRECT   tcp  --  anywhere             anywhere             tcp dpt:http redir ports 8080Chain INPUT (policy ACCEPT)target     prot opt source               destination         Chain OUTPUT (policy ACCEPT)target     prot opt source               destination         REDIRECT   tcp  --  anywhere             anywhere             tcp dpt:http redir ports 8080Chain POSTROUTING (policy ACCEPT)target     prot opt source               destination         

So in summary, when you load your CNAME, the load balancer is rerouting the traffic to a given instance on port 80, which is allowed through the security group, then iptables is forwarding that to port 8080, which is the port that nginx is using a proxy to pass the traffic to port 8081 which is the local port of NodeJS.

Here's a diagram:

incoming connections-> :80             - Load Balancer-> :80             - Security group-> :80   -> :8080  - EC2 instance, iptables forward-> :8080 -> :8081  - nginx, proxy pass-> :8081           - nodejs, your app

Hopefully that helps.