Never mind, I've found I've forgot to put
proxy_set_header Host $host:$server_port;proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
in my nginx config. Works fine now.