high performance application webserver in C/C++

I had the very same requirements for my job, so I evaluated a number of solutions: mongoose, libmicrohttpd, libevent. And I also was thinking about writing nginx modules. Here is the summary of my findings:

nginx

nginx project page

I love this server and use it a lot. Its performance and resource usage is much better than that of Apache, which I also still use but plan migrating to nginx.

• Very good tunable performance. Rich functionality. Portability.
• Module API is not documented and seems to be very verbose. See this nginx hello world module as example.
• Nginx does not use threads but uses multiple processes. This makes writing modules harder, need to learn nginx API for shared memory, etc.

mongoose

mongoose project page

• All server's code is in single mongoose.c file (about 130K), no dependencies. This is good.
• One thread per connection, so if you need concurrency you've got to configure lots of threads, ie. high RAM usage. Not too good.
• Performance is good, although not exceptional.
• API is simple but you have to compose all response HTTP headers yourself, ie. learn HTTP protocol in detail.

libmicrohttpd

libmicrohttpd project page

• Official GNU project.
• Verbose API, seems awkward to me, although much more simple than writing nginx modules.
• Good performance in keep-alive mode (link to my benchmarks below), not so good without keep-alive.

libevent

libevent project page

Libevent library has built-in web server called evhttp.

• It is event based, uses libevent for that.
• Easy API. Constructs HTTP headers automatically.
• Officially single-threaded. This is major disadvantage. I've found a hack, which makes several instances of evhttp run simultaneously accepting connections from the same socket. Not sure if it is all safe and robust.
• Performance of single-threaded evhttp is surprisingly poor. Multi-threaded hack works better, but still not good.

G-WAN

G-WAN project is not open source, but I'd like to say a few words about it.

• Very good performance, low memory usage, 150 KB executable.
• Very convenient 'servlet' deployment: just copy .c file into csp directory, and running server automatically compiles it. Code modifications also compiled on the fly.
• Simple API. Although constrained in some ways. Rich functionality (json, key-value store, etc.).
• Unstable. I had segfaults on static files. Hangs on some sample scripts. (Experienced on clean install. Never mixed files of different versions).
• Only 32-bit binary (not anymore).

So as you can see, none of existing alternatives have fully satisfied me. So I have developed my own server, which is ...

NXWEB

NXWEB project page

Feature highlights:

• Very good performance; see benchmarks on project page
• Can serve tens of thousands concurrent requests
• Small memory footprint
• Multi-threaded model designed to scale
• Exceptionally light code base
• Simple API
• Decent HTTP protocol handling
• Keep-alive connections
• SSL support (via GNUTLS)
• HTTP proxy (with keep-alive connection pooling)
• Non-blocking sendfile support (with configurable small file memory cache; gzip pre-encoded file serving)
• Modular design for developers
• Can be run as daemon; relaunches itself on error
• Open source

Limitations:

• Depends on libev library (not anymore)
• Only tested on Linux

I would suggest to write a FastCGI executable that can be used with many high performance web servers (even closed source ones).

mongoose: one file. simple and easy to use. not an asycn io but perfect for embedded and particular purposes.

gwan. excellent. no crashes. ultra well planned configuration. very smart and easy for c/c++ development in other words, very clean sensible api compared to nginx. provides a thread per core. or whatever you specify. a great choice. largest disadvantage (maybe im lacking in this area): cannot step thru code.

libevent: single thread is not a disadvantage on a single core machine. afterall its point is an async i/o. does have multithreads for other cores.

nginx: no personal experience. gaining serious ground on a-patchy server. (terribly confusing api)

boost asio: a c++ library for asynchio (asio). awesome. needs a friendly higher-level api for simpletons like myself. and others who come from php, java, javascript, node.js and other web languages.

python bottle: awesome 1 file lib (framework/system) that makes it easy to build python web apps. has/is a built in httpd server, like libevent and node.js

node.js: javascript asyncio server. an excellent selection. unfortunately, have to program in javascript that does become tedious. while there is something to be said for getting the job done; there is also something to be said for enjoying yourself during the process. hopefully no ones comes up with node.php