TNonblockingServer, TThreadedServer and TThreadPoolServer, which one fits best for my case?

Requests that take 50-600 milliseconds to complete are pretty long. The time it takes to create or destroy a thread is much less than that, so don't let that factor into your decision at this time. I would choose the one that is easiest to support and that is the least error-prone. You want to minimize the likelihood of subtle concurrency bugs.

This is why it is often easier to write single-threaded transaction handling code that blocks where it needs to, and have many of these running in parallel, than to have a more complex non-blocking model. A blocked thread may slow down an individual transaction, but it does not prevent the server from doing other work while it waits.

If your transaction load increases (i.e. more client transactions) or the requests become faster to process (approaching 1 millisecond per transaction), then transaction overhead becomes more of a factor. The metric to pay attention to is throughput: how many transactions complete per unit time. The absolute duration of a single transaction is less important than the rate at which they are being completed, at least if it stays well below one second.

One guy on Github has made a nice comparison

TThreadedServer

TThreadedServer spawns a new thread for each client connection, and each thread remains alive until the client connection is closed. This means that if there are 1000 concurrent client connections, TThreadedServer needs to run 1000 threads simultaneously.

TNonblockingServer

TNonblockingServer has one thread dedicated for network I/O. The same thread can also process requests, or you can create a separate pool of worker threads for request processing. The server can handle many concurrent connections with a small number of threads since it doesn’t need to spawn a new thread for each connection.

TThreadPoolServer (not benchmarked here)

TThreadPoolServer is similar to TThreadedServer; each client connection gets its own dedicated server thread. It’s different from TThreadedServer in 2 ways:

Server thread goes back to the thread pool after client closes the connection for reuse.There is a limit on the number of threads. The thread pool won’t grow beyond the limit.Client hangs if there is no more thread available in the thread pool. It’s much more difficult to use compared to the other 2 servers.