How can I handle multiple sockets within a Perl daemon with large memory usage? How can I handle multiple sockets within a Perl daemon with large memory usage? multithreading multithreading

How can I handle multiple sockets within a Perl daemon with large memory usage?


Instead of forking there are two other approaches to handle concurrent connections. Either you use threads or a polling approach.

In the thread approach for each connection a new thread is created that handles the I/O of a socket. A thread runs in the same virtual memory of the creating process and can access all of its data. Make sure to properly use locks to synchronize write access on your data.

An even more efficient approach is to use polling via select(). In this case a single process/thread handles all sockets. This works under the assumption that most work will be I/O and that the time spend with waiting for I/O requests to finish is spent handling other sockets.

Go research further on those two options and decide which one suits you best.

See for example: http://www.perlfect.com/articles/select.shtml


If you have that much data, I wonder why you don't simply use a database?


This architecture is unsuitable for Cygwin. Forking on real unix systems is cheap, but on fake unix systems like Cygwin it's terribly expensive, because all data has to be copied (real unices use copy-on-write). Using threads changes the memory usage pattern (higher base usage, but smaller increase per thread), but odds are it will still be inefficient.

I would advice you to use a single-process approach using polling, and maybe non-blocking IO too.