If you detect that a connection is closed, remove the socket from the fd set, otherwise select is going to report them (Socket 4 hung up).. select is not edge triggered, if you don't handle the event, it's going to report it again.
Indeed, if recv returns 0 (and not -1, with errno=EWOULDBLOCK), the socket is closed. You should call close() on it as well, and take it out of the select() call. Otherwise it will remain in WAIT1 and release select() each time.