How many concurrent requests should we multiplex in HTTP/2

The number of streams that client and server can initiate isn't unlimited, it's mandated by the SETTINGS_MAX_CONCURRENT_STREAMS parameter of the SETTINGS frame that each peer sends at the beginning of the connection: see section 6.5.2 of RFC 7540 The default is unlimited, and the RFC has the following recommendation:

It is recommended that this value be no smaller than 100, so as tonot unnecessarily limit parallelism.

The number of streams is however not the only parameter to take into account when considering parallelism in HTTP/2. Weights and stream dependencies also come into play.

Each stream is given a weight, and the RFC recommends that the server assign resources to streams based on their weight. Client side, Firefox assigns higher weights to the CSS than images. See this great presentation for more details about how each browser prioritizes and organizes its streams. Chrome uses dependencies so that the most important elements (CSS, HTML) are higher in the dependency chain than others. See this tool that illustrates the dependecy tree that Chrome generates. Server side, H2O, a new and fast HTTP server, implements an O(1) scheduler per connection in order to send the streams to the client based on the dependencies and weights. That means that if you request 500 elements with default dependency, each stream will get 1/500th of the server resources.

There shouldn't be any downside to request as much elements as possible (for regular web browsing). According to HTTPArchive, 40% of the pages contain more than 100 elements, and I would think it's reasonable to ask them all at the same time (provided the number of streams stays below SETTINGS_MAX_CONCURRENT_STREAMS. I believe that what matters is to be able to request them in the order that will allow the browser to render it as fast as possible.