Response streaming in Express does not work in Azure App Service

Like I described in the latter part of my question, directly piping the res (my response to the client) to dataStream (the data stream I got from the external API) allowed to stream without any issues.

Extending the same behavior, I created a Readable stream which is equivalent to the response I should send to my client. Then I piped it to res and it worked.

Here is my solution.

const https = require('https');const { withParser } = require('stream-json/filters/Pick');const { streamArray } = require('stream-json/streamers/StreamArray');const { chain } = require('stream-chain');const { Readable } = require('stream');exports.getStreamResponse = async function (req, res) {  const options = {    hostname,    port,    path,    method: 'GET',  };  return new Promise((resolve, reject) => {    https.request(options, (dataStream) => {      const pipeline = chain([        dataStream,        withParser({ filter: 'items' }),        streamArray()      ]);        // create a readable stream to collect data from response       const readable = new Readable({        // this empty method is to avoid 'ERR_METHOD_NOT_IMPLEMENTED'        // error when read method is called while there is no data in the        // readable stream        read(size) { }      });        let separator = '';        readable.pipe(res);      readable.push("[");      pipeline.on('data', data => {        readable.push(separator + JSON.stringify(data.value));        if (!separator) {          separator = ',';        }      });      pipeline.on('end', () => {        readable.push("]");        readable.push(null);        resolve();      });                  pipeline.on('error', reject);    });  })};

However, I noticed this solution requires more memory than the solution I had issues with. Probably because I am creating a readable stream that is redundant.