Script output is buffered into one message, despite separate echo statements?

You can use the readline interface provided as part of the node APIs. More information here https://nodejs.org/api/readline.html#readline_event_line. You will use spawn as it is however pass the stdout to readline so that it can parse the lines. Not sure if this is what you intend to do. Here is some sample code:

var process = require('child_process');const readline = require('readline');var child = process.spawn('./test.sh');// Use readline interfaceconst readlinebyline = readline.createInterface({ input: child.stdout });// Called when a line is receivedreadlinebyline.on('line', (line) => {    line = JSON.stringify(line.toString('utf8'));    console.log(line);});

Output:

"first message""second message""third message"

If you get an error like TypeError: input.on is not a function, make sure you have executing privileges on the test.sh script via chmod +x test.sh.

The C library that underlies bash and python is the one that does per-line buffering of stdout. stdbuf and unbuffer would deal with that, but not the buffering done by the operating system.

Linux, for example, allocates 4096 bytes as the buffer for the pipe between your node.js process and the bash process.

Truth is, there's no honest way for a process on one end of the pipe (node.js) to see individual writes (echo calls) on the other end. This isn't the right design (you could communicate via individual files instead of stdout).

If you insist, you can try and fool the OS scheduler: if nothing is even remotely close to writing to the pipe, then it will schedule-in the reader process (node.js) which will read what's currently in the OS buffer.

I tested this on Linux:

$ cat test.sh echo 'first message'sleep 0.1echo 'second message'sleep 0.1echo 'third message'$ cat test.js const  child_process  = require('child_process');var child = child_process.spawn(`./test.sh`);child.stdout.on('data', data => {   data = JSON.stringify(data.toString('utf8'));   global.process.stdout.write(data); // notice global object});$ node test.js"first message\n""second message\n""third message\n"

I ran into the same problem on a previous project. I used the interpretation switch on the echo statement and then split the string on a non-printable character.

Example:

echo -e 'one\u0016'echo -e "two\u0016"echo -e 'three\u0016'

Result:

"one\u0016\ntwo\u0016\nthree\u0016\n"

And the corresponding Javascript:

var child = process.spawn('./test.sh');child.stdout.on('data', data => {   var value = data.toString('utf8');   var values = value.split("\u0016\n").filter(item => item);   console.log(values);});