How can I redirect tail -f output to curl (HTTP stream) How can I redirect tail -f output to curl (HTTP stream) curl curl

How can I redirect tail -f output to curl (HTTP stream)


You can use a tool like buffer (or perhaps even better mbuffer) to combine both strategies.

tail -f  /var/logs/some-log.log | buffer | while read -r LINE; do curl -X POST --silent --data-binary $LINE "http://localhost:8080/"; done

This reduces the number of HTTP requests by curl to your webservice (but possibly makes you lose some logs when your connection dies).

However, I agree with Nick Russo that implementing websockets might be a better idea altogether.

Hope this helps!


Here's the beginning of a chunking strategy that doesn't care about line breaks:

while sleep .1; do uptime; done | while LINE=$(dd iflag=fullblock count=2); do echo "$LINE"; done

The first while loop is just a dummy data generator to make this example trivial to try out. The echo at the end stands in for the curl command.

And here's a strategy that does care about line breaks:

#!/bin/bashcount=0while sleep .1; do uptime; done | while read -r LINEdo  count=$[count+1]  if test "$count" -ge 10  then    echo "$ACCUMULATOR"    ACCUMULATOR="$LINE"    count=0  else    ACCUMULATOR+="$LINE"  fidone