How to continuously read pcap file while it is also still being written? How to continuously read pcap file while it is also still being written? shell shell

How to continuously read pcap file while it is also still being written?


Ignoring the pcap stuff (see end of post) to start, and answering the general question:

... here tcpdump and tcpstat only 2 example of reader and writer processes. They can be any other programs. Also note that reader processes can be more than one. What is the best way to accomplish that?

So one writer multiple reader problem.

What's the best solution in Bash? Probably tail -F my.log file. Example:

while true; do echo ${RANDOM} >> my.log; sleep 1; done & tail -F my.log &tail -F my.log &tail -F my.log &

There is also tee but that writes to multiple files not multiple processes. I guess you could setup a bunch of pipes to processes and use tee.

Note there are plenty of better solutions not in Bash that you probably want to use if you want a higher quality solution. One general issue with tail and UNIX pipes is that they are stream orientated, and you probably want a message orientated stream. UNIX message queues, a queue server, or maybe sockets can help with that.


On the pcap case

tail -F won't work because pcap file format has a header and you can't just start reading from the end of the file like tail -F does. You need to read the whole file start to finish then start tailing. -n option to tail does that:

 tail -n+0 -F foo.pcap  | tcpstat -r- -o "pps: %p\n"


tcpstat can read data from stdin:-r filenameRead all data from filename, which may be a regular file, a named pipe or "-" to read it's data from standard input.So:

tcpdump -i eth0 -w foo.pcap &tail -f foo.pcap | tcpstat -r - -o "pps: %p\n" 1

Also, you would run this commands in different termial windows