Use tee (or equivalent) but limit max file size or rotate to new file

use split:

my_program | tee >(split -d -b 100000 -)

Or if you don't want to see the output, you can directly pipe to split:

my_program | split -d -b 100000 -

As for the log rotation, there's no tool in coreutils that does it automatically. You could create a symlink and periodically update it using a bash command:

while ((1)); do ln -fns target_log_name $(ls -t | head -1); sleep 1; done

In package apache2-utils is present utility called rotatelogs, it fully meet to your requirements.

Synopsis:

rotatelogs [ -l ] [ -L linkname ] [ -p program ] [ -f ] [ -t ] [ -v ] [ -e ] [ -c ] [ -n number-of-files ] logfile rotationtime|filesize(B|K|M|G) [ offset ]

Example:

your_program | rotatelogs -n 5 /var/log/logfile 1M

Full manual you may read on this link.

or using awk

program | awk 'BEGIN{max=100} {n+=length($0); print $0 > "log."int(n/max)}'

It keeps lines together, so the max is not exact, but this could be nice especially for logging purposes. You can use awk's sprintf to format the file name.

Here's a pipable script, using awk

#!/bin/bashmaxb=$((1024*1024))    # default 1MiBout="log"              # output file namewidth=3                # width: log.001, log.002while getopts "b:o:w:" opt; do  case $opt in    b ) maxb=$OPTARG;;    o ) out="$OPTARG";;    w ) width=$OPTARG;;    * ) echo "Unimplented option."; exit 1  esacdoneshift $(($OPTIND-1))IFS='\n'              # keep leading whitespacesif [ $# -ge 1 ]; then # read from file  cat $1else                  # read from pipe  while read arg; do    echo $arg  donefi | awk -v b=$maxb -v o="$out" -v w=$width '{    n+=length($0); print $0 > sprintf("%s.%0.*d",o,w,n/b)}'

save this to a file called 'bee', run 'chmod +x bee' and you can use it as

program | bee

or to split an existing file as

bee -b1000 -o proglog -w8 file