Truncating a file while it's being used (Linux)

As of coreutils 7.0, there is a truncate command.

The interesting thing about those regrown files is that the first 128 KB or so will be all zeroes after you truncate the file by copying /dev/null over it. This happens because the file is truncated to zero length, but the file descriptor in the application still points immediately after its last write. When it writes again, the file system treats the start of the file as all zero bytes - without actually writing the zeroes to disk.

Ideally, you should ask the vendor of the application to open the log file with the O_APPEND flag. This means that after you truncate the file, the next write will implicitly seek to the end of the file (meaning back to offset zero) and then write the new information.

This code rigs standard output so it is in O_APPEND mode and then invokes the command given by its arguments (rather like nice runs a command after adjusting its nice-level, or nohup runs a command after fixing things so it ignores SIGHUP).

#include <stdlib.h>#include <unistd.h>#include <fcntl.h>#include <stdio.h>#include <stdarg.h>#include <string.h>#include <errno.h>static char *arg0 = "<unknown>";static void error(const char *fmt, ...){    va_list args;    int errnum = errno;    fprintf(stderr, "%s: ", arg0);    va_start(args, fmt);    vfprintf(stderr, fmt, args);    va_end(args);    if (errnum != 0)        fprintf(stderr, " (%d: %s)", errnum, strerror(errnum));    putc('\n', stderr);    fflush(0);    exit(1);}int main(int argc, char **argv){    int attr;    arg0 = argv[0];    if (argc < 2)        error("Usage: %s cmd [arg ...]", arg0);    if ((attr = fcntl(1, F_GETFL, &attr)) < 0)        error("fcntl(F_GETFL) failed");    attr |= O_APPEND;    if (fcntl(1, F_SETFL, attr) != 0)        error("fcntl(F_SETFL) failed");    execvp(argv[1], &argv[1]);    error("failed to exec %s", argv[1]);    return(1);}

My testing of it was somewhat casual, but just barely enough to persuade me that it worked.

Simpler alternative

Billy notes in his answer that '>>' is the append operator - and indeed, on Solaris 10, bash (version 3.00.16(1)) does use the O_APPEND flag - thereby making the code above unnecessary, as shown ('Black JL:' is my prompt on this machine):

Black JL: truss -o bash.truss bash -c "echo Hi >> x3.29"Black JL: grep open bash.trussopen("/var/ld/ld.config", O_RDONLY)             Err#2 ENOENTopen("/usr/lib/libcurses.so.1", O_RDONLY)       = 3open("/usr/lib/libsocket.so.1", O_RDONLY)       = 3open("/usr/lib/libnsl.so.1", O_RDONLY)          = 3open("/usr/lib/libdl.so.1", O_RDONLY)           = 3open("/usr/lib/libc.so.1", O_RDONLY)            = 3open("/platform/SUNW,Ultra-4/lib/libc_psr.so.1", O_RDONLY) = 3open64("/dev/tty", O_RDWR|O_NONBLOCK)           = 3stat64("/usr/openssl/v0.9.8e/bin/bash", 0xFFBFF2A8) Err#2 ENOENTopen64("x3.29", O_WRONLY|O_APPEND|O_CREAT, 0666) = 3Black JL:

Use append redirection rather than the wrapper ('cantrip') code above. This just goes to show that when you use one particular technique for other (valid) purposes, adapting it to yet another is not necessarily the simplest mechanism - even though it works.

Redirect the output using >> instead of >. This will allow you to truncate the file without the file growing back to its original size. Also, don't forget to redirect STDERR (2>&1).

So the end result would be: myprogram >> myprogram.log 2>&1 &