Why is data written to a file opened with O_APPEND flag, always written at the end, even with `lseek`?
When you open a file with O_APPEND
, all data gets written to the end, regardless of whatever the current file pointer is from the latest call to lseek(2)
or the latest read/write operation. From the open(2)
documentation:
O_APPEND
The file is opened in append mode. Before eachwrite(2)
, the file offset is positioned at the end of the file, as if withlseek(2)
.
If you want to write data the end of the file and then the beginning of it later, open it up without O_APPEND
, use fstat(2)
to get the file size (st_size
member within struct stat
), and then seek to that offset to write the end.
In effect, O_APPEND only affects the behavior of write
, but not that of read
. Whatever how the current position of a file is changed by lseek
, write
will always append-only
.
When you open
a file with O_RDWR | O_APPEND
, read
will still start from the beginning of the file.
In the manual of open
(man 2 open
),
O_APPEND The file is opened in append mode. Before each write(2), the file offset is positioned at the end of the file.
In the manual of write
(man 2 write
),
If the O_APPEND flag of the file status flags is set, the file offset shall be set to the end of the file prior to each write.
In the Linux kernel fs/ext4 syscall write
-> vfs_write
-> ext4_file_write_iter
,the ext4_file_write_iter
will call ext4_write_checks
then call generic_write_checks
you will find the place where setting the pos
= file.size
/* FIXME: this is for backwards compatibility with 2.4 */if (iocb->ki_flags & IOCB_APPEND) iocb->ki_pos = i_size_read(inode);pos = iocb->ki_pos;
The following demo can verify it.
cat open_append.cc#include <fcntl.h>#include <sys/stat.h>#include <unistd.h>#include <errno.h>#include <string.h>#include <string>#include <iostream>int main(int argc, char *argv[]) { std::string path = "./test.txt"; std::string content = "hello_world"; std::string read_buf(content.size(), 0x0); struct stat st_buf; ssize_t bytes_read = -1; ssize_t bytes_write = -1; int ret = -1; off_t cur_off = -1; int fd = ::open(path.c_str(), O_CREAT | O_RDWR | O_TRUNC, 0644); if (fd < 0) { std::cerr << "open err path " << path << " errno " << errno << std::endl; return -1; } std::cout << "open ok path " << path << " fd " << fd << std::endl; // Step 1 write some data into an empty file bytes_write = ::write(fd, content.data(), content.size()); if (bytes_write < 0) { std::cerr << "write err fd " << fd << " errno " << errno << std::endl; goto out; } std::cout << "write ok fd " << fd << " data " << content << " nbytes " << bytes_write << std::endl; ::close(fd); // Step 2 open the file again with O_APPEND fd = -1; fd = ::open(path.c_str(), O_CREAT | O_RDWR | O_APPEND, 0644); if (fd < 0) { std::cerr << "open again err path " << path << " errno " << errno << std::endl; return -1; } std::cout << "open again ok path " << path << " fd " << fd << std::endl; // Step 3 the current position of the file NOT affected by O_APPEND cur_off = ::lseek(fd, 0, SEEK_CUR); if (cur_off < 0) { std::cerr << "lseek err SEEK_CUR fd " << fd << " errno " << errno << std::endl; goto out; } // cur_off expected to be 0 std::cout << "lseek ok SEEK_CUR fd " << fd << " cur_off " << cur_off << std::endl; // Step 4 the read will start from the beginning of the file bytes_read = read(fd, (char*)read_buf.data(), content.size()); if (bytes_read < 0) { std::cerr << "read err fd " << fd << " errno " << errno << std::endl; goto out; } std::cout << "read ok fd " << fd << " data " << read_buf << " nbytes " << bytes_read << std::endl; // Step 5 change the position to the half of the file size cur_off = ::lseek(fd, content.size() / 2, SEEK_SET); if (cur_off < 0) { std::cerr << "lseek err SEEK_SET fd " << fd << " errno " << errno << std::endl; goto out; } // cur_off expected to be content.size() / 2 std::cout << "lseek ok SEEK_SET fd " << fd << " cur_off " << cur_off << std::endl; // Step 6 write will append data from the end of the file // the current position is ignored bytes_write = ::write(fd, content.data(), content.size()); if (bytes_write < 0) { std::cerr << "append write err fd " << fd << " errno " << errno << std::endl; goto out; } std::cout << "append write ok fd " << fd << " append data " << content << " append nbytes " << bytes_write << std::endl; // Step 7 the file size is double content.size() memset((void*)&st_buf, 0x0, sizeof(struct stat)); ret = lstat(path.c_str(), &st_buf); if (ret < 0) { std::cerr << "lstat err path " << path << " errno " << errno << std::endl; goto out; } std::cout << "lstat ok path " << path << " st_size " << st_buf.st_size << std::endl; ret = 0;out: if (fd >= 0) { close(fd); } return ret;}
Output result
open ok path ./test.txt fd 3write ok fd 3 data hello_world nbytes 11open again ok path ./test.txt fd 3lseek ok SEEK_CUR fd 3 cur_off 0read ok fd 3 data hello_world nbytes 11lseek ok SEEK_SET fd 3 cur_off 5append write ok fd 3 append data hello_world append nbytes 11lstat ok path ./test.txt st_size 22
O_APPEND flag forces file pointer to point at the end of file only. so if you do lseek from start of the file, it takes the updated file pointer position as a start of file i.e. end position of old file.