At what point does wrapping a FileOutputStream with a BufferedOutputStream make sense, in terms of performance? At what point does wrapping a FileOutputStream with a BufferedOutputStream make sense, in terms of performance? java java

At what point does wrapping a FileOutputStream with a BufferedOutputStream make sense, in terms of performance?


BufferedOutputStream helps when the writes are smaller than the buffer size e.g. 8 KB. For larger writes it doesn't help nor does it make it much worse. If ALL your writes are larger than the buffer size or you always flush() after every write, I would not use a buffer. However if a good portion of your writes are less that the buffer size and you don't use flush() every time, its worth having.

You may find increasing the buffer size to 32 KB or larger gives you a marginal improvement, or make it worse. YMMV


You might find the code for BufferedOutputStream.write useful

/** * Writes <code>len</code> bytes from the specified byte array * starting at offset <code>off</code> to this buffered output stream. * * <p> Ordinarily this method stores bytes from the given array into this * stream's buffer, flushing the buffer to the underlying output stream as * needed.  If the requested length is at least as large as this stream's * buffer, however, then this method will flush the buffer and write the * bytes directly to the underlying output stream.  Thus redundant * <code>BufferedOutputStream</code>s will not copy data unnecessarily. * * @param      b     the data. * @param      off   the start offset in the data. * @param      len   the number of bytes to write. * @exception  IOException  if an I/O error occurs. */public synchronized void write(byte b[], int off, int len) throws IOException {    if (len >= buf.length) {        /* If the request length exceeds the size of the output buffer,           flush the output buffer and then write the data directly.           In this way buffered streams will cascade harmlessly. */        flushBuffer();        out.write(b, off, len);        return;    }    if (len > buf.length - count) {        flushBuffer();    }    System.arraycopy(b, off, buf, count, len);    count += len;}


I have lately been trying to explore IO performance. From what I have observed, directly writing to a FileOutputStream has led to better results; which I have attributed to FileOutputStream's native call for write(byte[], int, int). Moreover, I have also observed that when BufferedOutputStream's latency begins to converge towards that of direct FileOutputStream, it fluctuates a lot more i.e. it can abruptly even double-up (I haven't yet been able to find out why).

P.S. I am using Java 8 and will not be able to comment right now on whether my observations will hold for previous java versions.

Here's the code I tested, where my input was a ~10KB file

public class WriteCombinationsOutputStreamComparison {    private static final Logger LOG = LogManager.getLogger(WriteCombinationsOutputStreamComparison.class);public static void main(String[] args) throws IOException {    final BufferedInputStream input = new BufferedInputStream(new FileInputStream("src/main/resources/inputStream1.txt"), 4*1024);    final ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();    int data = input.read();    while (data != -1) {        byteArrayOutputStream.write(data); // everything comes in memory        data = input.read();    }    final byte[] bytesRead = byteArrayOutputStream.toByteArray();    input.close();    /*     * 1. WRITE USING A STREAM DIRECTLY with entire byte array --> FileOutputStream directly uses a native call and writes     */    try (OutputStream outputStream = new FileOutputStream("src/main/resources/outputStream1.txt")) {        final long begin = System.nanoTime();        outputStream.write(bytesRead);        outputStream.flush();        final long end = System.nanoTime();        LOG.info("Total time taken for file write, writing entire array [nanos=" + (end - begin) + "], [bytesWritten=" + bytesRead.length + "]");        if (LOG.isDebugEnabled()) {            LOG.debug("File reading result was: \n" + new String(bytesRead, Charset.forName("UTF-8")));        }    }    /*     * 2. WRITE USING A BUFFERED STREAM, write entire array     */    // changed the buffer size to different combinations --> write latency fluctuates a lot for same buffer size over multiple runs    try (BufferedOutputStream outputStream = new BufferedOutputStream(new FileOutputStream("src/main/resources/outputStream1.txt"), 16*1024)) {        final long begin = System.nanoTime();        outputStream.write(bytesRead);        outputStream.flush();        final long end = System.nanoTime();        LOG.info("Total time taken for buffered file write, writing entire array [nanos=" + (end - begin) + "], [bytesWritten=" + bytesRead.length + "]");        if (LOG.isDebugEnabled()) {            LOG.debug("File reading result was: \n" + new String(bytesRead, Charset.forName("UTF-8")));        }    }}}

OUTPUT:

2017-01-30 23:38:59.064 [INFO] [main] [WriteCombinationsOutputStream] - Total time taken for file write, writing entire array [nanos=100990], [bytesWritten=11059]2017-01-30 23:38:59.086 [INFO] [main] [WriteCombinationsOutputStream] - Total time taken for buffered file write, writing entire array [nanos=142454], [bytesWritten=11059]