Why use tensorflow gfile? (for file I/O)
This comment:
File I/O wrappers without thread locking
...is a particularly unhelpful description for TensorFlow's tf.gfile
module!
The main roles of the tf.gfile
module are:
- To provide an API that is close to Python's
file
objects, and - To provide an implementation based on TensorFlow's C++
FileSystem
API.
The C++ FileSystem
API supports multiple file system implementations, including local files, Google Cloud Storage (using a gs://
prefix), and HDFS (using an hdfs://
prefix). TensorFlow exports these as tf.gfile
so that you can uses these implementations for saving and loading checkpoints, writing TensorBoard logs, and accessing training data (among other uses). However, if all of your files are local, you can use the regular Python file API without any problem.