TensorFlow: training on my own image

If you are interested in how to input your own data in TensorFlow, you can look at this tutorial.
I've also written a guide with best practices for CS230 at Stanford here.

New answer (with tf.data) and with labels

With the introduction of tf.data in r1.4, we can create a batch of images without placeholders and without queues. The steps are the following:

1. Create a list containing the filenames of the images and a corresponding list of labels
2. Create a tf.data.Dataset reading these filenames and labels
3. Preprocess the data
4. Create an iterator from the tf.data.Dataset which will yield the next batch

The code is:

# step 1filenames = tf.constant(['im_01.jpg', 'im_02.jpg', 'im_03.jpg', 'im_04.jpg'])labels = tf.constant([0, 1, 0, 1])# step 2: create a dataset returning slices of `filenames`dataset = tf.data.Dataset.from_tensor_slices((filenames, labels))# step 3: parse every image in the dataset using `map`def _parse_function(filename, label):    image_string = tf.read_file(filename)    image_decoded = tf.image.decode_jpeg(image_string, channels=3)    image = tf.cast(image_decoded, tf.float32)    return image, labeldataset = dataset.map(_parse_function)dataset = dataset.batch(2)# step 4: create iterator and final input tensoriterator = dataset.make_one_shot_iterator()images, labels = iterator.get_next()

Now we can run directly sess.run([images, labels]) without feeding any data through placeholders.

Old answer (with TensorFlow queues)

To sum it up you have multiple steps:

1. Create a list of filenames (ex: the paths to your images)
2. Create a TensorFlow filename queue
3. Read and decode each image, resize them to a fixed size (necessary for batching)
4. Output a batch of these images

The simplest code would be:

# step 1filenames = ['im_01.jpg', 'im_02.jpg', 'im_03.jpg', 'im_04.jpg']# step 2filename_queue = tf.train.string_input_producer(filenames)# step 3: read, decode and resize imagesreader = tf.WholeFileReader()filename, content = reader.read(filename_queue)image = tf.image.decode_jpeg(content, channels=3)image = tf.cast(image, tf.float32)resized_image = tf.image.resize_images(image, [224, 224])# step 4: Batchingimage_batch = tf.train.batch([resized_image], batch_size=8)

Based on @olivier-moindrot's answer, but for Tensorflow 2.0+:

# step 1filenames = tf.constant(['im_01.jpg', 'im_02.jpg', 'im_03.jpg', 'im_04.jpg'])labels = tf.constant([0, 1, 0, 1])# step 2: create a dataset returning slices of `filenames`dataset = tf.data.Dataset.from_tensor_slices((filenames, labels))def im_file_to_tensor(file, label):    def _im_file_to_tensor(file, label):        path = f"../foo/bar/{file.numpy().decode()}"        im = tf.image.decode_jpeg(tf.io.read_file(path), channels=3)        im = tf.cast(image_decoded, tf.float32) / 255.0        return im, label    return tf.py_function(_im_file_to_tensor,                           inp=(file, label),                           Tout=(tf.float32, tf.uint8))dataset = dataset.map(im_file_to_tensor)

If you are hitting an issue similar to:

ValueError: Cannot take the length of Shape with unknown rank

when passing tf.data.Dataset tensors to model.fit, then take a look at https://github.com/tensorflow/tensorflow/issues/24520. A fix for the code snippet above would be:

def im_file_to_tensor(file, label):    def _im_file_to_tensor(file, label):        path = f"../foo/bar/{file.numpy().decode()}"        im = tf.image.decode_jpeg(tf.io.read_file(path), channels=3)        im = tf.cast(image_decoded, tf.float32) / 255.0        return im, label    file, label = tf.py_function(_im_file_to_tensor,                                  inp=(file, label),                                  Tout=(tf.float32, tf.uint8))    file.set_shape([192, 192, 3])    label.set_shape([])    return (file, label)

2.0 Compatible Answer using Tensorflow Hub: Tensorflow Hub is a Provision/Product Offered by Tensorflow, which comprises the Models developed by Google, for Text and Image Datasets.

It saves Thousands of Hours of Training Time and Computational Effort, as it reuses the Existing Pre-Trained Model.

If we have an Image Dataset, we can take the Existing Pre-Trained Models from TF Hub and can adopt it to our Dataset.

Code for Re-Training our Image Dataset using the Pre-Trained Model, MobileNet, is shown below:

import itertoolsimport osimport matplotlib.pylab as pltimport numpy as npimport tensorflow as tfimport tensorflow_hub as hubmodule_selection = ("mobilenet_v2_100_224", 224) #@param ["(\"mobilenet_v2_100_224\", 224)", "(\"inception_v3\", 299)"] {type:"raw", allow-input: true}handle_base, pixels = module_selectionMODULE_HANDLE ="https://tfhub.dev/google/imagenet/{}/feature_vector/4".format(handle_base)IMAGE_SIZE = (pixels, pixels)print("Using {} with input size {}".format(MODULE_HANDLE, IMAGE_SIZE))BATCH_SIZE = 32 #@param {type:"integer"}#Here we need to Pass our Datasetdata_dir = tf.keras.utils.get_file(    'flower_photos',    'https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz',    untar=True)model = tf.keras.Sequential([    hub.KerasLayer(MODULE_HANDLE, trainable=do_fine_tuning),    tf.keras.layers.Dropout(rate=0.2),    tf.keras.layers.Dense(train_generator.num_classes, activation='softmax',                          kernel_regularizer=tf.keras.regularizers.l2(0.0001))])model.build((None,)+IMAGE_SIZE+(3,))model.summary()

Complete Code for Image Retraining Tutorial can be found in this Github Link.

More information about Tensorflow Hub can be found in this TF Blog.

The Pre-Trained Modules related to Images can be found in this TF Hub Link.

All the Pre-Trained Modules, related to Images, Text, Videos, etc.. can be found in this TF HUB Modules Link.

Finally, this is the Basic Page for Tensorflow Hub.