How does reduce_sum() work in tensorflow?

x has a shape of (2, 3) (two rows and three columns):

1 1 11 1 1

By doing tf.reduce_sum(x, 0) the tensor is reduced along the first dimension (rows), so the result is [1, 1, 1] + [1, 1, 1] = [2, 2, 2].

By doing tf.reduce_sum(x, 1) the tensor is reduced along the second dimension (columns), so the result is [1, 1] + [1, 1] + [1, 1] = [3, 3].

By doing tf.reduce_sum(x, [0, 1]) the tensor is reduced along BOTH dimensions (rows and columns), so the result is 1 + 1 + 1 + 1 + 1 + 1 = 6 or, equivalently, [1, 1, 1] + [1, 1, 1] = [2, 2, 2], and then 2 + 2 + 2 = 6 (reduce along rows, then reduce the resulted array).

The input is a 2-D tensor:

1 1 11 1 1

The 0 axis in tensorflow is the rows, 1 axis is the columns. The sum along the 0 axis will produce a 1-D tensor of length 3, each element is a per-column sum. The result is thus [2, 2, 2]. Likewise for the rows.

The sum along both axes is, in this case, the sum of all values in the tensor, which is 6.

Comparison to numpy:

a = np.array([[1, 1, 1], [1, 1, 1]])np.sum(a, axis=0)       # [2 2 2] np.sum(a, axis=1)       # [3 3]np.sum(a, axis=(0, 1))  # 6

As you can see, the output is the same.

In order to understand better what is going on I will change the values, and the results are self explanatory

import tensorflow as tfx = tf.constant([[1, 2, 4], [8, 16, 32]])a = tf.reduce_sum(x, 0)  # [ 9 18 36]b = tf.reduce_sum(x, 1)  # [ 7 56]c = tf.reduce_sum(x, [0, 1])  # 63with tf.Session() as sess:  output_a = sess.run(a)  print(output_a)  output_b = sess.run(b)  print(output_b)  output_c = sess.run(c)  print(output_c)