tensorflow einsum vs. matmul vs. tensordot tensorflow einsum vs. matmul vs. tensordot python python

tensorflow einsum vs. matmul vs. tensordot


Both tf.tensordot() and tf.einsum() are syntactic sugar that wrap one or more invocations of tf.matmul() (although in some special cases tf.einsum() can reduce to the simpler elementwise tf.multiply()).

In the limit, I'd expect all three functions to have equivalent performance for the same computation. However, for smaller matrices it may be more efficient to use tf.matmul() directly, because it would yield a simpler TensorFlow graph with fewer operations, and hence the per-operation invocation costs will be lower.