Most efficient way to map function over numpy array Most efficient way to map function over numpy array python python

# Most efficient way to map function over numpy array

I've tested all suggested methods plus `np.array(map(f, x))` with `perfplot` (a small project of mine).

Message #1: If you can use numpy's native functions, do that.

If the function you're trying to vectorize already is vectorized (like the `x**2` example in the original post), using that is much faster than anything else (note the log scale): If you actually need vectorization, it doesn't really matter much which variant you use. Code to reproduce the plots:

``import numpy as npimport perfplotimport mathdef f(x):    # return math.sqrt(x)    return np.sqrt(x)vf = np.vectorize(f)def array_for(x):    return np.array([f(xi) for xi in x])def array_map(x):    return np.array(list(map(f, x)))def fromiter(x):    return np.fromiter((f(xi) for xi in x), x.dtype)def vectorize(x):    return np.vectorize(f)(x)def vectorize_without_init(x):    return vf(x)perfplot.show(    setup=np.random.rand,    n_range=[2 ** k for k in range(20)],    kernels=[f, array_for, array_map, fromiter,             vectorize, vectorize_without_init],    xlabel="len(x)",)``

How about using `numpy.vectorize`.

``import numpy as npx = np.array([1, 2, 3, 4, 5])squarer = lambda t: t ** 2vfunc = np.vectorize(squarer)vfunc(x)# Output : array([ 1,  4,  9, 16, 25])``

# TL;DR

As noted by @user2357112, a "direct" method of applying the function is always the fastest and simplest way to map a function over Numpy arrays:

``import numpy as npx = np.array([1, 2, 3, 4, 5])f = lambda x: x ** 2squares = f(x)``

Generally avoid `np.vectorize`, as it does not perform well, and has (or had) a number of issues. If you are handling other data types, you may want to investigate the other methods shown below.

# Comparison of methods

Here are some simple tests to compare three methods to map a function, this example using with Python 3.6 and NumPy 1.15.4. First, the set-up functions for testing:

``import timeitimport numpy as npf = lambda x: x ** 2vf = np.vectorize(f)def test_array(x, n):    t = timeit.timeit(        'np.array([f(xi) for xi in x])',        'from __main__ import np, x, f', number=n)    print('array: {0:.3f}'.format(t))def test_fromiter(x, n):    t = timeit.timeit(        'np.fromiter((f(xi) for xi in x), x.dtype, count=len(x))',        'from __main__ import np, x, f', number=n)    print('fromiter: {0:.3f}'.format(t))def test_direct(x, n):    t = timeit.timeit(        'f(x)',        'from __main__ import x, f', number=n)    print('direct: {0:.3f}'.format(t))def test_vectorized(x, n):    t = timeit.timeit(        'vf(x)',        'from __main__ import x, vf', number=n)    print('vectorized: {0:.3f}'.format(t))``

Testing with five elements (sorted from fastest to slowest):

``x = np.array([1, 2, 3, 4, 5])n = 100000test_direct(x, n)      # 0.265test_fromiter(x, n)    # 0.479test_array(x, n)       # 0.865test_vectorized(x, n)  # 2.906``

With 100s of elements:

``x = np.arange(100)n = 10000test_direct(x, n)      # 0.030test_array(x, n)       # 0.501test_vectorized(x, n)  # 0.670test_fromiter(x, n)    # 0.883``

And with 1000s of array elements or more:

``x = np.arange(1000)n = 1000test_direct(x, n)      # 0.007test_fromiter(x, n)    # 0.479test_array(x, n)       # 0.516test_vectorized(x, n)  # 0.945``

Different versions of Python/NumPy and compiler optimization will have different results, so do a similar test for your environment. 