Vectorize iterative addition in NumPy arrays Vectorize iterative addition in NumPy arrays numpy numpy

Vectorize iterative addition in NumPy arrays


You could speed it up a little by using np.add.at, which correctly handles the case of duplicate indices:

def interadd(U, idx):    grids = np.zeros((U,U))          for i in range(len(idx)):        grids[idx[i,0],idx[i,1]] += 1    return gridsdef interadd2(U, idx):    grids = np.zeros((U,U))    np.add.at(grids, idx.T.tolist(), 1)    return gridsdef interadd3(U, idx):    # YXD suggestion    grids = np.zeros((U,U))    np.add.at(grids, (idx[:,0], idx[:,1]), 1)    return grids

which gives

>>> U = 100>>> idx = np.floor(np.random.random(size=(5000,2))*U).astype(np.int)>>> (interadd(U, idx) == interadd2(U, idx)).all()True>>> %timeit interadd(U, idx)100 loops, best of 3: 8.48 ms per loop>>> %timeit interadd2(U, idx)100 loops, best of 3: 2.62 ms per loop

And YXD's suggestion:

>>> (interadd(U, idx) == interadd3(U, idx)).all()True>>> %timeit interadd3(U, idx)1000 loops, best of 3: 1.09 ms per loop


Divakar's answer lead me to try the following, which looks to be the fastest method yet:

lin_idx = idx[:,0]*U + idx[:,1]grids = np.bincount(lin_idx, minlength=U**2).reshape(U, U)

Timings:

In [184]: U = 100      ...: input = np.random.random(size=(5000,2)) * U     ...: idx = np.floor(input).astype(np.int)In [185]: %timeit interadd3(U, idx)  # By DSM / XYD1000 loops, best of 3: 1.68 ms per loopIn [186]: %timeit unique_counts(U, idx)  # By Divakar1000 loops, best of 3: 676 µs per loopIn [187]: %%timeit     ...: lin_idx = idx[:,0]*U + idx[:,1]     ...: grids = np.bincount(lin_idx, minlength=U*U).reshape(U, U)     ...: 10000 loops, best of 3: 97.5 µs per loop


You can convert R,C indices from idx to linear indices, then find out the unique ones alongwith their counts and finally store them in the output grids as the final output. Here's the implementation to achieve the same -

# Calculate linear indices corressponding to idxlin_idx = idx[:,0]*U + idx[:,1]# Get unique linear indices and their countsunq_lin_idx,idx_counts = np.unique(lin_idx,return_counts=True)# Setup output array and store index counts in raveled/flattened versiongrids = np.zeros((U,U))  grids.ravel()[unq_lin_idx] = idx_counts

Runtime tests -

Here are the runtime tests covering all the approaches (including @DSM's approaches) and using the same definitions as listed in that solution -

In [63]: U = 100    ...: idx = np.floor(np.random.random(size=(5000,2))*U).astype(np.int)    ...: In [64]: %timeit interadd(U, idx)100 loops, best of 3: 7.57 ms per loopIn [65]: %timeit interadd2(U, idx)100 loops, best of 3: 2.59 ms per loopIn [66]: %timeit interadd3(U, idx)1000 loops, best of 3: 1.24 ms per loopIn [67]: def unique_counts(U, idx):    ...:     lin_idx = idx[:,0]*U + idx[:,1]    ...:     unq_lin_idx,idx_counts = np.unique(lin_idx,return_counts=True)    ...:     grids = np.zeros((U,U))      ...:     grids.ravel()[unq_lin_idx] = idx_counts    ...:     return grids    ...: In [68]: %timeit unique_counts(U, idx)1000 loops, best of 3: 595 µs per loop

Runtimes suggest that the proposed np.unique based approach is more than 50% faster than the second fastest approach.