Reserve memory for list in Python? Reserve memory for list in Python? python python

Reserve memory for list in Python?


Here's four variants:

  • an incremental list creation
  • "pre-allocated" list
  • array.array()
  • numpy.zeros()

 

python -mtimeit -s"N=10**6" "a = []; app = a.append;"\    "for i in xrange(N):  app(i);"10 loops, best of 3: 390 msec per looppython -mtimeit -s"N=10**6" "a = [None]*N; app = a.append;"\    "for i in xrange(N):  a[i] = i"10 loops, best of 3: 245 msec per looppython -mtimeit -s"from array import array; N=10**6" "a = array('i', [0]*N)"\    "for i in xrange(N):" "  a[i] = i"10 loops, best of 3: 541 msec per looppython -mtimeit -s"from numpy import zeros; N=10**6" "a = zeros(N,dtype='i')"\    "for i in xrange(N):" "  a[i] = i"10 loops, best of 3: 353 msec per loop

It shows that [None]*N is the fastest and array.array is the slowest in this case.


you can create list of the known length like this:

>>> [None] * known_number


Take a look at this:

In [7]: %timeit array.array('f', [0.0]*4000*1000)1 loops, best of 3: 306 ms per loopIn [8]: %timeit array.array('f', [0.0])*4000*1000100 loops, best of 3: 5.96 ms per loopIn [11]: %timeit np.zeros(4000*1000, dtype='f')100 loops, best of 3: 6.04 ms per loopIn [9]: %timeit [0.0]*4000*100010 loops, best of 3: 32.4 ms per loop

So don't ever use array.array('f', [0.0]*N), use array.array('f', [0.0])*N or numpy.zeros.