Share SciPy Sparse Array Between Process Objects Share SciPy Sparse Array Between Process Objects numpy numpy

Share SciPy Sparse Array Between Process Objects


I cannot help you with the multiprocessing part of your question, but a CSC sparse matrix is little more than three numpy arrays. You can instantiate another sparse matrix, b, sharing the same memory objects as a sparse matrix, a, by doing:

import scipy.sparse as spsb = sps.csc_matrix((a.data, a.indices, a.indptr), shape=a.shape, copy=False)

a.data, a.indices and a.indptr are the three numpy arrays you want to share between your processes, if you can do that, then instantiating a sparse matrix in each process will be an inexpensive operation.