Reading numpy arrays outside of Python

You should really look into OPeNDAP to simplify all aspects of scientific data networking. For Python, check out Pydap.

You can directly store your NumPy arrays into HDF5 format via h5py (or NetCDF), then stream the data to clients over HTTP using OPeNDAP.

I'd recommend using an existing data format for interchange of scientific data/arrays, such as NetCDF or HDF. In Python, you can use the PyNIO library which has numpy bindings, and there are several libraries for other languages. Both formats are built for handling large data and take care of language, machine representation problems, etc. They also work well with message passing, for example in parallel computing, so I suspect your use case is satisfied.

For something a little lighter-weight than HDF (though admittedly also more ad-hoc), you could also use JSON:

import jsonimport numpy as npx = np.arange(100, dtype=np.float64)print json.dumps(dict(data=x.tostring(),                      shape=x.shape,                      dtype=str(x.dtype)))

This would free your clients from needing to install HDF wrappers, at the expense of having to deal with a nonstandard protocol for data exchange (and possibly also needing to install JSON bindings !).

The tradeoff would be up to you to evaluate for your situation.