Input and output numpy arrays to h5py Input and output numpy arrays to h5py numpy numpy

Input and output numpy arrays to h5py


h5py provides a model of datasets and groups. The former is basically arrays and the latter you can think of as directories. Each is named. You should look at the documentation for the API and examples:

http://docs.h5py.org/en/latest/quick.html

A simple example where you are creating all of the data upfront and just want to save it to an hdf5 file would look something like:

In [1]: import numpy as npIn [2]: import h5pyIn [3]: a = np.random.random(size=(100,20))In [4]: h5f = h5py.File('data.h5', 'w')In [5]: h5f.create_dataset('dataset_1', data=a)Out[5]: <HDF5 dataset "dataset_1": shape (100, 20), type "<f8">In [6]: h5f.close()

You can then load that data back in using:'

In [10]: h5f = h5py.File('data.h5','r')In [11]: b = h5f['dataset_1'][:]In [12]: h5f.close()In [13]: np.allclose(a,b)Out[13]: True

Definitely check out the docs:

http://docs.h5py.org

Writing to hdf5 file depends either on h5py or pytables (each has a different python API that sits on top of the hdf5 file specification). You should also take a look at other simple binary formats provided by numpy natively such as np.save, np.savez etc:

http://docs.scipy.org/doc/numpy/reference/routines.io.html


A cleaner way to handle file open/close and avoid memory leaks:

Prep:

import numpy as npimport h5pydata_to_write = np.random.random(size=(100,20)) # or some such

Write:

with h5py.File('name-of-file.h5', 'w') as hf:    hf.create_dataset("name-of-dataset",  data=data_to_write)

Read:

with h5py.File('name-of-file.h5', 'r') as hf:    data = hf['name-of-dataset'][:]