Pickle incompatibility of numpy arrays between Python 2 and 3 Pickle incompatibility of numpy arrays between Python 2 and 3 python python

Pickle incompatibility of numpy arrays between Python 2 and 3


This seems like some sort of incompatibility. It's trying to load a "binstring" object, which is assumed to be ASCII, while in this case it is binary data. If this is a bug in the Python 3 unpickler, or a "misuse" of the pickler by numpy, I don't know.

Here is something of a workaround, but I don't know how meaningful the data is at this point:

import pickleimport gzipimport numpywith open('mnist.pkl', 'rb') as f:    u = pickle._Unpickler(f)    u.encoding = 'latin1'    p = u.load()    print(p)

Unpickling it in Python 2 and then repickling it is only going to create the same problem again, so you need to save it in another format.


If you are getting this error in python3, then, it could be an incompatibility issue between python 2 and python 3, for me the solution was to load with latin1 encoding:

pickle.load(file, encoding='latin1')


It appears to be an incompatibility issue between Python 2 and Python 3. I tried loading the MNIST dataset with

    train_set, valid_set, test_set = pickle.load(file, encoding='iso-8859-1')

and it worked for Python 3.5.2