Pandas msgpack vs pickle
Pickle is better for the following:
- Numerical data or anything that uses the buffer protocol (numpy arrays) (though only if you use a somewhat recent
protocol=) - Python specific objects like classes, functions, etc.. (although here you should look at
cloudpickle)
MsgPack is better for the following:
- Cross language interoperation. It's an alternative to JSON with some improvements
- Performance on text data and Python objects. It's a decent factor faster than Pickle at this under any setting.
As @Jeff noted above this blogpost may be of interest