Python JSON encoder convert NaNs to null instead
As @Gerrat points out, your hook
dumps(d, cls=NanConverter)
unfortunately won't work.@Alexander's
simplejson.dumps(d, ignore_nan=True)
works but introduces an additional dependency (simplejson
).
If we introduce another dependency (pandas):
Another obvious solution would be
dumps(pd.DataFrame(d).fillna(None))
, but Pandas issue 1972 notes thatd.fillna(None)
will have unpredictable behaviour:Note that
fillna(None)
is equivalent tofillna()
, which means the value parameter is unused. Instead, it uses the method parameter which is by default forward fill.So instead, use
DataFrame.where
:df = pd.DataFrame(d)dumps(df.where(pd.notnull(df), None)))
Unfortunately, you probably need to use @Bramar's suggestion. You're not going to be able to use this directly. The documentation for Python's JSON encoder states:
If specified, default is a function that gets called for objects that can’t otherwise be serialized
Your NanConverter.default
method isn't even being called, since Python's JSON encoder already knows how to serialize np.nan
. Add some print statements - you'll see your method isn't even being called.