Numpy isnan() fails on an array of floats (from pandas dataframe apply)
np.isnan
can be applied to NumPy arrays of native dtype (such as np.float64):
In [99]: np.isnan(np.array([np.nan, 0], dtype=np.float64))Out[99]: array([ True, False], dtype=bool)
but raises TypeError when applied to object arrays:
In [96]: np.isnan(np.array([np.nan, 0], dtype=object))TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
Since you have Pandas, you could use pd.isnull
instead -- it can accept NumPy arrays of object or native dtypes:
In [97]: pd.isnull(np.array([np.nan, 0], dtype=float))Out[97]: array([ True, False], dtype=bool)In [98]: pd.isnull(np.array([np.nan, 0], dtype=object))Out[98]: array([ True, False], dtype=bool)
Note that None
is also considered a null value in object arrays.
On top of @unutbu answer, you could coerce pandas numpy object array to native (float64) type, something along the line
import pandas as pdpd.to_numeric(df['tester'], errors='coerce')
Specify errors='coerce' to force strings that can't be parsed to a numeric value to become NaN. Column type would be dtype: float64
, and then isnan
check should work