Issues with try/except, attempting to convert strings to integers in pandas data frame where possible
insert the columm.append
into the try:
for col in list_of_columns: column = [] for row in list(df[col]): try: column.append(remove_html(row)) except ValueError: pass del df[col] df[col] = columnreturn df
consider the pd.DataFrame
df
df = pd.DataFrame(dict(A=[1, '2', '_', '4']))
You want to use the function pd.to_numeric
...
Notepd.to_numeric
operates on scalars and pd.Series
. It doesn't operate on a pd.DataFrame
Also
Use the parameter errors='coerce'
to get numbers where you can and NaN
elsewhere.
pd.to_numeric(df['A'], 'coerce')0 1.01 2.02 NaN3 4.0Name: A, dtype: float6
Or, to get numbers where you can, and what you already had elsewhere
pd.to_numeric(df['A'], 'coerce').combine_first(df['A'])0 11 22 _3 4Name: A, dtype: object
you can then assign it back to your df
df['A'] = pd.to_numeric(df['A'], 'coerce').combine_first(df['A'])
Works like this:
def clean_df(df):df = df.astype(str)list_of_columns = list(df.columns)for col in list_of_columns: column = [] for row in list(df[col]): try: column.append(int(remove_html(row))) except ValueError: column.append(remove_html(row)) del df[col] df[col] = columnreturn df