Pandas DataFrame stored list as string: How to convert back to list

As you pointed out, this can commonly happen when saving and loading pandas DataFrames as .csv files, which is a text format.

In your case this happened because list objects have a string representation, allowing them to be stored as .csv files. Loading the .csv will then yield that string representation.

If you want to store the actual objects, you should use DataFrame.to_pickle() (note: objects must be picklable!).

To answer your second question, you can convert it back with ast.literal_eval:

>>> from ast import literal_eval>>> literal_eval('[1.23, 2.34]')[1.23, 2.34]

You can directly use pandas -
df = pd.read_csv(df_name, converters={'column_name': eval})

This will read that column as a it's corresponding dtype in python instead of a string.

I just came across this problem and there is a very simple solution (pandas.eval()). I'm using pandas 0.20.0.

# SETUPimport pandas as pdimport iocsv = io.StringIO(u'''id  listA1  [1,2]A2  [3,4]A3  [5,6]''')df = pd.read_csv(csv, delim_whitespace = True)# TYPE CHECK <type 'str'>print type(df.at[0, 'list'])# MAIN CONVERSIONdf['list'] = pd.eval(df['list'])# TYPE CHECK <type 'list'>print type(df.at[0, 'list'])