Python Pandas add Filename Column CSV Python Pandas add Filename Column CSV python python

Python Pandas add Filename Column CSV


This should work:

import osfor csv in globbed_files:    frame = pd.read_csv(csv)    frame['filename'] = os.path.basename(csv)    data.append(frame)

frame['filename'] creates a new column named filename and os.path.basename() turns a path like /a/d/c.txt into the filename c.txt.


Mike's answer above works perfectly. In case any googlers run into the following error:

>>> TypeError: cannot concatenate object of type "<type 'str'>";     only pd.Series, pd.DataFrame, and pd.Panel (deprecated) objs are valid

It's possibly because the separator is not correct. I was using a custom csv file so the separator was ^. Becuase of that I needed to include the separator in the pd.read_csv call.

import osfor csv in globbed_files:    frame = pd.read_csv(csv, sep='^')    frame['filename'] = os.path.basename(csv)    data.append(frame)