Read in all csv files from a directory using Python Read in all csv files from a directory using Python numpy numpy

Read in all csv files from a directory using Python


That's how I'd do it:

import osdirectory = os.path.join("c:\\","path")for root,dirs,files in os.walk(directory):    for file in files:       if file.endswith(".csv"):           f=open(file, 'r')           #  perform calculation           f.close()


Using pandas and glob as the base packages

import globimport pandas as pdglued_data = pd.DataFrame()for file_name in glob.glob(directoryPath+'*.csv'):    x = pd.read_csv(file_name, low_memory=False)    glued_data = pd.concat([glued_data,x],axis=0)


I think you look for something like this

import globfor file_name in glob.glob(directoryPath+'*.csv'):    x = np.genfromtxt(file_name,delimiter=',')[:,2]    # do your calculations

Edit

If you want to get all csv files from a folder (including subfolder) you could use subprocess instead of glob (note that this code only works on linux systems)

import subprocessfile_list = subprocess.check_output(['find',directoryPath,'-name','*.csv']).split('\n')[:-1]for i,file_name in enumerate(file_list):    x = np.genfromtxt(file_name,delimiter=',')[:,2]    # do your calculations    # now you can use i as an index

It first searches the folder and sub-folders for all file_names using the find command from the shell and applies your calculations afterwards.