Loading a file with more than one line of JSON into Pandas Loading a file with more than one line of JSON into Pandas json json

Loading a file with more than one line of JSON into Pandas


From version 0.19.0 of Pandas you can use the lines parameter, like so:

import pandas as pddata = pd.read_json('/path/to/file.json', lines=True)


You have to read it line by line. For example, you can use the following code provided by ryptophan on reddit:

import pandas as pd# read the entire file into a python arraywith open('your.json', 'rb') as f:    data = f.readlines()# remove the trailing "\n" from each linedata = map(lambda x: x.rstrip(), data)# each element of 'data' is an individual JSON object.# i want to convert it into an *array* of JSON objects# which, in and of itself, is one large JSON object# basically... add square brackets to the beginning# and end, and have all the individual business JSON objects# separated by a commadata_json_str = "[" + ','.join(data) + "]"# now, load it into pandasdata_df = pd.read_json(data_json_str)


The following code helped me to load JSON content into a dataframe:

import jsonimport pandas as pdwith open('Appointment.json', encoding="utf8") as f:    data = f.readlines()    data = [json.loads(line) for line in data] #convert string to dict formatdf = pd.read_json(data) # Load into dataframe