How to print Pyspark Dataframe like pandas Dataframe in jupyter

pandas dataframe view pyspark jupyter

You can use the ability to convert a pyspark dataframe directly to a pandas dataframe. The command for the same would be -

df.limit(10).toPandas()

This should directly yield the result as a pandas dataframe and you just need to have pandas package installed.

pandas dataframe view pyspark jupyter

You have to use the below code

from IPython.display import displayimport pandas as pdimport numpy as npd = {'col1': [1, 2], 'col2': [3, 4]}df = pd.DataFrame(data=d)display(df)

CodeHunter