Using Pandas json_normalize on nested Json with arrays

Once we get past first normalization, I'd apply a lambda to finish the job.

from cytoolz.dicttoolz import mergepd.io.json.json_normalize(data).pipe(    lambda x: x.drop('ProductSMCP', 1).join(        x.ProductSMCP.apply(lambda y: pd.Series(merge(y)))    ))  Product.Currency Product.Description Product.Operational Product.TypeLevel1 Product.TypeLevel2  Xref.BBT Xref.SCSP Xref.TCK _id SMCP SMCP20              NaN              3 YEAR                 NaN           INTEREST               LONG       NaN        96      NaN  25   01   NaN1              USD             10 YEAR                True           INTEREST               LONG  CITITYM9       NaN       ZN  26   01    02

Trim Column Names

pd.io.json.json_normalize(data).pipe(    lambda x: x.drop('ProductSMCP', 1).join(        x.ProductSMCP.apply(lambda y: pd.Series(merge(y)))    )).rename(columns=lambda x: re.sub('(Product|Xref)\.', '', x))  Currency Description Operational TypeLevel1 TypeLevel2       BBT SCSP  TCK _id SMCP SMCP20      NaN      3 YEAR         NaN   INTEREST       LONG       NaN   96  NaN  25   01   NaN1      USD     10 YEAR        True   INTEREST       LONG  CITITYM9  NaN   ZN  26   01    02