How to shift dates in a pandas dataframe (add x months)? How to shift dates in a pandas dataframe (add x months)? pandas pandas

How to shift dates in a pandas dataframe (add x months)?


IIUC you could use apply with axis=1:

In [23]: df.apply(lambda x: x['mydate'] + pd.DateOffset(months = x['month shift']), axis=1)Out[23]:0   2000-03-011   2001-04-012   2002-05-013   2003-06-014   2004-07-015   2005-08-016   2006-09-017   2007-10-018   2008-11-019   2009-12-01dtype: datetime64[ns]


"one"-liner using the underlying numpy functionality:

df['my date shifted'] = (    df["mydate"].values.astype("datetime64[M]")     + df["month shift"].values.astype("timedelta64[M]"))


EdChurn's solution is indeed much faster than the answer of Anton Protopopov and in fact in my use case it executes in milliseconds versus the one with apply taking minutes. The problem is that the solution EdChurn posted in his comment gives slightly incorrect results. In the example:

import pandas as pdimport numpy as npimport datetimedf = pd.DataFrame()df['year'] = np.arange(2000,2010)df['month'] = 3df['mydate'] = pd.to_datetime((df.year * 10000 + df.month * 100 + 1).apply(str), format='%Y%m%d')df['month shift'] = np.arange(0,10)

The answer of:

df['my date shifted'] = df['mydate'] + pd.TimedeltaIndex( df['month shift'], unit='M')

gives:EdChurn solution

The correct solution can be obtained with:

def set_to_month_begin(series):    #Following doesn't work:    #  res = series.dt.floor("MS")    #This also doesn't work (it fails in case the date is already the first day of the month):    #  res = series - pd.offsets.MonthBegin(1)    res = pd.to_datetime(series).dt.normalize()    res = res - pd.to_timedelta(res.dt.day - 1, unit='d')    return resdef add_months(df, date_col, months_num_col):    """This function adds the number of months specified per each row in `months_num_col` to date in `date_col`.    This method is *significantly* faster than:        df.apply(lambda x: x[date_col] + pd.DateOffset(months = x[months_num_col]), axis=1)    """    number_of_days_in_avg_month = 365.24 / 12    time_delta = pd.TimedeltaIndex(df[months_num_col] * number_of_days_in_avg_month + 10, unit='D')    return set_to_month_begin(df[date_col] + time_delta)df['my date shifted'] = add_months(df, 'mydate', 'month shift')

This gives the following result:correct solution