Conditional forward fill in pandas Conditional forward fill in pandas pandas pandas

Conditional forward fill in pandas


groupby + ffill and bfill

df.Q=df.groupby('code').Q.apply(lambda x : x.ffill().bfill())dfOut[755]:           Date      S           E cp   Last       Q              code30  2017-11-10  22500  2017-11-17  P  170.0  828.47  11/17/2017P2250032  2017-11-10  22625  2017-11-17  P  180.0  646.91  11/17/2017P2262535  2017-11-10  22750  2017-11-17  C  145.0  651.84  11/17/2017C2275036  2017-11-13  22500  2017-11-17  P  245.0  828.47  11/17/2017P2250038  2017-11-13  22625  2017-11-17  P  315.0  646.91  11/17/2017P2262541  2017-11-13  22750  2017-11-17  C   35.0  651.84  11/17/2017C2275042  2017-11-14  22500  2017-11-17  P  215.0  828.47  11/17/2017P2250044  2017-11-14  22625  2017-11-17  P  305.0  646.91  11/17/2017P2262547  2017-11-14  22750  2017-11-17  C   26.0  651.84  11/17/2017C2275048  2017-11-15  22500  2017-11-17  P  490.0  828.47  11/17/2017P2250050  2017-11-15  22625  2017-11-17  P  605.0  646.91  11/17/2017P2262553  2017-11-15  22750  2017-11-17  C    4.0  651.84  11/17/2017C2275054  2017-11-16  22500  2017-11-17  P  140.0  828.47  11/17/2017P2250056  2017-11-16  22625  2017-11-17  P  295.0  646.91  11/17/2017P2262559  2017-11-16  22750  2017-11-17  C    4.0  651.84  11/17/2017C2275060  2017-11-17  22250  2017-11-24  P  165.0  707.57  11/24/2017P2225061  2017-11-17  22375  2017-11-24  P  195.0  607.16  11/24/2017P2237565  2017-11-17  22500  2017-11-24  C  175.0  666.56  11/24/2017C2250066  2017-11-20  22250  2017-11-24  P  175.0  707.57  11/24/2017P2225067  2017-11-20  22375  2017-11-24  P  225.0  607.16  11/24/2017P2237571  2017-11-20  22500  2017-11-24  C   75.0  666.56  11/24/2017C2250072  2017-11-21  22250  2017-11-24  P   70.0  707.57  11/24/2017P2225073  2017-11-21  22375  2017-11-24  P  120.0  607.16  11/24/2017P2237577  2017-11-21  22500  2017-11-24  C   95.0  666.56  11/24/2017C2250078  2017-11-22  22250  2017-11-24  P   15.0  707.57  11/24/2017P2225079  2017-11-22  22375  2017-11-24  P   35.0  607.16  11/24/2017P2237583  2017-11-22  22500  2017-11-24  C  125.0  666.56  11/24/2017C2250084  2017-11-24  22375  2017-12-01  P  140.0  834.13  12/01/2017P2237585  2017-11-24  22500  2017-12-01  P  185.0  763.76  12/01/2017P2250089  2017-11-24  22625  2017-12-01  C  165.0  750.45  12/01/2017C22625


You can use transform on the groupby object.

df.loc[:, 'Q'] = df.groupby('code')['Q'].transform(lambda group: group.ffill())

Timings

%timeit -n 1000 df.loc[:, 'Q'] = df.groupby('code')['Q'].transform(lambda group: group.ffill())# 1000 loops, best of 3: 2.41 ms per loop%timeit -n 1000 df.loc[:, 'Q'] = df.groupby('code')['Q'].ffill()# 1000 loops, best of 3: 3.66 ms per loop