Filtering a list based on a list of booleans Filtering a list based on a list of booleans numpy numpy

Filtering a list based on a list of booleans


You're looking for itertools.compress:

>>> from itertools import compress>>> list_a = [1, 2, 4, 6]>>> fil = [True, False, True, False]>>> list(compress(list_a, fil))[1, 4]

Timing comparisons(py3.x):

>>> list_a = [1, 2, 4, 6]>>> fil = [True, False, True, False]>>> %timeit list(compress(list_a, fil))100000 loops, best of 3: 2.58 us per loop>>> %timeit [i for (i, v) in zip(list_a, fil) if v]  #winner100000 loops, best of 3: 1.98 us per loop>>> list_a = [1, 2, 4, 6]*100>>> fil = [True, False, True, False]*100>>> %timeit list(compress(list_a, fil))              #winner10000 loops, best of 3: 24.3 us per loop>>> %timeit [i for (i, v) in zip(list_a, fil) if v]10000 loops, best of 3: 82 us per loop>>> list_a = [1, 2, 4, 6]*10000>>> fil = [True, False, True, False]*10000>>> %timeit list(compress(list_a, fil))              #winner1000 loops, best of 3: 1.66 ms per loop>>> %timeit [i for (i, v) in zip(list_a, fil) if v] 100 loops, best of 3: 7.65 ms per loop

Don't use filter as a variable name, it is a built-in function.


Like so:

filtered_list = [i for (i, v) in zip(list_a, filter) if v]

Using zip is the pythonic way to iterate over multiple sequences in parallel, without needing any indexing. This assumes both sequences have the same length (zip stops after the shortest runs out). Using itertools for such a simple case is a bit overkill ...

One thing you do in your example you should really stop doing is comparing things to True, this is usually not necessary. Instead of if filter[idx]==True: ..., you can simply write if filter[idx]: ....


With numpy:

In [128]: list_a = np.array([1, 2, 4, 6])In [129]: filter = np.array([True, False, True, False])In [130]: list_a[filter]Out[130]: array([1, 4])

or see Alex Szatmary's answer if list_a can be a numpy array but not filter

Numpy usually gives you a big speed boost as well

In [133]: list_a = [1, 2, 4, 6]*10000In [134]: fil = [True, False, True, False]*10000In [135]: list_a_np = np.array(list_a)In [136]: fil_np = np.array(fil)In [139]: %timeit list(itertools.compress(list_a, fil))1000 loops, best of 3: 625 us per loopIn [140]: %timeit list_a_np[fil_np]10000 loops, best of 3: 173 us per loop