Remove all the elements that occur in one list from another
Python has a language feature called List Comprehensions that is perfectly suited to making this sort of thing extremely easy. The following statement does exactly what you want and stores the result in l3
:
l3 = [x for x in l1 if x not in l2]
l3
will contain [1, 6]
.
One way is to use sets:
>>> set([1,2,6,8]) - set([2,3,5,8])set([1, 6])
Note, however, that sets do not preserve the order of elements, and cause any duplicated elements to be removed. The elements also need to be hashable. If these restrictions are tolerable, this may often be the simplest and highest performance option.
Performance Comparisons
Comparing the performance of all the answers mentioned here on Python 3.9.1 and Python 2.7.16.
Python 3.9.1
Answers are mentioned in order of performance:
Arkku's
set
difference using subtraction "-" operation - (91.3 nsec per loop)mquadri$ python3 -m timeit -s "l1 = set([1,2,6,8]); l2 = set([2,3,5,8]);" "l1 - l2"5000000 loops, best of 5: 91.3 nsec per loop
Moinuddin Quadri's using
set().difference()
- (133 nsec per loop)mquadri$ python3 -m timeit -s "l1 = set([1,2,6,8]); l2 = set([2,3,5,8]);" "l1.difference(l2)"2000000 loops, best of 5: 133 nsec per loop
Moinuddin Quadri's list comprehension with
set
based lookup- (366 nsec per loop)mquadri$ python3 -m timeit -s "l1 = [1,2,6,8]; l2 = set([2,3,5,8]);" "[x for x in l1 if x not in l2]" 1000000 loops, best of 5: 366 nsec per loop
Donut's list comprehension on plain list - (489 nsec per loop)
mquadri$ python3 -m timeit -s "l1 = [1,2,6,8]; l2 = [2,3,5,8];" "[x for x in l1 if x not in l2]" 500000 loops, best of 5: 489 nsec per loop
Daniel Pryden's generator expression with
set
based lookup and type-casting tolist
- (583 nsec per loop) : Explicitly type-casting to list to get the final object aslist
, as requested by OP. If generator expression is replaced with list comprehension, it'll become same as Moinuddin Quadri's list comprehension withset
based lookup.mquadri$ mquadri$ python3 -m timeit -s "l1 = [1,2,6,8]; l2 = set([2,3,5,8]);" "list(x for x in l1 if x not in l2)" 500000 loops, best of 5: 583 nsec per loop
Moinuddin Quadri's using
filter()
and explicitly type-casting tolist
(need to explicitly type-cast as in Python 3.x, it returns iterator) - (681 nsec per loop)mquadri$ python3 -m timeit -s "l1 = [1,2,6,8]; l2 = set([2,3,5,8]);" "list(filter(lambda x: x not in l2, l1))" 500000 loops, best of 5: 681 nsec per loop
Akshay Hazari's using combination of
functools.reduce
+filter
-(3.36 usec per loop) : Explicitly type-casting tolist
as from Python 3.x it started returned returning iterator. Also we need to importfunctools
to usereduce
in Python 3.xmquadri$ python3 -m timeit "from functools import reduce; l1 = [1,2,6,8]; l2 = [2,3,5,8];" "list(reduce(lambda x,y : filter(lambda z: z!=y,x) ,l1,l2))" 100000 loops, best of 5: 3.36 usec per loop
Python 2.7.16
Answers are mentioned in order of performance:
Arkku's
set
difference using subtraction "-" operation - (0.0783 usec per loop)mquadri$ python -m timeit -s "l1 = set([1,2,6,8]); l2 = set([2,3,5,8]);" "l1 - l2"10000000 loops, best of 3: 0.0783 usec per loop
Moinuddin Quadri's using
set().difference()
- (0.117 usec per loop)mquadri$ mquadri$ python -m timeit -s "l1 = set([1,2,6,8]); l2 = set([2,3,5,8]);" "l1.difference(l2)"10000000 loops, best of 3: 0.117 usec per loop
Moinuddin Quadri's list comprehension with
set
based lookup- (0.246 usec per loop)mquadri$ python -m timeit -s "l1 = [1,2,6,8]; l2 = set([2,3,5,8]);" "[x for x in l1 if x not in l2]" 1000000 loops, best of 3: 0.246 usec per loop
Donut's list comprehension on plain list - (0.372 usec per loop)
mquadri$ python -m timeit -s "l1 = [1,2,6,8]; l2 = [2,3,5,8];" "[x for x in l1 if x not in l2]" 1000000 loops, best of 3: 0.372 usec per loop
Moinuddin Quadri's using
filter()
- (0.593 usec per loop)mquadri$ python -m timeit -s "l1 = [1,2,6,8]; l2 = set([2,3,5,8]);" "filter(lambda x: x not in l2, l1)" 1000000 loops, best of 3: 0.593 usec per loop
Daniel Pryden's generator expression with
set
based lookup and type-casting tolist
- (0.964 per loop) : Explicitly type-casting to list to get the final object aslist
, as requested by OP. If generator expression is replaced with list comprehension, it'll become same as Moinuddin Quadri's list comprehension withset
based lookup.mquadri$ python -m timeit -s "l1 = [1,2,6,8]; l2 = set([2,3,5,8]);" "list(x for x in l1 if x not in l2)" 1000000 loops, best of 3: 0.964 usec per loop
Akshay Hazari's using combination of
functools.reduce
+filter
-(2.78 usec per loop)mquadri$ python -m timeit "l1 = [1,2,6,8]; l2 = [2,3,5,8];" "reduce(lambda x,y : filter(lambda z: z!=y,x) ,l1,l2)" 100000 loops, best of 3: 2.78 usec per loop