How to remove duplicates from Python list and keep order? [duplicate]

A list can be sorted and deduplicated using built-in functions:

myList = sorted(set(myList))

python list sorting unique

If your input is already sorted, then there may be a simpler way to do it:

from operator import itemgetterfrom itertools import groupbyunique_list = list(map(itemgetter(0), groupby(yourList)))

python list sorting unique

If you want to keep order of the original list, just use OrderedDict with None as values.

In Python2:

    from collections import OrderedDict    from itertools import izip, repeat    unique_list = list(OrderedDict(izip(my_list, repeat(None))))

In Python3 it's even simpler:

    from collections import OrderedDict    from itertools import repeat    unique_list = list(OrderedDict(zip(my_list, repeat(None))))

If you don't like iterators (zip and repeat) you can use a generator (works both in 2 & 3):

    from collections import OrderedDict    unique_list = list(OrderedDict((element, None) for element in my_list))

CodeHunter