Thursday, June 20, 2013

Python - Remove duplicate dictionaries in a list

Assuming I have a list of dictionaries, and I want to remove all of duplicate item from that list. The set built-in type of Python is what we need:

def remove_duplicate_items(my_list):
return_list = []
myset = set()
for i in my_list:
t = tuple(i.items())
if t not in myset:

return return_list

my_list = [{'a': 1, 'b': 2}, {'a': 4, 'b': 7}, {'a': 1, 'b': 2}]

my_final_list = remove_duplicate_items(my_list)

>>> [{'a': 1, 'b': 2}, {'a': 4, 'b': 7}]

Remember: a set only accepts tuples as its items.