How do you remove duplicates from a Python list while preserving order?

By | May 8, 2021

There are many different techniques and methods present in Python, to remove duplicate items from the list. To know about all the techniques check out this tutorial on How to remove duplicate items from a list.

For most cases when we remove the duplicate items from a list using predefined methods or techniques we generally disorder the list. Let’s if the list has 3 similar elements and we only wish to remove the duplicates 2 that’s come after the first one. With this, we manage to preserve the order of the list by removing the duplicate ones.

Vamware

In this Python tutorial, I will walk you through a simple Python function that accepts a list and return the list by removing all duplicate elements or items and preserve the order of the first occurrence of every element.

Python program to remove duplicate list items with preserving order

Now let’s write the program then we will break it in our Behind the code section.

def remove_dup(my_list):

    output_list = []

    for item in my_list:
        if item not in output_list:
            output_list.append(item)

    return output_list

my_list = [1,2,4,1,3,4,5,6,7,8,9,10,10,11,23,34,12,23]

print(remove_dup(my_list))

Output

[1, 2, 4, 3, 5, 6, 7, 8, 9, 10, 11, 23, 34, 12]

Behind the code

As you can see that the output is only containing unique elements and the order of every element is also preserved.

Our main method remove_dup(my_list) accept the list as a parameter.

Inside the function, we defined an empty list output_list.

Then we create a for loop that loops over every item of my_list. And inside the loop, we have a conditional statement that checks if the item does not present in the output_listlist then the element would not be appended to the list.

At last, we return the output_list that contain the unique elements of my_list and preserve order.

The time complexity of above program is O(N).

Conclusion

In this Python tutorial, we discussed how to remove duplicate items from a list while preserving the order of the list. To achieve our objective we have used an extra list that stores all the unique elements of our main list.

If you like this Python program or have any suggestions or queries related to the above program please let us know by commenting down below.

Leave a Reply

Your email address will not be published. Required fields are marked *