How do I remove duplicate items from an array in Perl?

You can do something like this as demonstrated in perlfaq4:

sub uniq {    my %seen;    grep !$seen{$_}++, @_;}my @array = qw(one two three two three);my @filtered = uniq(@array);print "@filtered\n";

Outputs:

one two three

If you want to use a module, try the uniq function from List::MoreUtils

The Perl documentation comes with a nice collection of FAQs. Your question is frequently asked:

% perldoc -q duplicate

The answer, copy and pasted from the output of the command above, appears below:

Found in /usr/local/lib/perl5/5.10.0/pods/perlfaq4.pod How can I remove duplicate elements from a list or array?   (contributed by brian d foy)   Use a hash. When you think the words "unique" or "duplicated", think   "hash keys".   If you don't care about the order of the elements, you could just   create the hash then extract the keys. It's not important how you   create that hash: just that you use "keys" to get the unique elements.       my %hash   = map { $_, 1 } @array;       # or a hash slice: @hash{ @array } = ();       # or a foreach: $hash{$_} = 1 foreach ( @array );       my @unique = keys %hash;   If you want to use a module, try the "uniq" function from   "List::MoreUtils". In list context it returns the unique elements,   preserving their order in the list. In scalar context, it returns the   number of unique elements.       use List::MoreUtils qw(uniq);       my @unique = uniq( 1, 2, 3, 4, 4, 5, 6, 5, 7 ); # 1,2,3,4,5,6,7       my $unique = uniq( 1, 2, 3, 4, 4, 5, 6, 5, 7 ); # 7   You can also go through each element and skip the ones you've seen   before. Use a hash to keep track. The first time the loop sees an   element, that element has no key in %Seen. The "next" statement creates   the key and immediately uses its value, which is "undef", so the loop   continues to the "push" and increments the value for that key. The next   time the loop sees that same element, its key exists in the hash and   the value for that key is true (since it's not 0 or "undef"), so the   next skips that iteration and the loop goes to the next element.       my @unique = ();       my %seen   = ();       foreach my $elem ( @array )       {         next if $seen{ $elem }++;         push @unique, $elem;       }   You can write this more briefly using a grep, which does the same   thing.       my %seen = ();       my @unique = grep { ! $seen{ $_ }++ } @array;

Install List::MoreUtils from CPAN

Then in your code:

use strict;use warnings;use List::MoreUtils qw(uniq);my @dup_list = qw(1 1 1 2 3 4 4);my @uniq_list = uniq(@dup_list);