Remove Duplicates From ArrayList

Introduction

In Java, an ArrayList can store duplicate elements. If we want to remove the duplicates and only keep unique elements, we have to take specific steps. In this Lab, we will learn how to remove duplicates from an ArrayList in two ways: using a HashSet and using the distinct() method of Stream.

Skills Graph

Create an ArrayList

Create an ArrayList and add some elements to it. Here is an example:

import java.util.ArrayList;

public class RemoveDuplicates {
    public static void main(String[] args) {
        ArrayList<Integer> arrayList = new ArrayList<>();
        arrayList.add(1);
        arrayList.add(2);
        arrayList.add(3);
        arrayList.add(1);
        arrayList.add(4);
        arrayList.add(2);
        System.out.println("Original ArrayList: " + arrayList);
    }
}

To run the code in the terminal, navigate to the directory containing the file and enter the following command:

javac RemoveDuplicates.java && java RemoveDuplicates

Remove duplicates using HashSet

To remove duplicates using a HashSet, first create a HashSet and pass the ArrayList to its constructor. Since HashSet only contains unique elements, this eliminates all duplicates. Convert the resulting HashSet back into an ArrayList.

import java.util.ArrayList;
import java.util.HashSet;

public class RemoveDuplicates {
    public static void main(String[] args) {
        ArrayList<Integer> arrayList = new ArrayList<>();
        arrayList.add(1);
        arrayList.add(2);
        arrayList.add(3);
        arrayList.add(1);
        arrayList.add(4);
        arrayList.add(2);
        System.out.println("Original ArrayList: " + arrayList);

        // Remove duplicates
        HashSet<Integer> hashSet = new HashSet<>(arrayList);
        arrayList.clear();
        arrayList.addAll(hashSet);

        System.out.println("ArrayList with duplicates removed using HashSet: " + arrayList);
    }
}

Remove duplicates using distinct() method of Stream

To remove duplicates using the Stream API, first use the stream() method of ArrayList to create a Stream. Then use the distinct() method to return a new stream with distinct elements. Finally, use the collect() method to return the stream elements as an ArrayList.

import java.util.ArrayList;

public class RemoveDuplicates {
    public static void main(String[] args) {
        ArrayList<Integer> arrayList = new ArrayList<>();
        arrayList.add(1);
        arrayList.add(2);
        arrayList.add(3);
        arrayList.add(1);
        arrayList.add(4);
        arrayList.add(2);
        System.out.println("Original ArrayList: " + arrayList);

        // Remove duplicates
        arrayList = (ArrayList<Integer>) arrayList.stream().distinct().collect(Collectors.toList());

        System.out.println("ArrayList with duplicates removed using Stream: " + arrayList);
    }
}

Test Both Approaches

To test both approaches, compile and run both code examples in the terminal.

javac RemoveDuplicates.java && java RemoveDuplicates

Create a Function

To make it easier to remove duplicates from multiple ArrayLists in our program, we can create a function. Here is an example:

import java.util.ArrayList;
import java.util.HashSet;
import java.util.stream.Collectors;

public class RemoveDuplicates {
    public static void main(String[] args) {
        ArrayList<Integer> arrayList1 = new ArrayList<>();
        arrayList1.add(1);
        arrayList1.add(2);
        arrayList1.add(3);
        arrayList1.add(1);
        arrayList1.add(4);
        arrayList1.add(2);
        System.out.println("Original ArrayList: " + arrayList1);

        ArrayList<Integer> arrayList2 = new ArrayList<>();
        arrayList2.add(5);
        arrayList2.add(6);
        arrayList2.add(1);
        arrayList2.add(7);
        arrayList2.add(5);
        System.out.println("Original ArrayList: " + arrayList2);

        // Remove duplicates using HashSet
        removeDuplicatesUsingHashSet(arrayList1);
        removeDuplicatesUsingHashSet(arrayList2);

        // Remove duplicates using Stream
        removeDuplicatesUsingStream(arrayList1);
        removeDuplicatesUsingStream(arrayList2);

    }

    public static void removeDuplicatesUsingHashSet(ArrayList<Integer> arrayList) {
        HashSet<Integer> hashSet = new HashSet<>(arrayList);
        arrayList.clear();
        arrayList.addAll(hashSet);

        System.out.println("ArrayList with duplicates removed using HashSet: " + arrayList);
    }

    public static void removeDuplicatesUsingStream(ArrayList<Integer> arrayList) {
        arrayList = (ArrayList<Integer>) arrayList.stream().distinct().collect(Collectors.toList());
        System.out.println("ArrayList with duplicates removed using Stream: " + arrayList);
    }
}

Compile and Run

To run the code in the terminal, navigate to the directory containing the file and enter the following command:

javac RemoveDuplicates.java && java RemoveDuplicates

Modify Data Type

The removeDuplicatesUsingHashSet() function only works for ArrayList<Integer>. To make the function more flexible, we can modify the data type to ArrayList<T>.

import java.util.ArrayList;
import java.util.HashSet;
import java.util.stream.Collectors;

public class RemoveDuplicates<T> {
    public static void main(String[] args) {
        ArrayList<Integer> arrayList1 = new ArrayList<>();
        arrayList1.add(1);
        arrayList1.add(2);
        arrayList1.add(3);
        arrayList1.add(1);
        arrayList1.add(4);
        arrayList1.add(2);
        System.out.println("Original ArrayList: " + arrayList1);

        ArrayList<String> arrayList2 = new ArrayList<>();
        arrayList2.add("Hello");
        arrayList2.add("World");
        arrayList2.add("Java");
        arrayList2.add("Hello");
        arrayList2.add("Mars");
        System.out.println("Original ArrayList: " + arrayList2);

        // Remove duplicates using HashSet
        removeDuplicatesUsingHashSet(arrayList1);
        removeDuplicatesUsingHashSet(arrayList2);

        // Remove duplicates using Stream
        removeDuplicatesUsingStream(arrayList1);
        removeDuplicatesUsingStream(arrayList2);

    }

    public static <T> void removeDuplicatesUsingHashSet(ArrayList<T> arrayList) {
        HashSet<T> hashSet = new HashSet<>(arrayList);
        arrayList.clear();
        arrayList.addAll(hashSet);

        System.out.println("ArrayList with duplicates removed using HashSet: " + arrayList);
    }

    public static <T> void removeDuplicatesUsingStream(ArrayList<T> arrayList) {
        arrayList = (ArrayList<T>) arrayList.stream().distinct().collect(Collectors.toList());
        System.out.println("ArrayList with duplicates removed using Stream: " + arrayList);
    }
}

Compile and Run the Modified Version

To run the code in the terminal, navigate to the directory containing the file and enter the following command:

javac RemoveDuplicates.java && java RemoveDuplicates

Add User Input

To make our program more interactive, we can allow the user to enter the integers they want to add to the ArrayList. Here is an example:

import java.util.ArrayList;
import java.util.HashSet;
import java.util.Scanner;
import java.util.stream.Collectors;

public class RemoveDuplicates<T> {
    public static void main(String[] args) {
        ArrayList<Integer> arrayList1 = new ArrayList<>();
        int input = getUserInput();
        while (input != -1) {
            arrayList1.add(input);
            input = getUserInput();
        }
        System.out.println("Original ArrayList: " + arrayList1);

        // Remove duplicates using HashSet
        removeDuplicatesUsingHashSet(arrayList1);

        // Remove duplicates using Stream
        removeDuplicatesUsingStream(arrayList1);

    }

    public static <T> void removeDuplicatesUsingHashSet(ArrayList<T> arrayList) {
        HashSet<T> hashSet = new HashSet<>(arrayList);
        arrayList.clear();
        arrayList.addAll(hashSet);

        System.out.println("ArrayList with duplicates removed using HashSet: " + arrayList);
    }

    public static <T> void removeDuplicatesUsingStream(ArrayList<T> arrayList) {
        arrayList = (ArrayList<T>) arrayList.stream().distinct().collect(Collectors.toList());
        System.out.println("ArrayList with duplicates removed using Stream: " + arrayList);
    }

    public static int getUserInput() {
        Scanner scanner = new Scanner(System.in);
        System.out.print("Enter an integer to add to the ArrayList, or -1 to quit: ");
        if (scanner.hasNextInt()) {
            return scanner.nextInt();
        }
        return -1;
    }
}

Compile and Run the Final Version

To run the code in the terminal, navigate to the directory containing the file and enter the following command:

javac RemoveDuplicates.java && java RemoveDuplicates

Summary

In this lab, we learned how to remove duplicates from an ArrayList in two ways: using a HashSet and using the distinct() method of Stream. We also created a function to make it easier to remove duplicates from multiple ArrayLists in our program, modified the data type to make the function more flexible, and allowed the user to enter the integers they want to add to the ArrayList.