Now that we understand different methods for finding common elements and have seen practical applications, let's compare the performance of these methods and establish some best practices.
Let's create a script to measure the performance of the different methods for finding common elements:
- Create a new file called
performance_test.py with the following content:
import time
import random
def measure_time(func, *args):
"""Measure the execution time of a function"""
start_time = time.time()
result = func(*args)
end_time = time.time()
return result, end_time - start_time
## Create large test lists with some overlap
def create_test_lists(size, overlap_percent):
"""Create two lists of the given size with specified overlap percentage"""
overlap_size = int(size * overlap_percent / 100)
## Create list1 with random numbers
list1 = random.sample(range(1, size * 10), size)
## Create list2 with some elements from list1 (overlap) and some new elements
overlap_elements = random.sample(list1, overlap_size)
remaining_size = size - overlap_size
new_elements = random.sample([x for x in range(1, size * 10) if x not in list1], remaining_size)
list2 = overlap_elements + new_elements
random.shuffle(list2)
return list1, list2
## Methods to find common elements
def using_loop(list1, list2):
common = []
for item in list1:
if item in list2:
common.append(item)
return common
def using_sets(list1, list2):
return list(set(list1) & set(list2))
def using_comprehension(list1, list2):
return [item for item in list1 if item in list2]
## Test with different list sizes
sizes = [100, 1000, 10000]
overlap = 50 ## 50% overlap
print("Performance comparison for finding common elements:")
print("-" * 60)
print(f"{'Size':<10}{'Loop (s)':<15}{'Set (s)':<15}{'Comprehension (s)':<20}")
print("-" * 60)
for size in sizes:
list1, list2 = create_test_lists(size, overlap)
## Measure time for each method
_, loop_time = measure_time(using_loop, list1, list2)
_, set_time = measure_time(using_sets, list1, list2)
_, comp_time = measure_time(using_comprehension, list1, list2)
print(f"{size:<10}{loop_time:<15.6f}{set_time:<15.6f}{comp_time:<20.6f}")
- Run the script:
python3 performance_test.py
You should see a performance comparison between the three methods, with execution times for different list sizes. The output will look something like this (actual times will vary):
Performance comparison for finding common elements:
------------------------------------------------------------
Size Loop (s) Set (s) Comprehension (s)
------------------------------------------------------------
100 0.000134 0.000050 0.000117
1000 0.008561 0.000247 0.009018
10000 0.910376 0.001944 0.915267
Best Practices
Based on what we've learned, let's create a file with best practices for finding common elements in Python lists:
- Create a new file called
best_practices.py with the following content:
"""
Best Practices for Finding Common Elements in Python Lists
This file demonstrates recommended approaches for different scenarios
when finding common elements between lists.
"""
## Sample lists for demonstration
small_list1 = [1, 2, 3, 4, 5]
small_list2 = [4, 5, 6, 7, 8]
large_list1 = list(range(1, 10001))
large_list2 = list(range(5001, 15001))
print("Best Practices for Finding Common Elements in Python Lists")
print("=" * 60)
print("\n1. For small lists (less than ~100 elements):")
print(" Any method works well, but set intersection is still recommended for clarity:")
common_small = list(set(small_list1) & set(small_list2))
print(f" Common elements: {common_small}")
print("\n2. For large lists (100+ elements):")
print(" Always use set intersection for performance:")
## Using set method for large lists
start_time = __import__('time').time()
common_large = list(set(large_list1) & set(large_list2))
end_time = __import__('time').time()
print(f" Found {len(common_large)} common elements in {end_time - start_time:.6f} seconds")
print("\n3. When order matters:")
print(" Use list comprehension with a set for lookup efficiency:")
lookup_set = set(small_list2) ## Convert the second list to a set for O(1) lookups
ordered_common = [item for item in small_list1 if item in lookup_set]
print(f" Common elements (preserving order from list1): {ordered_common}")
print("\n4. When dealing with duplicates:")
print(" Standard set intersection removes duplicates. If you need to keep them:")
list1_with_duplicates = [1, 2, 2, 3, 4, 4, 5]
list2_with_duplicates = [2, 2, 4, 5, 5, 6]
print(f" List 1 with duplicates: {list1_with_duplicates}")
print(f" List 2 with duplicates: {list2_with_duplicates}")
## Method that preserves duplicates
def find_common_with_duplicates(list1, list2):
result = []
list2_copy = list2.copy() ## Create a copy to modify
for item in list1:
if item in list2_copy:
result.append(item)
list2_copy.remove(item) ## Remove to avoid matching the same item again
return result
common_with_duplicates = find_common_with_duplicates(list1_with_duplicates, list2_with_duplicates)
print(f" Common elements (preserving duplicates): {common_with_duplicates}")
print("\nSummary:")
print("1. For most cases: Use set intersection -> list(set(list1) & set(list2))")
print("2. When order matters: Convert smaller list to set, use list comprehension on larger list")
print("3. When duplicates matter: Use custom functions that check and remove matched elements")
print("4. Always consider the specific requirements of your use case")
- Run the script:
python3 best_practices.py
This will display a comprehensive guide to best practices for finding common elements in different scenarios.
Create a Utility Module
Finally, let's create a reusable utility module that we can import in future projects:
- Create a new file called
list_utils.py with the following content:
"""
List Utilities Module
A collection of functions for working with lists, including finding common elements.
"""
def find_common_elements(list1, list2, method='set', preserve_order=False, preserve_duplicates=False):
"""
Find common elements between two lists.
Parameters:
list1 (list): First list
list2 (list): Second list
method (str): Method to use ('set', 'loop', or 'comprehension')
preserve_order (bool): Whether to preserve the order of items from list1
preserve_duplicates (bool): Whether to preserve duplicate common elements
Returns:
list: List of common elements
"""
## Handle the case with duplicates
if preserve_duplicates:
result = []
list2_copy = list2.copy()
for item in list1:
if item in list2_copy:
result.append(item)
list2_copy.remove(item)
return result
## When order matters but duplicates don't
if preserve_order and not preserve_duplicates:
lookup_set = set(list2)
return [item for item in list1 if item in lookup_set]
## When neither order nor duplicates matter
if method == 'set':
return list(set(list1) & set(list2))
elif method == 'loop':
common = []
for item in list1:
if item in list2 and item not in common:
common.append(item)
return common
elif method == 'comprehension':
seen = set()
return [item for item in list1 if item in list2 and not (item in seen or seen.add(item))]
else:
raise ValueError("Method must be 'set', 'loop', or 'comprehension'")
def list_difference(list1, list2):
"""
Find elements in list1 that are not in list2.
Parameters:
list1 (list): First list
list2 (list): Second list
Returns:
list: Elements in list1 but not in list2
"""
return list(set(list1) - set(list2))
def list_union(list1, list2):
"""
Find all unique elements from both lists combined.
Parameters:
list1 (list): First list
list2 (list): Second list
Returns:
list: All unique elements from both lists
"""
return list(set(list1) | set(list2))
## Usage examples
if __name__ == "__main__":
## Sample lists
list1 = [1, 2, 3, 4, 5, 5]
list2 = [4, 5, 5, 6, 7]
print("Original lists:")
print(f"List 1: {list1}")
print(f"List 2: {list2}")
print("\nCommon elements (using different methods):")
print(f"Set method: {find_common_elements(list1, list2, method='set')}")
print(f"Loop method: {find_common_elements(list1, list2, method='loop')}")
print(f"Comprehension method: {find_common_elements(list1, list2, method='comprehension')}")
print("\nWith order and duplicates preserved:")
print(f"Preserving order: {find_common_elements(list1, list2, preserve_order=True)}")
print(f"Preserving duplicates: {find_common_elements(list1, list2, preserve_duplicates=True)}")
print("\nOther list operations:")
print(f"Elements in list1 but not in list2: {list_difference(list1, list2)}")
print(f"Elements in list2 but not in list1: {list_difference(list2, list1)}")
print(f"All unique elements from both lists: {list_union(list1, list2)}")
- Run the script to see the utility module in action:
python3 list_utils.py
- Now let's test our utility module by importing it in a new script. Create a file called
test_utils.py:
## Import our utility functions
from list_utils import find_common_elements, list_difference, list_union
## Creating some test data
fruits1 = ["apple", "banana", "cherry", "date", "banana"]
fruits2 = ["banana", "cherry", "fig", "grape", "banana"]
print("Fruits List 1:", fruits1)
print("Fruits List 2:", fruits2)
## Find common elements with different options
common_default = find_common_elements(fruits1, fruits2)
common_order = find_common_elements(fruits1, fruits2, preserve_order=True)
common_duplicates = find_common_elements(fruits1, fruits2, preserve_duplicates=True)
print("\nCommon fruits (default):", common_default)
print("Common fruits (preserving order):", common_order)
print("Common fruits (preserving duplicates):", common_duplicates)
## Find differences between lists
only_in_fruits1 = list_difference(fruits1, fruits2)
only_in_fruits2 = list_difference(fruits2, fruits1)
print("\nFruits only in list 1:", only_in_fruits1)
print("Fruits only in list 2:", only_in_fruits2)
## Find union of lists
all_unique_fruits = list_union(fruits1, fruits2)
print("\nAll unique fruits from both lists:", all_unique_fruits)
- Run the test script:
python3 test_utils.py
You should see output demonstrating our utility functions with various options.
By completing this step, you've not only learned about the performance implications of different methods but also created a reusable utility module that you can incorporate into future Python projects.
