Advanced Topics: Creating Custom Iterable Objects
In this step, we'll explore how to create your own iterable objects in Python. This is an important skill that allows you to design custom data structures that work seamlessly with Python's iteration mechanisms.
Understanding the Iterator Protocol
To create a custom iterable object, you need to implement the iterator protocol. This involves:
- Implementing the
__iter__() method that returns an iterator object
- The iterator object must implement a
__next__() method that returns the next value in the sequence
Let's create a simple custom iterable class to demonstrate this:
- Create a new file called
custom_iterable.py
- Add the following code:
## custom_iterable.py
class CountDown:
"""
A custom iterable class that counts down from a specified number to 1.
"""
def __init__(self, start):
"""Initialize with the starting number."""
self.start = start
def __iter__(self):
"""Return an iterator object."""
## This is a simple case where the class is both the iterable and iterator
## In more complex cases, you might return a separate iterator class
self.current = self.start
return self
def __next__(self):
"""Return the next value in the sequence."""
if self.current <= 0:
## Signal the end of iteration
raise StopIteration
## Decrement the counter and return the previous value
self.current -= 1
return self.current + 1
## Test the custom iterable
countdown = CountDown(5)
print("Custom iterable countdown from 5:")
for number in countdown:
print(number, end=" ")
print()
## We can iterate through it again
print("Iterating again:")
for number in countdown:
print(number, end=" ")
print()
## We can also check if it's iterable using our utility
from iteration_utils import is_iterable, get_iterable_info
print("\nChecking if CountDown is iterable:")
print(f"Is CountDown(5) iterable? {is_iterable(countdown)}")
print("Detailed info:", get_iterable_info(countdown))
- Save the file and run it:
python3 custom_iterable.py
You should see the countdown sequence from 5 to 1, and then again when we iterate a second time. This demonstrates that our custom class is indeed iterable.
Creating a More Complex Iterable: Fibonacci Sequence
Let's create a more interesting iterable that generates the Fibonacci sequence up to a specified limit:
- Create a new file called
fibonacci_iterable.py
- Add the following code:
## fibonacci_iterable.py
class Fibonacci:
"""An iterable that generates Fibonacci numbers up to a specified limit."""
def __init__(self, limit):
"""
Initialize with a limit (the maximum Fibonacci number to generate).
Args:
limit: The maximum value in the sequence
"""
self.limit = limit
def __iter__(self):
"""Return a fresh iterator."""
return FibonacciIterator(self.limit)
class FibonacciIterator:
"""Iterator for the Fibonacci sequence."""
def __init__(self, limit):
self.limit = limit
self.previous = 0
self.current = 1
def __next__(self):
"""Return the next Fibonacci number."""
## Check if we've reached the limit
if self.previous > self.limit:
raise StopIteration
## Save the current value to return
result = self.previous
## Update for the next iteration
self.previous, self.current = self.current, self.previous + self.current
return result
## Test the Fibonacci iterable
print("Fibonacci sequence up to 100:")
for number in Fibonacci(100):
print(number, end=" ")
print()
## Converting to a list
fib_list = list(Fibonacci(50))
print("\nFibonacci sequence up to 50 as a list:")
print(fib_list)
## Using it in a list comprehension
fib_squared = [x**2 for x in Fibonacci(30)]
print("\nSquared Fibonacci numbers up to 30:")
print(fib_squared)
## Checking iterability
from iteration_utils import is_iterable, get_iterable_info
print("\nChecking if Fibonacci is iterable:")
fib = Fibonacci(100)
print(f"Is Fibonacci(100) iterable? {is_iterable(fib)}")
print("Detailed info:", get_iterable_info(fib))
- Save the file and run it:
python3 fibonacci_iterable.py
This example demonstrates a more sophisticated iterable class that separates the iterable (the Fibonacci class) from the iterator (the FibonacciIterator class). This is a common pattern in more complex iterables.
Practical Use Case: Data Processing Pipeline
Finally, let's create a simple data processing pipeline using our knowledge of iterables:
- Create a new file called
data_pipeline.py
- Add the following code:
## data_pipeline.py
class DataSource:
"""
A data source that can yield data records.
This simulates reading from a file, database, or API.
"""
def __init__(self, data):
self.data = data
def __iter__(self):
return iter(self.data)
class DataProcessor:
"""
A data processor that transforms data records.
"""
def __init__(self, source, transform_func):
self.source = source
self.transform_func = transform_func
def __iter__(self):
## Iterate through the source and apply the transformation
for item in self.source:
yield self.transform_func(item)
class DataSink:
"""
A data sink that collects processed records.
"""
def __init__(self):
self.collected_data = []
def collect(self, processor):
"""Collect all data from the processor."""
if not isinstance(processor, DataProcessor):
raise TypeError("Expected a DataProcessor")
for item in processor:
self.collected_data.append(item)
return self.collected_data
## Sample data - a list of dictionaries representing people
sample_data = [
{"name": "Alice", "age": 25, "city": "New York"},
{"name": "Bob", "age": 30, "city": "Los Angeles"},
{"name": "Charlie", "age": 35, "city": "Chicago"},
{"name": "Diana", "age": 40, "city": "Houston"},
{"name": "Eve", "age": 45, "city": "Phoenix"}
]
## Create a data source
source = DataSource(sample_data)
## Define a transformation function
def transform_record(record):
## Create a new record with transformed data
return {
"full_name": record["name"].upper(),
"age_in_months": record["age"] * 12,
"location": record["city"]
}
## Create a data processor
processor = DataProcessor(source, transform_record)
## Create a data sink and collect the processed data
sink = DataSink()
processed_data = sink.collect(processor)
## Display the results
print("Original data:")
for record in sample_data:
print(record)
print("\nProcessed data:")
for record in processed_data:
print(record)
- Save the file and run it:
python3 data_pipeline.py
This example demonstrates a practical application of iterables in creating a data processing pipeline. Each component in the pipeline (source, processor, sink) is designed to work with Python's iteration mechanisms, making the code clean and efficient.
