Iterators & Generators

Learn how to use Python's iteration protocol and build memory-efficient generators for real-world use

Understanding Iterators

An iterator is an object that implements __iter__() and __next__()
iter() function creates an iterator from an iterable
next() function retrieves the next value from an iterator
Raises StopIteration when no more items are available

# Using iter() and next()
my_list = [1, 2, 3, 4]
iterator = iter(my_list)

print(next(iterator))  # 1
print(next(iterator))  # 2
print(next(iterator))  # 3
print(next(iterator))  # 4

# Handling StopIteration
try:
    print(next(iterator))
except StopIteration:
    print("No more items")

Creating Custom Iterators

Implement __iter__() to return self
Implement __next__() to return the next value
Raise StopIteration when iteration is complete
Useful for custom iteration logic

# Custom iterator class
class Counter:
    def __init__(self, limit: int):
        self.limit = limit
        self.current = 0
    
    def __iter__(self):
        return self
    
    def __next__(self):
        if self.current < self.limit:
            self.current += 1
            return self.current
        raise StopIteration

# Using custom iterator
counter = Counter(5)
for num in counter:
    print(num, end=" ")

Generator Functions

Use yield keyword instead of return
Automatically implements the iterator protocol
State is preserved between calls
More memory efficient than lists

from typing import Generator

# Simple generator function
def count_up_to(limit: int) -> Generator[int, None, None]:
    count = 1
    while count <= limit:
        yield count
        count += 1

# Using generator
counter = count_up_to(5)
for num in counter:
    print(num, end=" ")

# Generator with multiple yields
def fibonacci(n: int) -> Generator[int, None, None]:
    a, b = 0, 1
    for _ in range(n):
        yield a
        a, b = b, a + b

print("\nFibonacci:")
for fib in fibonacci(10):
    print(fib, end=" ")

Generator Benefits

Memory efficient – values are generated on demand
Can represent infinite sequences
Lazy evaluation – compute only when needed
Enables pipeline processing with multiple generators

from typing import Generator

# Infinite generator
def infinite_sequence() -> Generator[int, None, None]:
    num = 0
    while True:
        yield num
        num += 1

# Use with limit
gen = infinite_sequence()
for _ in range(5):
    print(next(gen), end=" ")

# Generator pipeline
def numbers() -> Generator[int, None, None]:
    for i in range(10):
        yield i

def squares(nums: Generator[int, None, None]) -> Generator[int, None, None]:
    for n in nums:
        yield n ** 2

def evens(nums: Generator[int, None, None]) -> Generator[int, None, None]:
    for n in nums:
        if n % 2 == 0:
            yield n

# Chain generators
result = evens(squares(numbers()))
print("\nEven squares:", list(result))

Real-life Use Case

Generators are especially useful when working with large data
They allow you to process data lazily without loading everything into memory

# Real-life use case: Reading large files line by line
from typing import Generator

def read_large_file(file_path: str) -> Generator[str, None, None]:
    with open(file_path, "r") as file:
        for line in file:
            yield line.strip()

# Process file lazily without loading it fully into memory
for line in read_large_file("data.txt"):
    print(line)