In Python, iterator is an object which has a __next__ method. Iterators are used to represent sequences like lists, tuples, strings etc. When calling the __next__ method, an iterator gives the next element in the sequence. Iterators will raise a StopIteration exception when there is no more elements to iterate.

Python’s built in sequences like list, tuple, string etc. can be converted to iterator objects by calling the built in function iter(). The built in function next() can be used to get the next element from an iterator. This is illustrated below.

str_obj = 'abc'
str_iterator = iter(str_obj)  # Creates iterator object
next(str_iterator)  # Returns 'a'
next(str_iterator)  # Returns 'b'
next(str_iterator)  # Returns 'c'
next(str_iterator)  # Raises StopIteration exception

A for loop uses this behind the scenes. The operation of a for loop can be roughly summarised as follows. When you give a sequence to a for loop, it calls the iter() function and converts the sequence into an iterator object. On each iteration, the for loop uses next() function to get the next element in the sequence. The StopIteration exception is a signal for the for loop to terminate.

Creating Your Own Iterators

It is fairly simple to create your own iterators since iterators are normal Python objects with some special methods. All you need a class which contains an iter() method and a next() method. The Python interpreter calls iter() method when you call the iter() with an iterator object and calls the next() method when you call the next() method with an iterator object.

The following example illustrates a simple iterator to generate a sequence of numbers.

class Counter:
    def __init__(self, limit):
        self.limit = limit
        self.count = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self.count == self.limit:
            raise StopIteration
        self.count += 1
        return self.count

counter = Counter(3)  # Creates counter object
iterator = iter(counter)  # Creates iterator object
next(iterator)  # Retruns 1
next(iterator)  # Retruns 2
next(iterator)  # Retruns 3
next(iterator)  # Raises StopIteration exception

We defined a class Counter with methods iter() and next(). When initialising a new object, we are creating a variable count to keep track of the iteration. Since our class itself is an iterator class, the iter() method can return the current object (self).

In the above example, we called iter() and next() method manually. In real world programs, we rarely do this. A more real use case will be using a for loop to iterate over our counter objects.

# Uses the same Counter class from previous example

counter = Counter(5)
for number in counter:
    print(number)
Output:

1
2
3
4
5

A slightly more complex iterator to reverse a string is given below.

class StrReverse:
    def __init__(self, string):
        self.string = string
        self.tracker = len(string)

    def __iter__(self):
        return self

    def __next__(self):
        if self.tracker == 0:
            raise StopIteration
        self.tracker -= 1
        return self.string[self.tracker]

reverser = StrReverse('abcd')
for char in reverser:
    print(char)
Output:

d
c
b
a

Generators

Generators are special type of functions used to create iterators. Generators use yield statement instead of return. The counter iterator in the previous section can be implemented using generators as follows:

def counter(number):
    for n in range(number):
        yield n

counter_obj = counter(5)
for count in counter_obj:
    print(count)
Output:

0
1
2
3
4

Generators greatly simply the creation of iterators by implementing iter() and next() methods for you. You may have noticed a difference in the output of Counter iterator in the previous section and the above generator. The first one gave you 1 to 5 while the second one gave 0 to 4. Try modifying the generator to give 1 to 5.

We can convert the string reverse iterator to a generator as follows:

def str_reverse(string):
    for tracker in range(len(string)-1, -1, -1):
        yield string[tracker]

for char in str_reverse('abcd'):
    print(char)
Output:

d
c
b
a

Inside generator function, we are using the range to count from last index of string to 0 and uses tracker to keep track of this count.

Generator Expression

Generator expressions are similar to list comprehensions but returns a generator instead of a list. Generator expressions use parentheses just like list comprehensions use square brackets. It can be used in anywhere a list (or any iterable) is expected.

For example, the built in sum function which accepts a list of numbers can also accept a generator expression. The following code calculates the sum of numbers from 1 to 10.

sum(number for number in range(1, 10))  # Outputs 45

This is equivalent to

numbers = [number for number in range(1, 10)]
sum(numbers)  # Outputs 45

Posted on
Category: Python
Tags: Python, Iterators, Generators, Generator Expressions