In Python, an iterator is an object which has a __next__ method. Iterators are used to represent sequences like lists, tuples, strings etc. When calling the __next__ method, an iterator gives the next element in the sequence. Iterators will raise a StopIteration exception when there are no more elements to iterate.
Python’s built-in sequences like the list, tuple, string etc. can be converted to iterator objects by calling the built-in function iter(). The built-in next() function can be used to get the next element from an iterator. This is illustrated below.
str_obj = 'abc'
str_iterator = iter(str_obj) # Creates iterator object
next(str_iterator) # Returns 'a'
next(str_iterator) # Returns 'b'
next(str_iterator) # Returns 'c'
next(str_iterator) # Raises StopIteration exception
A for loop uses this behind the scenes. The operation of a for loop can be roughly summarised as follows. When you give a sequence to a for loop, it calls the iter() function and converts the sequence into an iterator object. On each iteration, the for loop uses next() function to get the next element in the sequence. The StopIteration exception is a signal for the for loop to terminate.
Creating Your Own Iterators
It is fairly simple to create your own iterators. Iterators are normal Python classes with some special methods. All you need is a class which contains an __iter__() method and a __next__() method. The Python interpreter will call __iter__() method when you call the iter() function with an iterator object. When you call the next() function with an iterator object, the interpreter will call the __next__ method in your class.
The following example illustrates a simple iterator to generate a sequence of numbers.
class Counter:
def __init__(self, limit):
self.limit = limit
self.count = 0
def __iter__(self):
return self
def __next__(self):
if self.count == self.limit:
raise StopIteration
self.count += 1
return self.count
counter = Counter(3) # Creates counter object
iterator = iter(counter) # Creates iterator object
next(iterator) # Retruns 1
next(iterator) # Retruns 2
next(iterator) # Retruns 3
next(iterator) # Raises StopIteration exception
We defined a class Counter with methods __iter__ and __next__. When initialising a new object, we are creating a variable count to keep track of the iteration. Since our class itself is an iterator class, the __iter__ method can return the current object(self).
In the above example, we called iter() and next() functions manually. In real-world programs, we rarely do this. A more realistic use case will be using a for loop to iterate over our counter objects.
class Counter:
def __init__(self, limit):
self.limit = limit
self.count = 0
def __iter__(self):
return self
def __next__(self):
if self.count == self.limit:
raise StopIteration
self.count += 1
return self.count
counter = Counter(5)
for number in counter:
print(number)
Output:
1
2
3
4
5
A slightly more complex iterator to reverse a string is given below.
class StrReverse:
def __init__(self, string):
self.string = string
self.tracker = len(string)
def __iter__(self):
return self
def __next__(self):
if self.tracker == 0:
raise StopIteration
self.tracker -= 1
return self.string[self.tracker]
reverser = StrReverse('abcd')
for char in reverser:
print(char)
Output:
d
c
b
a
Generators
Generators are a special type of functions used to create iterators. Generators use yield statement instead of return. The counter iterator in the previous section can be implemented using generators as follows:
def counter(number):
for n in range(number):
yield n
counter_obj = counter(5)
for count in counter_obj:
print(count)
Output:
0
1
2
3
4
Generators greatly simplify the creation of iterators by implementing __iter__ and __next__ methods for you. You may have noticed that the output is different from Counter iterator in the previous section. The iterator gave you 1 to 5 while the generator gave you 0 to 4. You may try modifying the generator to give 1 to 5.
We can convert the string reverse iterator to a generator as follows:
def str_reverse(string):
for tracker in range(len(string)-1, -1, -1):
yield string[tracker]
for char in str_reverse('abcd'):
print(char)
Output:
d
c
b
a
Inside the generator function, we are using the range function to count from the last index of string to 0 and uses the tracker to keep track of this count.
Generator Expression
Generator expressions are similar to list comprehensions but return a generator instead of a list. Generator expressions use parentheses just like list comprehensions use square brackets. It can be used in anywhere a list (or any iterable) is expected.
For example, the built-in sum function which accepts a list of numbers can also accept a generator expression. The following code calculates the sum of numbers from 1 to 10.
sum(number for number in range(1, 10)) # Outputs 45
This is equivalent to
numbers = [number for number in range(1, 10)]
sum(numbers) # Outputs 45