Page 1 of 3
Iterators are one of the great simplifications of design that make Python so productive. Find out how it all works in this extract from my new book Programmer's Python: Everything is Data.
Everything is Data
Is now available as a print book: Amazon
- Python – A Lightning Tour
- The Basic Data Type – Numbers
- Truthy & Falsey
- Dates & Times
- Sequences, Lists & Tuples
Extract Unicode Strings
- Regular Expressions
- The Dictionary
Extract The Dictionary ***NEW!!!
- Iterables, Sets & Generators
- Data Structures & Collections
- Bits & Bit Manipulation
Extract Bytes And Strings
Extract Byte Manipulation
- Binary Files
- Text Files
- Creating Custom Data Classes
- Python and Native Code
Extract Native Code
Appendix I Python in Visual Studio Code
Appendix II C Programming Using Visual Studio Code
Although iterables are a very basic data type and very simple, many Python data types are iterables. Sequences and mappings which include lists, tuples and dictionaries are all iterables. An iterable is a mixin that you can add to almost any data class without having to worry about an inheritance hierarchy. They are so ubiquitous that it makes it difficult to focus on the idea in isolation. It is also difficult to find a “pure” example of an iterable, but the set comes closest. Finally iterables provide a way of generating sets, sequences and dictionaries via the powerful idea of a comprehension, which is the subject of the next chapter.
Iterables, Enumerables and Sequences
An iteration is simply the act of taking each item stored in a container in turn. It is typified by a for loop. A container is a class that, as its name suggests, contains other classes. Whenever you write something like:
for x in container:
you are performing an iteration. Iteration is simple but we tend to give it more properties than are absolutely necessary.
Python has three distinct types – iterable, enumerable and sequence.
Iterable – a container that you can access one element at a time in an arbitrary order. An iterable only has to have the ability to give you the next element.
Enumerable – an iterable that also has an integer associated with each element. That is, you can access each element in the container by asking for the first, second, third and so on. Notice that this doesn’t mean that you can go to any particular element without accessing all of the elements that come before it. You can think of an enumerable as a container that only allows sequential access.
Sequence – an enumerable that also allows you to access any element in any order using indexing. A sequence is a container that allows random access to any element.
A sequence is necessarily an enumerable, which in turn is necessarily an iterable, but the reverse isn’t true. An iterable doesn’t have to be an enumerable and neither does it have to be a sequence. For example, a list is a sequence and you can write:
to display element number 42.
You can iterate through a sequence as if it was enumerable using the index:
for i in range(0,42):
and this is the way it is done in other languages. Notice that you are iterating through a range and in turn using this to index the list.
Enumeration implies an order, first, second and so on, but iteration doesn’t – it just means next. Of course, you can iterate through containers that do have an order such as a list:
for x in myList:
but there are containers that you can iterate through that don’t support indexing or enumeration.
Iteration is more basic than enumeration or sequencing.
Any Python object can be made into an iterable. All it has to do is support the __iter__ method, which returns an iterator object. An iterator object has to have a __next__ method, which returns elements one after another until there are no more when it throws a StopIteration exception. An iterator doesn’t promise to deliver the items in any particular order and indeed might return them in a different order each time. All an iterator promises is to provide the next element until it runs out of elements. The reason that the iterator has to be an object rather than just a function that gives you the next element is explained later.
An iterator object should also have an __iter__ method, which returns a reference to the iterator object. This allows the iterator object to be used as if it was an iterable in its own right.
For example, we can create a random iterable which simply returns a random number for each of its elements:
In this case the __iter__ method returns an iterator object which provides the random numbers each time it is used.
To make this work we need to define the randItIterator class:
The iterator class simply returns a random number as the result of __next__. Putting this together we can now write a for loop that uses an instance of randIt:
for x in numbers:
This will print random numbers until you stop it by pressing the Break key.
The numbers iterable behaves as if it was a container with an infinite series of random numbers stored within it. You can explicitly call __next__ to get the next item in the iteration or you can use the next built-in function to do the same job:
displays two random numbers or, more generally, the first two elements in the iteration.
You can also specify a default as a second parameter to next, which will be used if the iteration is complete and there isn’t a next element. Notice that next simply calls __next__ and substitutes the default if specified and there is no pending item.
If you define a simple __iter__ method within the iterator it too can be used as an iterable, in a for loop for example. That is, if we add:
to randItItertor it can be used in a for loop in place of the iterable:
for x in numberIterator:
In this case we explicitly return the iterator and then use it in the for loop in place of the iterable. This is really the only use for the __iter_ method within the iterator, but it is still advisable to include it.
The full program is:
for x in numbers: