Programmer's Python Data  Comprehensions 
Written by Mike James  
Monday, 30 May 2022  
Page 2 of 2
Using ComprehensionsComprehensions are simple to write and mostly easy to understand, however, they can be used in many “creative” ways. The key is generally the expression you use which you need to think of as transforming the iterable. For example to create a zeroed list you could use: myList = [0 for i in range(1,11)] sets the list elements to 0. In this case the transformation of the iteration is to map every element to zero. If you want to set the list to random numbers you could use: myList = [random.random() for i in range(1,11)] In the case where the expression doesn’t depend on the variable all the iterable does is determine the number of elements in the result. In most cases the expression will depend on the iterable and this will result in a true transformation of the elements. For example: myList = [i*2 for i in range(1,11)] multiplies each element by 2 to convert them into even values 2, 4,6 and so on. In addition to the transformation caused by the expression you can also use a conditional to filter the elements, for example: myList=[i for i in range(1,11) if i%2 == 0] In this case i%2 is zero only if i is even and so the final list is: [2, 4, 6, 8, 10] That is, we have picked out the even elements from the iteration. So far we have only used range as an example of an iterable, but you can use any iterable and this means you can transform text strings. For example: s = set(("mike","lucy","harry")) myList = [str.upper(w) for w in s] In this case the iteration is provided by a set and each element is converted to upper case, displaying: ['MIKE','LUCY', 'HARRY'] Also notice that we started with a set and transformed it into a list. Dictionary comprehensions are just as easy, but now we have the possibility of constructing two expressions, one for the key and one for the value. For example: myList = ["mike","lucy","harry"] myDict = {str.upper(w):0 for w in myList} creates a dictionary set to: {'MIKE': 0, 'LUCY': 0, 'HARRY': 0} The usual problem in using a dictionary comprehension is deriving two expressions for the key and value. You can make use of data stored in other data structures for either part of the new element. For example: myList = ["mike","lucy","harry"] ages = [18,16,23] myDict = {w[1]:ages[w[0]] for w in enumerate(myList)} In this case the key is derived from each item in myList, but the value is derived from the ages list. Notice the way that enumerate is used to derive an index that can be used to select the appropriate element of ages. The result is that myDict contains: {'mike': 18, 'lucy': 16, 'harry': 23} Finally a generator comprehension can involve infinite iterations, for example: def genSquared(): x = 0 while True: x = x+1 yield x**2 genOddSquare = (s for s in genSquared() if s%2 != 0) for i in genOddSquare: print(i) In this case the generator function genSquared produces the squares of the integers and it runs for as long as its __next__ method is called. The generator comprehension transforms genSquared into an iteration of just the squares that are odd. The work in this case is done by the conditional which selects only the odd elements generated by genSquare. The final for loop that uses the generator object runs until you hit the Break key. Notice that in the case of a generator comprehension it is possible to write a condition that will never be satisfied and hence the call to the generator object will never return. Finally it is worth noting that comprehensions can occur in unexpected places. In fact, anywhere a for loop occurs there is the possibility of using a comprehension. For example, if you want to print a row of numbers formatted with two digits you might use: for i in range(20): which displays: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
This can be written as a single print using a generator comprehension: print(*(f"{i:2}" for i in range(20))) Notice that we need the unpacking operator to both evaluate the comprehension and to provide the results as parameters to the print function. As there is only a single print function, we also don’t need the end = " ". Another difference is that as the numbers are passed to the print function as separate parameters they are displayed with a single space between them. Is this better? As long as you are very happy about reading comprehensions, it is more compact, but it is arguably more difficult to understand. Nested ComprehensionsThe expression in a comprehension can be any valid expression and this includes another comprehension. This gives rise to the idea of a nested comprehension and these can be difficult to understand. The key idea is that the expression in a comprehension is fully evaluated to an object and this object is added to the data structure being created. What this means is that when the expression is itself a comprehension it is fully evaluated and it returns an object of the appropriate type, for example: m = [[j for j in range(3)] for i in range(5)] This is a nested list comprehensions, as you can tell from the fact that the expression for the outer comprehension is itself a comprehension. The way to understand this is to first look at the inner comprehension: [j for j in range(3)] This creates a list [0,1,2] and it does this every time the outer comprehension evaluates its expression which it does for i = 0,1,2,3,4. You can see that the result is the list: [[0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2]] You can see that this is a list of five repeats of [0, 1, 2], which is what you would expect if you understand the basic idea of nesting. In the more general case, the inner expression can involve the result of the outer comprehension, for example: m=[[j*i for j in range(3)] for i in range(5)] In this case the inner comprehension uses the current value of i when it is evaluated. The result is the list: [[0, 0, 0], [0, 1, 2], [0, 2, 4], [0, 3, 6], [0, 4, 8]]
In book but not in this extract
Summary
Programmer's Python


Last Updated ( Monday, 09 January 2023 ) 