Page 3 of 3
Computing Pi using AsyncResult
Using a process pool makes our example computation of Pi simpler in that we don’t have to create individual processes, but also because we don’t need to use locks to control access and shared data structures to return the values. All of the communication and timing between the processes is taken care of by the AsyncResult objects:
pi = 0
for k in range(m,n+1):
s = 1 if k%2 else -1
pi += s / (2 * k - 1)
if __name__ == '__main__':
with multiprocessing.pool.Pool(2) as p:
This program runs a little faster than the version that explicitly creates its own process and provides its own locks, but the difference is small. The main advantage of using a Pool is that the process are set up by the time you want to use them.
In chapter but not in this extract
- Immediate Results – imap
- Sharing and Locking
A process pool, a set of processes created and ready to run, can be used to avoid the high cost of repeatedly creating processes.
The Pool object creates a specified number of processes ready to be used.
The apply_async method will run a function using a process from the pool.
All of the Pool methods return an AsyncResult object which acts like what would usually be called a deferred, a promise or, in Python’s terminology, a future. This can be used to wait for a result.
There are a range of map-like functions which allow you to run a set of functions on a set of processes from the process pool.
map_async maps a single function onto a range of values, each run on a process from the pool.
starmap_async works like map_async, but automates the passing of multiple parameters.
imap works like map_async, but returns an iterator that allows you to step through results as they become available.
The map functions are one half of a standard approach to parallel programming called MapReduce. The functools module provides a reduce function.
You can’t pass objects like Lock and Value to a process from the process pool because the process is already running and parameters are passed as pickled data. Lock, Value and any object that has state cannot be pickled.
To share important resources like locks you have to use the initialization parameter when the Pool is first created to supply a function which creates the shared resources in each of the processes.
Threads, processes, asyncio & more
Is now available as a print book: Amazon
1) A Lightning Tour of Python.
2) Asynchronous Explained
3) Processed-Based Parallelism
Extract 1 Process Based Parallism
Extract 1 -- Threads
5) Locks and Deadlock
7) Sharing Data
Extract 1 - Pipes & Queues
8) The Process Pool
Extract 1 -The Process Pool 1 ***NEW!
9) Process Managers
Extract 1 Futures,
12) Basic Asyncio
Extract 1 Basic Asyncio
13) Using asyncio
Extract 1 Asyncio Web Client
14) The Low-Level API
Extract 1 - Streams & Web Clients
Appendix I Python in Visual Studio Code