To illustrate the problem of keeping a Value object safe, consider the following example which simply sets two processes to count up to 2000:

import time
import multiprocessing
import ctypes

def myUpdate(val):
    for i in range(1000):
            time.sleep(0.005) 
            val.value += 1      

if __name__ == '__main__':
    myValue= multiprocessing.Value(ctypes.c_int, 0)
    p1=multiprocessing.Process(target=myUpdate,
                                      args=(myValue,))
    p2=multiprocessing.Process(target=myUpdate,
                                      args=(myValue,))  
    p2.start()
    p1.start()
    p1.join()
    p2.join()
    print(myValue.value)

If you run this program you should find that the total is less than 2000. If it isn’t, increase the number of times the for loop executes or the number of threads. The point is that, even though the get implied in val.value+1 is protected by the lock, the lock is released and reacquired before the result is stored back in val.value. The solution is to apply the lock explicitly on the instruction you need to protect. Change the myUpdate function to read:

def myUpdate(val):
    for i in range(1000): 
            time.sleep(0.005)  
            with val:
                val.value += 1  

As already mentioned, using val in the with instruction acquires the lock which is released when the with ends. This now works and you should always see 2000.

In nearly all cases the automatic locking provided by Value is insufficient and you need to add your own explicit lock. As the lock provided by the Value object is still applied, even when it isn’t needed, the fact that it is slow makes this all the more irritating. What is worse is that the lock is acquired twice and while this is permitted with an Rlock it isn’t with a Lock. For example, if you change the creation of myValue to:

myValue = multiprocessing.Value(ctypes.c_int, 0,
                          lock = multiprocessing.Lock())

and run the modified program then you will discover that it hangs due to a deadlock. The with clause acquires the lock and then the access to val.value tries to acquire it a second time and hence the process waits forever.

It is a much better idea to create a Value object without a lock, set lock = False in the constructor and use an explicit lock. Alternatively you can use a raw Value object, see later.

The Array object works in much the same way as Value, but for an array of the specified type. To create an Array you use:

multiprocessing.Array(type, size_or_initializer,
                                         lock = True)

You can specify the size of the array or provide a sequence to initialize the array. If you create an array of c_char then you can treat the value attribute as a string. For other data types you can treat the object as a sequence. For example:

myArray = multiprocessing.Array(ctypes.c_int, 10)

creates an array with ten integer elements initialized to zero. You can sum the array using:

for i in range(10):
        sum += myArray[i]

or:

for v in myArray:
        sum += v

Notice that in the both cases you might need additional locking to stop the value in the array changing during the operation. 

Shared ctypes

The Value and Array classes are capable of representing almost any ctypes object. They are based on a set of more basic routines in multiprocessing.sharedctypes and in most cases you don’t need to know about them, even though they are documented as end-user classes in the documentation. In particular, you don’t need to use RawValue or RawArray as they are returned if you use Value and Array with lock set to False and you don’t need to use synchronized as this is what is returned with lock set to None or True. For example, you can use Value to represent a ctypes Structure:

class Point(ctypes.Structure):
    _fields_ = [('x', ctypes.c_double),
                        ('y', ctypes.c_double)]

Point has two fields x and y, which are both floating-point numbers. Once you have the Structure class you can use Value to create a shared instance:

mystruct = multiprocessing.Value(Point,1.0,2.0)

This creates a Structure with the x and y fields initialized to 1.0 and 2.0 respectively. The general principle is that the parameters that follow the type parameter are passed to the ctypes constructor without change. We can now access the Structure in the usual way from any process, safe in the knowledge that a lock is used for get and set operations for any of the fields. So expressions like:

mystruct.x = 3.0

can be regarded as atomic but:

mystruct.x += 1.0

isn’t as it involves two atomic get/set operations. To make the increment safe we have to explicitly lock it:

with mystruct:
    mystruct.x+=1.0

In most cases it is more efficient to create a synchronized shared object protected by a separate lock:

mystruct = multiprocessing.Value(Point,1.0,2.0.lock = False)
myLock = multiprocessing.Rlock()
with myLock:
    mystruct.x += 1.0

You can create most ctypes as shared objects in this way, but a notable exception is Pointer. The reason is that it will share a pointer to a specific memory location, but if the shared object is used in different processes it will reference an area of memory used for a different purpose. Processes have independent memory arrangements and do not share addresses.