Page 1 of 2
Closure - what's that all about. As it happens it's all surprisingly logical once you know why Python implements closure. Closure motivated and explained in this extract from my book, Programmer's Python: Everything is an Object.
Everything is an Object
Is now available as a print book: Amazon
- Hello Python World
- Variables, Objects and Attributes
- The Function Object
Extract - Function Objects
- Scope, Lifetime and Closure
Extract - Local and Global
Extract - Closure ***NEW
- Advanced Functions
Extract - Parameters
Extract - Decorators
- Class Methods and Constructors
Extract - Objects Become Classes
- Inside Class
- Advanced Attributes
Extract - Properties
- Custom Attribute Access
Extract - Custom Attributes
Extract - Default Methods
- Single Inheritance
- Multiple Inheritance
- Class and Type
Extract - Class & Type
- Type Annotation
Extract - Type Annotation
- More Magic - Operator Overloading
Not included in this extract:
- Global v Local
- Inner Functions
- Local, Global and Nonlocal
Closure is often treated as if it was a mysterious theoretical device when in fact it is a simple consequence of defining functions as objects.
To be more precise, it is a consequence of the fact that a function object can exist even when the function isn’t being executed.
Function objects have a life of their own even when not being used as functions.
When you define a function it has access to the local variables of any containing function. This is fine and causes no problem, but notice that access to the local variables of the containing function occurs when the inner function is executed. This normally isn’t an issue.
Where things become more complicated is if an inner function is executed after its containing function has finished. This is perfectly possible as an inner function can be assigned to a global variable or returned as an object reference by its containing function. In this case the inner function object continues to exist after the outer function has completed – it is an object and it still has a reference to it. Now imagine what happens if you call the function. The local variables of the containing function no longer exist and so the function crashes.
This isn’t reasonable – when you wrote the inner function it was perfectly correct but when you ran the function it crashed. A function shouldn’t depend on when it is run for its correctness.
The solution to the problem is to create a closure. The terminology comes from functional programming and it isn’t entirely meaningful in a wider context.
A closure captures the variables that are in scope at the time the function is defined and makes them available for use when the function is executed.
Another way to say this is that a function’s execution context is the set of variables that are in scope when it is defined and a closure makes its execution context available to it at any time in the future.
An example will make the idea clear:
Notice that the only new feature here is the way a reference to the inner function is returned. Keep in mind that myInnerFunction is just a reference to the function object that is created when we use def myInnerFunction.
The function can be used as you might expect:
MyClosure now contains a reference to the function object created by MyFunction. You can call this using the invocation operator in the usual way:
You will see 1 displayed. If you didn’t know about closure this would be amazing. The function manages to use a variable, myVar, that is local to MyFunction which finished executing and had all of its local variables destroyed before MyClosure was used to call it.
Any local variables of the containing function that are used by the inner function are stored as part of the closure.
The details of the closure are stored in the __closure__ attribute of the function object – see the section on closure and cells for details of how this works.
Now we come to some finer points of the Python closure.
The first thing that should be obvious is that if the inner function assigns to the variable then it is converted into a local variable and the variable isn’t included in the closure.
The intent is to add one to myVar to keep a count of how many times the function is called. However, this doesn’t work and the function fails reporting an error of trying to use a variable before it has been assigned to. The point is the assignment creates myVar as a local variable to myInnerFunction and stops the closure.
The solution is to declare myVar nonlocal:
Now everything works and myVar is available to myInnerFunction as part of a closure. The nonlocal declaration stops the Python system creating a local variable.
If you want to assign to a variable you want to include in the closure then make sure it is declared nonlocal.