Since the release of Python 2.2 in 2001, all Python functions have closed over bindings in outer scopes. Before this release variables referenced from outer scopes had to be passed in as arguments. A common workaround to this limiting behavior was to set these at function definition time with keyword arguments:
Since 2001 this workaround is no longer necessary and the references
name in the body of the lambda function are resolved using the
current value in the outer
However Python functions could not modify these bindings: although an object referred to by a variable from an outer scope can be modified, like appending an item to a list,
a function could not change to which object a variable from an outer scope referred.
One of my favorite changes in Python 3 is the addition of the nonlocal keyword. Described by PEP 31041 as “access to names in outer scopes” and in other places as “rebinding closures” and “read/write closures,” this allows functions to rebind variables in outer scopes. By marking a variable name nonlocal we specify that when that variable name is assigned to in this function, we mean the already existing binding from outside this function. The nonlocal keyword works analogously to the global keyword which allows reassignment of global variables in Python 2 and 3.
Folks who write Python code, what do you think of the new nonlocal keyword in Python 3?— Thomas Ballinger (@ballingt) January 5, 2016
Here are some thoughts on why this situation doesn’t come up very much in Python and how people worked around not having it in Python 2.
Global scope isn’t
Python 2 functions can rebind some variables:
the module-level variables known as globals.
global, available in Python 2, marks a variable name
in a function as referring to the global variable in that module.
This provides some fraction of the power of full closures
since every Python function closes over the “global” variables
of the module in which it was defined.
I also find I write less local functions in Python because of
top-level functions have no enclosing scope besides the global
scope to use
Community norm: no data hiding
that same pattern in Python 3 would be considered pretty weird code:
Mutable object hack
Writing to a closed over variable is useful for producing a callback which records somewhere that it was called. Here’s a signal handing example in Python 3:
The most direct translation of the above is to use
the simplest possible mutable object to store
Read-only closures with mutable objects seem
to provide an (ugly) equivalent to full closures:
This works in every case I can think of, but is ugly because the list only obfuscates what’s going on. When I see this pattern in Python 2 code I wish it were Python 3 so it could be fixed, but it seems pretty rare.
Mutability and method binding:
It’s much more common to find a function that could rebind an outer variable, but instead mutates an object more descriptive than a one-element list.
When passing in a callback to a function a Python method is particularly convenient because that method will be bound to that object at attribute lookup time. This more clearly communicates what state might be modified because methods of objects often change the state of the objects to which the belong.
Less callback-oriented interfaces
Python codebases seem to prefer passing objects that conform to informal interfaces to passing functions directly, using attributes of that object for the state being changed.
- PEP 3104: Access to Names in Outer Scopes
- PEP 227: Statically Nested Scopes
- Stack Overflow posts about workarounds for Python 2: 1 2 3 4 5
Thanks Julia Evans and Lindsey Kuper for comments and corrections.