In this post I will show you how to pass data to places where you cannot pass it as arguments.
As an example, let’s say we have a function that we cannot change which calls
our code (MyAction.action). Because we cannot change
function_i_cannot_change, we cannot make it pass dynamic_value through to
our code. Also note that we cannot pass something else than a number as the
arg parameter because of its implementation. (Yes, in this simple example
there would be alternative solutions, see note at the end.)
from typing import Callable
def function_i_cannot_change(callback : Callable[[int], int], arg : int) -> int:
arg += 1
# lots of code
result = callback(arg)
# more code
return result
def action(arg):
# hmm, how do I get `dynamic_value` into this method?
return arg + dynamic_value
def usage():
dynamic_value = 100
function_i_cannot_change(action, 1)We can solve this by passing the data through a global variable.
DYNAMIC_GLOBAL = 1
def action(arg):
return arg + DYNAMIC_GLOBAL
def usage():
global DYNAMIC_GLOBAL
DYNAMIC_GLOBAL = 100
function_i_cannot_change(action, 1) # returns 102Yes, you’re right… Using globals is bad. It makes your code hard to follow and breaks thread-safety.
To address the threading issue, we can use a threading.local instead:
import threading
DYNAMIC_TL = threading.local()
DYNAMIC_TL.value = 1
def action(arg):
return arg + DYNAMIC_TL.value
def usage():
DYNAMIC_TL.value = 100
function_i_cannot_change(action, 1) # returns 102This solves threading issues, but we still have to know the thread local in the
consuming code and we have to trust all consumers to be well-behaved concerning
what they set on DYNAMIC_TL.value and that they reset it to the original
value.
Let’s use a context manager to make the usage more safe and clear for the consumers.
import threading
import contextlib
DYNAMIC_TL = threading.local()
DYNAMIC_TL.value = 1
@contextlib.contextmanager
def override_dynamic_value(value):
# we can do assertions on the value here
old_value = DYNAMIC_TL.value
DYNAMIC_TL.value = value
yield
DYNAMIC_TL.value = old_value
def get_dynamic_value():
return DYNAMIC_TL.value
def action(arg):
return arg + get_dynamic_value()
def usage():
with override_dynamic_value(100):
function_i_cannot_change(action, 1) # returns 102I also added get_dynamic_value to hide the usage of the thread local from the
implementation of action, too.
This sounds a bit crazy, does anybody really do this?
Yes. Have you ever wondered how
flask.request
works?
Can I really use this?
Well, I would say: Yes, if there is no simpler alternative. The pattern comes with the general problem of global variables:
a) They are available everywhere and thus people will starting using them in more and more places. This leads to unintended coupling.
b) The dataflow is not obvious. You will get surprising and hard to debug errors.
For me, this means that I only use the pattern in a very explicit way that makes sure it will be used responsibly. I would move the thread local and it’s accessors to their own module and make the thread local itself private.
I often find myself using this technique in tests, when I need to get 3rd party code to cooperate. In production code I would try very hard to find an alternative.
Note: Alternative solutions for the simple example
The simple example has a couple pretty simple, preferable solutions,
E.g. there is partial application of functions:
import functools
def action(dynamic_value, arg):
return arg + dynamic_value
def usage():
function_i_cannot_change(functools.partial(action, 100), 1) # returns 102Or a function that returns a function:
def action_factory(dynamic_value):
def action(arg):
return arg + dynamic_value
def usage():
function_i_cannot_change(action_factory(100), 1) # returns 102Or a class:
class MyAction:
def __init__(self, dynamic_value):
self.dynamic_value = dynamic_value
def action(self, arg):
return arg + self.dynamic_value
def usage():
function_i_cannot_change(MyAction(100), 1) # returns 102