So you want to create a function that will do the following:
- Call an API.
- If the call succeeds, call another function passing the result as argument.
A naive solution
You try the following:
def process_data(data): # save data to db or something here. pass def fetch_data(callback): result = http_client.get('someurl') if result.ok: callback(result) fetch_data(process_data)
fetch_data will run and if the request succeeds, it will call
process_data, passing the result as argument.
Now, a new requirement came up where you need
process_data to know whether it should overwrite existing data or not. So you change the signature like so:
def process_data(data, force=False): if force: # force processing pass else: # Don't force pass
How are you going to pass this to
force defaults to
False which is fine but what about when you want it to be
Will you change the caller’s signature as follows?
def fetch_data(callback, force_processing=False): result = http_client.get('someurl') if result.ok: callback(result, force_processing) fetch_data(process_data, force_processing)
Cool, problem solved.
fetch_data can take any callback function, so you receive a new requirement where fetched data should only be logged instead of processed.
You try this out:
def log_data(data, destination): if destination == 'console': print(data) else: # Log data somewhere else pass
Now you need to modify
fetch_data again to account for the
def fetch_data(callback, force_processing=False, destination='console'): ...
Every time you add callbacks, your
fetch_data caller’s signature will need to be updated as well. Then you have to go through all calling sites to update the signature. This is error-prone and makes refactoring hard.
A not so naive solution
Okay, what if you simply use
**kwargs instead of fixed parameters.
def fetch_data(callback, *callback_args, **callback_kwargs): callback(*callback_args, **callback_kwargs) fetch_data(process_data, True) fetch_data(log_data, destination='syslog')
Ah, this looks much better.
Now, what happens if
fetch_data itself needs an argument now?
def fetch_data(url, callback, *callback_args, **callback_kwargs): result = http_client.get(url) if result.ok: callback(*callback_args, **callback_kwargs) fetch_data('myurl', process_data, True)
This solution works but notice how when you call
fetch_data, you don’t really know which arguments are for the callback function and which ones are for the caller.
As your requirements grow, this will become even more confusing.
Enter partial objects
Fortunalety, the Python standard library provides a concept called partial objects.
Instead of passing the function itself as a callback, you turn it into a partial object first and pass that instead.
Let’s reimplement everything:
def process_data(data, force=False): pass def log_data(data, destination): pass def fetch_data(url, callback): result = http_client.get(url) if result.ok: callback()
Notice how we’re now calling callback directly without any arguments in fetch_data. But how can we use this now? Check this out:
from functools import partial fetch_data('someapi', partial(process_data, force=true)) fetch_data('anotherapi', partial(log_data, destination='console'))
By passing a partial object instead of the function itself,
fetch_data doesn’t need to know anything about the callback’s signature. It can simply call it because the partial object has already been prepared to be called with the proper arguments.
Your code is now much cleaner and easier to refactor.