Flattening your decorators

I figured out this really funny (cursed?) trick you can do with Python decorators, since people seem to trip over nesting functions constantly. So on this page I figured I'd explain what exactly is going on.

Let's first look at an example decorator you might use in a real project. This piece of code lets you easily mark functions for deprecation:

def deprecated(instead):
    def decorator(func):
        def wrapper(*args, **kwargs):
            warnings.warn(
                f'Call to deprecated function {func.__qualname__}. Use {instead} instead.',
                category=DeprecationWarning,
                stacklevel=2
            )
            return func(*args, **kwargs)
        return wrapper
    return decorator

For those familiar with decorators, this code shouldn't be too hard to digest. But even for such a seemingly simple task as adding a line of logging, we need to create a highly nested function. The varying depth makes the code noisy, not to mention how it pushes the actual logic further to the right on the screen.

And now for something completely different

Luckily Python is an incredibly flexible programming language, and with some out of the box thinking, we can do something very sneaky:

@partial(partial, partial, partial)
def deprecated(instead, func, *args, **kwargs):
    warnings.warn(
        f'Call to deprecated function {func.__qualname__}. Use {instead} instead.',
        category=DeprecationWarning,
        stacklevel=2
    )
    return func(*args, **kwargs)

Using functools.partial, we have gotten rid of all those nasty nested functions and lifted the core logic out of the inner-most wrapper function directly into the decorator body.

But this is the kind code that makes you squint your eyes and scratch your head. It's very difficult to try and keep track of how all of the functools.partials interact together here.

But as with most things, when you write it out the long way and look at it step by step, the function becomes simpler to understand. The key to what is going on is remembering that partial(a, b)(c) is equivalent to a(b, c). I've attached a simple animation to help illustrate this.

The decorator on the decorator

First, let's shortly look at how the perplexing @partial(partial, partial, partial) decorator affects the deprecated decorator.

@partial(partial, partial, partial)
def deprecated(...):
    ...

deprecated = partial(partial, partial, partial)(deprecated)
deprecated = partial(partial, partial, deprecated)

The deprecated function still looks quite daunting after applying the intimidating partial decorator. Next, let's take a look at what happens when we use this decorator to decorate a function.

The decorated function

@deprecated('new_function')
def old_function(...):
    ...

old_function = deprecated('new_function')(old_function)

# we replace `partial` with the transformation we applied above
old_function = partial(partial, partial, deprecated)('new_function')(old_function)
old_function = partial(partial, deprecated, 'new_function')(old_function)

# since there's a second function call, let's apply the same process again
old_function = partial(partial, deprecated, 'new_function')(old_function)
old_function = partial(deprecated, 'new_function', old_function)

This is what old_function looks like now. It is perhaps even more noisy than what we started with. But lastly, let's look at what happens in user code, when this function is called.

User code using the decorated function

key = old_function('data', hash=sha512)

# we replace `old_function` with the transformation we applied above
key = partial(deprecated, 'new_function', old_function)('data', hash=sha512)
key = deprecated('new_function', old_function, 'data', hash=sha512)

Hopefully when you contrast this line with the function we originally declared, it should now be clearer how all the parameters are passed in a flattened form.