[Python Series 14] Understanding Decorators and Higher-Order Functions

Understand decorators and higher-order functions

After compressing code with comprehensions, we now focus on higher-order functions: simply put, functions that handle other functions. There are many new terms, so start with the simple idea that "functions can be passed like values." Whenever you wrap "log before calling an API" into a helper, you are already using higher-order ideas. Python’s decorator syntax is built on top of that intuition.

Key terms

Higher-order function: accepts functions as arguments or returns a new function
Closure: an inner function that remembers variables from its surrounding scope
Decorator: the @ syntax that layers shared pre/post logic onto functions
functools.wraps: a helper that preserves original names and docstrings when writing decorators

Core ideas

Study notes

Time: 60 minutes

Prereqs: defining/calling functions, closure basics, writing simple pytest tests

Goal: implement decorators with functools.wraps to build retry and timer patterns

Higher-order functions treat functions as values.
Closures capture outer variables for later use.
Decorators inject shared pre- and post-processing.
functools.wraps keeps metadata intact.
"Core" sections get you production-ready; save "Optional" for later.

Code walkthrough

Higher-order basics (Core)

def apply_twice(func, value):
    return func(func(value))


def increment(x):
    return x + 1


result = apply_twice(increment, 3)  # 5

The func argument receives any callable.
Python’s flexibility comes from passing functions like any other value.
Stay with this pattern until it feels natural before moving on.

Inner functions and closures (Core)

def make_multiplier(factor):
    def multiply(value):
        return value * factor

    return multiply


double = make_multiplier(2)
double(5)  # 10

multiply remembers the factor from its outer scope. This closure pattern is exactly how decorators are implemented.

Build a decorator (Core)



def timer(func):
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        elapsed = time.perf_counter() - start
        print(f"{func.__name__} ran in {elapsed:.3f}s")
        return result

    return wrapper


@timer
def fetch_data():
    time.sleep(0.2)
    return {"status": "ok"}

@timer is equivalent to fetch_data = timer(fetch_data).
wrapper calls the original function and surrounds it with common behavior.
*args/**kwargs forward positional and keyword arguments unchanged.

The diagram shows that a decorator simply wraps a function so shared code can run before and after each call.

Preserve metadata with functools.wraps (Core)



def timer(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        ...

    return wrapper

functools.wraps copies the original function’s name and docstring to the wrapper. It matters for debugging, IDE hints, and documentation.

Parameterized decorators (Core → Plus)

When you want to pass options into a decorator, add one more layer. It looks intimidating, but the only rule is "return a function."

def retry(max_attempts=3):
    def decorator(func):
        def wrapper(*args, **kwargs):
            for attempt in range(1, max_attempts + 1):
                try:
                    return func(*args, **kwargs)
                except Exception as exc:
                    print(f"Attempt {attempt} failed: {exc}")
            raise RuntimeError("Retry limit exceeded")

        return wrapper

    return decorator


@retry(max_attempts=5)
def unstable_call():
    ...

Decorators that need arguments are "decorator factories"—functions that create decorators. Keep names and returns explicit to avoid getting lost; sketch the call stack on paper if needed.

Real-world example: retry Slack notifications (Core → Plus)



def retry_webhook(max_attempts=3):
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            for attempt in range(1, max_attempts + 1):
                try:
                    return func(*args, **kwargs)
                except httpx.HTTPError as exc:
                    print(f"#{attempt} Slack send failed", exc)
                    time.sleep(0.5)
            raise RuntimeError("Webhook retries exhausted")

        return wrapper

    return decorator


@retry_webhook(max_attempts=5)
def send_slack(payload: dict):
    httpx.post(WEBHOOK_URL, json=payload, timeout=3)

When to use it: repeated API error handling, shared caching, auth pre-checks—anything that needs consistent prep.
What you gain: reuse the retry logic from the CLI, web apps, or anywhere else.

Higher-order tools in the standard library (Optional)

map(func, iterable): apply a function to every element.
filter(func, iterable): keep only elements that satisfy the predicate.
functools.partial(func, **fixed_args): create a new function with some arguments preset.
functools.lru_cache: a decorator that memoizes return values.

Use higher-order helpers sparingly. They shine when you want to remove duplication or control side effects. If you do not need them today, skim and move on.

Why it matters

Higher-order functions treat behavior as data.
Decorators provide syntactic sugar for shared pre/post hooks.
functools offers batteries for performance and clarity.

Practice

Follow along: implement a timer decorator and print duration from a decorated function.
Extend: adapt the retry decorator for HTTP requests and log failures instead of printing them.
Debug: remove functools.wraps, observe how the function name/docstring change, then reinstate it to fix IDE autocompletion.
Definition of done: you have built both parameterless and parameterized decorators and applied them to real code.

Wrap-up

Next we will implement iterators and generators so values are produced lazily only when needed.