[Python Series 16] Resource Management with Context Managers and the with Statement

If you already controlled data flow with iterators, it is time to handle resources that must be released—files, sockets, database locks—in a safe way. Python's context managers pair with the with statement so that the setup and teardown code for a resource runs automatically. There is nothing mysterious here: you are just bundling “open → use → close” into one block.

Key terms

1. Context manager: a structure that groups the code run when a with block starts and ends
2. with statement: syntax that opens and closes a resource automatically through a context manager
3. __enter__/__exit__: the methods a context manager class must implement to prepare and clean up a resource
4. ExitStack: a contextlib tool that stacks as many contexts as you need and exits them together

Core ideas

Study notes

• Time required: 50–60 minutes
• Prerequisites: file I/O, exception handling, functions/classes
• Goal: implement both a class-based context manager and a @contextmanager version that automate cleanup

• A context manager collects the code that runs right before and after a with block.
• __enter__ opens or acquires the resource; __exit__ takes care of cleanup and exception handling.
• contextlib.contextmanager lets you build a manager from a generator by wrapping the code around yield.
• ExitStack dynamically stacks multiple contexts at runtime.
• Mastering the core sections already solves most resource-management needs; the optional parts expand your toolbox.

direction: right

open: "__enter__()
acquire resource"
work: "with block
do the work"
cleanup: "__exit__()
cleanup/rollback"

open -> work: "return handle"
work -> cleanup: "success/error"
cleanup -> open: "prepare next resource"

Seeing the lifecycle makes it easier to remember that __enter__ and __exit__ always run as a pair.

Code examples

Basic usage (Core)

with open("scores.csv", "r", encoding="utf-8") as f:
    data = f.read()

• The file opens when the with block begins and automatically closes when it ends.
• Even if an exception occurs, close() still runs so you avoid leaks.

The context manager protocol (Core)

Context managers implement two methods:

• __enter__(self): runs at the start of the with block. The return value is bound to the name after as.
• __exit__(self, exc_type, exc_val, exc_tb): runs at the end of the block. It receives exception info; returning True suppresses the exception.

class DatabaseSession:
    def __enter__(self):
        self.conn = connect()
        return self.conn

    def __exit__(self, exc_type, exc_val, exc_tb):
        if exc_type:
            self.conn.rollback()
        else:
            self.conn.commit()
        self.conn.close()


with DatabaseSession() as conn:
    conn.execute("INSERT ...")

Build one with contextlib (Core)

If writing a class feels heavy, start here. Just remember what to place before and after yield.

contextlib lets you skip the class entirely:

from contextlib import contextmanager


@contextmanager
def temporary_directory(path):
    os.makedirs(path, exist_ok=True)
    try:
        yield path
    finally:
        shutil.rmtree(path)


with temporary_directory("./tmp-build") as build_dir:
    ...

• @contextmanager builds a generator-based context manager for you.
• Code before yield plays the __enter__ role; code after yield acts like __exit__.

Managing multiple contexts (Core → Plus)

from contextlib import ExitStack


with ExitStack() as stack:
    files = [stack.enter_context(open(path)) for path in file_list]
    ...

ExitStack lets you add contexts dynamically at runtime. It shines when you must open many files inside a loop. If you currently only juggle one or two resources, just remember the name for later.

Exception-handling strategy (Optional)

• Suppressing exceptions inside __exit__ can hide bugs, so return True only when you truly intend to swallow them.
• Perform logging or cleanup in a finally block.
• contextlib.suppress(*exceptions) can intentionally ignore specific errors, but only use it when you understand the cause.

Real-world uses (Optional)

• File handlers: open, gzip.open
• Database sessions: SQLAlchemy Session
• Concurrency control: threading.Lock, asyncio.Lock
• Temporary environment tweaks: stack contexts with contextlib.ExitStack so settings revert automatically

Why it matters

• Context managers give structure to setup/teardown code.
• You can author them with __enter__/__exit__ or via @contextmanager.
• When you need to juggle several resources at once, stack them dynamically with ExitStack.

Practice

• Follow along: Implement the DatabaseSession class so the with block logs commits and rollbacks.
• Extend: Use @contextmanager to create a temporary directory and verify it disappears after the block ends.
• Debug: Force __exit__ to suppress exceptions with return True, observe how logs vanish, then let exceptions propagate again.
• Definition of done: You have resource-management code that works in both class-based and generator-based forms and passes your test script.

Wrap-up

Up next: use type hints and typing.Protocol to make interfaces explicit and unlock better tooling.