Python Generators, yield, Lazy Evaluation and How Decorators Work

Explain what generators and decorators are in Python and why you would use them. These came as a quick oral sequence in the fundamentals half of a technical screen for a junior quantitative developer role, and each answer drew a follow-up on the mechanism underneath. ```python squares_list = [x * x for x in range(10_000_000)] squares_gen = (x * x for x in range(10_000_000)) @timed def load_prices(path): ... ``` ### Clarifying Questions - When comparing a generator with an "ordinary iterator", is that a hand-written class with `__iter__` and `__next__`, or a sequence such as a list? - For decorators, should the answer include decorators that take arguments, or only the basic form? - Should the memory discussion include approximate sizes, or only the reason? ### Part 1 — Generators, `yield` and lazy evaluation 1. What is a generator? 2. What advantages does a generator have over an ordinary iterator or a sequence? 3. What does `yield` do? 4. Why is a generator more memory efficient? 5. What is lazy evaluation? ```hint Where the function pauses Call `next()` on a generator twice and track what happens to the function's local variables in between. Then count how many values exist in memory at once, compared with the list. ``` #### What This Part Should Cover - A generator as a function whose execution is suspended and resumed, and the exact role of `yield` - How it satisfies the iterator protocol with far less code than a hand-written iterator class - Memory use and lazy evaluation, with a concrete use such as streaming a large file - The limitations: a single pass, no length or indexing, deferred errors ### Part 2 — Decorators 1. Have you used decorators? Give an example. 2. What is a decorator really doing? ```hint Remove the @ sign Rewrite `@timed` placed above `def load_prices` as an ordinary assignment statement, and ask what object the name `load_prices` refers to afterwards. ``` #### What This Part Should Cover - What the `@` syntax is equivalent to, and when the decorator runs compared with the wrapped function - A correctly written wrapper that passes arguments and the return value through - Preserving the wrapped function's metadata, and decorators that take arguments - Real examples from the standard library or your own code ### What a Strong Answer Covers - Precise mechanics: suspended execution for generators, functions that take and return functions for decorators - Memory and laziness claims backed by a reason, not just "it is faster" - The costs of generators (single pass, deferred errors) as well as the benefits - A working decorator written from memory, including `functools.wraps` - Examples relevant to data work, such as streaming records or timing and retrying calls ### Follow-up Questions - You loop over the same generator twice. What does the second loop produce, and how do you handle a computation that needs two passes? - Write a decorator that retries a function up to a given number of times when it raises a given exception. Where do those settings go? - A function opens a file with `with` and returns a generator expression over its lines. What goes wrong when the caller iterates? - How would you chain generators to read a very large CSV file, filter its rows and keep a running average without loading the whole file?