Python Internals: How dict Works, When Not to Use It, and NumPy Arrays vs Lists

After a resume discussion, a 30-minute first-round screen for a systematic trading team moved to quick Python fundamentals questions. Answer each part as you would in the interview: explain the mechanism precisely, then say what it means in practice for code that processes large amounts of market data. ### Clarifying Questions - Should the answers describe CPython specifically, or Python as a language independent of its implementation? - Which Python version should be assumed, given that the internal layout of `dict` changed in recent versions? ### Part 1 — What is a Python dict under the hood, and when is it a poor fit? Explain how a Python `dict` is implemented and what that implies for the cost of lookups, insertions and deletions. Then describe the scenarios a `dict` is not well suited for, and what you would use instead. ```hint Start from the hash Follow a single lookup from the key's hash to the stored value, and ask what happens when two keys land in the same place or the table fills up. ``` #### What This Part Should Cover - The hash-table structure, collision handling, resizing, and the resulting average and worst-case costs - Requirements on keys, and how ordering behaves - Concrete poor-fit scenarios, each with a better alternative ### Part 2 — NumPy array versus Python list Explain the differences between a NumPy array and a Python list. ```hint Look at the memory layout Picture how one million floating-point numbers are laid out in memory in each case, and ask what a loop over them must do for each element. ``` #### What This Part Should Cover - Memory layout, element types, and memory footprint - Vectorized computation versus per-element interpreter work, and the semantics of operators and slicing - When a list is still the better choice ### What a Strong Answer Covers - Mechanisms explained correctly rather than as slogans such as "dicts are O(1)" or "NumPy is faster" - Costs stated with their conditions: average versus worst case, amortized versus per operation - Practical consequences for large numeric data, with a concrete example - Awareness of behaviors that surprise people, such as views versus copies and fixed-width integer overflow ### Follow-up Questions - Why must dictionary keys be hashable, and what goes wrong if a key object's hash changes after insertion? - How would you look up the most recent price at or before a given timestamp, and why is a plain `dict` the wrong tool? - When does NumPy slicing return a view and when a copy, and how can a view cause a bug? - Why can `np.append` inside a loop be dramatically slower than appending to a list and converting once at the end?