Implement blocked time-series cross-validation
Company: Freddie Mac
Role: Data Scientist
Category: Coding & Algorithms
Difficulty: medium
Interview Round: Onsite
Quick Answer: This question evaluates competence in designing panel-aware time-series cross-validation with embargo handling, grouped blocking, deterministic splitting, scalability analysis, and adversarial unit-test design for issues like duplicate timestamps and entity migration.
Constraints
- 0 <= N <= 10^5 rows; each row is [loan_id (int), msa (str), month (int)].
- 1 <= k <= number of distinct months (k <= 0 or empty rows -> return []).
- embargo is a non-negative integer number of month units; consecutive integer months are one unit apart.
- month values may repeat (duplicate timestamps) and need not be contiguous (missing months).
- The same loan_id may appear under different msa values (loan migration) — rows are treated independently by month.
- Output must be invariant to the ordering of the input rows.
Examples
Input: ([[1, "A", 1], [1, "A", 2], [2, "A", 3], [2, "B", 4], [3, "B", 5], [3, "B", 6], [4, "C", 7], [4, "C", 8]], 2, 1)
Expected Output: [{"train": [[1, "A", 1], [1, "A", 2]], "test": [[2, "A", 3], [2, "B", 4], [3, "B", 5], [3, "B", 6]]}, {"train": [[1, "A", 1], [1, "A", 2], [2, "A", 3], [2, "B", 4], [3, "B", 5]], "test": [[3, "B", 6], [4, "C", 7], [4, "C", 8]]}]
Explanation: 8 distinct months. k=2 -> cutoffs at months[round(1*7/3)=2]=3 and months[round(2*7/3)=5]=6. Fold 0: train = month<=3-1=2; test = (2, 6]. Fold 1: train = month<=6-1=5; test = (5, 8]. The embargo of 1 removes month 2 from fold-1 test even though it would otherwise border the train edge.
Input: ([[4, "C", 8], [2, "A", 3], [1, "A", 1], [3, "B", 6], [1, "A", 2], [4, "C", 7], [3, "B", 5], [2, "B", 4]], 2, 1)
Expected Output: [{"train": [[1, "A", 1], [1, "A", 2]], "test": [[2, "A", 3], [2, "B", 4], [3, "B", 5], [3, "B", 6]]}, {"train": [[1, "A", 1], [1, "A", 2], [2, "A", 3], [2, "B", 4], [3, "B", 5]], "test": [[3, "B", 6], [4, "C", 7], [4, "C", 8]]}]
Explanation: The exact same rows as the previous case but shuffled. The output is identical, demonstrating determinism and stability under shuffled input (requirement 3).
Hints
- Sort the rows once by (month, loan_id, msa) up front; every fold then reads from this canonical order, which is what makes the output independent of input shuffling.
- Compute the K cutoff months from the DISTINCT sorted months using idx = round(j*(m-1)/(k+1)) for j = 1..k, then clamp to [0, m-1].
- For each fold, the train window ends at cutoff - embargo (inclusive) and the test window is the half-open band (cutoff - embargo, next_cut]; the last fold's next_cut is the maximum month.
- Always emit exactly k folds: a fold whose train or test band is empty still contributes an entry with an empty list.