Freddie Mac·Data Scientist·Technical Phone Screen·Senior

SeniorPrefer not to say

Apr 2026Remote

Summary

Interviewed for a Data Scientist role at Freddie Mac and got hit with a deeply technical ML engineering question about time-series cross-validation for mortgage data. One question, very long, very specific. The kind of thing where you either know the domain or you're fumbling through it live.

Questions Asked (1)

Design a panel-aware blocked time-series cross-validation splitter with an embargo period. The input is a DataFrame with loan_id, MSA, and month columns. Requirements include: K=5 expanding-window folds with embargo logic preventing any loan_id from appearing in both train and test within 90 days; grouped blocking so at least one fold holds out an entire MSA as test; determinism under shuffled input; pseudocode plus time/memory complexity analysis; and unit tests covering duplicate timestamps, missing months, loan migration across MSAs, and highly imbalanced MSAs.

Algorithms & Data StructuresSystem DesignTechnical Trade-offs

Author's notes

This question is basically a take-home disguised as a verbal question.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the requirements and edge cases, then outline a deterministic algorithm that sorts by time, assigns folds with expanding windows, enforces embargo via a 90-day buffer, and ensures at least one fold holds out an entire MSA. Provide pseudocode, complexity analysis, and a unit test plan covering the specified scenarios.

Pro tip: Emphasize determinism by sorting on stable keys (e.g., month, MSA, loan_id) and using a fixed random seed if any randomness is needed; also discuss how to handle loan migration by treating each loan_id's timeline independently and applying the embargo per loan_id.

1. Clarify requirements and edge cases

Restate the problem: K=5 expanding-window folds, 90-day embargo per loan_id, at least one fold with entire MSA held out, determinism under shuffled input, and unit tests for duplicate timestamps, missing months, loan migration, and imbalanced MSAs.

2. Design the algorithm

Sort data by month, MSA, and loan_id for determinism. Assign each loan_id to a fold based on its first appearance month, ensuring expanding windows. For each fold, define train as all data before the test period minus a 90-day embargo, and test as the fold's period. For the MSA holdout fold, select an MSA and assign all its loans to test, with train excluding that MSA and applying embargo.

3. Write pseudocode

Provide clear pseudocode that iterates over folds, computes train/test indices, applies embargo by filtering out any loan_id that appears in test within 90 days of train, and handles the MSA holdout by overriding fold assignment for one MSA.

4. Analyze complexity

Time complexity: O(N log N) due to sorting, plus O(N) for fold assignment and filtering. Memory: O(N) for storing indices and intermediate structures. Discuss potential optimizations for large datasets.

5. Outline unit tests

Describe tests for: duplicate timestamps (ensure deterministic ordering), missing months (handle gaps in time series), loan migration across MSAs (ensure embargo applies per loan_id regardless of MSA), and highly imbalanced MSAs (ensure MSA holdout still works and folds are balanced).

Key Points to Mention

Determinism: sort by stable keys and avoid randomness; if randomness is needed, use a fixed seed.
Embargo logic: for each loan_id, exclude any training samples that fall within 90 days of a test sample for that loan_id.
Expanding window: each fold's training set includes all data prior to the test period, growing with each fold.
MSA holdout: at least one fold must have an entire MSA as test, requiring careful selection to avoid leakage.
Loan migration: a loan_id may appear in multiple MSAs; embargo must be applied per loan_id across all its records.
Complexity: O(N log N) time and O(N) memory, with potential for optimization using groupby operations.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.