Thumbtack·Data Scientist·Technical Phone Screen
Jul 2026
Thumbtack data scientist interview that went deep on model evaluation and statistical fundamentals. The technical questions were genuinely hard and required connecting theory to practical constraints like temporal data and class imbalance.
- Walk through the main cross-validation strategies (k-fold, stratified k-fold, leave-one-out, nested CV, time-series rolling and blocked CV) and explain when you'd use each one.
- You have a dataset with temporal ordering and class imbalance. Design an evaluation framework that avoids data leakage while still giving stable performance estimates. Justify your fold construction and your choice of metrics.
- Explain the bias-variance tradeoff quantitatively. How do model complexity, training set size, and regularization each affect bias and variance? What do CV error curves tell you about underfitting vs overfitting, and what levers do you pull depending on which problem you have?
“I actually felt okay on this until they pushed on nested CV.”