Snowflake·Data Scientist·Onsite - System Design / Architecture
- Design an end-to-end training and evaluation pipeline for a 7-day purchase classifier that avoids label leakage given a 10-day label delay. Specify your time-based cross-validation scheme including fold boundaries and feature/label windows.
- Which offline metrics would you use for this imbalanced classifier, how would you calibrate the model's probabilities, and what formula would you use to pick a decision threshold that maximizes expected profit given the stated costs?
- How would you detect and respond to distribution shift in both input features and model calibration over time? Describe the monitoring dashboard and any guardrails you'd put in place.
- Given a 50ms p95 latency budget and 64MB RAM per request, what model architecture and featurization strategy would you choose for deployment, and what fallback rule would you use if the model is unavailable?
- How would you explain this model and its threshold choice to a non-technical stakeholder, and what would you do if they pushed back and insisted on using a different threshold?
“This is where I spent the most time and also made the most mistakes early in my answer.”