OneMain Financial·Data Scientist·Technical Phone Screen·Senior

SeniorPrefer not to say

Jun 2026

Summary

OneMain Financial data scientist interview that went deep on preprocessing pipelines for churn modeling. One question, but it was a beast covering imputation, outlier treatment, calibration testing, and MNAR bias all at once. Left feeling like I'd answered 60% of it well and glossed over the rest.

Questions Asked (1)

You're building a customer churn model with messy features: right-skewed spend data with extreme outliers, zero-inflated count features, and categorical plan types. Missingness is a mix of MAR and MNAR (high-spend users sometimes skip income). Walk through your full preprocessing pipeline for both linear models and tree ensembles, explain the tradeoffs of different imputation and outlier strategies, describe how you'd test the pipeline's effect on calibration and SHAP without introducing bias, and address what you'd do if about 10% of records are MNAR on a key feature.

Data ModelingTechnical Trade-offsRoot Cause Analysis

Author's notes

This was basically four questions stapled together and they wanted a coherent answer, not four separate answers.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Structure your answer around a decision framework that starts with diagnosing missingness mechanisms and data distributions, then branches into model-specific preprocessing for linear models versus tree ensembles. Emphasize that imputation and outlier handling must be validated for their downstream effects on calibration and SHAP, and propose concrete strategies for the 10% MNAR scenario such as missingness indicators, pattern submodels, or sensitivity analysis.

Pro tip: Always tie preprocessing choices back to the business objective—churn prediction for a financial institution demands well-calibrated probabilities, so prioritize calibration-preserving techniques and avoid imputation methods that distort the target relationship.

1. Diagnose Data Issues and Missingness Mechanisms

Quantify skewness, outliers, zero-inflation, and missingness patterns. Test whether missingness is MAR or MNAR by comparing distributions of observed variables across missing vs. non-missing groups, and consult domain experts to understand why high-spend users skip income.

2. Design Model-Specific Preprocessing Pipelines

For linear models, use robust scaling, log or Box-Cox transformations, and imputation with missingness indicators. For tree ensembles, rely on their native handling of skew and outliers, but still address missingness via surrogate splits or explicit imputation with indicators.

3. Evaluate Imputation and Outlier Strategies with Tradeoffs

Compare mean/median, MICE, and model-based imputation for bias and variance. For outliers, consider winsorizing, trimming, or robust transformations, noting that tree models are less sensitive but can still overfit to extreme values.

4. Validate Pipeline Effects on Calibration and SHAP

Use cross-validation to assess calibration (e.g., reliability diagrams, Brier score) and compute SHAP values on held-out data. Ensure imputation is done within CV folds to avoid leakage, and check for SHAP instability across imputation methods.

5. Handle MNAR with Sensitivity Analysis and Robust Methods

For the 10% MNAR feature, add a missingness indicator, consider pattern submodels or Heckman-style correction, and perform sensitivity analysis (e.g., tipping point) to quantify the impact of MNAR on model performance and fairness.

Key Points to Mention

Missingness indicators for MNAR features to allow the model to learn the missingness mechanism.
Log or Box-Cox transformations for right-skewed spend data, and robust scaling for linear models.
Tree ensembles' inherent robustness to outliers and skew, but need for explicit handling of missing values.
Within-cross-validation imputation to prevent data leakage and ensure unbiased performance estimates.
Calibration metrics (reliability diagrams, Brier score) and SHAP value stability across preprocessing choices.
Sensitivity analysis and pattern submodels as strategies for MNAR, especially when missingness is related to the target.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.