← Capital One Interview Insights
This is the kind of question that sounds like a clean engineering task until you actually read the spec.
Start by clarifying the requirements: the functions must fit imputers on training data only and apply them to validation/test data without leakage. Then outline a modular design that handles numeric, categorical, and time-ordered features separately, using appropriate imputation strategies for each. Finally, discuss how to store the fitted imputers and ensure they are used consistently during transformation.
Pro tip: Emphasize that imputation parameters (e.g., mean, median, mode) must be learned solely from the training set and then applied to validation/test sets to prevent data leakage. Also, mention that for time-ordered features, forward-fill or backward-fill should be done within each dataset separately to avoid using future information.
Identify the types of features (numeric, categorical, time-ordered) and the binary label. Confirm that the functions must prevent leakage by fitting imputers only on training data.
For each feature type, compute imputation values from the training data: mean/median for numeric, mode or constant for categorical, and forward/backward fill for time-ordered. Store these values in a dictionary or object.
Apply the stored imputation values to the validation/test data. For time-ordered features, apply fill within the validation set only, not using training data beyond the last value if appropriate.
Consider missing values in validation that were not present in training, and ensure imputation values are applied consistently. Validate that no leakage occurs by checking that imputation statistics are identical for train and validation.
Mention alternative imputation methods (e.g., KNN, iterative imputer) and why simple methods might be preferred for leakage prevention and simplicity. Discuss how to handle time-ordered features without introducing future information.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
The age part is fine, group by country on train rows, store the medians, apply them.
Start by clarifying the goal: impute missing age and income without leakage from the target or validation data. For age, use country-level medians computed only from the training data; for income, build a regression model using features other than the target, trained on complete cases within the training set. Emphasize that all imputation must be done within cross-validation folds to avoid data leakage.
Pro tip: When imputing income with regression, include the imputed age as a feature—but ensure the age imputation itself is done within the same fold to prevent leakage. Also, consider adding a missingness indicator for age and income, as missingness itself can be informative.
Confirm that imputation must not use the target label or validation data, and that all statistics/models should be learned only from the training set. Emphasize the importance of avoiding leakage by performing imputation within cross-validation folds.
Compute the median age per country using only the training data. For missing age values, fill them with the median of the corresponding country. If a country has no training data, fall back to the global median.
Select features (excluding the target) that are predictive of income, such as age (now imputed), education, occupation, etc. Train a regression model (e.g., linear regression, random forest) on complete cases in the training set where income is observed.
Use the trained regression model to predict income for rows with missing income. Apply the same model within each cross-validation fold to avoid leakage.
Assess imputation quality using holdout data or cross-validation, but without using the target. Consider adding missingness indicators and compare different imputation strategies.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by clarifying the data structure and requirements, then outline a window-function-based solution that partitions by user and orders by event date, using conditional logic to propagate values only when the date gap is ≤14 days. Emphasize handling edge cases like multiple events on the same day and the first event per user.
Pro tip: Mention that in production, you'd validate the 14-day gap logic with a quick sanity check on a sample of users to ensure the fill stops correctly, and consider performance implications for large datasets by using efficient window functions or iterative approaches if needed.
Confirm the input table structure (e.g., user_id, event_date, last_purchase_days_ago, session_length) and define what 'time-ordered forward-fill' means: fill missing values with the most recent non-null value within the same user, but only if the gap between consecutive event dates is ≤14 days.
Use window functions to compute the date difference between consecutive events per user. Create a flag or group identifier that increments when the gap exceeds 14 days, effectively segmenting the user's timeline into blocks where forward-fill is allowed.
Within each user and group, use LAST_VALUE or FIRST_VALUE with IGNORE NULLS (or equivalent) to propagate the last non-null value forward. Ensure ordering by event_date and handle ties (same date) appropriately.
Address cases like the first event having null values (no fill), multiple events on the same day (order by a secondary key if needed), and verify that propagation stops correctly after a >14-day gap. Test with sample data.
Mention that window functions are efficient for large datasets, but if the database lacks IGNORE NULLS support, an iterative approach or self-join may be needed. Also note the importance of indexing on (user_id, event_date).
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Easier than the other parts but the tiebreaker detail is easy to skip.
Explain a two-level imputation strategy: first compute each user's most frequent category (mode) for the column, then fill missing values with that user-specific mode. For users with no non-missing values or ties, fall back to the global mode computed from the training set. Emphasize that the global mode must be derived only from training data to avoid leakage.
Pro tip: Mention that you would store the global mode as a fitted parameter and apply it consistently to validation/test sets, and consider adding a binary indicator for imputed values to preserve missingness information.
Group the training data by user_id and calculate the mode of the categorical column for each user, ignoring missing values. This captures individual user preferences.
Calculate the overall mode of the categorical column across the entire training set. This serves as a fallback for users with no observed values or ambiguous modes.
For each missing value, fill it with the user's mode if available; otherwise use the global mode. Handle ties in per-user mode by using the global mode as a tiebreaker.
Use the same per-user modes and global mode computed from the training set to impute missing values in validation and test sets, ensuring consistency and preventing data leakage.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.