Disney·Machine Learning Engineer·Take-home Assignment·Intermediate

Intermediate

Jul 2026

Summary

Disney ML Engineer take-home that was basically a spam classifier build-out in a Jupyter notebook. Pretty self-contained but there were a few gotchas around class imbalance and leakage that I think they were specifically watching for.

Questions Asked (1)

Build a spam detection classifier using Naive Bayes on a provided text dataset. Handle preprocessing, feature extraction, model training, and evaluation with F1 score, then run predictions on a few test examples.

Technical Trade-offsAlgorithms & Data StructuresData Modeling

Author's notes

The F1 part tripped me up a bit.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by outlining the end-to-end pipeline: preprocessing, feature extraction, training, evaluation, and prediction. Emphasize the choice of Multinomial Naive Bayes with TF-IDF or CountVectorizer, and justify using F1 score due to class imbalance. Walk through each step concisely, mentioning trade-offs and potential pitfalls.

Pro tip: Mention that you would use a held-out test set and cross-validation to ensure robust evaluation, and discuss how you'd handle class imbalance by tuning the decision threshold or using class weights. This shows practical maturity beyond just implementing the algorithm.

1. Data Preprocessing

Clean the text by lowercasing, removing punctuation, stopwords, and applying stemming/lemmatization. Handle missing values and split data into training and test sets.

2. Feature Extraction

Convert text to numerical features using TF-IDF or CountVectorizer. Consider n-grams and vocabulary size trade-offs.

3. Model Training

Train a Multinomial Naive Bayes classifier on the training data. Optionally tune hyperparameters like smoothing (alpha) via cross-validation.

4. Evaluation

Evaluate using F1 score, precision, recall, and confusion matrix. Discuss why F1 is appropriate for imbalanced spam datasets.

5. Prediction

Run predictions on a few test examples, showing the predicted class and probability. Explain how to interpret results.

Key Points to Mention

Choice of Multinomial Naive Bayes for text classification and its assumptions (conditional independence, multinomial distribution).
Text preprocessing techniques: tokenization, stopword removal, stemming/lemmatization, and handling of special characters.
Feature extraction with TF-IDF vs. CountVectorizer, including the impact of n-grams and vocabulary size.
Evaluation metrics: F1 score, precision, recall, and why accuracy can be misleading for imbalanced data.
Handling class imbalance: techniques like resampling, class weights, or threshold tuning.
Potential pitfalls: overfitting, data leakage during preprocessing, and the need for cross-validation.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.