Pinterest·Machine Learning Engineer·Technical Phone Screen·Senior

Senior

May 2026

Summary

Pinterest ML Engineer interview that centered on a fairly involved evaluation metrics problem, the kind where you're handed multiple confusion matrix tables and asked to compute accuracy, precision, recall, F1, specificity, and sometimes AUC, then pick the best model under a constraint. Not a vibe check round at all.

Questions Asked (1)

You are given multiple tables showing model predictions on labeled datasets (true positives, false positives, true negatives, false negatives). For each table, compute evaluation metrics like accuracy, precision, recall, F1, and specificity, write out the formulas, show your work, and then identify which model or group satisfies a given requirement such as highest precision while keeping recall above a certain threshold.

Product Analytics & MetricsA/B Testing & ExperimentationTechnical Trade-offs

Author's notes

This is the kind of question that looks straightforward until you're actually doing it live and second-guessing whether precision is TP/(TP+FP) or the other way around.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clearly defining each metric and its formula, then systematically compute the confusion matrix components for each model. Use a structured table to compare metrics across models, and finally apply the given constraint (e.g., recall threshold) to filter and select the best model based on the primary objective (e.g., highest precision).

Pro tip: Always double-check your calculations and consider the business context—sometimes a slight trade-off in precision for higher recall is acceptable if it aligns with user experience goals. Also, mention that in practice, you'd use libraries like scikit-learn to avoid manual errors, but here you're demonstrating understanding.

1. Define Metrics and Formulas

Write down the formulas for accuracy, precision, recall, F1, and specificity in terms of TP, FP, TN, FN. This ensures clarity and shows you know the fundamentals.

2. Compute Confusion Matrix Components

For each model, extract TP, FP, TN, FN from the given tables. If not directly provided, calculate them from the raw counts.

3. Calculate Metrics for Each Model

Plug the values into the formulas and compute each metric. Present results in a clear table for easy comparison.

4. Apply Constraints and Select Best Model

Identify models that meet the recall threshold, then among those, select the one with the highest precision. If multiple, consider F1 or other tie-breakers.

5. Interpret and Sanity Check

Briefly interpret the results in the context of the problem (e.g., Pinterest's use case) and verify that the chosen model makes sense.

Key Points to Mention

Formulas: Accuracy = (TP+TN)/(TP+FP+TN+FN), Precision = TP/(TP+FP), Recall = TP/(TP+FN), F1 = 2*(Precision*Recall)/(Precision+Recall), Specificity = TN/(TN+FP)
Importance of understanding the business context: e.g., in content moderation, high recall might be prioritized to catch all violations, while precision matters for user experience.
Trade-offs between precision and recall, and how F1 balances them.
The impact of class imbalance on accuracy and why precision/recall are more informative.
Use of a structured approach to avoid calculation errors, such as creating a comparison table.
Mention that in practice, you'd use tools like scikit-learn's classification_report, but manual calculation demonstrates deeper understanding.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.