Pinterest·Machine Learning Engineer·Technical Phone Screen·Senior

SeniorPrefer not to say

Jun 2026

Summary

Pinterest ML engineer interview with a meaty bagging/ensemble coding question that also required me to talk through theory. Not a leetcode grind session, more of a 'show me you actually understand the algorithm' kind of thing.

Questions Asked (1)

Implement bootstrap aggregation (bagging) from scratch in Python without NumPy, given an existing DecisionTree with fit and predict methods. Then discuss bias-variance trade-offs, the effect of the number of bootstrap rounds B, and out-of-bag estimation.

Algorithms & Data StructuresTechnical Trade-offs

Author's notes

The coding part was fine once I remembered to use random.choices for sampling with replacement.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by outlining the bagging algorithm: for each of B rounds, create a bootstrap sample by sampling with replacement, fit a DecisionTree on that sample, and aggregate predictions via majority vote (classification) or averaging (regression). Then discuss the bias-variance trade-off, emphasizing that bagging reduces variance without increasing bias, and explain how B and out-of-bag estimation affect performance.

Pro tip: Mention that out-of-bag samples provide a nearly unbiased estimate of generalization error without a separate validation set, and that increasing B reduces variance but yields diminishing returns, so B is often chosen based on computational budget.

1. Clarify the algorithm and data structures

Explain that bagging trains multiple decision trees on bootstrap samples and aggregates their predictions. Use a list to store models and a list of lists for out-of-bag indices.

2. Implement bootstrap sampling

For each round, generate a bootstrap sample by randomly selecting n indices with replacement from the training set. Track which indices are not selected as out-of-bag for that round.

3. Train and aggregate

Fit a DecisionTree on each bootstrap sample. For prediction, collect outputs from all trees and combine via majority vote (classification) or mean (regression).

4. Discuss bias-variance trade-off

Explain that bagging reduces variance by averaging decorrelated trees, without substantially increasing bias. The bias remains similar to a single tree, but variance decreases as B increases.

5. Explain effect of B and OOB estimation

Describe how increasing B reduces variance but with diminishing returns, and how out-of-bag samples provide a validation-like error estimate without extra data.

Key Points to Mention

Bootstrap sampling with replacement and its role in creating diverse training sets.
Aggregation methods: majority vote for classification, averaging for regression.
Bias-variance trade-off: bagging primarily reduces variance, bias remains roughly constant.
Effect of B: more rounds reduce variance but with diminishing returns; computational cost increases linearly.
Out-of-bag estimation: using samples not in the bootstrap sample to estimate generalization error.
Comparison to random forests: bagging uses all features, while random forests add feature subsampling.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.