Amazon·Machine Learning Engineer·Technical Phone Screen
May 2026
Deep technical screen for an ML Engineer role at Amazon, covering everything from evaluation design to optimizer theory. The interviewer clearly wanted to stress-test whether you actually understand the tradeoffs behind your choices, not just recite definitions. Pretty intense across all four sections.
- Walk me through how you evaluate a supervised ML model end-to-end, including your data split strategy, validation protocol, and how you use the test set.
- Which evaluation metrics do you use for your model and why, considering both ML and business tradeoffs?
- Give me the mathematical definitions for metrics like accuracy, precision, recall, F1, ROC-AUC, PR-AUC, log loss, MSE, MAE, and any calibration metrics you'd use.
- How would you design a more robust evaluation workflow than a single holdout set?
- If you could incorporate human labels into your evaluation process, what would you label, how would you ensure label quality, and how does it improve your evaluation signal?
- If you have no labels at all, what is the simplest way to estimate whether two model outputs are similar?
- Compare Transformer and RNN/LSTM/GRU architectures. For very long sequences, what are the tradeoffs of each in terms of training stability, long-range dependencies, and compute?
- Why can attention mechanisms capture long-range dependencies while vanilla RNNs struggle with them?
- You have two sets of images, Set A and Set B. How would you test whether they come from the same underlying distribution?
- Compare SGD with and without momentum, RMSProp, Adam, and AdamW. When would you prefer AdamW over Adam?
“I laid out the standard train/val/test split and explained why you never touch the test set until the very end.” The rest of the author's notes on Machine Learning Engineer interview at Amazon, Technical Phone Screen round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post