Amazon·Machine Learning Engineer·Technical Phone Screen·Intermediate

Intermediate

Apr 2026

Summary

Amazon ML Engineer interview, technical phone screen focused on core PyTorch fundamentals. Pretty straightforward if you've done any real model training, but the order of operations inside the loop is where people slip up.

Questions Asked (1)

Write a complete PyTorch training loop function that handles device placement, multiple epochs, forward pass, loss computation, backpropagation, and gradient zeroing in the correct order.

Technical Trade-offsAlgorithms & Data StructuresSystem Design

Author's notes

The actual code isn't that hard but I second-guessed myself on where to zero the gradients.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the problem: define a function that takes a model, data loader, optimizer, loss function, and device, then loops over epochs and batches. For each batch, move data to the device, perform forward pass, compute loss, zero gradients, backpropagate, and update weights. Emphasize correct order of operations and device placement.

Pro tip: Mention that you typically set the model to training mode (model.train()) at the start and consider using a learning rate scheduler or gradient clipping for robustness, but keep the core loop simple and correct.

1. Function Signature and Setup

Define the function with parameters: model, dataloader, optimizer, loss_fn, device, and num_epochs. Move the model to the device and set it to training mode.

2. Epoch Loop

Iterate over the specified number of epochs. Optionally, you can shuffle data or use a sampler, but the dataloader typically handles that.

3. Batch Loop and Device Placement

For each batch, move inputs and targets to the device. This ensures computations happen on the correct hardware.

4. Forward Pass, Loss, and Backward Pass

Perform forward pass, compute loss, zero gradients, backpropagate, and update weights. The correct order is: optimizer.zero_grad(), loss.backward(), optimizer.step().

5. Return or Logging

Optionally, return the trained model or track and return loss metrics. You might also include validation after each epoch.

Key Points to Mention

Device placement: move model and data to the same device (e.g., 'cuda' or 'cpu').
Correct order: zero gradients before backward pass to prevent accumulation.
Use model.train() to set training mode, especially important for dropout and batch norm.
Handle variable batch sizes or last batch appropriately (dataloader does this).
Consider using torch.no_grad() for validation to save memory and speed up computation.
Mention that optimizer.step() updates weights after gradients are computed.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.