Amazon·Machine Learning Engineer·Technical Phone Screen·Senior

Senior

Apr 2026

Summary

Amazon ML engineer technical screen, basically one big coding problem: build a neural net from scratch using only NumPy. No PyTorch, no autograd, nothing. It was a lot to cover in one session and I definitely underestimated how much they'd push on the math.

Questions Asked (1)

Implement a feed-forward neural network from scratch using only NumPy, including forward pass, backpropagation, cross-entropy loss, and mini-batch SGD. Train it on a small dataset like XOR and be ready to walk through tensor shapes at every step.

Algorithms & Data StructuresTechnical Trade-offsSystem Design

Author's notes

This is the whole interview, basically.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the architecture (e.g., 2-2-1 with sigmoid activations) and then walk through the forward pass with explicit tensor shapes. Next, derive the backpropagation using the chain rule, implement cross-entropy loss, and finally describe mini-batch SGD training on XOR, highlighting shape consistency at each step.

Pro tip: Emphasize that you always verify tensor shapes by printing them during implementation, and mention that for XOR you need a hidden layer to capture non-linearity. This shows practical debugging skills and deep understanding.

1. Define Architecture and Initialize Weights

Choose a simple feed-forward network (e.g., 2 input, 2 hidden, 1 output) with sigmoid activations. Initialize weights randomly with small values and biases to zero, noting shapes: W1 (2x2), b1 (1x2), W2 (2x1), b2 (1x1).

2. Forward Pass with Shape Tracking

For a batch of inputs X (batch_size x 2), compute Z1 = X·W1 + b1 (batch_size x 2), A1 = sigmoid(Z1), Z2 = A1·W2 + b2 (batch_size x 1), A2 = sigmoid(Z2). Track shapes at each operation.

3. Compute Cross-Entropy Loss

For binary classification, use binary cross-entropy: L = -1/N * sum(y*log(A2) + (1-y)*log(1-A2)). Note that for multi-class, softmax + categorical cross-entropy would be used, but XOR is binary.

4. Backpropagation via Chain Rule

Compute gradients: dZ2 = A2 - y (batch_size x 1), dW2 = A1.T·dZ2 (2x1), db2 = sum(dZ2, axis=0) (1x1). Then dA1 = dZ2·W2.T (batch_size x 2), dZ1 = dA1 * sigmoid_derivative(Z1) (batch_size x 2), dW1 = X.T·dZ1 (2x2), db1 = sum(dZ1, axis=0) (1x2).

5. Mini-Batch SGD Training Loop

Shuffle data, split into mini-batches (e.g., size 2 for XOR). For each epoch, iterate over batches: forward pass, compute loss, backprop, update weights: W -= learning_rate * dW. Repeat until convergence, monitoring loss.

Key Points to Mention

Importance of non-linear activation (sigmoid) in hidden layer to solve XOR
Tensor shape consistency: e.g., (batch_size, features) for inputs, (features, units) for weights
Derivation of backpropagation using chain rule and vectorized operations
Choice of binary cross-entropy loss for binary classification
Mini-batch SGD: batch size, learning rate, and number of epochs as hyperparameters
Potential issues: vanishing gradients with sigmoid, need for proper weight initialization

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.