← Amazon Interview Insights

Amazon·Machine Learning Engineer·Technical Phone Screen·Senior

Senior
May 2026

Summary

Deep learning fundamentals interview at Amazon for an ML Engineer role. Every topic had a follow-up that went two or three levels deeper than I expected, so if you're prepping, don't stop at the conceptual layer.

Questions Asked (5)

Q1

Explain FlashAttention: what problem it solves compared to standard attention, how the IO-aware tiling works, and what the memory and speed tradeoffs look like in practice.

Technical Trade-offsSystem Design
Author's notes

I started with the quadratic memory issue in vanilla attention and thought that was enough.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the core problem: standard attention is memory-bound due to materializing the N×N attention matrix, causing high HBM traffic. Then explain how FlashAttention uses IO-aware tiling and recomputation to keep intermediate results in SRAM, reducing HBM reads/writes. Finally, discuss the practical tradeoffs: memory savings enable longer sequences, speedups come from reduced memory bandwidth, but there is extra compute from recomputation and implementation complexity.

Pro tip: Emphasize that FlashAttention is not approximate—it computes exact attention—and that the speedup is primarily from reducing memory bandwidth, not FLOPs. This shows you understand the real bottleneck in modern hardware.

1. Problem with standard attention

Explain that standard attention computes S = QK^T and P = softmax(S), materializing N×N matrices in HBM. This leads to O(N^2) memory and excessive HBM traffic, making it memory-bound and slow for long sequences.

2. IO-aware tiling and recomputation

Describe how FlashAttention tiles Q, K, V into blocks that fit in SRAM. It computes attention block-by-block, keeping intermediate results on-chip and recomputing the attention matrix during the backward pass instead of storing it.

3. Memory and speed tradeoffs

Highlight that memory usage drops from O(N^2) to O(N), enabling much longer sequences. Speed improves because HBM accesses are reduced, but extra compute from recomputation may increase FLOPs. The net effect is faster and more memory-efficient for typical sequence lengths.

4. Practical implications

Mention that FlashAttention is exact, not approximate, and is widely used in production (e.g., in transformers). It requires custom CUDA kernels and may have overhead for very short sequences, but overall it's a major win for long-context models.

Key Points to Mention

  • Standard attention is memory-bound due to O(N^2) intermediate matrices in HBM.
  • FlashAttention uses tiling to keep blocks of Q, K, V in SRAM, reducing HBM reads/writes.
  • It recomputes attention during backward pass instead of storing the N×N matrix, saving memory.
  • Memory complexity reduces from O(N^2) to O(N), enabling longer sequences.
  • Speedup comes from reduced memory bandwidth, not reduced FLOPs; it may even increase FLOPs.
  • FlashAttention is exact (not approximate) and is implemented as a fused CUDA kernel.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

Walk through LoRA: the low-rank decomposition idea, where adapters plug into a transformer, how rank and alpha interact, and how you'd actually serve or merge adapters at inference time.

Technical Trade-offsSystem Design
Author's notes

This one went better.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by explaining the core low-rank decomposition idea and why it reduces trainable parameters, then describe where adapters are inserted in a transformer (typically in attention projections) and how rank and alpha control capacity and scaling. Finally, discuss inference-time options: merging adapters into base weights for zero-latency serving or keeping them separate for multi-adapter serving, and mention trade-offs like memory and latency.

Pro tip: Emphasize that LoRA's rank and alpha are not just hyperparameters but directly affect the effective learning rate and capacity; also highlight that merging is only possible when there's no additional nonlinearity between adapter and base weights, which is a common pitfall.

1. Explain low-rank decomposition

Describe how LoRA approximates weight updates as the product of two low-rank matrices, reducing trainable parameters from d×k to r×(d+k).

2. Describe adapter placement

Detail where adapters are inserted in a transformer, typically in the query and value projections of self-attention, and why these locations are effective.

3. Discuss rank and alpha

Explain that rank controls the expressiveness of the update, while alpha scales the update relative to the base weights, often with a fixed ratio like alpha/r.

4. Cover inference serving options

Compare merging adapters into base weights for latency-sensitive single-task serving versus keeping them separate for dynamic multi-adapter serving, noting memory and compute trade-offs.

5. Summarize trade-offs

Conclude with key trade-offs: parameter efficiency vs. expressiveness, mergeability vs. flexibility, and the impact on serving infrastructure.

Key Points to Mention

  • Low-rank decomposition reduces trainable parameters and memory footprint.
  • Adapters are typically added to attention query and value projections.
  • Rank (r) controls the bottleneck dimension; alpha scales the update.
  • Common practice: set alpha to r or a multiple, and tune both.
  • Merging adapters into base weights eliminates inference overhead but prevents dynamic switching.
  • Serving multiple adapters requires separate forward passes or batched inference with adapter-specific weights.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q3

Describe the RNN forward and backward pass, explain vanishing and exploding gradients, and compare LSTM and GRU gating. When would you still reach for an RNN over a transformer?

Technical Trade-offsAlgorithms & Data Structures
Author's notes

The gating mechanisms I had cold.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Structure your answer by first explaining the RNN forward and backward pass mathematically, then define vanishing/exploding gradients with their causes and solutions, and finally compare LSTM and GRU gating mechanisms. Conclude by discussing scenarios where RNNs are still preferred over transformers, emphasizing trade-offs like sequential data, low latency, and resource constraints.

Pro tip: Relate the concepts to real-world applications and Amazon's scale, e.g., mention how vanishing gradients impact long sequences in customer behavior modeling, and how GRUs offer efficiency for on-device inference. This shows practical insight beyond textbook knowledge.

1. Explain RNN Forward Pass

Describe the step-by-step computation: hidden state update using input and previous hidden state, and output generation. Use equations like h_t = tanh(W_hh h_{t-1} + W_xh x_t + b_h) and y_t = softmax(W_hy h_t + b_y).

2. Explain RNN Backward Pass (BPTT)

Detail backpropagation through time: unrolling the network, computing gradients for each time step, and accumulating them. Mention the chain rule and the role of the Jacobian of the hidden state.

3. Discuss Vanishing and Exploding Gradients

Explain how repeated multiplication of gradients (especially with tanh/sigmoid) causes gradients to shrink (vanishing) or grow (exploding). Mention solutions: gradient clipping, careful initialization, and gated architectures.

4. Compare LSTM and GRU Gating

Contrast LSTM's three gates (input, forget, output) and cell state with GRU's two gates (update, reset) and lack of separate cell state. Highlight trade-offs: LSTM more expressive, GRU simpler and faster.

5. When to Use RNNs Over Transformers

Discuss scenarios: streaming/real-time data, limited memory/compute, small datasets, and tasks where sequential inductive bias helps. Mention transformers' quadratic complexity and need for large data.

Key Points to Mention

  • RNN forward pass equations and hidden state recurrence
  • Backpropagation through time (BPTT) and gradient accumulation
  • Vanishing/exploding gradients: causes (repeated Jacobians, activation functions) and solutions (gates, clipping, initialization)
  • LSTM vs. GRU: gate count, cell state, computational complexity, and performance trade-offs
  • RNN advantages: sequential processing, constant memory per step, low latency for streaming
  • Transformer limitations: quadratic attention complexity, large data requirements, and less effective for short sequences

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q4

Explain the RLHF pipeline: how the reward model gets trained, how PPO is applied to fine-tune the policy, what the KL constraint against the SFT policy is doing, and what tends to go wrong.

Technical Trade-offsSystem Design
Author's notes

Spent a lot of prep time on this and it showed, mostly.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Walk through the RLHF pipeline in three stages: supervised fine-tuning (SFT), reward model training, and PPO fine-tuning. For each stage, explain the objective, the data, and the key algorithmic details. Then discuss the KL constraint's role in preventing reward hacking and the common failure modes like reward model overoptimization and mode collapse.

Pro tip: Emphasize that the KL penalty is a regularizer that trades off reward maximization for staying close to the SFT policy, and that tuning its coefficient is critical—too low leads to gibberish, too high prevents learning. Mention that in practice, people monitor KL divergence and reward scores together to detect issues early.

1. Supervised Fine-Tuning (SFT)

Start by describing how a pretrained language model is fine-tuned on human demonstrations to produce the initial policy. This step provides a strong initialization for subsequent RLHF stages.

2. Reward Model Training

Explain that human preferences between pairs of model outputs are collected, and a reward model is trained to predict these preferences using a Bradley-Terry model. The reward model serves as a proxy for human judgment.

3. PPO Fine-Tuning with KL Constraint

Detail how PPO optimizes the policy to maximize the reward model's score while a KL divergence penalty keeps the policy close to the SFT model. This prevents the policy from drifting into regions where the reward model is inaccurate.

4. Failure Modes and Mitigations

Discuss common issues: reward hacking (overoptimizing the reward model), mode collapse (reduced diversity), and training instability. Mention mitigations like KL coefficient tuning, reward model ensembles, and early stopping.

Key Points to Mention

  • SFT provides a strong prior and prevents the policy from starting from scratch.
  • Reward model is trained on pairwise human preferences using a Bradley-Terry model.
  • PPO is used to optimize the policy against the reward model, with a KL penalty to the SFT policy.
  • The KL constraint prevents reward hacking and maintains output diversity.
  • Common failure modes: reward model overoptimization, mode collapse, and training instability.
  • Hyperparameters like KL coefficient and PPO clip range require careful tuning.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q5

Compare GRPO and GSPO directly: what each method averages over, why GSPO uses sequence-level importance ratios instead of token-level ones, and what that means for training stability.

Technical Trade-offsAlgorithms & Data Structures
Author's notes

Completely humbled by this.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining what each method averages over: GRPO averages token-level importance ratios, while GSPO averages sequence-level ratios. Then explain that GSPO uses sequence-level ratios to reduce variance and improve stability, especially for long sequences, by treating the entire generated sequence as a single action. Finally, discuss the trade-offs in bias and variance, and how this impacts training stability in practice.

Pro tip: Emphasize that GSPO's sequence-level ratio is not just a variance reduction trick but also aligns better with the actual objective of generating coherent sequences, which is crucial for tasks like dialogue or code generation. Mention that in production systems, this often leads to more stable convergence and less need for reward shaping.

1. Define the averaging units

Clearly state that GRPO computes importance ratios per token and averages them across tokens, whereas GSPO computes a single importance ratio per sequence and averages across sequences.

2. Explain the rationale for sequence-level ratios

Discuss how token-level ratios can lead to high variance because each token's ratio is noisy, and errors compound over long sequences. Sequence-level ratios treat the whole sequence as one action, reducing variance and making the ratio more stable.

3. Connect to training stability

Explain that lower variance in importance ratios leads to more stable gradient estimates, reducing the risk of divergence or erratic updates. This is particularly important for long sequences where token-level ratios can explode.

4. Discuss trade-offs and practical implications

Acknowledge that sequence-level ratios may introduce bias if the policy changes significantly within a sequence, but in practice, the variance reduction often outweighs this bias. Mention that GSPO may require fewer samples or less clipping to achieve stable training.

Key Points to Mention

  • GRPO: token-level importance ratios, averaged over tokens; GSPO: sequence-level importance ratios, averaged over sequences.
  • Token-level ratios have high variance because each token's ratio is based on a single sample and can be extreme.
  • Sequence-level ratios reduce variance by aggregating information across the entire sequence, leading to more stable updates.
  • High variance in token-level ratios can cause unstable training, especially for long sequences, due to compounding errors.
  • GSPO's approach aligns with the sequence-level objective of many RL tasks, such as dialogue generation.
  • Trade-off: sequence-level ratios may introduce bias if the policy changes within a sequence, but this is often acceptable for stability gains.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.