OpenAI·Machine Learning Engineer·Technical Phone Screen
Jul 2026
Hands-on PyTorch screen for an MLE role at OpenAI, two parts back-to-back in a shared code editor. Part A was debugging a broken miniGPT and adding KV caching, Part B was implementing matmul forward and backward from scratch plus a parallel scan discussion. About an hour total and it moves fast.
- You're given a small decoder-only transformer that runs without crashing but produces garbage text during autoregressive generation. Find and fix the logical bug(s), then walk through your debugging process step by step.
- After fixing the generation bug, implement key-value caching so that each decode step only attends over the cached prefix rather than recomputing the full sequence.
- What is the memory footprint of a KV cache as context length grows, and what approaches exist to reduce it?
- How does batched generation with sequences of different lengths interact with a shared KV cache and the causal mask?
- Implement a custom matrix multiply op as a torch.autograd.Function, including both the forward pass and a from-scratch derivation of the backward gradients for both operands.
- When A or B is broadcast in the matmul (say, one weight matrix multiplied against a batch of inputs), how does the backward change and over which axes do you sum the gradient?
- Explain how a parallel prefix scan like Hillis-Steele could be applied to parallelize the tiled accumulation in matrix multiply, and be honest about when it actually helps versus when it doesn't.
- Hillis-Steele is step-efficient but not work-efficient at O(n log n) work. When would a Blelloch scan be preferable and what is the tradeoff?
“This was the part that stressed me out most.” The rest of the author's notes on Machine Learning Engineer interview at OpenAI, Technical Phone Screen round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post