OpenAI·Software Engineer·Technical Phone Screen
Jun 2026
OpenAI SWE interview with a heavy focus on PyTorch autograd internals, covering in-place vs out-of-place ops, hand-written backward passes, and parallel prefix scan. The questions were genuinely hard and the kind of thing that exposes whether you've actually read the docs or just skimmed a tutorial.
- Implement matrix multiplication in-place, then explain why it breaks PyTorch's autograd.
- Now implement the same matrix multiplication out-of-place so that autograd works correctly.
- Write the backward pass for matrix multiplication by hand, without relying on PyTorch's autograd to do it for you.
- Implement Hillis-Steele parallel prefix scan for the forward pass, then derive and implement its backward pass.
“The interviewer ran my code and I got a cryptic version-counter error.” The rest of the author's notes on Software Engineer interview at OpenAI, Technical Phone Screen round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post