Scale AI·Machine Learning Engineer·Technical Phone Screen
May 2026
Scale.ai ML engineer coding round done entirely on Google Colab, no IDE, just NumPy and a starter file with some softmax scaffolding already in place. Two-part attention implementation problem, single-head then multi-head. No LLM allowed, which they apparently enforce somehow.
- Implement scaled dot-product attention from scratch using only NumPy, given Q, K, and V matrices. Include support for an optional additive mask applied before the softmax.
- Extend your single-head attention to full multi-head attention: split the embedding dimension across h heads, project Q/K/V separately per head, run attention in parallel, then concatenate and apply an output projection.
“The formula itself isn't the hard part, it's remembering to divide by sqrt(d_k) before softmax and not after.” The rest of the author's notes on Machine Learning Engineer interview at Scale AI, Technical Phone Screen round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post