xAI·Software Engineer·Onsite - Coding / Algorithms
Apr 2026
Technical round for an ML Infra role at xAI. The whole thing was a coding exercise around distributed matrix multiplication, two strategies, and you had to actually implement both with a provided skeleton. More systems-flavored than pure algo, which I appreciated but also was not fully prepared for.
- Complete the data parallel matrix multiplication function: each worker has a full copy of the weights and a shard of the input batch. Compute the local output per worker and assemble the final result correctly.
- Implement fully sharded data parallel matrix multiplication from scratch: shard the weight matrix across workers, use an all-gather collective to retrieve needed shards at compute time, and keep per-worker memory bounded by the shard size.
- Walk through the correctness guarantees, memory footprint, and communication overhead for each of the two parallelism strategies you implemented.
“This part felt more approachable.” The rest of the author's notes on Software Engineer interview at xAI, Onsite - Coding / Algorithms round, covers how they worked through the question, what the panel pushed back on, and what they would do differently.
View Post