instagram·Machine Learning Engineer·Onsite - System Design / Architecture
Jul 2026
Heavy ML infra system design round at Instagram. They go deep on GPU internals and if you've only ever called high-level training APIs you will feel it.
- Walk me through the communication bottlenecks you'd encounter in multi-GPU training, covering collective communication libraries, All-Reduce implementations, and the tradeoffs between ring and tree topologies as well as intra-node versus inter-node interconnects.
- How does the PyTorch caching allocator work under the hood, and how do memory fragmentation, activation checkpointing, and gradient accumulation factor into your memory management strategy during training?
- You're hitting OOM errors in a multi-GPU training or inference job. Walk me through exactly how you'd diagnose the root cause and what steps you'd take to resolve it, including any profiling tools, memory inspection techniques, and architectural changes like sharding or offloading.
“This is where I started to feel the gap between knowing the terms and actually knowing the stuff.”