NVIDIA·Software Engineer·Technical Phone Screen
- Explain PyTorch Dynamo and how it fits into inference acceleration, including how it captures and compiles computation graphs and how it connects to backends like TorchInductor and TensorRT.
- Walk through multiple techniques for speeding up model inference, covering things like parallelism strategies, batching, operator fusion, quantization, kernel autotuning, CUDA Graphs, overlapping compute with data transfer, sparsity, caching, and compiler-level optimizations.
- How would you design a rigorous and fair inference benchmark? Cover what metrics you'd track, how you'd set up the test environment, what baselines you'd use, and how you'd present the results.
“This one tripped me up a bit because I kept conflating what Dynamo does versus what the backend does.”