ReflectionAI·Software Engineer·Technical Phone Screen
- If you have a single decoding request to serve (no batching), which parallelism strategy would you pick among data parallelism, tensor parallelism, and pipeline parallelism, and why? Walk through the tradeoffs in latency, memory, and underutilization.
“This is the kind of question that sounds easy until you start talking and realize you've been thinking about parallelism mostly in the training context.”