Netflix·Machine Learning Engineer·Technical Phone Screen
- What is SentencePiece and how does it work under the hood?
- Which tokenizers do BERT and typical Transformer-based language models use, and what's the reasoning behind those choices?
- Walk me through the core components inside a Transformer block and what each one does.
- Compare a standard Transformer to LLaMA and Qwen architectures. What design choices differ and what are the trade-offs, specifically around things like MoE, RMSNorm, and RoPE?
“I knew the high level answer but stumbled a bit on the specifics of the training algorithm.”