Cohere·Machine Learning Engineer·Technical Phone Screen
Jun 2026
Cohere ML Engineer interview with a coding question around implementing decoding strategies for a toy GPT model. Pretty focused on getting the details right, not just the high-level idea.
- Given a simplified GPT-like class with a fixed vocab size, an EOS token, and a forward method returning deterministic 1D logits, implement two autoregressive decoding functions that stop at EOS: one using greedy argmax selection, and one sampling from the top-k renormalized softmax distribution.
“The greedy part was fine, argmax at each step until you hit EOS, straightforward.”