← Microsoft Interview Insights

Microsoft·Machine Learning Engineer·Technical Phone Screen·Senior

Senior
May 2026

Summary

Microsoft ML Engineer interview that went deep on beam search internals. The follow-up question on length normalization was the kind of thing you either know cold or you're fumbling through on the spot.

Questions Asked (1)

Q1

Beam search assigns scores using cumulative log-probabilities, which tend to favor shorter sequences. How would you fix the scoring to make it fair across different sequence lengths, and what are the trade-offs involved?

Technical Trade-offsAlgorithms & Data Structures
Author's notes

I knew the basic fix (divide by length) but fumbled when they pushed on the alpha parameter.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by explaining why cumulative log-probabilities bias toward shorter sequences, then propose length normalization as the primary fix. Discuss variants like Google's length penalty and coverage penalty, and analyze trade-offs such as over-penalizing long sequences or introducing hyperparameters. Conclude with practical recommendations for tuning and evaluation.

Pro tip: Mention that length normalization can be applied during beam search or only at final ranking, and that the choice affects diversity and latency. Also note that in production, you might combine it with other heuristics like n-gram blocking.

1. Diagnose the length bias

Explain that summing log-probabilities makes longer sequences have lower (more negative) scores, so beam search favors shorter outputs. This is because each additional token adds a negative log-probability.

2. Introduce length normalization

Propose dividing the cumulative log-probability by a length penalty term, such as length^α or ((5+len)/6)^α, to make scores comparable across lengths. Mention that α is a hyperparameter controlling the strength of normalization.

3. Discuss alternative scoring methods

Cover other approaches like coverage penalty to encourage attending to all input tokens, or using a weighted combination of length and coverage. Also mention that some models use a learned length predictor.

4. Analyze trade-offs

Highlight that length normalization can lead to overly long or short sequences if α is mis-tuned, and may reduce diversity. Also note that it adds hyperparameter tuning overhead and can affect inference speed if applied during search.

5. Recommend practical implementation

Suggest tuning α on a validation set using metrics like BLEU or ROUGE, and applying normalization only at final ranking to avoid search distortion. Emphasize monitoring output length distribution and task-specific needs.

Key Points to Mention

  • Cumulative log-probability bias: longer sequences have more negative scores.
  • Length normalization: divide by length^α or use Google's length penalty.
  • Hyperparameter α: controls trade-off between length and likelihood; typically tuned between 0.6 and 1.0.
  • Coverage penalty: encourages model to attend to all input tokens, often used with length normalization.
  • Trade-offs: risk of over/under-penalizing, reduced diversity, and additional tuning complexity.
  • Evaluation: use task-specific metrics (BLEU, ROUGE) and monitor length distribution.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.