Anthropic·Software Engineer·Technical Phone Screen
- Given a helper function that returns per-token probabilities from an LLM, design a complete binary text classifier: construct prompts for each class, produce a continuous probability score per input, handle numerical stability with log-probabilities, aggregate token-level scores to sequence level, define batching, select thresholds, evaluate with ROC-AUC and F1, propose improvements like prompt ensembling and calibration, discuss failure modes, and give pseudocode with complexity analysis.
“This was one question but it kept expanding.”