Meta·Data Scientist·Technical Phone Screen·Senior

Senior

Jun 2026

Summary

Got a deep ML system design question for a Data Scientist role at Meta. One question, but it had like six sub-parts packed into it, so it basically ran the whole session. Dense and a bit exhausting, but also kind of interesting if you're into autonomous systems stuff.

Questions Asked (1)

You have trajectory data (position, velocity, maybe acceleration) for multiple vehicles at an intersection over the last several seconds. Build an ML system to predict each vehicle's intention: left turn, straight, or right turn. Walk through labeling strategy, feature engineering vs sequence models, multi-agent interactions, evaluation metrics, class imbalance, distribution shift, and how you'd use uncertainty estimates in downstream planning.

System DesignTechnical Trade-offsData Modeling

Author's notes

This question is basically six questions stapled together.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the problem as a multi-class classification task with sequential inputs, then systematically address each sub-topic: labeling, features vs sequence models, multi-agent interactions, evaluation, imbalance, distribution shift, and uncertainty. Emphasize trade-offs and justify choices based on data characteristics and deployment constraints. Conclude with how uncertainty informs downstream planning.

Pro tip: Show awareness that intention prediction is inherently uncertain and that calibration and proper scoring rules matter more than raw accuracy; also mention that in production, you'd likely need to handle real-time constraints and integrate with a planner that can handle probabilistic outputs.

1. Labeling Strategy

Define intention labels based on future trajectory (e.g., turn direction from map or path curvature) and discuss automatic labeling from map-matched routes, manual annotation, or weak supervision. Address label noise and ambiguity (e.g., vehicles that go straight then turn).

2. Feature Engineering vs Sequence Models

Compare hand-crafted features (kinematics, heading change, distance to intersection) with end-to-end sequence models (RNN, Transformer) that capture temporal dependencies. Discuss trade-offs in data efficiency, interpretability, and performance.

3. Multi-Agent Interactions

Incorporate interactions via social pooling, graph neural networks, or attention mechanisms to model how vehicles influence each other. Mention the importance of relative positions, velocities, and right-of-way rules.

4. Evaluation, Imbalance, and Shift

Choose metrics like macro-F1, per-class recall, and calibration (e.g., ECE) due to class imbalance. Address imbalance via resampling, class weights, or focal loss. Discuss distribution shift (new intersections, weather) and mitigation via domain adaptation or continual learning.

5. Uncertainty in Downstream Planning

Explain how to quantify uncertainty (e.g., Bayesian NNs, ensembles, MC dropout) and use it in planning: conservative behavior under high uncertainty, risk-aware trajectory optimization, and human-in-the-loop fallback.

Key Points to Mention

Labeling from future trajectories using map data and handling ambiguous cases
Trade-offs between feature-based models and sequence models (e.g., LSTMs, Transformers)
Modeling multi-agent interactions with graph neural networks or attention
Evaluation metrics robust to class imbalance (macro-F1, per-class recall) and calibration
Distribution shift challenges and techniques like domain adaptation or online learning
Uncertainty estimation methods and their integration into downstream planning for safety

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.