Weride·Data Scientist·Technical Phone Screen·Senior

Senior

May 2026

Summary

WeRide data scientist interview with a meaty domain adaptation question about expanding an autonomous driving eval system from Beijing to Guangzhou. One question, lots of sub-parts, and a follow-up that honestly caught me working harder than expected.

Questions Asked (1)

You built an autonomous-driving evaluation system using labeled data from one city. The company now wants to expand to a new city where you can only collect a small amount of data. How would you evaluate whether the system is likely to perform well there? Cover how you'd define the target population and success criteria, assess whether your existing metrics and calibration transfer, detect distribution shift, combine the large source dataset with the small target sample, and make a go/no-go call under limited evidence. Follow-up: if the new city has important scenarios that barely exist in your source data (different road layouts, higher scooter density, different weather, local signage, etc.), how does that change your data collection strategy and edge-case prioritization?

Product Analytics & MetricsTechnical Trade-offsAdaptability & Ambiguity

Author's notes

This one took me a minute to even organize in my head.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Frame the answer around a structured evaluation plan that starts with defining the target population and success criteria, then systematically assesses metric transferability, detects distribution shift, and leverages both datasets. Emphasize a risk-aware go/no-go decision that acknowledges limited evidence and proposes a phased rollout with monitoring.

Pro tip: Propose a 'shadow mode' deployment in the new city where the system runs but doesn't control the vehicle, allowing safe collection of real-world performance data and edge cases before full commitment. This demonstrates practical risk management and a test-and-learn mindset.

1. Define Target Population and Success Criteria

Specify the operational design domain (ODD) for the new city, including geography, weather, traffic density, and scenario types. Establish clear, measurable success criteria (e.g., disengagement rate, precision/recall on critical objects) that align with business goals and safety requirements.

2. Assess Metric and Calibration Transfer

Evaluate whether existing metrics (e.g., precision, recall, calibration error) remain valid in the new city. Check model calibration on the small target sample and compare to source; if miscalibrated, consider re-calibration or domain adaptation.

3. Detect and Quantify Distribution Shift

Use statistical tests (e.g., KS test, MMD) and visualizations to compare feature distributions between source and target. Identify which scenarios are underrepresented or missing, and prioritize them for data collection.

4. Combine Datasets for Robust Evaluation

Leverage the large source dataset for training and the small target sample for validation and fine-tuning. Employ techniques like importance weighting, domain adaptation, or transfer learning to bridge the gap, and use cross-validation to estimate performance uncertainty.

5. Make a Go/No-Go Call Under Uncertainty

Synthesize evidence from steps 1-4 to decide whether to proceed, with a focus on risk. If evidence is insufficient, propose a phased rollout with extensive monitoring and a clear fallback plan, and define triggers for re-evaluation.

Key Points to Mention

Operational design domain (ODD) definition and its role in scoping evaluation
Metric transferability: checking if precision/recall/calibration hold in new city
Distribution shift detection methods (e.g., KS test, MMD, adversarial validation)
Domain adaptation techniques (e.g., fine-tuning, importance weighting, transfer learning)
Active learning and targeted data collection for underrepresented scenarios
Risk-aware decision-making: shadow mode, phased rollout, and monitoring

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.