PayPal·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Jun 2026

Summary

PayPal ML engineer interview focused entirely on a deep system design problem around fraud detection. One question but it covered a lot of ground, and I left feeling like I only scratched the surface on a few parts.

Questions Asked (1)

Design a complete fraud detection system from scratch, covering how you'd label training data given that fraud confirmations are delayed and rare, how you'd handle extreme class imbalance, what features you'd use (behavioral, graph-based, device signals, merchant patterns), what models you'd pick and why, how you'd serve predictions in real time with latency constraints, how you'd set decision thresholds given the precision/recall tradeoff, what metrics you'd evaluate on, and how you'd monitor for drift and adversarial adaptation over time.

System DesignTechnical Trade-offsProduct Analytics & Metrics

Author's notes

This was basically a 45-minute design marathon in one prompt.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Structure your answer as an end-to-end ML system design, starting with business objectives and data challenges, then walking through the ML lifecycle: labeling, feature engineering, modeling, deployment, and monitoring. Emphasize trade-offs at each stage, especially latency vs. accuracy and precision vs. recall, and tie decisions back to PayPal's scale and fraud patterns.

Pro tip: Show you understand that fraud detection is an adversarial, cost-sensitive problem: the optimal threshold isn't just about F1, but about minimizing expected financial loss, and models must be retrained frequently to adapt to new fraud tactics.

1. Clarify requirements and constraints

Ask about scale (transactions per second), latency budget (e.g., <100ms), fraud rate, cost of false positives vs. false negatives, and regulatory constraints. This shapes all subsequent design choices.

2. Address labeling and class imbalance

Discuss delayed labels: use proxy labels (chargebacks, manual reviews) and techniques like PU learning or weak supervision. For imbalance, consider resampling, class weights, or anomaly detection, and evaluate with precision-recall curves.

3. Design features and models

Cover behavioral (velocity, amount patterns), graph-based (shared devices, IPs, merchant networks), device signals (fingerprinting, emulator detection), and merchant patterns. Choose models balancing accuracy and interpretability: gradient boosting for tabular, graph neural networks for relational, and possibly online learning for adaptation.

4. Plan real-time serving and thresholding

Propose a low-latency architecture: precomputed features in a feature store, model serving with caching, and fallback rules. Set thresholds by optimizing expected cost (false positive cost vs. false negative cost) and possibly use a tiered decision system (block, review, allow).

5. Define metrics and monitoring

Use business metrics (fraud loss rate, false positive rate, review rate) and ML metrics (AUPRC, recall@fixed FPR). Monitor for data drift, concept drift, and adversarial adaptation via performance tracking, feature distribution shifts, and retraining triggers.

Key Points to Mention

Handling delayed labels with proxy labels, PU learning, or semi-supervised techniques
Class imbalance strategies: resampling, class weighting, anomaly detection, and evaluation with AUPRC
Feature engineering: behavioral, graph-based, device, and merchant features, with real-time computation
Model selection: gradient boosting for tabular, GNNs for graph, and online learning for adaptation
Real-time serving with low latency: feature store, model caching, and fallback rules
Threshold optimization based on expected cost and business impact, not just F1
Monitoring for drift and adversarial adaptation: performance tracking, feature drift detection, and retraining pipelines

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.