TikTok·Data Scientist·Onsite - System Design / Architecture·Senior

Senior

May 2026

Summary

TikTok data scientist interview with a meaty ML system design question about fraud detection. The kind of question that sounds focused until you realize how many moving parts they actually want you to cover.

Questions Asked (1)

Design a real-time credit card fraud detection system for an online payments company. Walk through data sources, feature engineering, model selection, real-time architecture, how often you'd retrain, and how you'd catch model drift.

System DesignTechnical Trade-offsData Modeling

Author's notes

This question sprawls in every direction and I did not do a great job scoping it before diving in.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Structure your answer around the end-to-end ML lifecycle: start with data sources and feature engineering, then justify model choice and real-time serving architecture, and finish with retraining cadence and drift detection. Emphasize the latency-accuracy trade-off and how you'd monitor both model and business metrics in production.

Pro tip: Anchor your design to concrete latency and throughput targets (e.g., <100ms p99, thousands of TPS) and explicitly discuss how you'd handle extreme class imbalance and false positive costs, since fraud systems live or die by precision-recall trade-offs.

1. Data Sources & Feature Engineering

Identify transactional, behavioral, and contextual data sources (card, device, IP, merchant, historical user behavior) and engineer features like velocity counts, time-since-last-transaction, and geolocation anomalies. Distinguish between batch-computed and streaming features.

2. Model Selection & Training

Choose models suited to tabular, imbalanced, real-time data—e.g., gradient-boosted trees (XGBoost/LightGBM) for accuracy, or a two-stage approach with a fast filter model and a heavier scorer. Address class imbalance with techniques like SMOTE, class weighting, or focal loss.

3. Real-Time Architecture

Design a streaming pipeline (e.g., Kafka for ingestion, Flink/Spark Streaming for feature computation, a low-latency feature store, and a model server like TensorFlow Serving or a custom microservice). Ensure sub-100ms scoring and fallback rules for failures.

4. Retraining Strategy

Propose a hybrid schedule: periodic retraining (e.g., daily/weekly) on recent data plus triggered retraining when performance degrades. Use shadow deployment and A/B testing to validate new models before full rollout.

5. Drift Detection & Monitoring

Monitor data drift (PSI, KL divergence on feature distributions), concept drift (performance metrics like precision/recall on delayed labels), and business metrics (fraud rate, false positive rate). Set up alerts and automated retraining triggers.

Key Points to Mention

Latency vs. accuracy trade-off: real-time scoring requires low-latency models, possibly with a cascade of models.
Class imbalance: fraud is rare (<1%), so use precision-recall AUC, not accuracy, and techniques like undersampling or cost-sensitive learning.
Feature store: enables consistent features between training and serving, reducing training-serving skew.
Concept drift: fraud patterns evolve quickly, so retraining frequency should be high (e.g., daily) and drift detection must be automated.
Feedback loop: labels are delayed (chargebacks), so use proxy labels or human review for immediate feedback.
Scalability: design for high throughput (thousands of TPS) with horizontal scaling and graceful degradation.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.