Meta·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Jul 2026

Summary

ML system design round at Meta for an MLE role. The whole thing centered on a single meaty problem involving weapon-related content on a marketplace, and they wanted you to go deep on basically every layer of the ML stack.

Questions Asked (1)

Design a machine learning system to detect illicit firearm listings on a marketplace, or alternatively build a recommendation or pricing system for legal firearm sales. Walk through problem framing, label definition, data sources, feature engineering, model selection (text classifier, image model, or multi-modal), evaluation metrics, deployment, and monitoring with feedback loops.

System DesignTechnical Trade-offsProduct Analytics & Metrics

Author's notes

This one sprawls in every direction if you let it.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Choose the illicit firearm detection system as it aligns with Meta's trust and safety priorities and demonstrates handling of high-stakes, imbalanced classification. Structure your answer around the ML lifecycle: problem framing, label definition, data sources, feature engineering, model selection, evaluation, deployment, and monitoring with feedback loops. Emphasize multi-modal approaches and practical trade-offs between precision and recall in a policy-driven context.

Pro tip: Frame the problem as a high-recall first-pass filter with human review to balance safety and scalability, and discuss how to handle adversarial evasion and concept drift in monitoring. Show awareness of legal and policy constraints, such as avoiding over-blocking legal sales while catching illicit ones.

1. Problem Framing and Label Definition

Define the task as binary classification of listings as illicit or legal, and specify what constitutes 'illicit' (e.g., prohibited firearms, illegal modifications, sales to prohibited persons). Address label ambiguity by creating a detailed annotation guideline and using human review for ground truth.

2. Data Sources and Feature Engineering

Collect text (title, description), images, seller metadata, and user reports. Engineer features from text (TF-IDF, embeddings), images (pre-trained CNN features), and structured data (price, location, seller history). Consider multi-modal fusion early.

3. Model Selection and Training

Start with a text classifier (e.g., fine-tuned BERT) as a baseline, then explore image models (e.g., ResNet) and multi-modal architectures (e.g., CLIP or late fusion). Handle class imbalance with techniques like focal loss or resampling, and use cross-validation.

4. Evaluation Metrics and Thresholds

Use precision, recall, F1, and AUC-ROC, but prioritize recall at a fixed precision to minimize missed illicit listings. Set thresholds based on business requirements and human review capacity, and evaluate with a holdout set and slice-based analysis.

5. Deployment, Monitoring, and Feedback Loops

Deploy as a real-time service with a human-in-the-loop review queue. Monitor for data drift, adversarial evasion, and performance degradation. Collect reviewer feedback to retrain models periodically and update labels.

Key Points to Mention

Class imbalance and the need for high recall in safety-critical applications
Multi-modal fusion of text and images for better detection
Human-in-the-loop review to handle false positives and provide labels
Adversarial evasion and concept drift monitoring
Evaluation metrics beyond accuracy: precision-recall trade-offs and slice-based analysis
Legal and policy constraints, such as avoiding over-blocking legal sales

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.