PayPal·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

Senior

May 2026

Summary

PayPal ML Engineer interview with a deep reinforcement learning system design question focused on per-user spending limits. The scope was broad and the expectations were clearly senior-level.

Questions Asked (1)

Design a reinforcement learning system to set personalized spending limits for users. Walk through the state representation, action space, transition dynamics, reward signal, training approach, exploration strategy under risk constraints, off-policy evaluation, safety guardrails, cold start handling, and non-stationarity.

System DesignTechnical Trade-offsProduct Analytics & Metrics

Author's notes

This one had a lot of moving parts and I think I underestimated how much they wanted me to connect the pieces rather than just list them.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by framing the problem as a constrained Markov Decision Process (CMDP) where the agent learns to set personalized spending limits that balance user utility and risk. Walk through each component systematically, emphasizing safety, cold start, and non-stationarity, and tie design choices to PayPal's business metrics like fraud reduction and user engagement.

Pro tip: Emphasize that in a fintech setting, safety and interpretability often trump pure reward maximization; propose a conservative offline training pipeline with human-in-the-loop validation before any online deployment.

1. Define the CMDP and State Representation

Formalize the problem as a constrained MDP with states capturing user features, transaction history, and contextual signals. Discuss feature engineering and dimensionality reduction to handle high-cardinality data.

2. Design Action Space and Reward Signal

Define actions as discrete spending limit adjustments (e.g., percentage changes) and craft a reward that balances user utility (e.g., successful transactions) with risk penalties (e.g., overdrafts, fraud). Incorporate constraints as costs.

3. Training, Exploration, and Safety

Choose an off-policy RL algorithm (e.g., Conservative Q-Learning) trained on historical logs. Implement constrained exploration (e.g., safe exploration via shielding) and safety guardrails like hard limits and fallback policies.

4. Off-Policy Evaluation and Cold Start

Use importance sampling and doubly robust estimators for OPE. For cold start, leverage meta-learning or hierarchical models to generalize from similar users, and start with conservative limits.

5. Handling Non-Stationarity and Deployment

Address non-stationarity via continual learning, change detection, and periodic retraining. Deploy with A/B testing and monitor for distribution shifts, ensuring compliance and fairness.

Key Points to Mention

Constrained MDP formulation with safety constraints (e.g., expected overdraft rate below threshold)
State representation: user demographics, transaction sequences, temporal patterns, and external context
Action space: discrete limit adjustments (e.g., -20% to +20%) with guardrails
Reward shaping: utility from successful transactions minus penalties for overdrafts/fraud, with constraint costs
Off-policy evaluation techniques: importance sampling, doubly robust, and their assumptions
Cold start: meta-learning, hierarchical Bayesian models, and conservative initialization
Non-stationarity: continual learning, change point detection, and periodic retraining

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.