← Plaid Interview Insights

Plaid·Software Engineer·Onsite - System Design / Architecture·Senior

Senior
May 2026

Summary

System design round at Plaid for a software engineer role. The prompt was basically 'build ChatGPT' which sounds exciting until you realize how many moving parts they actually want you to cover in 45 minutes.

Questions Asked (1)

Q1

Design a conversational AI assistant similar to ChatGPT. Your design should support multi-turn conversations with history, token-by-token streaming responses, high availability under peak load, safety controls like toxicity filtering and PII handling, and optional tool use such as calling a search API or calculator. Walk through the high-level architecture, key components, storage choices, scaling strategy, and how you'd evaluate quality and safety in production.

System DesignTechnical Trade-offsProduct Analytics & Metrics
Author's notes

I started with the happy path, which in hindsight was the wrong move.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and scale (e.g., QPS, latency, model size), then sketch a high-level architecture with clear separation of concerns: API gateway, conversation service, model inference, safety layer, and tool orchestration. Walk through data flow for a multi-turn streaming request, then dive into storage, scaling, and evaluation, emphasizing trade-offs and Plaid-relevant concerns like PII and reliability.

Pro tip: Anchor your design around the streaming and stateful nature of conversations: treat conversation history as a first-class entity with its own storage and caching strategy, and explicitly discuss how you'd handle partial failures and retries without duplicating or corrupting turns.

1. Clarify Requirements and Scope

Ask about expected QPS, peak load, latency SLOs, model size/hosting, and whether tools are mandatory. Confirm safety and compliance requirements (e.g., PII, toxicity thresholds) and evaluation goals.

2. High-Level Architecture

Draw the main components: client, API gateway, conversation service (manages history and orchestration), model inference service (with streaming), safety service, tool executor, and storage layers. Explain the request flow for a multi-turn streaming response.

3. Storage and State Management

Choose storage for conversation history (e.g., DynamoDB/Cassandra for durability, Redis for hot cache), model artifacts (object store), and tool results. Discuss token limits, summarization, and how to retrieve history efficiently.

4. Scaling and Reliability

Describe horizontal scaling of stateless services, autoscaling for inference (GPU/CPU), load balancing, rate limiting, and graceful degradation. Cover high availability via multi-AZ, retries, idempotency, and fallback models.

5. Safety, Tools, and Evaluation

Explain toxicity filtering (pre/post), PII detection and redaction, and tool invocation with sandboxing. Outline offline and online evaluation: automated metrics, human review, A/B tests, and monitoring for drift and safety violations.

Key Points to Mention

  • Token-by-token streaming via SSE/WebSockets and backpressure handling
  • Conversation history storage with TTL, summarization, and token limit management
  • Safety layers: input/output toxicity classifiers, PII detection/redaction, and policy enforcement
  • Tool use orchestration: function calling, sandboxed execution, and result caching
  • Scaling strategies: autoscaling inference, caching, rate limiting, and multi-region deployment
  • Evaluation metrics: latency, throughput, toxicity rate, PII leakage, user satisfaction, and A/B testing

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.