Openai·Software Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

May 2026

Summary

System design round at OpenAI for a software engineer role. The question was one massive prompt covering basically every dimension of a GPT-facing web app you can think of, streaming, snapshots, search, sharing, auth, scaling, testing, the whole thing. Left feeling like I'd only scratched the surface on half of it.

Questions Asked (1)

Design an end-to-end web application for interacting with a GPT-like model that supports real-time token streaming, snapshot saving, full-text search over saved sessions, and link-based sharing with role-based access controls. Walk through the full architecture including frontend, backend, data stores, APIs, and all the operational concerns.

System DesignTechnical Trade-offsAPI & Integrations

Author's notes

This was the entire interview, one question, no breaks.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and scale, then present a high-level architecture before diving into each component. Focus on the streaming pipeline, data model for sessions and snapshots, and security for sharing. Discuss trade-offs and operational concerns like monitoring and cost.

Pro tip: Emphasize how you would handle backpressure and partial failures in the streaming path, as real-time token streaming is the most critical and challenging aspect. Also, mention using OpenAI's own APIs and best practices, showing familiarity with the company's ecosystem.

1. Clarify Requirements and Scale

Ask questions to understand expected user load, token throughput, latency requirements, and security constraints. Define functional and non-functional requirements.

2. High-Level Architecture

Sketch the main components: frontend, backend API gateway, streaming service, model inference service, data stores, and search service. Explain data flow for streaming, saving, searching, and sharing.

3. Deep Dive into Key Components

Detail the streaming protocol (e.g., WebSockets or SSE), backend services (e.g., Node.js or Python), data stores (e.g., PostgreSQL for metadata, Redis for caching, Elasticsearch for search), and sharing/access control mechanisms.

4. API Design and Integration

Define RESTful or GraphQL APIs for session management, snapshot saving, search, and sharing. Discuss how to integrate with the GPT model API, including rate limiting and error handling.

5. Operational Concerns and Trade-offs

Cover scalability, monitoring, logging, deployment, cost optimization, and security. Discuss trade-offs such as consistency vs. availability, and latency vs. cost.

Key Points to Mention

Real-time streaming: use WebSockets or Server-Sent Events (SSE) for token streaming, with backpressure handling and reconnection logic.
Data model: store sessions, messages, and snapshots in a relational database; use a document store or search engine for full-text search.
Sharing and RBAC: implement link-based sharing with signed URLs and role-based access control (e.g., viewer, editor) using JWT or OAuth.
Scalability: design for horizontal scaling of streaming servers, use load balancers, and consider sharding or partitioning for data stores.
Operational concerns: monitoring with Prometheus/Grafana, logging with ELK, tracing with Jaeger, and cost management for model inference.
Trade-offs: discuss consistency vs. latency in search indexing, and the cost of maintaining persistent connections vs. polling.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.