Google·Software Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Jun 2026

Summary

System design round at Google for a software engineer role, focused entirely on building a ChatGPT-style conversational AI system. Dense question with a lot of moving parts, and I felt like I was constantly playing catch-up trying to cover everything they wanted.

Questions Asked (1)

Design a chat system similar to ChatGPT, where users can have multi-turn conversations with an LLM and their conversation history is persisted and can be restored later.

System DesignData ModelingTechnical Trade-offs

Author's notes

This question is basically five questions in a trench coat.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and scale, then design the high-level architecture covering client, API, LLM integration, and storage. Focus on data modeling for conversations and messages, and discuss trade-offs in persistence, retrieval, and streaming responses.

Pro tip: Emphasize idempotency and ordering of messages to handle retries and concurrent updates, and discuss how to efficiently paginate and restore long conversation histories without loading everything into memory.

1. Clarify Requirements and Scale

Ask about expected user base, message volume, latency requirements, and whether conversations are private or shareable. Establish functional and non-functional requirements.

2. High-Level Architecture

Outline components: client, API gateway, conversation service, LLM service, and storage. Describe data flow for sending a message and receiving a streamed response.

3. Data Modeling and Storage

Design schemas for users, conversations, and messages. Choose storage solutions (e.g., SQL for metadata, NoSQL for messages) and discuss indexing for efficient retrieval.

4. Conversation Restoration and Pagination

Explain how to restore a conversation by fetching messages in order, using pagination or cursors to handle long histories. Discuss caching strategies for active conversations.

5. Trade-offs and Scalability

Discuss trade-offs: consistency vs. availability, SQL vs. NoSQL, streaming vs. batch responses. Address scaling with sharding, replication, and rate limiting.

Key Points to Mention

Use of WebSockets or Server-Sent Events (SSE) for streaming LLM responses to the client.
Data model: conversations table with user_id, title, timestamps; messages table with conversation_id, role, content, sequence number.
Idempotency keys for message sending to avoid duplicates on retries.
Pagination using cursor-based approach (e.g., timestamp or sequence number) for efficient history retrieval.
Caching recent messages in Redis or similar to reduce latency for active conversations.
Handling LLM context window limits by truncating or summarizing older messages.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.