Decagon·Software Engineer·Onsite - System Design / Architecture·Senior

Senior

Apr 2026

Summary

Decagon software engineer interview with a meaty system design question around building an enterprise chatbot from scratch. One question, but it covers a lot of ground so expect to stay organized or you'll lose the thread fast.

Questions Asked (1)

Design an enterprise chatbot system for a single customer and business domain. It should handle multi-turn conversations, answer questions from approved documents with citations, avoid making things up, allow admins to update the knowledge base, fall back to a human agent when unsure, and meet reasonable latency and cost targets. Walk through the full design including APIs, session management, document ingestion, retrieval, prompt construction, model serving, safety, monitoring, and scaling.

System DesignTechnical Trade-offsAPI & Integrations

Author's notes

This one sprawls in every direction and the hard part is not going down a rabbit hole on any single piece.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and constraints, then walk through the end-to-end architecture from ingestion to serving, emphasizing how each component addresses the specific needs (multi-turn, citations, no hallucinations, admin updates, human fallback, latency/cost). Use a layered approach: data layer, retrieval layer, orchestration layer, and serving layer, and discuss trade-offs at each step.

Pro tip: Explicitly call out how you would measure and enforce 'no making things up'—e.g., via retrieval grounding, confidence thresholds, and fallback logic—and tie it to business metrics like containment rate and CSAT. This shows you understand the real-world impact beyond just technical implementation.

1. Clarify Requirements and Constraints

Ask about expected traffic, document types, latency targets, cost limits, and compliance needs. Define success metrics like answer accuracy, containment rate, and fallback frequency.

2. Design Data Ingestion and Knowledge Base

Outline the pipeline for ingesting approved documents: parsing, chunking, embedding, indexing, and versioning. Include admin APIs for updates and a review workflow to ensure only approved content is used.

3. Architect Retrieval and Conversation Orchestration

Describe how to retrieve relevant chunks using hybrid search (keyword + vector), maintain session state for multi-turn context, and construct prompts with citations. Include confidence scoring and fallback triggers.

4. Model Serving, Safety, and Monitoring

Explain model selection (e.g., fine-tuned vs. API-based), latency optimization (caching, batching), and safety layers (moderation, PII redaction). Detail monitoring for quality, drift, and cost.

5. Scaling and Trade-offs

Discuss horizontal scaling of retrieval and serving, cost management via caching and tiered models, and trade-offs between latency, accuracy, and cost. Mention human handoff integration and feedback loops.

Key Points to Mention

Multi-turn session management with context windowing and state storage (e.g., Redis).
Retrieval-augmented generation (RAG) with hybrid search and citation generation.
Confidence estimation and fallback to human agents when unsure.
Admin APIs for document CRUD, versioning, and re-indexing.
Latency and cost optimization: caching, batching, model distillation, and tiered serving.
Monitoring: logging, tracing, feedback collection, and A/B testing for continuous improvement.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.