Atlassian·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

May 2026

Summary

Atlassian ML Engineer system design round focused on building an enterprise RAG chatbot. The question was dense and covered basically every layer of the stack, from ingestion to eval. Left feeling like I did okay on the retrieval side but probably undersold the ACL enforcement piece.

Questions Asked (1)

Design an internal company chatbot that answers employee questions by pulling from heterogeneous internal data sources like wikis, docs, tickets, code repos, and HR systems. The system should support multi-turn conversations, cite sources, enforce per-document access controls, and meet latency and freshness requirements at scale.

System DesignTechnical Trade-offsAPI & Integrations

Author's notes

This was a single big question that kept expanding.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and constraints, then propose a high-level architecture that separates ingestion, indexing, retrieval, and generation. Emphasize how you would handle access control, multi-turn context, citations, and latency/freshness trade-offs, and discuss evaluation and iteration.

Pro tip: Treat access control as a first-class concern by enforcing permissions at query time and filtering retrieved documents before generation, rather than relying on post-hoc filtering. Also, design for incremental indexing and caching to balance freshness and latency.

1. Clarify Requirements and Constraints

Ask about scale (users, queries per second, data volume), latency targets, freshness requirements, and security/compliance needs. Confirm the types of data sources and their update frequencies.

2. Design Data Ingestion and Indexing Pipeline

Propose connectors for each source (wikis, docs, tickets, code, HR) that extract and normalize content, then chunk and embed it. Use incremental indexing with change data capture to meet freshness requirements.

3. Architect Retrieval and Access Control

Implement a hybrid retrieval system (e.g., dense + sparse) with metadata filters. Enforce per-document access control by integrating with source systems' permission APIs and filtering results at query time based on the user's identity.

4. Build Multi-Turn Conversation and Generation Layer

Maintain conversation state and use a retrieval-augmented generation (RAG) approach to produce answers with citations. Ensure the model can handle follow-up questions by incorporating context and re-retrieving as needed.

5. Address Latency, Freshness, and Scalability

Optimize with caching, pre-computation, and distributed serving. Discuss trade-offs between freshness and latency, and propose monitoring and evaluation metrics (e.g., answer accuracy, citation correctness, latency percentiles).

Key Points to Mention

Hybrid retrieval (dense + sparse) for better recall and precision
Per-document access control enforced at query time via integration with source permission systems
Incremental indexing and change data capture to meet freshness requirements
Caching and pre-computation to reduce latency for frequent queries
Multi-turn context management and retrieval-augmented generation with citations
Evaluation metrics and monitoring for answer quality, citation accuracy, and system performance

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.