Adobe·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

May 2026

Summary

Adobe ML Engineer onsite focused entirely on one massive system design question about building an AI assistant for their Experience Platform. No coding, no behavioral, just two hours of whiteboarding an end-to-end RAG architecture with some very pointed follow-ups.

Questions Asked (1)

Design a natural-language assistant for Adobe Experience Platform that can both answer questions about product documentation and answer questions about a customer's own tenant data (like dataset counts or audience sizes). Walk through the full architecture, security model, guardrails, latency targets, privacy handling, and evaluation strategy.

System DesignTechnical Trade-offsAPI & Integrations

Author's notes

This is a monster of a question and I did not scope it fast enough.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and constraints, then present a layered architecture that separates the two data domains (public docs vs. tenant data) with a router and distinct retrieval pipelines. Emphasize security, privacy, and evaluation as first-class concerns, and discuss trade-offs between latency, accuracy, and cost.

Pro tip: Explicitly call out the need for a permission-aware retrieval layer that enforces row-level security at query time, and mention how you would handle stale or cached tenant data to avoid privacy leaks.

1. Clarify Requirements and Constraints

Ask about expected query volume, latency SLOs, data freshness, tenant isolation requirements, and compliance needs (e.g., GDPR, CCPA). Confirm whether the assistant must support multi-turn conversations and which Adobe Experience Platform APIs are available.

2. Design the High-Level Architecture

Propose a modular system with an intent classifier/router that directs queries to either a documentation QA pipeline (RAG over public docs) or a tenant data QA pipeline (text-to-API/SQL over tenant-specific data). Include components like query understanding, retrieval, generation, and response post-processing.

3. Detail Security, Privacy, and Guardrails

Describe authentication/authorization (OAuth, IMS), tenant isolation, encryption in transit/at rest, and permission-aware retrieval. Outline guardrails such as PII redaction, output filtering, rate limiting, and audit logging.

4. Define Latency Targets and Optimization Strategies

Set latency budgets for each pipeline (e.g., p95 < 2s for docs, < 5s for tenant data) and discuss caching, pre-computation, streaming, and model selection (small vs. large LLMs) to meet them.

5. Outline Evaluation and Monitoring

Propose offline metrics (retrieval precision/recall, answer correctness, faithfulness) and online metrics (user feedback, task success, latency). Include A/B testing, canary deployments, and continuous monitoring for drift and security incidents.

Key Points to Mention

Retrieval-Augmented Generation (RAG) for documentation QA, with vector search and re-ranking.
Text-to-API or text-to-SQL for tenant data, with schema awareness and query validation.
Permission-aware retrieval and row-level security to enforce tenant isolation.
PII detection and redaction, plus output filtering to prevent data leaks.
Latency optimization via caching, streaming, and model distillation.
Evaluation framework combining automated metrics and human-in-the-loop feedback.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.