Figma·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Apr 2026

Summary

Figma ML engineer interview with a meaty system design question around building a text-to-HTML generation service. Covered a lot of ground, from model training to serving infrastructure, and I felt pretty good about some parts and totally lost on others.

Questions Asked (1)

Design a service that takes a natural language description of a webpage and generates the corresponding HTML (and optionally CSS and JS). Walk through the full system: model choice and training, alignment, evaluation, and serving.

System DesignTechnical Trade-offsProduct Sense & Ideation

Author's notes

This one had a lot of layers.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and constraints, then walk through the full ML lifecycle: data collection, model choice (likely a fine-tuned LLM or multimodal model), training, alignment, evaluation, and serving. Emphasize trade-offs between quality, latency, and cost, and tie decisions back to Figma's design-centric user needs.

Pro tip: Anchor your answer in a concrete user story (e.g., 'a designer types a description and gets editable HTML/CSS in Figma') and consistently refer back to it to show product sense and prioritization.

1. Clarify Requirements and Scope

Ask about expected input/output formats, target users, quality metrics, latency constraints, and whether the output must be editable or production-ready. Define success criteria and constraints early.

2. Data and Model Selection

Discuss data sources (web scraping, synthetic generation, human annotations), preprocessing, and model choices (e.g., fine-tuned LLM like Codex, or multimodal model). Justify trade-offs between model size, quality, and inference cost.

3. Training and Alignment

Outline training pipeline: pretraining/fine-tuning on HTML/CSS/JS corpora, then alignment via RLHF or DPO to match human preferences for aesthetics, correctness, and editability. Mention safety and bias mitigation.

4. Evaluation Strategy

Define offline metrics (e.g., BLEU, exact match, rendering similarity) and online metrics (user engagement, edit rate). Include human evaluation for visual fidelity and usability, and A/B testing for iterative improvement.

5. Serving and Deployment

Design serving architecture: model quantization, caching, batching, and fallback strategies. Discuss latency vs. quality trade-offs, monitoring, and continuous learning from user feedback.

Key Points to Mention

Choice of model architecture (e.g., transformer-based LLM, multimodal) and justification based on task complexity and Figma's ecosystem.
Data collection and annotation challenges, including synthetic data generation and handling of diverse web designs.
Alignment techniques like RLHF/DPO to ensure outputs are not only syntactically correct but also visually appealing and editable.
Evaluation metrics that combine automatic (e.g., rendering similarity, code validity) and human judgment (e.g., designer preference).
Serving considerations: latency, cost, scalability, and integration with Figma's existing infrastructure.
Safety and ethical concerns: avoiding malicious code generation, respecting copyright, and ensuring accessibility.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.