← Runway Interview Insights

Runway·Machine Learning Engineer·Onsite - System Design / Architecture·Senior

Senior
Apr 2026

Summary

Runway MLE interview that went deep on video search infrastructure. One long design question that covered basically everything from model architecture to abuse detection. Pretty intense for a single session.

Questions Asked (1)

Q1

Design a video search system from scratch that supports both text queries and video queries, covering model architecture, embedding space, retrieval infrastructure, and scalability to billions of clips.

System DesignTechnical Trade-offsA/B Testing & Experimentation
Author's notes

This one was massive.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying requirements and scale, then propose a dual-encoder architecture that maps text and video into a shared embedding space, followed by a two-stage retrieval pipeline (ANN search + re-ranking). Discuss infrastructure choices for billion-scale indexing, latency, and how to evaluate and iterate with A/B tests.

Pro tip: Emphasize that the embedding space must be trained with contrastive losses on large-scale video-text pairs, and that hard negative mining and modality gap reduction are critical for cross-modal retrieval quality. Also, mention that you'd start with a simpler baseline (e.g., CLIP-style) and iterate based on offline metrics before scaling.

1. Clarify Requirements and Constraints

Ask about query types (text, video, image), latency targets, scale (billions of clips), and update frequency. Define success metrics like recall@k, mAP, and user engagement.

2. Design Model Architecture

Propose a dual-encoder (two-tower) model: one for text and one for video, mapping to a shared embedding space. Use contrastive learning with hard negatives and consider temporal modeling for video.

3. Plan Retrieval Infrastructure

Use a two-stage retrieval: approximate nearest neighbor (ANN) search (e.g., FAISS, ScaNN) for candidate generation, then a cross-encoder re-ranker for top-k. Discuss sharding, quantization, and distributed indexing for billion-scale.

4. Address Scalability and Efficiency

Cover embedding generation at scale (batch processing, GPU inference), index updates (incremental vs. full rebuild), and latency optimization (caching, tiered storage).

5. Evaluate and Iterate with A/B Testing

Define offline metrics (recall, precision) and online A/B tests (CTR, watch time). Discuss how to handle cold-start and feedback loops, and how to monitor model drift.

Key Points to Mention

  • Dual-encoder architecture with shared embedding space and contrastive loss (e.g., InfoNCE).
  • Hard negative mining and modality gap reduction techniques (e.g., temperature scaling, projection heads).
  • Two-stage retrieval: ANN (HNSW, IVF-PQ) for efficiency and cross-encoder for precision.
  • Scalability considerations: sharding, quantization (PQ, OPQ), and distributed systems (e.g., FAISS with GPU).
  • Evaluation metrics: offline (recall@k, mAP) and online (A/B tests on CTR, engagement).
  • Handling video queries: temporal aggregation (e.g., mean pooling, attention) and query expansion.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.