Google·Software Engineer·Onsite - System Design / Architecture·Senior

SeniorPrefer not to say

Apr 2026

Summary

Google SWE system design round, one question about building a visual landmark recognition system. Pretty open-ended and I spent a lot of time second-guessing my architecture choices.

Questions Asked (1)

Design a system that can identify real-world landmarks from images or photos submitted by users.

System DesignTechnical Trade-offsAPI & Integrations

Author's notes

I jumped straight into the ML pipeline without scoping the problem first, which I think hurt me.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying functional and non-functional requirements, then sketch a high-level architecture that separates the image ingestion pipeline, the landmark recognition service, and the metadata/result storage. Dive into the ML model choice (e.g., a CNN-based classifier or embedding-based retrieval), scalability, and trade-offs around accuracy, latency, and cost.

Pro tip: Emphasize that landmark recognition is a retrieval problem, not just classification—using embeddings and approximate nearest neighbor search (e.g., ScaNN) can handle millions of landmarks and new additions without retraining. Also, mention the importance of handling false positives with a confidence threshold and fallback to reverse image search or human review.

1. Clarify Requirements

Ask about scale (QPS, number of landmarks), latency targets, accuracy expectations, and whether the system should handle user-submitted photos in real-time or batch. Also clarify if the system needs to identify landmarks globally or in specific regions.

2. High-Level Architecture

Outline components: an API gateway for image upload, a preprocessing service (resize, normalize), a recognition service (ML model + landmark database), a metadata store (landmark info), and a caching layer for popular landmarks. Consider asynchronous processing for large images.

3. Recognition Approach

Choose between classification (fixed set of landmarks) and retrieval (embedding + ANN search). For scalability and flexibility, recommend a two-stage approach: first, a lightweight model to detect if the image contains a landmark; second, generate embeddings and search a vector database for nearest neighbors.

4. Scalability & Trade-offs

Discuss scaling the recognition service horizontally, using GPU/TPU for inference, and sharding the landmark database. Trade-offs: accuracy vs. latency (larger models slower), cost of GPUs vs. using precomputed embeddings, and freshness of landmark data (retraining vs. adding new embeddings).

5. Data & Feedback Loop

Explain how to handle new landmarks: allow adding new embeddings without full retraining. Incorporate user feedback (e.g., 'not this landmark') to improve the model and update the database. Also mention monitoring for drift and accuracy metrics.

Key Points to Mention

Use of pre-trained CNN models (e.g., ResNet, EfficientNet) for feature extraction and embedding generation.
Approximate Nearest Neighbor (ANN) search with libraries like ScaNN or FAISS for efficient retrieval at scale.
Handling of false positives: confidence threshold, fallback to reverse image search, or human-in-the-loop.
Caching frequently requested landmarks to reduce latency and load on the recognition service.
Data pipeline for ingesting and indexing new landmarks, including metadata (name, location, description).
Monitoring and A/B testing to measure accuracy and user satisfaction, with feedback loops for continuous improvement.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.