Google·Product Manager·Onsite - Product Sense / Strategy·Senior

Senior

Apr 2026

Summary

Google PM interview with a classic infrastructure/product trade-off question. Short and focused, one question that sounds deceptively simple but has a lot of layers depending on how deep you go.

Questions Asked (1)

If you're building an app that needs to serve millions of users, would you run the ML model on the server or on the user's device? Walk through your reasoning.

Technical Trade-offsSystem DesignProduct Strategy

Author's notes

I started with latency and cost tradeoffs, which felt right, but I kept second-guessing myself mid-answer.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clarifying the product context and key requirements, then compare on-device vs. server-side ML across dimensions like latency, cost, privacy, and scalability. Conclude with a hybrid recommendation that balances trade-offs and aligns with business goals.

Pro tip: Acknowledge that the decision isn't binary—many successful products use a hybrid approach, such as running a lightweight model on-device for real-time inference and a heavier model on the server for complex tasks. This shows you understand practical constraints and can optimize for both user experience and cost.

1. Clarify Requirements and Constraints

Ask questions to understand the app's use case, latency needs, privacy requirements, and user base characteristics (e.g., device diversity, network conditions). This ensures your recommendation is grounded in the specific context.

2. Evaluate On-Device ML

Discuss pros like low latency, offline functionality, and privacy, and cons like limited compute, model size constraints, and update challenges. Consider if the model can be compressed and if the device ecosystem supports it.

3. Evaluate Server-Side ML

Discuss pros like scalability, easier updates, and access to powerful hardware, and cons like network latency, bandwidth costs, and privacy concerns. Consider if the app can tolerate round-trip delays and if data transfer is acceptable.

4. Compare Trade-offs and Consider Hybrid

Weigh the factors from steps 2 and 3 against the requirements. Propose a hybrid solution if appropriate, such as on-device for real-time features and server-side for heavy computation or personalization.

5. Make a Recommendation and Justify

State your recommendation clearly, explaining how it aligns with user needs and business goals. Mention potential risks and mitigation strategies, and suggest metrics to monitor success.

Key Points to Mention

Latency and real-time responsiveness: on-device avoids network delays, crucial for interactive features.
Cost and scalability: server-side can be more cost-effective for complex models but incurs bandwidth and infrastructure costs; on-device shifts compute to users.
Privacy and security: on-device keeps sensitive data local, reducing compliance risks; server-side requires data transfer and storage safeguards.
Model updates and maintenance: server-side allows seamless updates; on-device requires app releases or over-the-air updates, which can be slow.
Device heterogeneity: on-device must handle diverse hardware capabilities; server-side provides consistent performance but may exclude offline users.
Hybrid approaches: combining both can optimize for latency, cost, and privacy, e.g., on-device for inference and server-side for training or heavy lifting.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.