Lyft·AI Engineer·Technical Phone Screen·Senior

Senior

Jun 2026

Summary

Lyft AI Engineer technical screen, pretty deep on ML fundamentals. One question but it had a lot of surface area and they clearly wanted more than a textbook answer.

Questions Asked (1)

Walk me through the difference between a general-purpose transformer model (like GPT or BERT) and an embedding model (like sentence-BERT or OpenAI's text-embedding series). Cover what each is trained to do, the training objectives involved, what the outputs actually represent, typical use cases for each, and when you'd pick one over the other in a real system like RAG.

System DesignTechnical Trade-offs

Author's notes

This looked like a single question but it was really five questions jammed together.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining the core distinction: general-purpose transformers are trained for generation or understanding tasks, while embedding models are trained to produce fixed-size vector representations for semantic similarity. Then compare their training objectives, output representations, and use cases, and finally explain how they complement each other in a RAG system.

Pro tip: Emphasize that embedding models are often fine-tuned from general-purpose transformers, and in production RAG systems, you typically use both: an embedding model for retrieval and a general-purpose model for generation. Mention that the choice depends on latency, cost, and whether you need generation or just similarity.

1. Define the two model types

Clarify that general-purpose transformers (e.g., GPT, BERT) are large language models trained on broad text corpora for generation or masked language modeling, while embedding models (e.g., Sentence-BERT, text-embedding-ada-002) are specialized to map text to dense vectors for similarity comparison.

2. Compare training objectives

Explain that GPT is trained with causal language modeling (predict next token), BERT with masked language modeling and next sentence prediction, whereas embedding models are trained with contrastive learning (e.g., triplet loss, multiple negatives ranking loss) to pull similar texts together and push dissimilar ones apart.

3. Describe output representations

Highlight that general-purpose transformers output token-level representations or generated text, while embedding models output a single fixed-dimensional vector (e.g., 768 or 1536 dimensions) that captures semantic meaning of the entire input.

4. Discuss typical use cases

Mention that general-purpose models are used for text generation, summarization, question answering, and classification, while embedding models are used for semantic search, clustering, recommendation, and retrieval-augmented generation (RAG).

5. Explain selection criteria in RAG

In RAG, use an embedding model to index and retrieve relevant documents based on semantic similarity, and a general-purpose model to generate the final answer from the retrieved context. Choose based on whether the task requires generation (general-purpose) or efficient similarity search (embedding).

Key Points to Mention

Training objectives: causal LM vs. masked LM vs. contrastive learning
Output types: token-level vs. sentence-level fixed vectors
Use cases: generation/understanding vs. semantic similarity/search
RAG architecture: embedding model for retrieval, general-purpose model for generation
Trade-offs: embedding models are smaller, faster, and cheaper for retrieval; general-purpose models are larger and more expensive but can generate text
Fine-tuning: embedding models can be fine-tuned from general-purpose transformers for domain-specific retrieval

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.