← Point72 Asset Management Interview Insights

Point72 Asset Management·Machine Learning Engineer·Technical Phone Screen·Intermediate

Intermediate
Jun 2026

Summary

Technical phone screen for an ML Engineer role at Point72. Just one question from what I can tell, but it went deep enough that I felt it.

Questions Asked (1)

Q1

What is the key difference between an encoder and a decoder in transformer architectures?

Technical Trade-offsSystem Design
Author's notes

The interviewer seemed pretty satisfied once I brought up the causal mask.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by defining the core architectural difference: encoders use bidirectional self-attention to build rich contextual representations, while decoders use causal (masked) self-attention to generate sequences autoregressively. Then explain how this difference drives their distinct roles in tasks like understanding vs. generation, and mention cross-attention when they are combined in sequence-to-sequence models.

Pro tip: Tie the architectural difference to practical trade-offs in latency, memory, and training objective, and note that many modern models (e.g., BERT, GPT) are specialized for one or the other, while T5 and BART combine both for seq2seq tasks.

1. Define the encoder

Explain that an encoder processes the entire input sequence at once with bidirectional self-attention, allowing each token to attend to all others. Its output is a set of contextualized representations, not a generated sequence.

2. Define the decoder

Explain that a decoder generates output tokens one at a time, using masked (causal) self-attention to prevent attending to future tokens. It can also use cross-attention to attend to encoder outputs in seq2seq models.

3. Highlight the key difference

State that the fundamental difference is the attention mask: encoders are bidirectional, decoders are unidirectional (causal). This dictates whether the model is suited for understanding tasks or generation tasks.

4. Connect to use cases and trade-offs

Discuss how encoders excel at tasks like classification, sentiment analysis, and masked language modeling, while decoders excel at generation, such as text completion and translation. Mention that decoders are slower at inference due to sequential generation.

5. Mention combined architectures

Note that encoder-decoder models (e.g., T5, BART) use both: the encoder processes the input, and the decoder generates the output while attending to the encoder via cross-attention. This is common in sequence-to-sequence tasks.

Key Points to Mention

  • Bidirectional vs. causal (masked) self-attention
  • Encoder outputs contextualized representations; decoder generates tokens autoregressively
  • Cross-attention in encoder-decoder models
  • Training objectives: masked language modeling (encoder) vs. next-token prediction (decoder)
  • Inference speed: encoders parallelize, decoders are sequential
  • Examples: BERT (encoder-only), GPT (decoder-only), T5/BART (encoder-decoder)

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.