← Openai Interview Insights

Openai·Machine Learning Engineer·Technical Phone Screen·Senior

SeniorPrefer not to say
May 2026

Summary

Technical screen for an ML Engineer role at OpenAI, heavy on NumPy internals and numerical computing. The whole thing felt like a deep dive into whether you actually understand what's happening under the hood, not just whether you can call the right functions.

Questions Asked (3)

Q1

Write vectorized NumPy code to compute pairwise cosine similarity between two matrices X (shape n×d) and Y (shape m×d) without using any Python loops. Analyze the time and space complexity of your solution.

Algorithms & Data StructuresTechnical Trade-offs
Author's notes

I knew the formula cold but fumbled the normalization step at first.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

First, explain the mathematical formulation of cosine similarity using dot products and norms. Then, present vectorized NumPy code that computes the dot product matrix and normalizes by the outer product of norms. Finally, analyze time and space complexity, highlighting the O(n*m*d) time and O(n*m) space.

Pro tip: Mention that for large matrices, you can avoid the full n×m matrix by using a chunked or approximate approach, but for exact computation, the vectorized method is optimal. Also, note that normalizing rows first can simplify the code and improve numerical stability.

1. Mathematical Formulation

Write the cosine similarity formula: sim(i,j) = (X_i · Y_j) / (||X_i|| * ||Y_j||). Explain that this can be computed as a matrix product followed by normalization.

2. Vectorized Implementation

Provide NumPy code: compute dot product matrix using X @ Y.T, compute norms using np.linalg.norm along axis=1, then divide by outer product of norms. Ensure no Python loops.

3. Complexity Analysis

Analyze time complexity: O(n*m*d) for matrix multiplication. Space complexity: O(n*m) for the similarity matrix. Mention that norms take O(n*d + m*d) time and O(n+m) space.

4. Edge Cases and Optimizations

Discuss handling zero norms (add epsilon), and potential memory optimizations like chunking or using float32. Mention that if n or m is huge, the O(n*m) matrix may be prohibitive.

Key Points to Mention

  • Cosine similarity formula and its matrix form
  • Use of broadcasting for normalization
  • Time complexity O(n*m*d) and space complexity O(n*m)
  • Numerical stability: adding epsilon to norms
  • Memory trade-offs and chunking for large matrices
  • Avoiding explicit loops by leveraging NumPy's vectorized operations

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q2

Implement a numerically stable softmax for a 2D array along the last axis using NumPy. Why does numerical stability matter here and what pitfall does your implementation avoid?

Technical Trade-offsAlgorithms & Data Structures
Author's notes

The subtract-the-max trick is pretty well known so I got through this one without much drama.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by explaining the mathematical definition of softmax and why the naive implementation can overflow. Then describe the standard stabilization trick: subtract the maximum value along the last axis before exponentiation, and show how to implement it efficiently in NumPy using broadcasting. Finally, discuss the pitfalls avoided, such as overflow and underflow, and mention any edge cases like all -inf inputs.

Pro tip: Mention that in practice, you might also clip the logits or use a log-softmax for numerical stability in loss functions, and that frameworks like PyTorch have built-in stable implementations. This shows awareness of real-world ML engineering.

1. Define softmax and its purpose

Briefly state that softmax converts logits to probabilities by exponentiating and normalizing. Emphasize that it's used in classification and attention mechanisms.

2. Identify numerical instability

Explain that large logits cause overflow in exp, leading to inf or nan, and small logits cause underflow, leading to zero probabilities and division by zero.

3. Describe the stabilization trick

Subtract the maximum value along the last axis before exponentiation: exp(x - max(x)) / sum(exp(x - max(x))). This keeps the exponent arguments <= 0, preventing overflow.

4. Implement in NumPy

Use np.max with keepdims=True to maintain shape for broadcasting, then np.exp and np.sum along the last axis with keepdims=True for normalization.

5. Discuss pitfalls and edge cases

Mention that subtracting the max avoids overflow and underflow, but if all inputs are -inf, the result is nan; handle by returning uniform distribution or using a small epsilon.

Key Points to Mention

  • Overflow and underflow in exponentiation
  • Subtracting the maximum for numerical stability
  • Broadcasting with keepdims=True in NumPy
  • Division by zero when all exponentiated values underflow
  • Handling -inf inputs (e.g., masked softmax)
  • Efficiency: vectorized operations and avoiding loops

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.

Q3

If X has shape (n, 1, d) and Y has shape (1, m, d), explain how NumPy broadcasting works when you compute operations between them. What is the resulting shape and what are the memory implications?

System DesignTechnical Trade-offs
Author's notes

Honestly my weakest moment in the interview.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by explaining the broadcasting rules: align shapes from the right, and dimensions of size 1 are stretched to match the other. Then apply these rules to X (n,1,d) and Y (1,m,d) to determine the resulting shape (n,m,d). Finally, discuss memory implications, emphasizing that broadcasting avoids copying data but can lead to large intermediate arrays if not careful.

Pro tip: Mention that while broadcasting is memory-efficient in terms of not replicating the original arrays, the resulting array can be large, and using in-place operations or libraries like NumPy's einsum can mitigate memory overhead.

1. State broadcasting rules

Explain that broadcasting aligns shapes from the right, and dimensions of size 1 are expanded to match the other array's dimension.

2. Apply rules to X and Y

Show that X (n,1,d) and Y (1,m,d) broadcast to (n,m,d) by expanding the second dimension of X and the first dimension of Y.

3. Determine resulting shape

Conclude that the result has shape (n, m, d).

4. Discuss memory implications

Explain that broadcasting does not copy the original arrays, but the operation may create a large output array of size n*m*d, which can be memory-intensive.

5. Suggest optimizations

Mention alternatives like using einsum, chunking, or in-place operations to reduce memory usage when n and m are large.

Key Points to Mention

  • Broadcasting rules: align from right, expand size-1 dimensions.
  • Resulting shape: (n, m, d).
  • No copying of original arrays; broadcasting is a view-like mechanism.
  • Output array size can be large (n*m*d), leading to high memory usage.
  • Potential optimizations: einsum, chunking, in-place operations.
  • Example: X[:, None, :] * Y[None, :, :] yields (n, m, d).

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.