Knew this cold in theory but fumbled the delivery a bit.
Start by defining both tests and their core difference: the Z-test uses the known population standard deviation, while the t-test estimates it from the sample and uses the t-distribution. Then explain the decision rule based on sample size and whether the population variance is known, and finally connect it to A/B testing at Google, emphasizing practical implications for experiment analysis.
Pro tip: Mention that in practice, with large samples (e.g., n > 30 or 50), the t-test and Z-test yield nearly identical results, so many tech companies default to t-tests for simplicity and robustness—this shows you understand real-world experimentation beyond textbook theory.
Explain that a Z-test is used when the population standard deviation is known and the sample size is large, relying on the standard normal distribution.
Explain that a t-test is used when the population standard deviation is unknown and estimated from the sample, especially with small sample sizes, using the t-distribution with heavier tails.
Highlight that the t-distribution approaches the normal distribution as sample size increases, and that the t-test is more conservative for small samples due to added uncertainty.
State that Z-tests are appropriate when population variance is known and n is large; t-tests are used when variance is unknown, regardless of sample size, but especially for small n.
Connect to Google's context: in A/B tests, population variance is rarely known, so t-tests (or variations like Welch's t-test) are standard, but with large samples the choice often doesn't matter.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.