← OneMain Financial Interview Insights
I knew the formula going in but fumbled the plug-in.
Start by stating the two-proportion z-test sample size formula, then plug in the given baseline (p1=0.10) and target (p2=0.11) with standard assumptions (α=0.05 two-sided, power=80%). Compute the required sample size per arm, showing intermediate calculations and final rounded number.
Pro tip: Mention that this is a simplified calculation and that in practice you'd use simulation or tools like power.prop.test in R, and consider continuity correction or unequal allocation if needed.
Present the standard sample size formula for comparing two proportions: n = (Z_{α/2} + Z_β)^2 * (p1(1-p1) + p2(1-p2)) / (p2 - p1)^2.
Set p1 = 0.10 (baseline), p2 = 0.11 (target), α = 0.05 (two-sided), power = 0.80. Thus Z_{α/2} = 1.96, Z_β = 0.84.
Calculate p1(1-p1) = 0.09, p2(1-p2) = 0.0979, sum = 0.1879. Difference (p2-p1) = 0.01, squared = 0.0001. Sum of Z's = 2.80, squared = 7.84.
Multiply: n = 7.84 * 0.1879 / 0.0001 = 1473.136. Round up to 1474 per arm.
State that approximately 1,474 users per arm are needed. Mention that this assumes no continuity correction, equal allocation, and that the test is two-sided.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Bonferroni is straightforward, alpha divided by number of tests, so 0.05 / 10 = 0.005.
First, state the Bonferroni-adjusted significance level as α/10 (e.g., 0.05/10 = 0.005). Then explain that this stricter threshold reduces power, so you must either increase sample size or accept a larger minimum detectable effect, and quantify the trade-off using the standard sample size formula for two proportions.
Pro tip: Mention that Bonferroni is conservative and can be overly punitive with correlated metrics; suggest alternatives like Holm-Bonferroni or false discovery rate (FDR) control if the business context allows, showing you understand practical trade-offs beyond textbook formulas.
Calculate the Bonferroni-corrected alpha as the original alpha divided by the number of metrics (e.g., 0.05/10 = 0.005). Clearly explain that this controls the family-wise error rate at 5%.
A smaller alpha reduces statistical power for a fixed sample size. To maintain the same power (e.g., 80%), the required sample size per variant increases, typically by a factor related to the ratio of z-scores squared.
Use the sample size formula for two proportions: n = 2*(z_{α/2} + z_β)^2 * p(1-p) / Δ^2. Show that z_{α/2} increases from ~1.96 to ~2.81 for α=0.005, leading to a larger n.
Acknowledge that the required sample size may be infeasible. Suggest prioritizing a smaller set of primary metrics, using sequential testing, or applying less conservative corrections like Holm or FDR.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
I had studied this but my explanation came out muddled.
Start by acknowledging the multiple testing problem: peeking daily at 14 days inflates Type I error. Then describe a sequential testing approach like O'Brien-Fleming or Pocock alpha-spending, explaining how it adjusts the stopping boundary at each interim look to control the overall false positive rate. Finally, discuss the trade-offs between the two methods in terms of early stopping and power.
Pro tip: Mention that O'Brien-Fleming is conservative early and more permissive later, making it ideal when early results are unlikely to be definitive, while Pocock maintains a constant boundary and is better for detecting large effects early. Also note that these methods require pre-specification of the number and timing of looks.
Explain that daily peeking over 14 days means 14 interim analyses, which increases the chance of a false positive if no adjustment is made.
Describe alpha-spending as a method to allocate the overall Type I error rate across interim looks, ensuring the total alpha remains at the desired level (e.g., 0.05).
Explain that O'Brien-Fleming uses very stringent boundaries early (e.g., p < 0.0001) and relaxes them later, while Pocock uses constant boundaries (e.g., p < 0.02 at each look).
Detail how the stopping boundary changes: for O'Brien-Fleming, the critical Z-value decreases over time; for Pocock, it remains relatively constant.
Mention that O'Brien-Fleming is better for early stopping when effects are large, but Pocock may stop earlier for moderate effects; both control Type I error but affect power and sample size.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
This was the part I actually felt okay about.
Start by defining the specific test context (e.g., A/B test on loan application conversion) and the practical consequences of a Type II error (missing a true improvement). Then propose a concrete method to reduce beta without increasing alpha, such as increasing sample size or reducing variance, and quantify the trade-off in terms of required sample size or test duration.
Pro tip: In financial services, always tie the cost of a Type II error to business metrics like lost revenue or missed opportunities, and mention that reducing beta often requires more traffic or time, which has its own cost. Quantify the trade-off using power analysis to show you understand the resource implications.
Specify the A/B test scenario (e.g., testing a new loan offer page) and what a Type II error means: failing to detect a true increase in conversion. State the practical cost, such as foregone revenue from not implementing the better version.
Choose one concrete approach: increasing sample size, reducing variance (e.g., via stratification or blocking), or using a more sensitive metric. Explain how it reduces beta without changing alpha.
Use power analysis to show the relationship: e.g., to reduce beta from 0.20 to 0.10 while keeping alpha at 0.05, the required sample size increases by a certain factor (often ~1.33x for a one-sided test). Mention the cost in terms of additional traffic or test duration.
Connect the trade-off to business decisions: e.g., the extra sample size may delay the test by a week, but the cost of missing a 1% lift in conversion could be $X in lost revenue. Show that the trade-off is worthwhile if the expected value of detecting the effect exceeds the cost of additional data collection.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.