Start by clearly stating the null and alternative hypotheses, then compute the pooled proportion and standard error to calculate the z-statistic and p-value. Finally, compute the 95% confidence interval for the difference in proportions (lift) and interpret the results in the context of the business decision.
Pro tip: Always check the assumptions for the two-proportion z-test (independent samples, random assignment, and sufficient sample size) and mention them briefly to show rigor. Also, interpret the confidence interval in terms of practical significance, not just statistical significance.
Define null hypothesis H0: pA = pB (or pB - pA = 0) and alternative hypothesis H1: pB > pA (one-sided) or pB ≠ pA (two-sided). Since the question asks if B is significantly better, use a one-sided test.
Calculate the pooled proportion p̂ = (700+800)/(1000+1000) = 0.75. Then compute the standard error SE = sqrt(p̂(1-p̂)(1/nA + 1/nB)) = sqrt(0.75*0.25*(0.001+0.001)) = sqrt(0.000375) ≈ 0.01936. The z-statistic is (0.80 - 0.70)/SE = 0.10/0.01936 ≈ 5.16.
For a one-sided test, p-value = P(Z > 5.16) ≈ 0.00000012 (essentially 0). Since p-value < 0.05, reject H0. Conclude Model B is significantly better.
Calculate the 95% confidence interval for the difference in proportions (pB - pA). Use unpooled SE for CI: SE_diff = sqrt(pA(1-pA)/nA + pB(1-pB)/nB) = sqrt(0.7*0.3/1000 + 0.8*0.2/1000) = sqrt(0.00021 + 0.00016) = sqrt(0.00037) ≈ 0.01924. The 95% CI is (0.10 ± 1.96*0.01924) = (0.10 ± 0.0377) = (0.0623, 0.1377). So, we are 95% confident the lift is between 6.23% and 13.77%.
Explain that the result is statistically significant and the confidence interval suggests a positive lift. Discuss practical significance: a 10% absolute lift (from 70% to 80%) is substantial and likely meaningful for the business.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.