OneMain Financial·Data Scientist·Technical Phone Screen·Intermediate

Intermediate

May 2026

Summary

Technical screen for a Data Scientist role at OneMain Financial where they walked through a past modeling project and then pivoted into stats fundamentals. Pretty conversational but they definitely wanted to see if you actually understood the theory behind the work you claimed to have done.

Questions Asked (1)

What are Type I and Type II errors in hypothesis testing? Walk through the difference and give a real-world example of each. Then explain how you'd think about balancing the two in an A/B test.

A/B Testing & ExperimentationProduct Analytics & Metrics

Author's notes

I started with the textbook definitions, false positive vs false negative, which was fine.

Create a free account to read the full note

AI HintsAI Generated

Suggested Approach

Start by clearly defining Type I (false positive) and Type II (false negative) errors, then contrast them with a concrete example for each. Next, discuss the trade-off between them in A/B testing, emphasizing how to balance based on business context and costs. Conclude with a practical framework for setting significance level, power, and sample size.

Pro tip: In A/B testing, always consider the business impact of each error type—sometimes a Type II error (missing a true effect) can be more costly than a Type I error, so adjust your thresholds accordingly. Also, mention that increasing sample size reduces both errors simultaneously, but there's a trade-off with time and resources.

1. Define Type I and Type II Errors

Clearly state that Type I error (α) is rejecting a true null hypothesis (false positive), while Type II error (β) is failing to reject a false null hypothesis (false negative). Mention that power is 1-β.

2. Provide Real-World Examples

Give a concrete example for each: e.g., Type I: A drug trial concludes a drug works when it doesn't; Type II: A trial concludes a drug doesn't work when it actually does. For A/B testing: Type I: Declaring a new feature improves conversion when it doesn't; Type II: Missing a true improvement.

3. Explain the Trade-off and Balancing in A/B Tests

Discuss how α and β are inversely related for a fixed sample size. In A/B testing, balance depends on business costs: if false positives are costly (e.g., rolling out a bad feature), lower α; if false negatives are costly (e.g., missing a revenue boost), increase power (lower β).

4. Outline Practical Steps for Balancing

Describe how you'd set significance level (α) typically at 0.05, power at 0.8, and calculate required sample size. Mention using sequential testing or Bayesian methods to adaptively balance errors, and considering multiple testing corrections.

5. Conclude with Business Context

Emphasize that the optimal balance is not purely statistical but driven by the relative costs of each error and the company's risk tolerance. For OneMain Financial, consider regulatory and financial impacts of false positives vs. false negatives.

Key Points to Mention

Definitions of Type I (α) and Type II (β) errors, and power (1-β).
Real-world examples: medical trials, spam filtering, or A/B testing scenarios.
Trade-off: decreasing α increases β for a fixed sample size.
Factors influencing balance: cost of false positive vs. false negative, business impact, risk tolerance.
Methods to control errors: sample size calculation, significance level, power analysis, sequential testing, Bayesian approaches.
Practical A/B testing considerations: minimum detectable effect, duration, multiple comparisons, and guardrail metrics.

AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.