I knew the punchline intuitively but fumbled the algebra mid-explanation.
First, set up the algebra by writing the OLS estimator for the duplicated dataset and show that the coefficients are unchanged because the normal equations scale identically. Then derive the variance of the estimator, showing it is divided by k, so standard errors shrink by 1/√k. Finally, explain the downstream effects: t-statistics inflate by √k, p-values decrease, and confidence intervals narrow, but emphasize that this is an artifact of artificially inflating sample size without new information.
Pro tip: Connect this to real-world implications: duplicating data is like p-hacking because it artificially shrinks standard errors and inflates significance without adding information. Mention that in practice, you should never duplicate data to boost significance; instead, focus on collecting more data or using proper statistical methods.
Write the OLS estimator β̂ = (XᵀX)⁻¹Xᵀy for the original data. For the duplicated dataset, the design matrix becomes X_dup = [X; X; ...; X] (k times) and similarly for y. Show that X_dupᵀX_dup = k XᵀX and X_dupᵀy_dup = k Xᵀy, so β̂_dup = (k XᵀX)⁻¹ (k Xᵀy) = β̂.
Under the Gauss-Markov assumptions, Var(β̂) = σ² (XᵀX)⁻¹. For the duplicated data, Var(β̂_dup) = σ² (X_dupᵀX_dup)⁻¹ = σ² (k XᵀX)⁻¹ = (1/k) σ² (XᵀX)⁻¹ = (1/k) Var(β̂). Thus, standard errors are scaled by 1/√k.
The t-statistic for a coefficient is β̂_j / SE(β̂_j). Since β̂_j is unchanged and SE(β̂_j) is divided by √k, the t-statistic is multiplied by √k. Consequently, p-values decrease (become more significant) because the t-distribution with more degrees of freedom (since n increases) and larger t-statistic yields smaller tail probabilities.
A confidence interval for β_j is β̂_j ± t_{α/2, df} * SE(β̂_j). The standard error shrinks by 1/√k, and the critical value t_{α/2, df} also changes slightly due to increased degrees of freedom (df = k*n - p). However, for large n, the critical value is approximately unchanged, so the interval width shrinks roughly by 1/√k.
Conclude that duplicating data artificially inflates significance without adding new information, leading to overconfidence in estimates. This is a form of data leakage or p-hacking. In practice, one should not duplicate data; instead, collect more data or use appropriate methods like bootstrapping or regularization.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
First, recognize that the original t-statistic is 5.0 = β/0.20, so β = 1.0. Then, treating 4 duplicates as independent inflates the sample size by a factor of 4, which incorrectly reduces the standard error by sqrt(4) = 2, giving a new SE of 0.10. Finally, compute the new t-statistic as 1.0/0.10 = 10.0 and find the two-sided p-value for t=10 with the (incorrectly) inflated degrees of freedom, which is approximately <0.0001 (or 2e-23 for large df).
Pro tip: Emphasize that this mistake artificially inflates significance by ignoring the correlation among duplicates, leading to false positives. Always check for and account for clustering or repeated measures in A/B tests.
Use the given original SE and t-statistic to back out the coefficient: β = t * SE = 5.0 * 0.20 = 1.0.
Treating 4 duplicates as independent multiplies the effective sample size by 4, so the standard error is scaled by 1/sqrt(4) = 1/2.
New SE = original SE / sqrt(4) = 0.20 / 2 = 0.10.
New t = β / new SE = 1.0 / 0.10 = 10.0.
For t=10 with large degrees of freedom (due to inflated n), the two-sided p-value is extremely small, roughly <0.0001 (e.g., 2e-23). Use a t-distribution or normal approximation.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Tripped up a bit connecting bootstrapping to this.
Start by defining a p-value precisely as the probability of observing data at least as extreme as the observed, assuming the null hypothesis is true. Then explain that its validity hinges on independent observations, and show how duplication violates this by artificially inflating the effective sample size and shrinking standard errors. Finally, connect to real analyst workflows where duplication can occur unintentionally, such as oversampling, data augmentation, and bootstrapping, and suggest ways to detect and avoid the issue.
Pro tip: Emphasize that p-values are not probabilities of the null hypothesis being true, and that duplication can make p-values artificially small even when there is no real effect. Mention that techniques like bootstrapping are valid for inference but require careful handling to avoid treating resampled data as independent new observations.
State that a p-value is the probability, under the null hypothesis, of obtaining a test statistic at least as extreme as the one observed. Clarify that it is not the probability that the null hypothesis is true.
Highlight that p-values rely on assumptions like independence of observations and correct specification of the null model. Violating independence, e.g., through duplication, invalidates the null distribution and leads to misleading p-values.
Explain that duplicating data points makes observations non-independent, artificially increases the sample size, and underestimates variance. This inflates test statistics and produces p-values that are too small, increasing false positive rates.
Give examples: oversampling minority classes in classification can duplicate rows; data augmentation in ML creates near-duplicates; bootstrapping resamples with replacement, and if treated as new independent data, it can mimic duplication. Discuss how each can accidentally lead to the same problem if not handled properly.
Recommend using methods that account for non-independence, such as mixed-effects models, or adjusting sample size calculations. For bootstrapping, emphasize that it is a resampling technique for estimating variability, not for increasing sample size. Always check for duplicates and consider the effective sample size.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
This pivot to chi-square caught me off guard after the OLS thread.
Acknowledge the sample size issue and emphasize that statistical significance does not imply practical significance. Then discuss effect size measures like Cramér's V and odds ratios with confidence intervals to quantify the magnitude and uncertainty of the effect. Finally, compare and contrast penalized (e.g., ridge) and Bayesian approaches, explaining when each is preferable based on the goal (e.g., prediction vs. inference) and context (e.g., prior information availability).
Pro tip: Frame the answer around decision-making: the goal is not just to detect an effect but to decide if it's large enough to matter for the business. Mention that at Google, you'd often combine effect sizes with confidence intervals and consider the cost-benefit trade-off.
Explain that with large samples, chi-square tests can detect trivial deviations from the null, leading to tiny p-values that are not practically meaningful.
Discuss Cramér's V for categorical association and odds ratios for 2x2 tables, including confidence intervals to convey precision and practical significance.
Describe how penalized methods (e.g., ridge regression) shrink estimates to avoid overfitting, and Bayesian methods incorporate priors to regularize and provide credible intervals.
Explain when each is preferable: effect sizes for communication and practical significance; penalized methods for prediction with many predictors; Bayesian for incorporating prior knowledge and quantifying uncertainty.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.