Start by writing the two model equations clearly, defining each term and the interaction. Then interpret the treatment coefficient and interaction in each model, emphasizing the difference between additive (LPM) and multiplicative (logit) scales. Finally, explain how to convert logit coefficients to odds ratios and then to marginal effects at the mean, noting the dependence on other covariates.
Pro tip: When interpreting the interaction in a logit model, remember that the interaction coefficient does not directly give the interaction effect on the probability scale; you need to compute marginal effects or use the difference in differences of predicted probabilities. Also, mention that fixed effects for signup date can absorb time trends, so careful with collinearity with treatment if treatment is time-varying.
Write the linear probability model as P(Y=1) = α + β1*Treatment + β2*Day1WatchTime + β3*NewUser + β4*(Treatment*NewUser) + γ*Country + δ*SignupDate + ε. Write the logistic regression as logit(P(Y=1)) = α + β1*Treatment + β2*Day1WatchTime + β3*NewUser + β4*(Treatment*NewUser) + γ*Country + δ*SignupDate.
In the LPM, β1 is the average difference in retention probability between treatment and control for existing users (NewUser=0), holding other covariates constant. For new users, the treatment effect is β1 + β4.
In the logit, β1 is the change in log-odds of retention for treatment vs control for existing users. Exponentiating gives the odds ratio: exp(β1). For new users, the odds ratio is exp(β1 + β4).
In LPM, β4 is the difference in treatment effect (probability points) between new and existing users. In logit, β4 is the difference in log-odds ratios; exp(β4) is the ratio of odds ratios (treatment effect for new users relative to existing users).
Exponentiate the coefficient to get odds ratio. To get marginal effect at the mean, compute p = 1/(1+exp(-(α + β1*mean(Treatment) + ...))) and then the marginal effect of Treatment is β1 * p * (1-p) for a continuous variable, but for binary treatment, it's the difference in predicted probabilities when Treatment=1 vs 0, evaluated at means of other covariates.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Went through heteroskedasticity and HC-robust SEs fine.
Structure your answer by first addressing each diagnostic (heteroskedasticity, perfect separation, multicollinearity, misspecification, calibration) with specific tests and remedies, then discuss standard error choices and clustering decisions. Emphasize how these choices depend on the model type (e.g., logistic vs. linear) and the data structure (e.g., panel, time series, hierarchical). Conclude with a practical framework for deciding on clustering based on the data generating process and potential correlation.
Pro tip: Always connect diagnostics to business impact—e.g., heteroskedasticity can inflate false positives in A/B tests, while miscalibration can mislead targeting decisions. Mention that at TikTok, with massive user-level data, clustering by user or content ID is often essential to avoid underestimated standard errors.
Clarify whether the model is linear, logistic, or another GLM, and note the data's hierarchical or panel structure (e.g., users nested in regions, repeated observations). This determines which diagnostics and standard error adjustments are relevant.
For heteroskedasticity: plot residuals vs. fitted, use Breusch-Pagan or White test; for perfect separation: check for extreme coefficients or use penalized regression; for multicollinearity: compute VIF or condition number; for misspecification: use RESET test, link test, or compare with flexible models; for calibration: plot observed vs. predicted probabilities and compute Brier score or calibration slope.
Decide between homoskedastic, heteroskedasticity-robust (e.g., White), or cluster-robust standard errors based on diagnostics. If clustering, specify the cluster variable (e.g., user ID, session ID) and justify it by the correlation structure.
Cluster when observations within groups are correlated (e.g., repeated measures per user). Use cluster-robust standard errors with the cluster variable that captures the dependence; test sensitivity by comparing with and without clustering.
Explain how each diagnostic and standard error choice affects inference and business decisions. Highlight that ignoring these issues can lead to biased coefficients, invalid p-values, or poor predictions.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
First, clarify the endogeneity problem: day-1 watch time is correlated with unobserved user preferences that also affect the outcome (e.g., long-term engagement). Then, present two approaches: (1) instrumental variables (IV) and (2) proxy variables or fixed effects. For each, explicitly state the assumptions required for validity and discuss potential instruments/proxies in the TikTok context, such as random exposure to content or app-level variations.
Pro tip: Acknowledge that no instrument is perfect; discuss the trade-off between bias and variance, and suggest sensitivity analyses (e.g., plausible exogenous variation) to strengthen your answer.
Explain that day-1 watch time is endogenous because unobserved user preferences (e.g., content taste, engagement propensity) affect both day-1 watch time and the outcome (e.g., long-term retention). This creates omitted variable bias.
Propose using an instrument that affects day-1 watch time but is uncorrelated with unobserved preferences. State the assumptions: relevance (instrument affects day-1 watch time) and exogeneity (instrument is uncorrelated with the error term).
Suggest using proxies for unobserved preferences (e.g., historical behavior, device type) or fixed effects (e.g., user fixed effects) to control for time-invariant unobserved heterogeneity. Assumptions: proxy is correlated with the unobserved preference and uncorrelated with the error; fixed effects assume unobserved preferences are time-invariant.
Discuss valid instruments/proxies in TikTok context: random content exposure (e.g., A/B tests, algorithmic variations), app version updates, network outages, or natural experiments. For proxies: pre-period engagement metrics, demographics, or device characteristics.
Acknowledge that instruments may be weak or violate exogeneity; proxies may be imperfect. Suggest robustness checks, sensitivity analyses, and triangulation with multiple methods.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Start by explaining that count data like daily videos watched is non-negative and often skewed, making OLS inappropriate due to violated assumptions. Then discuss when Poisson or negative binomial is preferred, focusing on the mean-variance relationship and overdispersion. Finally, cover how to check for overdispersion and interpret exponentiated coefficients in practical terms.
Pro tip: Emphasize that in practice, negative binomial is often the default for count data because overdispersion is the norm, not the exception. Also, mention that exponentiated coefficients represent multiplicative effects on the expected count, which is directly interpretable for product metrics like video watches.
Explain that OLS assumes continuous, normally distributed errors with constant variance, which is violated for count data that is non-negative, integer, and often heteroscedastic.
Discuss that Poisson assumes mean equals variance, while negative binomial relaxes this by adding a dispersion parameter. Prefer Poisson when variance equals mean; otherwise, negative binomial.
Describe methods such as comparing residual deviance to degrees of freedom, using Pearson chi-square statistic, or conducting a dispersion test (e.g., Cameron & Trivedi's test).
Explain that exponentiating a coefficient gives the multiplicative change in the expected count for a one-unit increase in the predictor, holding other variables constant.
Connect the modeling choice to TikTok's metrics, e.g., understanding what drives daily video watches and how changes in features affect engagement.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.