Start by defining the truncated normal distribution for a standard normal Z restricted to (a, b). Then derive the normalized PDF by dividing the standard normal PDF by the probability mass in the interval, and express the CDF as a difference of standard normal CDFs divided by that same probability. Emphasize the role of the normalizing constant and the connection to the standard normal.
Pro tip: Mention that the truncated normal is often used in survival analysis and econometrics, and that the normalizing constant ensures the density integrates to 1. Also, note that if a = -∞ or b = ∞, it reduces to a one-sided truncation.
State that Z is standard normal (mean 0, variance 1) and we condition on a < Z < b. The truncation interval is (a, b) with a < b.
Recall the standard normal PDF φ(z) = (1/√(2π)) e^{-z²/2} and CDF Φ(z). These are the building blocks for the truncated distribution.
The probability that Z falls in (a, b) is P(a < Z < b) = Φ(b) - Φ(a). This constant ensures the truncated PDF integrates to 1.
The PDF is f_Z(z | a < Z < b) = φ(z) / (Φ(b) - Φ(a)) for a < z < b, and 0 otherwise.
The CDF is F_Z(z | a < Z < b) = (Φ(z) - Φ(a)) / (Φ(b) - Φ(a)) for a ≤ z ≤ b, with F(z) = 0 for z < a and F(z) = 1 for z > b.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
This is where things got interesting and also where I started sweating.
For each truncated normal case, analyze the acceptance probability of naive rejection sampling from a standard normal and compare it to the efficiency and exactness of inverse-CDF sampling. Discuss the trade-offs in terms of computational cost, implementation complexity, and numerical stability, especially for extreme truncation regions. Conclude with a recommendation for each case based on the analysis.
Pro tip: Quantify the acceptance rates for the rejection sampler (e.g., for case (i) it's about 15.9%, for (ii) it's ~0.0001%, and for (iii) it's ~0.00003%) to concretely demonstrate why rejection sampling is impractical for the latter two. Also, mention that inverse-CDF sampling requires evaluating the normal CDF and its inverse, which are readily available in standard libraries but may have numerical issues in extreme tails.
Restate the three cases: (i) X ~ N(0,1) truncated to [1, ∞), (ii) truncated to [4, 4.05], (iii) truncated to [4, ∞). Clarify that the goal is to sample from these truncated distributions efficiently.
For each case, compute the acceptance probability (the probability mass of the truncation region under the standard normal). Discuss that the number of trials needed follows a geometric distribution with mean 1/p, leading to high computational cost when p is very small.
Explain that inverse-CDF sampling involves generating U ~ Uniform(0,1) and setting X = Φ^{-1}(Φ(a) + U*(Φ(b)-Φ(a))). Discuss its exactness and constant time per sample, but note potential numerical challenges when Φ(a) and Φ(b) are extremely close to 0 or 1.
For case (i), both methods are feasible; rejection sampling has ~15.9% acceptance, but inverse-CDF is still more efficient. For case (ii), rejection sampling is extremely inefficient (acceptance ~0.0001%), so inverse-CDF is preferred despite numerical precision issues. For case (iii), rejection sampling is even worse (~0.00003%), so inverse-CDF is necessary, but consider using specialized algorithms for extreme tails.
Mention that for extreme truncation, inverse-CDF may suffer from loss of precision; alternatives include using a shifted exponential approximation or specialized samplers like the one by Robert (1995). Also, note that for case (ii), the interval is so narrow that the distribution is nearly uniform, so one could approximate by sampling uniformly and adjusting.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
I knew the general idea: use an exponential proposal shifted to the truncation point and find a bound on the density ratio.
Start by framing the problem: we need to sample from a target distribution's extreme tail where direct sampling is inefficient. Propose an acceptance-rejection algorithm with an exponential or half-normal proposal, derive the acceptance probability, and explain why it outperforms alternatives like uniform proposals or MCMC in terms of efficiency and tail accuracy.
Pro tip: Emphasize the importance of the proposal's tail behavior matching the target's—exponential or half-normal proposals have heavier tails than many targets, ensuring bounded likelihood ratios and high acceptance rates. Also, mention that for very extreme tails, you might need to use a shifted or truncated proposal to avoid underflow.
Clearly state that we want to sample from the extreme tail of a distribution (e.g., beyond a high threshold) where direct methods are impractical. Specify the target density f(x) and the tail region of interest.
Select an exponential or half-normal proposal g(x) that is easy to sample from and has tails at least as heavy as the target in the region of interest. Justify the choice based on tail behavior and computational simplicity.
Compute the bound M such that f(x)/g(x) ≤ M for all x in the tail. The acceptance probability is f(x)/(M g(x)). Describe the steps: sample Y ~ g, sample U ~ Uniform(0,1), accept if U ≤ f(Y)/(M g(Y)).
Discuss the acceptance rate and computational cost. Compare with alternatives like uniform proposals (which have low acceptance in tails), importance sampling (which may have high variance), or MCMC (which may mix poorly in tails). Highlight that the exponential/half-normal proposal yields higher acceptance rates and lower variance.
Mention how to handle very extreme tails (e.g., using a shifted exponential or a Pareto proposal), numerical stability (log-scale computations), and potential adaptations for multivariate or heavy-tailed targets.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Honestly the part I was least prepared for.
Start by acknowledging the challenges of numerical stability in extreme tails, then discuss specific techniques like using erfc for the normal CDF and rational approximations for the inverse. Emphasize the importance of avoiding catastrophic cancellation and leveraging asymptotic expansions, while also mentioning practical implementations in libraries like SciPy.
Pro tip: Mention that in production, you'd use well-tested libraries rather than implementing from scratch, but understanding the underlying algorithms helps debug edge cases and optimize performance.
Explain why direct computation of the normal CDF and its inverse fails near machine precision, such as underflow/overflow and loss of precision in extreme tails.
Describe using the complementary error function (erfc) for the upper tail and erfcx for scaled versions, and how these avoid cancellation and underflow.
Mention rational approximations (e.g., Acklam's algorithm) and Newton refinement, and how to handle extreme tail regions using asymptotic expansions.
Talk about accuracy vs. speed, and the importance of using established libraries (e.g., SciPy's ndtr, ndtri) that implement these techniques.
Give a concrete example, such as computing p-values in hypothesis testing or quantiles for risk models, where numerical stability is critical.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.
Back-transformed uniforms via a KS test plus moment checks.
Start by outlining a multi-layered validation strategy that combines theoretical checks, statistical tests, and empirical comparisons. Emphasize the importance of testing edge cases and using known properties of the truncated normal distribution. Conclude by discussing how you would integrate these tests into a continuous validation pipeline.
Pro tip: Leverage the fact that the truncated normal distribution has a known CDF and moments; use them to create precise statistical tests rather than relying solely on visual inspection. Also, consider using a reference implementation from a trusted library (e.g., scipy.stats.truncnorm) for comparison.
Verify that the sampler's output matches the known theoretical properties of the truncated normal distribution, such as the mean, variance, and CDF. Use analytical formulas for these moments and compare with empirical estimates from a large sample.
Apply goodness-of-fit tests like Kolmogorov-Smirnov or Anderson-Darling to check if the sampled data follows the expected truncated normal distribution. Also, test for independence and randomness if the sampler is supposed to produce i.i.d. samples.
Test scenarios where the truncation bounds are extreme (e.g., very narrow or wide intervals, bounds far in the tails) to ensure numerical stability and correctness. Check behavior when the distribution is nearly normal or highly truncated.
Compare your sampler's output with a trusted reference implementation (e.g., scipy.stats.truncnorm) using metrics like KL divergence, Wasserstein distance, or visual QQ-plots. Ensure that any differences are within acceptable tolerance.
Incorporate these validation checks into an automated test suite that runs on every code change. Include tests for reproducibility (with fixed seeds) and performance to catch regressions.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.