Start by clarifying the problem setup and then present the golden-section search algorithm as the optimal method for finding the minimum of a convex function using only function evaluations. Explain why gradient-based methods are inapplicable due to the black-box nature, analyze the convergence rate to achieve an uncertainty interval below epsilon, and discuss how noise would necessitate robust or stochastic optimization techniques.
Pro tip: Mention that golden-section search is optimal in the worst case for this problem, and that with noise, one can use repeated evaluations or stochastic approximation methods like Robbins-Monro, showing awareness of practical constraints.
Restate the problem: minimize a convex function on a closed interval using at most K function evaluations, no gradients. Confirm that the function is deterministic and that we need to output an interval containing the minimum.
Describe the golden-section search algorithm: iteratively narrow the interval by evaluating at two interior points, using the golden ratio to maintain a constant reduction factor per iteration.
Gradient-based methods require gradient information, which is unavailable in a black-box setting. Additionally, numerical differentiation would require extra evaluations and may be inaccurate, especially with noise.
Derive the number of evaluations needed to reduce the interval length below epsilon. Golden-section search reduces the interval by a factor of about 0.618 per evaluation, so K ≈ log(epsilon / (b-a)) / log(0.618) evaluations are needed.
With noise, function evaluations are stochastic. Simple golden-section search may fail; instead, use repeated evaluations at each point to average out noise, or employ stochastic approximation methods. The uncertainty interval will be larger for the same budget, and convergence guarantees become probabilistic.
AI-generated suggestions, not part of the candidate's original notes. May be inaccurate — verify before relying on them.