We introduce a novel approach for analyzing the performance of first-order black-box optimization methods. Following the seminal work of Nemirovski and Yudin (1983) in the complexity analysis of convex optimization methods, we measure the computational cost based on the oracle model of optimization. Building on this model, our approach relies on the observation that by definition, the worst case behavior of a black-box optimization method is by itself an optimization problem, which we call the Performance Estimation Problem (PEP). We analyze the properties of the resulting PEP for various black-box first order schemes. This allows us to prove a new tight analytical bound for the classical gradient method, as well as to derive numerical bounds that can be efficiently computed for a broad class of first order schemes. Moreover, we derive an efficient procedure for finding step sizes which produces a first-order black-box method that achieves best performance. |