Nesterov's method

Problem formulation[1]:

$$\begin{equation} {\rm minimize} \quad f(x), \quad x \in {\mathbb R}^n \end{equation}$$

Nesterovs method generates a sequence of search points $x_i, i=0,1,...,$ and the sequence of approximate solutions $y_i,i=1,2,...,$ along with auxiliary sequences of vectors $q_i$, positive reals $L_i$ abd reals $t_i \geq 1$, according to the following rules:
Initialization: Set

$$x_0=y_1=q_0=0;t_0=1;$$
choose as $L_0$ an arbitrary positive initial estimate of ${\mathcal L}(f)$.
$i$-th step, $i \geq 1$: Given $x_{i-1},y_i,q_{i-1},L_{i-1},t_{i-1}$, act as follows:

1. set
   $$x_i=y_i-\frac{1}{t_{i-1}}(y_i+q_{i-1})$$
   and compute $f(x_i),f'(x_i);$
2. Testing sequentially the values $l=2^j L_{i-1},j=0,1,...,$ find the first value of $l$ such that
   $$f(x_i-\frac{1}{l}f'(x_i)) \leq f(x_i)-\frac{1}{2l}|f'(x_i)|^2$$
   set $L_i$ equal to the resulting value of $l$.
3. Set
   $$y_{i+1}=x_i-\frac{1}{L_i}f'(x_i), \\ q_i=q_{i-1}+\frac{t_{i-1}}{L_i}f'(x_i),$$
   define $t_i$ as the larger root of the equation
   $$t^2-t=t^2_{i-1}$$
   and loop.

Convergence rate: O($N^{-2}$).

[1] Y. Nesterov. Introductory Lectures on Convex Optimization. Kluwer Academic Publishers, 2003.