The Nelder-Mead Algorithm in Two Dimensions

http://people.duke.edu/~hpgavin/cee201/Nelder-Mead-2D.pdf

Steps for one iteration of the Nelder-Mead Algorithm

1. Sort the vertices such that f(u) < f(v) < f(w). Point u is the best point, point v is the  next-to-worst point, and point w is the worst point.

2. Reflect the worst point, w, through the centroid of the remaining points (u and v) to obtain  the reflected point, r, and evaluate f(r).

If the cost at the reflected point, f(r), is between the best and next-to-worst cost (f(u) <f(r) < f(v)) then replace the worst point, w, with the reflected point, r, and go to step 5

3. If the cost at the reflected point, f(r), is better than f(u) (f(r) < f(u)) then extend the  reflected point, r, further past the average of u and v, to point e, and evaluate f(e).

(a) If the cost at the extended point, f(e), is better than the reflected point cost, f(r), then  replace the worst point, w, with the extended point, e, and go to step 5.

(b) Otherwise replace the worst point, w with the reflected point, r, and go to step 5.

4. If the inequalities of steps 2 and 3 are not satisfied, then it is certain that the reflected point, r, is worse than the next-to-worst point, v, (f(r) > f(v)) and, a smaller value of f might be  found between points w and r. So try to contract the worst point, w, to a point c between  w and r, and evaluate f(w). The best distance along the line from w to r can be hard to  

determine, and, in general, it is not worth trying too hard to find this minimum. Typicalvalues of c are one-quarter and three-quarters of the way from w to r. These are called insideand outside contraction points, ci and co.