Nelder–Mead method

The Nelder–Mead method (also downhill simplex method, amoeba method, or polytope method) is a numerical method used to find a local minimum or maximum of an objective function in a multidimensional space. It is a direct search method (based on function comparison) and is often applied to nonlinear optimization problems for which derivatives may not be known. However, the Nelder–Mead technique is a heuristic search method that can converge to non-stationary points on problems that can be solved by alternative methods.

The Nelder–Mead technique was proposed by John Nelder and Roger Mead in 1965, as a development of the method of Spendley et al.

Overview

The method uses the concept of a simplex, which is a special polytope of n + 1 vertices in n dimensions. Examples of simplices include a line segment in one-dimensional space, a triangle in two-dimensional space, a tetrahedron in three-dimensional space, and so forth.

The method approximates a local optimum of a problem with n variables when the objective function varies smoothly and is unimodal. Typical implementations minimize functions, and we maximize

f

(

x

)

{\displaystyle f(\mathbf {x} )}

by minimizing

−

f

(

x

)

{\displaystyle -f(\mathbf {x} )}

.

For example, a suspension bridge engineer has to choose how thick each strut, cable, and pier must be. These elements are interdependent, but it is not easy to visualize the impact of changing any specific element.