Normal cone (variational analysis)

In variational analysis, set-valued analysis and optimization, the concept of a normal cone to a subset of a space generalizes that of the orthogonal complement / annihilator of a vector space, (outward) normal vector fields to surfaces — or more generally of the normal bundle of an embedded submanifold — to possibly non-smooth subsets of vector spaces.

Normal cones provide, among other things, the geometrical foundation for generalizing the convex subdifferential to non-convex functions. Of particular note is also their role in generalizing Fermat's rule to give necessary (and sometimes also sufficient) first order optimality conditions for constrained and non-smooth optimization problems. Moreover, they play a role in defining coderivatives of set-valued maps.

In the non-convex case there are several inequivalent definitions for a normal cone that all turn out to be useful and interesting for different problems — whereas in the convex case these all coincide which greatly simplifies things. For clarity and approachability this article first discusses the convex case over Hilbert spaces before going into the non-convex case over more general spaces.

Conventions

All vector spaces in this article are assumed to be real. The set

R

¯

{\displaystyle {\bar {\mathbb {R} }}}

denotes the extended real numbers and a function

f

:

X

→

R

¯

{\displaystyle f:X\to {\bar {\mathbb {R} }}}

is called proper if it is nowhere equal to

−

∞

{\displaystyle -\infty }

and also somewhere finite. We also assume that all cones contain zero. Sums of sets throughout the article are to be interpreted as Minkowski sums.