Gram–Schmidt process

In mathematics, particularly linear algebra and numerical analysis, the Gram–Schmidt process or Gram-Schmidt algorithm is a way of finding a set of two or more vectors that are perpendicular to each other.

By technical definition, it is a method of constructing an orthonormal basis from a set of vectors in an inner product space, most commonly the Euclidean space

R

n

{\displaystyle \mathbb {R} ^{n}}

equipped with the standard inner product. The Gram–Schmidt process takes a finite, linearly independent set of vectors

S

=

{

v

1

,

…

,

v

k

}

{\displaystyle S=\{\mathbf {v} _{1},\ldots ,\mathbf {v} _{k}\}}

for k ≤ n and generates an orthogonal set

S

′

=

{

u

1

,

…

,

u

k

}

{\displaystyle S'=\{\mathbf {u} _{1},\ldots ,\mathbf {u} _{k}\}}

that spans the same

k

{\displaystyle k}

-dimensional subspace of

R

n

{\displaystyle \mathbb {R} ^{n}}

as

S

{\displaystyle S}

.

The method is named after Jørgen Pedersen Gram and Erhard Schmidt, but Pierre-Simon Laplace had been familiar with it before Gram and Schmidt. In the theory of Lie group decompositions, it is generalized by the Iwasawa decomposition.

The application of the Gram–Schmidt process to the column vectors of a full column rank matrix yields the QR decomposition (it is decomposed into an orthogonal and a triangular matrix).

Description

The vector projection of a vector

v

{\displaystyle \mathbf {v} }

on a nonzero vector

u

{\displaystyle \mathbf {u} }

is defined as

proj

u

⁡

(

v

)

=

⟨

v

,

u

⟩

⟨

u

,

u

⟩

u

,

{\displaystyle \operatorname {proj} _{\mathbf {u} }(\mathbf {v} )={\frac {\langle \mathbf {v} ,\mathbf {u} \rangle }{\langle \mathbf {u} ,\mathbf {u} \rangle }}\,\mathbf {u} ,}

where

⟨

v

,

u

⟩

{\displaystyle \langle \mathbf {v} ,\mathbf {u} \rangle }

denotes the dot product of the vectors

u

{\displaystyle \mathbf {u} }

and

v

{\displaystyle \mathbf {v} }

. This means that

proj

u

⁡

(

v

)

{\displaystyle \operatorname {proj} _{\mathbf {u} }(\mathbf {v} )}

is the orthogonal projection of

v

{\displaystyle \mathbf {v} }

onto the line spanned by

u

{\displaystyle \mathbf {u} }

. If

u

{\displaystyle \mathbf {u} }

is the zero vector, then

proj

u

⁡

(

v

)

{\displaystyle \operatorname {proj} _{\mathbf {u} }(\mathbf {v} )}

is defined as the zero vector.

Given

k

{\displaystyle k}

nonzero linearly-independent vectors

v

1

,

…

,

v

k

{\displaystyle \mathbf {v} _{1},\ldots ,\mathbf {v} _{k}}

the Gram–Schmidt process defines the vectors

u

1

,

…

,

u

k

{\displaystyle \mathbf {u} _{1},\ldots ,\mathbf {u} _{k}}

as follows:

u

1

=

v

1

,

e

1

=

u

1

‖

u

1

‖

u

2

=

v

2

−

proj

u

1

⁡

(

v

2

)

,

e

2

=

u

2

‖

u

2

‖

u

3

=

v

3

−

proj

u

1

⁡

(

v

3

)

−

proj

u

2

⁡

(

v

3

)

,

e

3

=

u

3

‖

u

3

‖

u

4

=

v

4

−

proj

u

1

⁡

(

v

4

)

−

proj

u

2

⁡

(

v

4

)

−

proj

u

3

⁡

(

v

4

)

,

e

4

=

u

4

‖

u

4

‖

⋮

⋮

u

k

=

v

k

−

∑

j

=

1

k

−

1

proj

u

j

⁡

(

v

k

)

,

e

k

=

u

k

‖

u

k

‖

.