Processing math: 10%
Definition (projection): The projection of a vector →x onto a vector →v in an inner product space is given by
proj→v→x=⟨→x,→v⟩⟨→v,→v⟩→v.
Definition (norm,normed space): Let V be a vector space. We say that a function ‖ is a norm for V if the following three properties hold:
(i) for all nonzero vectors \vec{v}, \lVert \vec{v} \rVert > 0 and \lVert \vec{v} \rVert = 0 if and only if \vec{v}=\vec{0},
(ii) for scalar \alpha and any \vec{v} \in V,
\lVert \alpha \vec{v} \rVert = |\alpha| \lVert \vec{v} \rVert,
and
(iii) the "Triangle inequality" holds:
\lVert \vec{x} + \vec{y} \rVert \leq \lVert \vec{x} \rVert + \lVert \vec{y} \rVert.
If \lVert \cdot \rVert is a norm on a vector space V then we say that (V,\lVert \cdot \rVert) is a normed space.
Theorem: (Cauchy-Schwarz inequality) In an inner product space (V, \langle \cdot,\cdot \rangle) the following inequality holds:
\left|\langle \vec{x},\vec{y} \rangle \right|^2 \leq \langle \vec{x},\vec{x} \rangle \langle \vec{y},\vec{y} \rangle.
Theorem: Let V be a vector space and (V,\langle \cdot,\cdot \rangle) be an inner product space. If we define \lVert \cdot \rVert by
\lVert \vec{v} \rVert = \sqrt{\langle \vec{v},\vec{v} \rangle},
then (V, \lVert \cdot \rVert) is a normed space.
Def (orthonormal sequence): Let (V,\langle \cdot,\cdot \rangle) be an inner product space and let (V,\lVert \cdot \rVert) be the normed space associated with \langle \cdot,\cdot \rangle from the prior theorem. Let (\vec{v}_n)_{n=0}^{\infty} be an orthogonal sequence of nonzero vectors. If \lVert \vec{v}_n \rVert = 1 for every n=0,1,\ldots, then we say that (\vec{v}_n)_{n=0}^{\infty} is an orthonormal sequence of vectors. If it turns out that \lVert \vec{v}_n \rVert \neq 1 for every n then we may create an orthonormal sequence by defining
\vec{u}_n = \dfrac{\vec{v}_n}{\lVert \vec{v}_n \rVert},
then (\vec{u})_{n=0}^{\infty} is an orthonormal sequence of vectors.
Theorem (The Gram-Schmidt Process): Let (V,\langle \cdot,\cdot \rangle) be an inner product space. Let \{\vec{v}_1,\vec{v}_2,\ldots\} be a set of vectors (which may or may not be a set of orthogonal vectors). Define a sequence of vectors by the following formula:
\left\{ \begin{array}{ll}
\vec{u}_1 &= \vec{v}_1 \\
\vec{u}_2 &= \vec{v}_2 - \mathrm{proj}_{\vec{u}_1} (\vec{v}_2) \\
\vec{u}_3 &= \vec{v}_3 - \mathrm{proj}_{\vec{u}_1} (\vec{v}_3) - \mathrm{proj}_{\vec{u}_2} (\vec{v}_3) \\
\vec{u}_4 &= \vec{v}_4 - \mathrm{proj}_{\vec{u}_1} (\vec{v}_4) - \mathrm{proj}_{\vec{u}_2} (\vec{v}_4) - \mathrm{proj}_{\vec{u}_3}(\vec{v}_4) \\
\vdots \\
\vec{u}_n &= \vec{v}_n - \displaystyle\sum_{k=1}^{n-1} \mathrm{proj}_{\vec{u}_k} (\vec{v}_n) \\
\vdots \\
\end{array} \right.
then the sequence (\vec{u}_n)_{n=1}^{\infty} is an orthogonal sequence of vectors. Moreover if we define the vectors
\left\{ \begin{array}{ll}
\vec{w}_1 &= \dfrac{\vec{u}_1}{\lVert \vec{u}_1 \rVert} \\
\vec{w}_2 &= \dfrac{\vec{w}_2}{\lVert \vec{w}_2 \rVert} \\
\vdots \\
\vec{w}_n &= \dfrac{\vec{w}_n}{\lVert \vec{w}_n \rVert}, \\
\vdots
\end{array} \right.
then (w_n)_{n=1}^{\infty} is an orthonormal sequence of vectors.
