KE8QZC | SFW | TSW

Definition (projection): The projection of a vector $\vec{x}$ onto a vector $\vec{v}$ in an inner product space is given by $$\mathrm{proj}_{\vec{v}}\vec{x} = \dfrac{\langle \vec{x},\vec{v} \rangle}{\langle \vec{v},\vec{v} \rangle} \vec{v}.$$ Definition (norm,normed space): Let $V$ be a vector space. We say that a function $\lVert \cdot \rVert \colon V \rightarrow \mathbb{R}$ is a norm for $V$ if the following three properties hold:
(i) for all nonzero vectors $\vec{v}, \lVert \vec{v} \rVert > 0$ and $\lVert \vec{v} \rVert = 0$ if and only if $\vec{v}=\vec{0}$,
(ii) for scalar $\alpha$ and any $\vec{v} \in V$, $$\lVert \alpha \vec{v} \rVert = |\alpha| \lVert \vec{v} \rVert,$$ and
(iii) the "Triangle inequality" holds: $$\lVert \vec{x} + \vec{y} \rVert \leq \lVert \vec{x} \rVert + \lVert \vec{y} \rVert.$$ If $\lVert \cdot \rVert$ is a norm on a vector space $V$ then we say that $(V,\lVert \cdot \rVert)$ is a normed space.
Theorem: (Cauchy-Schwarz inequality) In an inner product space $(V, \langle \cdot,\cdot \rangle)$ the following inequality holds: $$\left|\langle \vec{x},\vec{y} \rangle \right|^2 \leq \langle \vec{x},\vec{x} \rangle \langle \vec{y},\vec{y} \rangle.$$ Theorem: Let $V$ be a vector space and $(V,\langle \cdot,\cdot \rangle)$ be an inner product space. If we define $\lVert \cdot \rVert$ by $$\lVert \vec{v} \rVert = \sqrt{\langle \vec{v},\vec{v} \rangle},$$ then $(V, \lVert \cdot \rVert)$ is a normed space.
Def (orthonormal sequence): Let $(V,\langle \cdot,\cdot \rangle)$ be an inner product space and let $(V,\lVert \cdot \rVert)$ be the normed space associated with $\langle \cdot,\cdot \rangle$ from the prior theorem. Let $(\vec{v}_n)_{n=0}^{\infty}$ be an orthogonal sequence of nonzero vectors. If $\lVert \vec{v}_n \rVert = 1$ for every $n=0,1,\ldots$, then we say that $(\vec{v}_n)_{n=0}^{\infty}$ is an orthonormal sequence of vectors. If it turns out that $\lVert \vec{v}_n \rVert \neq 1$ for every $n$ then we may create an orthonormal sequence by defining $$\vec{u}_n = \dfrac{\vec{v}_n}{\lVert \vec{v}_n \rVert},$$ then $(\vec{u})_{n=0}^{\infty}$ is an orthonormal sequence of vectors.

Theorem (The Gram-Schmidt Process): Let $(V,\langle \cdot,\cdot \rangle)$ be an inner product space. Let $\{\vec{v}_1,\vec{v}_2,\ldots\}$ be a set of vectors (which may or may not be a set of orthogonal vectors). Define a sequence of vectors by the following formula: $$\left\{ \begin{array}{ll} \vec{u}_1 &= \vec{v}_1 \\ \vec{u}_2 &= \vec{v}_2 - \mathrm{proj}_{\vec{u}_1} (\vec{v}_2) \\ \vec{u}_3 &= \vec{v}_3 - \mathrm{proj}_{\vec{u}_1} (\vec{v}_3) - \mathrm{proj}_{\vec{u}_2} (\vec{v}_3) \\ \vec{u}_4 &= \vec{v}_4 - \mathrm{proj}_{\vec{u}_1} (\vec{v}_4) - \mathrm{proj}_{\vec{u}_2} (\vec{v}_4) - \mathrm{proj}_{\vec{u}_3}(\vec{v}_4) \\ \vdots \\ \vec{u}_n &= \vec{v}_n - \displaystyle\sum_{k=1}^{n-1} \mathrm{proj}_{\vec{u}_k} (\vec{v}_n) \\ \vdots \\ \end{array} \right.$$ then the sequence $(\vec{u}_n)_{n=1}^{\infty}$ is an orthogonal sequence of vectors. Moreover if we define the vectors $$\left\{ \begin{array}{ll} \vec{w}_1 &= \dfrac{\vec{u}_1}{\lVert \vec{u}_1 \rVert} \\ \vec{w}_2 &= \dfrac{\vec{w}_2}{\lVert \vec{w}_2 \rVert} \\ \vdots \\ \vec{w}_n &= \dfrac{\vec{w}_n}{\lVert \vec{w}_n \rVert}, \\ \vdots \end{array} \right.$$ then $(w_n)_{n=1}^{\infty}$ is an orthonormal sequence of vectors.