> [!definition] > > Let $\ch$ be a [[Complex Numbers|complex]] [[Vector Space|vector space]]. $\ch$ is a **pre-Hilbert space** if it is equipped with an [[Inner Product|inner product]]. > [!definition] > > Let $\ch$ be a pre-Hilbert space, and define > $ > \norm{x} = \sqrt{\angles{x, x}} > $ > then $\norm{x}$ is a [[Normed Vector Space|norm]] on $\ch$. If $\ch$ is [[Banach Space|complete]] with respect to $\norm{\cdot}$, then $\ch$ is a **Hilbert space**. > > *Proof*. Since $\angles{x, x} = 0$ if and only if $x = 0$, $\norm{x} = 0$ if and only if $x = 0$. Let $\lambda \in \complex$. then > $ > \angles{\lambda x, \lambda x} = \lambda \ol{\lambda}\angles{x, x} = \abs{\lambda}^2 \angles{x, x} > $ > Now let $x, y \in \ch$, then > $ > \begin{align*} > \angles{x + y, x + y} &= \angles{x + y, x} + \angles{x + y, y} \\ > &= \angles{x, x} + \angles{y, x} + \angles{x, y} + \angles{y, y} \\ > &= \angles{x, x} + \ol{\angles{x, y}} + \angles{x, y} + \angles{y, y} \\ > &= \angles{x, x} + 2\re{\angles{x, y}} + \angles{y, y} \\ > &\le \angles{x, x} + 2\abs{\angles{x, y}} + \angles{y, y} \\ > &\le \angles{x, x} + 2\norm{x}\norm{y} + \angles{y, y} \\ > &\le \angles{x, x} + \angles{y, y} > \end{align*} > $ > [!theorem] > > Let $\ch$ be a pre-Hilbert space and $\seq{x_n}, \seq{y_n} \subset \ch$. If $x_n \to x$ and $y_n \to y$, then $\angles{x_n, y_n} \to \angles{x, y}$. > > *Proof*. > $ > \begin{align*} > \abs{\angles{x_n, y_n} - \angles{x, y}} &= \abs{ > \angles{x_n - x, y_n} + \angles{x, y_n} - \angles{x, y} > } \\ > &= \abs{\angles{x_n - x, y_n} + \angles{x, y_n - y}} \\ > &\le \norm{x_n - x}\norm{y_n} + \norm{x} \norm{y_n - y} > \end{align*} > $ > which approaches $0$ as $n \to \infty$. > [!theorem] > > Let $\ch$ be a Hilbert space, and $\cm \subset \ch$ be a [[Closed Set|closed]] subspace. Let > $ > \cm^\perp = \bracs{x \in \ch: \angles{x, y} = 0 \quad \forall y \in \cm} > $ > be its [[Orthogonal Complement|orthogonal complement]], then $\ch = \cm \oplus \cm^\perp$ is an inner [[Direct Sum|direct sum]]. Moreover, when decomposing $x \in \ch$ to $y + z$ where $y \in \cm$ and $z \in \cm^\perp$, $y$ and $z$ are the unique elements of $\cm$ and $\cm^\perp$ whose distance to $x$ is minimal. > > *Proof*. In regular normed spaces, there is no good way of getting a sequence in $\cm$ to approach a vector that minimises its distance from $x$. However, since the geometry of Hilbert spaces resemble Euclidean spaces thanks to the [[Pythagorean Theorem|Pythagorean theorem]] and the Parallelogram law (and the space being complete), the exact vector that minimises distance may be constructed. > > ### Decomposition > > Let $x \in \ch$, then there exists $y \in \cm$ such that > $ > \norm{x - y} = \inf_{y' \in \cm}\norm{x - y'} > $ > > *Proof.* Let $x \in \ch$, and let $\delta = \inf_{y \in \cm}\norm{x - y}$. Let $\seq{y_n} \subset \cm$ be a sequence such that $\norm{x - y_n} \to \delta$. By the Parallelogram law, > $ > 2\norm{y_n - x}^2 + 2\norm{y_m - x}^2 = \norm{y_n - y_m}^2 + \norm{y_n + y_m - 2x}^2 > $ > where since $\frac{1}{2}(y_m + y_n) \in \cm$, > $ > \begin{align*} > \norm{y_n - y_m}^2 &= 2\norm{y_n - x}^2 + 2\norm{y_m - x}^2 - \norm{y_n + y_m - 2x}^2 \\ > &= 2\norm{y_n - x}^2 + 2\norm{y_m - x}^2 - 4\norm{\frac{1}{2}(y_n + y_m) - x}^2 \\ > &\le 2\norm{y_n - x}^2 + 2\norm{y_m - x}^2 - 4\delta^2 > \end{align*} > $ > As $\norm{y_n - x} \downto \delta$, there exists $N \in \nat$ such that $\norm{y_n - x} < \delta + \varepsilon$ for all $n \ge N$, in which case > $ > 2\norm{y_n - x}^2 + 2\norm{y_m - x}^2 - 4\delta^2 < 4(\delta + \varepsilon)^2 - 4\delta^2 > $ > Therefore $\norm{y_n - y_m}$ is a [[Cauchy Sequence|Cauchy sequence]], and as $\ch$ is complete, $y_n \to y \in \ch$. > > > ### Orthogonal > > Let $x \in \ch$ and $y$ as above, then $z = x - y \in \cm^\perp$. > > *Proof*. Let $z' \in \cm$. If $\angles{z, z'} \not\in\real$, then $\angles{z, \sgn(\angles{z, z'})z'} \in \real$, so we can assume that $\angles{z, z'} \in \real$. Then the function > $ > f(t) = \norm{z + tz'}^2 = \angles{z, z} + 2t\angles{z, z'} + t^2\angles{z', z'} > $ > has a minimum at $t = 0$ (because $z + tz' = x - (y - tz')$, and $y$ minimises $\norm{x - y}$), meaning that $f'(0) = 2\angles{z, z'} = 0$ (bottom of parabola). > > > ### Unique > > Let $x \in \ch$ and $y, z$ as above, then $z$ is the unique vector in $\cm^\perp$ and $y$ is the unique vector in $\cm$ that minimise distance to $x$. > > *Proof*. Let $z' \in \cm^\perp$, then by the [[Pythagorean Theorem|Pythagorean theorem]] > $ > \norm{x - z'}^2 = \underbrace{\norm{x - z}^2}_{= y \in \cm} + \underbrace{\norm{z - z'}^2}_{\in \cm^\perp} \ge \norm{x - z}^2 > $ > with equality if and only if $z = z'$. > > Let $y' \in \cm$, then by the Pythagorean theorem again, > $ > \norm{x - y'}^2 = \underbrace{\norm{x - y}^2}_{= z \in \cm^\perp} + \underbrace{\norm{y - y'}^2}_{\in \cm} \ge \norm{x - y}^2 > $ > with equality if and only if $y = y'$. > > > ### Direct Sum > > $ > \ch = \cm \oplus \cm^\perp > $ > *Proof*. Since for each $x \in \ch$ there exists $y \in \cm$ and $z \in \cm^\perp$ such that $x = y + z$, $\ch = \cm + \cm^\perp$. Let $x \in \cm \cap \cm^\perp$, then $\angles{x, x} = 0$ and $x = 0$. > [!theorem] > > Let $\ch$ be a Hilbert space and $\seqi{u} \subset \ch$ be an [[Orthonormal Set|orthonormal]] set, then the following are equivalent: > 1. **Completeness**: If $\angles{x, u_i} = 0$ for all $i \in I$, then $x = 0$. > 2. For each $x \in \ch$, $x = \sum_{i \in I}\angles{x, u_i}u_i$, where the sum has countably many non-zero terms, and converges in the norm topology regardless of the ordering. > 3. **Parseval's Identity**: $\norm{x}^2 = \sum_{i \in I}\abs{\angles{x, u_i}}^2$ for all $x \in \ch$. > > An orthonormal set satisfying these properties is an **orthonormal basis** of $\ch$. > > *Proof*. > > # Completeness to Decomposition > > ### Countable Reduction > > Let $x \in \ch$, then $\angles{x, u_i} \ne 0$ for only countably many $i$s. > > *Proof*. By [[Bessel's Inequality|Bessel's inequality]], $\norm{x}^2 \ge \sum_{i \in I}\abs{\angles{x, u_i}}^2$ and $\abs{\angles{x, u_i}}^2$ is non-zero for only countably many $i$s. > > ### Evaluating the Sum > > Let $\seq{u_n}$ be an enumeration of $u_i$s where $\angles{x, u_i} \ne 0$, then > $ > x = \sum_{n = 1}^{\infty}\angles{x, u_n}u_n > $ > *Proof*. By [[Bessel's Inequality|Bessel's inequality]], the series $\sum_{n \in \nat}\abs{\angles{x, u_n}}^2$ converges. By the [[Pythagorean Theorem|Pythagorean theorem]], > $ > \norm{\sum_{k = n}^{m}\angles{x, u_k}u_k} = \sum_{k = n}^{m}\abs{\angles{x, u_k}}^2 \le \sum_{k = n}^{\infty}\abs{\angles{x, u_k}}^2 \to 0 > $ > the series $\sum_{n = 1}^{\infty}\angles{x, u_n}u_n$ is [[Cauchy Sequence|Cauchy]], and therefore converges. Let $y = x - \sum_{n = 1}^{\infty}\angles{x, u_n}u_n$, then > $ > \begin{align*} > \angles{y, u_j} &= \angles{x - \sum_{n = 1}^{\infty}\angles{x, u_n}u_n, u_j} \\ > &= \limv{n}\angles{x - \sum_{k = 1}^{n}\angles{x, u_k}u_k, u_j} \\ > &= \angles{x, u_j} - \angles{\angles{x, u_j}u_j, u_j} &(&\forall n \ge j) \\ > &= 0 > \end{align*} > $ > for all $j \in \nat$ and as $x, \sum_{n = 1}^{\infty}\angles{x, u_n}u_n \perp u_i$ for all $u_i \not\in \bracs{u_n: n \in \nat}$, $y \perp u_i$ for all $i \in I$. Therefore $y = 0$, and $x = \sum_{n = 1}^{\infty}\angles{x, u_n}u_n$. > > > # Decomposition to Parseval's Identity > > Suppose that each $x \in \ch$ can be written as $\sum_{i \in I}\angles{x, u_i}u_i$, then > $ > \norm{x}^2 = \sum_{i \in I}\abs{\angles{x, u_i}}^2 \quad \forall x \in \ch > $ > *Proof*. Let $\seq{u_n}$ be an enumeration of $u_i$s such that $\angles{x, u_i} \ne 0$, then > $ > \sum_{i \in I}\abs{\angles{x, u_i}}^2 = \sum_{n \in \nat}\abs{\angles{x, u_n}}^2 = \sum_{n = 1}^{\infty}\abs{\angles{x, u_n}}^2 > $ > By the [[Pythagorean Theorem|Pythagorean theorem]], > $ > \norm{x}^2= \norm{x - \sum_{k = 1}^{n}\angles{x, u_k}u_k}^2 + \norm{\sum_{k = 1}^{n}\angles{x, u_k}u_k}^2 > $ > Since $x = \sum_{n = 1}^{\infty}\angles{x, u_n}u_n$, > $ > \begin{align*} > \norm{x}^2 - \norm{\sum_{k = 1}^{n}\angles{x, u_k}u_k}^2= \norm{x - \sum_{k = 1}^{n}\angles{x, u_k}u_k}^2 \to 0 > \end{align*} > $ > as $n \to \infty$. Therefore > $ > \norm{x}^2 = \norm{\sum_{n = 1}^{\infty}\angles{x, u_n}u_n}^2 = \sum_{n = 1}^{\infty}\abs{\angles{x, u_n}}^2 > $ > > # Parseval's Identity to Completeness > > Suppose that Parseval's identity holds, then $\angles{x, u_i} = 0$ for all $i \in I$ implies that $x = 0$. > > *Proof*. Let $x \in \ch$ and suppose that $\angles{x, u_i} = 0 \forall i \in I$, then > $ > \norm{x}^2 = \sum_{i \in I}\abs{\angles{x, u_i}}^2 = 0 > $ > and $x = 0$. > [!theorem] > > Every Hilbert space has an orthonormal basis. > > *Proof*. A maximal orthonormal set is complete, and therefore an orthonormal basis. > [!theorem] > > Let $\ch$ be a Hilbert space. $\ch$ is [[Separable Topological Space|separable]] if and only if $\ch$ has a countable orthonormal basis, in which case every orthonormal basis is countable. > > ### Separable to Countable Orthonormal Basis > > Suppose that $\ch$ is separable, then $\ch$ has a countable orthonormal basis. > > *Proof*. Let $\seq{x_n}$ be a countable [[Dense|dense]] subset of $\ch$. Construct inductively a linearly independent subset of $\seq{x_n}$, then apply the [[Gram-Schmidt Process]] to obtain an orthonormal set $\seq{u_n}$, then the span of $\seq{u_n}$ is dense in $\ch$. > > Let $x \in \ch$, then there exists $\seq{x_n}$ such that $x_n \to x$ with $x_n = \sum_{k = 1}^{\infty}\angles{x_n, u_k}u_k$. For any $\varepsilon$, choose $m$ and $n$ such that > $ > \sum_{k = 1}^{m}\angles{x, u_k}u_k \approx \sum_{k = 1}^{m}\angles{x_n, u_k}u_k \approx \sum_{k = 1}^{\infty}\angles{x_n, u_k}u_k = x_n \approx x > $ > where each $\approx$ represents a distance of less than $\varepsilon/4$, then $\sum_{k = 1}^{m}\angles{x, u_k}u_k \to x$ as $m \to \infty$. > > > ### Countable Orthonormal Basis to Separable > > If $\ch$ has a countable orthonormal basis, then $\ch$ is separable. > > *Proof*. Let $F \subset \complex$ be a countable dense subset, then the collection of finite linear combinations of $\seq{x_n}$ with $F$ coefficients would be our countable dense subset.