If $V$ is a complex vector space, by an inner product, also called a positive definite Hermitian form, we mean a map $H : V \times V \longrightarrow \mathbb{C}$ which has the following properties. First, it is linear in the first variable, and antilinear in the second; that is, \[ H (a u + a' u', v) = a H (u, v) + a' H (u', v), \hspace{2em} H (u, a v + a' v') = \overline{a} H (u, v) + \overline{a'} H (u', v), \] where $\overline{a}$ is the complex conjugate of $a$. Second, we have \[ H (u, v) = \overline{H (v, w)} . \] Note that this implies that $H (v, v)$ is real. The last condition to be satisfied is that $H (v, v) > 0$ if $v \neq 0$. For example, we can put an inner product on $\mathbb{C}^n$ by the rule \[ H (v, w) = \sum v_i \overline{w_i}, \hspace{2em} v = \left(\begin{array}{c} v_1\\ \vdots\\ v_n \end{array}\right), \hspace{2em} v = \left(\begin{array}{c} w_1\\ \vdots\\ w_n \end{array}\right) . \]
We call a vector space with an inner product an inner product space. We will usually fix the inner product, and use the notation $\left\langle v, w \right\rangle = H (v, w)$. Moreover, we write $\|v\|= \sqrt{\left\langle v, v \right\rangle}$ and call it the length of the vector $v$. Much of our intuition about Euclidean geometry, such as the triangle inequality \[ \|v + w\| \le \|v\|+\|w\| \] and the Pythagorean theorem are applicable. To formulate this, we call vectors $v$ and $w$ orthogonal if \[ \left\langle v, w \right\rangle = 0. \] Then the Pythagorean theorem asserts that: \[ \text{if $v, w$ are orthogonal, we have $| | v + w\|^2 =\|v\|^2 +\|w\|^2 $} . \]
Exercise 2.1.1: Prove the triangle inequality and the Pythagorean theorem.
We digress to call the reader's attention to an important analytic concept, that of a Hilbert space. An inner product space has a natural topology. Indeed, using the triangle inequality, it is a metric space with the distance function $d (v, w) =\|v - w\|$, and so we have a topology in which a sequence $\{x_n \}$ converges to a point $x$ if $\|x_n - x\| \longrightarrow 0$ as $n \longrightarrow \infty$. If it is a complete metric space – that is, if every Cauchy sequence converges – then the space is called a Hilbert space. (Recall that $\{x_n \}$ is a Cauchy sequence if $\|x_n - x_m \| \longrightarrow 0$ as $n, m \longrightarrow \infty$.)
Finite dimensional inner product spaces are always Hilbert spaces, and the topology defined by this method is always the same as the usual topology on a finite-dimensional complex vector space. But for infinite-dimensional vector spaces, Hilbert spaces bring with them a few new ideas that are mostly outside the scope of this text. Nevertheless, if one goes on to develop representation theory for topological groups, Hilbert spaces immediately arise, and what one finds is that the ideas from the representation theory of finite groups generalizes very nicely. The representation theory of finite abelian groups generalizes to the theory of locally compact abelian groups, a theory that contains classical Fourier analysis as a special case. For nonabelian groups, the generalizations are more difficult, but well worth study. The representation theory of Lie groups is truly one of the fundamental areas of mathematics, with applications everywhere.
We will sometimes describe a finite-dimensional inner product space as a Hilbert space. The reader should remember that the definition of a Hilbert space has a topological component (completeness as a metric space) that is automatic for finite-dimensional vector spaces.
Returning to finite-dimensional inner product spaces, a basis $y_1, \cdots, y_n$ is called orthogonal if the $y_i$ are orthogonal to each other; in other words $\left\langle y_i, y_j \right\rangle = 0$ if $i \neq j$. It is called orthonormal if the $y_i$ have length 1, so that
| \[ \left\langle y_i, y_j \right\rangle = \left\{ \begin{array}{ll} 1 & \text{if $i = j$,}\\ 0 & \text{if $i \neq j$.} \end{array} \right. \] | (2.1.1) |
Lemma 2.1.1:
Let $y_i$ be an orthonormal set, and let $v =
\sum_i a_i y_i$, with $a_i \in \mathbb{C}$. Then
\[
a_i = \left\langle v, y_i \right\rangle .
\]
(2.1.2)
Proof. (Click to Expand/Collapse)
Proposition 2.1.1: An orthonormal set is linearly independent.
Proof. (Click to Expand/Collapse)
We will soon see that an orthonormal basis is a very good thing. The Gram-Schmidt process guarantees the existence of orthonormal bases for finite-dimensional vector spaces with inner products. We won't need it to construct an orthonormal basis for $L^2 (G)$ when $G$ is abelian, though it will be needed when $G$ is nonabelian. The idea is that we can start with an arbitrary basis and modify it so that it becomes orthonormal.
Proposition 2.1.2:
(Gram-Schmidt algorithm.) If $V$ is a finite-dimensional vector space
with an inner product, and if $x_1, \cdots, x_n$ are a basis of $V$, then
there is a basis $y_1, \cdots, y_n$ such that $y_i$ equals a multiple of
$x_i$ plus a linear combination of $x_1, \cdots, x_{i - 1}$, and such that
$y_i$ are orthogonal. Thus
\[
\begin{array}{rcl}
y_1 & = & \lambda_1 x_1 \\
y_2 & = & \lambda_2 x_2 + c_{12} x_1 \\
y_3 & = & \lambda_3 x_3 + c_{13} x_1 + c_{23} x_2 \\
& \vdots &
\end{array}
\]
(2.1.3)
Proof. (Click to Expand/Collapse)
Now we show that $y_1, \cdots, y_{k - 1}, y_k'$ and $x_1, \cdots, x_k$ span the same vector space. By the induction hypothesis, $y_1, \cdots, y_{k - 1}$ and $x_1, \cdots, x_{k - 1}$ span the same vector space, so what we need to know is that $y_k'$ can be expressed as a linear combination of $x_1, \cdots, x_{k - 1}, x_k$, and that $x_k$ can be expressed as a linear combination of $y_1, \cdots, y_{k - 1}, y_k'$. For the first, $y_k'$ is a linear combination of $y_1, \cdots, y_{k - 1}, x_k$, and since $y_1, \cdots, y_{k - 1}$ and $x_1, \cdots, x_{k - 1}$ span the same vector space, it is also a linear combination of $x_1, \cdots, x_{k - 1}, x_k$. For the second, we can write \[ x_k = y_k' + \sum_{i = 1}^{k - 1} \left\langle x_k, y_i \right\rangle \, y_i . \]
Although $y_k'$ is orthogonal to $y_i$, we also need $y_k$ to have length 1. At least it is nonzero, since if $y_k' = 0$, then span of $x_1, \cdots, x_k$ is the span of $y_1, \cdots, y_{k - 1}, y_k'$, that is, of $y_1, \cdots, y_{k - 1}$. But the $x_i$ are linearly independent, so $x_1, \cdots, x_k$ span a $k$-dimensional vector space, and they cannot be the linear combination of only $k - 1$ vectors. Now since $y_k' \neq 0$, we can define $y_k = y_k' /\|y_k' \|.$ The $y_k$ are now orthonormal.
Since part of what we have established by induction is that the span of $y_1, \cdots, y_k$ is the same as the span of $x_1, \cdots, x_k$, we can write $y_k$ as a linear combination of $x_1, \cdots, x_k$, proving (2.1.3).
Now we know that a finite-dimensional inner product space $V$ always has an orthonormal basis. Given an orthonormal basis $y_1, \cdots y_n$ of $V$, by we can write any vector as a linear combination of the $y_i$, and Lemma 2.1.1 identifies the coefficients. Thus
| \[ v = \sum_i a_i y_i, \hspace{2em} a_i = \left\langle v, y_i \right\rangle . \] | (2.1.4) |
Exercise 2.1.2: Let $V$ be a finite-dimensional Hilbert space, and let $U$ be a vector subspace. Let \[W =\{w \in V| \text{$\left\langle w, u \right\rangle = 0$ for all $u \in U$} \}.\] Show that $V$ is the direct sum $U \oplus W$.