If $V$ is a complex vector space, by an *inner product*, also called a
*positive definite Hermitian form*, we mean a map $H : V \times V
\longrightarrow \mathbb{C}$ which has the following properties. First, it is
linear in the first variable, and antilinear in the second; that is,
\[ H (a u + a' u', v) = a H (u, v) + a' H (u', v), \hspace{2em} H (u, a v + a'
v') = \overline{a} H (u, v) + \overline{a'} H (u', v), \]
where $\overline{a}$ is the complex conjugate of $a$. Second, we have
\[ H (u, v) = \overline{H (v, w)} . \]
Note that this implies that $H (v, v)$ is real. The last condition to be
satisfied is that $H (v, v) > 0$ if $v \neq 0$. For example, we can put an
inner product on $\mathbb{C}^n$ by the rule
\[ H (v, w) = \sum v_i \overline{w_i}, \hspace{2em} v = \left(\begin{array}{c}
v_1\\
\vdots\\
v_n
\end{array}\right), \hspace{2em} v = \left(\begin{array}{c}
w_1\\
\vdots\\
w_n
\end{array}\right) . \]

We call a vector space with an inner product an *inner product space*.
We will usually fix the inner product, and use the notation $\left\langle v, w
\right\rangle = H (v, w)$. Moreover, we write $\|v\|= \sqrt{\left\langle v, v
\right\rangle}$ and call it the *length* of the vector $v$. Much of our
intuition about Euclidean geometry, such as the triangle inequality
\[ \|v + w\| \le \|v\|+\|w\| \]
and the Pythagorean theorem are applicable. To formulate this, we call vectors
$v$ and $w$ *orthogonal* if
\[ \left\langle v, w \right\rangle = 0. \]
Then the Pythagorean theorem asserts that:
\[ \text{if $v, w$ are orthogonal, we have $| | v + w\|^2 =\|v\|^2 +\|w\|^2 $}
. \]

**Exercise 2.1.1:***
Prove the triangle inequality and the Pythagorean theorem.
*

We digress to call the reader's attention to an important analytic concept,
that of a *Hilbert space*. An inner product space has a natural
topology. Indeed, using the triangle inequality, it is a metric space with the
distance function $d (v, w) =\|v - w\|$, and so we have a topology in which a
sequence $\{x_n \}$ converges to a point $x$ if $\|x_n - x\| \longrightarrow
0$ as $n \longrightarrow \infty$. If it is a complete metric space – that is,
if every Cauchy sequence converges – then the space is called a
*Hilbert space*. (Recall that $\{x_n \}$ is a *Cauchy sequence*
if $\|x_n - x_m \| \longrightarrow 0$ as $n, m \longrightarrow \infty$.)

Finite dimensional inner product spaces are *always* Hilbert spaces,
and the topology defined by this method is always the same as the usual
topology on a finite-dimensional complex vector space. But for
infinite-dimensional vector spaces, Hilbert spaces bring with them a few new
ideas that are mostly outside the scope of this text. Nevertheless, if one
goes on to develop representation theory for topological groups, Hilbert
spaces immediately arise, and what one finds is that the ideas from the
representation theory of finite groups generalizes very nicely. The
representation theory of finite *abelian* groups generalizes to the
theory of locally compact abelian groups, a theory that contains classical
Fourier analysis as a special case. For *nonabelian* groups, the
generalizations are more difficult, but well worth study. The representation
theory of Lie groups is truly one of the fundamental areas of mathematics,
with applications everywhere.

We will sometimes describe a finite-dimensional inner product space as a
*Hilbert space*. The reader should remember that the definition of a
Hilbert space has a topological component (completeness as a metric space)
that is automatic for finite-dimensional vector spaces.

Returning to finite-dimensional inner product spaces, a basis $y_1, \cdots,
y_n$ is called *orthogonal* if the $y_i$ are orthogonal to each other;
in other words $\left\langle y_i, y_j \right\rangle = 0$ if $i \neq j$. It is
called *orthonormal* if the $y_i$ have length 1, so that

\[ \left\langle y_i, y_j \right\rangle = \left\{ \begin{array}{ll} 1 & \text{if $i = j$,}\\ 0 & \text{if $i \neq j$.} \end{array} \right. \] | (2.1.1) |

**Lemma 2.1.1:***
Let $y_i$ be an orthonormal set, and let $v =
\sum_i a_i y_i$, with $a_i \in \mathbb{C}$. Then
*

**Proof. **(Click to Expand/Collapse)

We have
\[ \left\langle v, y_i \right\rangle = \left\langle \sum_j a_j y_j, y_i
\right\rangle = \sum_j a_j \left\langle y_j, y_i \right\rangle . \]
In this sum, orthonormality (2.1.1) implies that only one
term is nonzero, namely $j = i$; that term contributes $a_i$, and all the
others contribute zero, proving (2.1.2).

**Proposition 2.1.1:***
An orthonormal set is linearly independent.
*

**Proof. **(Click to Expand/Collapse)

If $\sum a_i y_i = 0$, then $a_i = 0$ by (2.1.2).

We will soon see that *an orthonormal basis is a very good thing*. The
Gram-Schmidt process guarantees the existence of orthonormal bases for
finite-dimensional vector spaces with inner products. We won't need it to
construct an orthonormal basis for $L^2 (G)$ when $G$ is abelian, though it
will be needed when $G$ is nonabelian. The idea is that we can start with an
arbitrary basis and modify it so that it becomes orthonormal.

**Proposition 2.1.2:***
( Gram-Schmidt algorithm.) If $V$ is a finite-dimensional vector space
with an inner product, and if $x_1, \cdots, x_n$ are a basis of $V$, then
there is a basis $y_1, \cdots, y_n$ such that $y_i$ equals a multiple of
$x_i$ plus a linear combination of $x_1, \cdots, x_{i - 1}$, and such that
$y_i$ are orthogonal. Thus
*

**Proof. **(Click to Expand/Collapse)

Suppose that $y_1, \cdots, y_{k - 1}$ have been constructed. Our induction
hypothesis is that they are orthonormal, and that the span of $y_1, \cdots,
y_k$ is the span of $x_1, \cdots, x_k$. Let
\[ y_k' = x_k - \sum_{i = 1}^{k - 1} \left\langle x_k, y_i \right\rangle \,
y_i . \]
We claim that $y_k'$ is orthogonal to $y_1, \cdots, y_{k - 1}$. Indeed,
write $y_k' = x_k - v$ where
\[ v = \sum_{i = 1}^{k - 1} a_i \, y_i, \hspace{2em} a_i = \left\langle x_k,
y_i \right\rangle . \]
By (2.1.2), we have, for $i < k$
\[ \left\langle v, y_i \right\rangle = a_i = \left\langle x_k, y_i
\right\rangle, \]
so
\[ \left\langle y_k', y_i \right\rangle = \left\langle x_k, y_i
\right\rangle - \left\langle v, y_i \right\rangle = 0. \]

Now we show that $y_1, \cdots, y_{k - 1}, y_k'$ and $x_1, \cdots, x_k$ span the same vector space. By the induction hypothesis, $y_1, \cdots, y_{k - 1}$ and $x_1, \cdots, x_{k - 1}$ span the same vector space, so what we need to know is that $y_k'$ can be expressed as a linear combination of $x_1, \cdots, x_{k - 1}, x_k$, and that $x_k$ can be expressed as a linear combination of $y_1, \cdots, y_{k - 1}, y_k'$. For the first, $y_k'$ is a linear combination of $y_1, \cdots, y_{k - 1}, x_k$, and since $y_1, \cdots, y_{k - 1}$ and $x_1, \cdots, x_{k - 1}$ span the same vector space, it is also a linear combination of $x_1, \cdots, x_{k - 1}, x_k$. For the second, we can write \[ x_k = y_k' + \sum_{i = 1}^{k - 1} \left\langle x_k, y_i \right\rangle \, y_i . \]

Although $y_k'$ is orthogonal to $y_i$, we also need $y_k$ to have length 1. At least it is nonzero, since if $y_k' = 0$, then span of $x_1, \cdots, x_k$ is the span of $y_1, \cdots, y_{k - 1}, y_k'$, that is, of $y_1, \cdots, y_{k - 1}$. But the $x_i$ are linearly independent, so $x_1, \cdots, x_k$ span a $k$-dimensional vector space, and they cannot be the linear combination of only $k - 1$ vectors. Now since $y_k' \neq 0$, we can define $y_k = y_k' /\|y_k' \|.$ The $y_k$ are now orthonormal.

Since part of what we have established by induction is that the span of $y_1, \cdots, y_k$ is the same as the span of $x_1, \cdots, x_k$, we can write $y_k$ as a linear combination of $x_1, \cdots, x_k$, proving (2.1.3).

Now we know that a finite-dimensional inner product space $V$ always has an orthonormal basis. Given an orthonormal basis $y_1, \cdots y_n$ of $V$, by we can write any vector as a linear combination of the $y_i$, and Lemma 2.1.1 identifies the coefficients. Thus

\[ v = \sum_i a_i y_i, \hspace{2em} a_i = \left\langle v, y_i \right\rangle . \] | (2.1.4) |

**Exercise 2.1.2:***
Let $V$ be a finite-dimensional Hilbert space, and
let $U$ be a vector subspace. Let
\[W =\{w \in V| \text{$\left\langle w, u \right\rangle = 0$ for all $u \in U$} \}.\]
Show that $V$ is the direct sum $U \oplus W$.
*