Mackey theory asks the following question: if $H_1$ and $H_2$ are subgroups of $G$ and $\psi_1$ and $\psi_2$ are characters, then what is $\left\langle \psi_1^G, \psi_2^G \right\rangle$? This may seem like a technical question, and indeed many accounts of Mackey theory may not do much to dispel this impression. However Mackey's theorem is extremely important and useful, and properly understood it has a conceptual basis, which we hope to convey.
For simplicity, we will limit ourselves to the special case where $\psi_1$ and $\psi_2$ are linear characters, which makes for a minor simplification, and is already enough for some important examples.
We go back to the viewpoint that $\mathbb{C}[G]$ is the ring (under convolution) of complex valued functions on $G$. We recall the right regular representation $\rho : G \longrightarrow \operatorname{End} (\mathbb{C}[G])$, which is the action $(\rho (g) f) (x) = f (x g)$.
Lemma 4.3.1: Let $T : \mathbb{C}[G] \longrightarrow \mathbb{C}[G]$ be a linear transformation that commutes with $\rho (g)$; that is, $T (\rho (g) f) = \rho (g) T (f)$. Then there exists a unique $\lambda \in \mathbb{C}[G]$ such that $T (f) = \lambda \ast f$.
Proof. (Click to Expand/Collapse)
\[ f = \sum_{g \in G} f (g) \rho (g^{- 1}) \delta_0 . \] | (4.3.1) |
Now applying $T$ to (4.3.1) gives \[ T f = \sum_{g \in G} f (g) T \left( \rho (g^{- 1}) \delta_0 \right) = \sum_{g \in G} f (g) \rho (g^{- 1}) T \left( \delta_0 \right) = \sum_{g \in G} f (g) \rho (g^{- 1}) \lambda . \] Thus \[ T f (x) = \sum_g (\rho (g^{- 1}) \lambda) (x) f (g) = \sum_g \lambda (x g^{- 1}) f (g) = (\lambda \ast f) (x) . \]
We may regard $\psi_i$ as the character of $H_i$ acting on $V_i =\mathbb{C}$ (since $V_i$ is one-dimensional) with the representation $\pi_i (g) v = \psi_i (g) v$ when $g \in G$ and $v \in V_i =\mathbb{C}$. Then $\psi_i^G$ acts on the space $V_i^G$ of all functions $f_i : G \longrightarrow \mathbb{C}$ $(= V_i$) such that $f_i (h_i g) = \psi_i (h_i) f (g)$ when $h_i \in H_i$. The subspaces $V_i^G$ are thus invariant subspaces of $\mathbb{C}[G]$ under the action $\rho$, and the action of $G$ on $V_i^G$ is given by $\rho$.
Theorem 4.3.1:
(Geometric form of Mackey's Theorem.) Let
$\Lambda \in \operatorname{Hom}_G (V_1^G, V_2^G)$. Then there exists a function
$\Delta \in \mathbb{C}[G]$ such that
\[
\Delta (h_2 g h_1) = \psi_2 (h_2) \Delta (g)
\psi_1 (h_1), \hspace{2em} h_i \in H_i,
\]
(4.3.2)
Proof. (Click to Expand/Collapse)
\[ \Delta (g h_1) = \Delta (g) \psi_1 (h_1), \hspace{2em} h_1 \in H_1, \] | (4.3.3) |
\[ \Delta (h_2 g) = \psi_2 (h_2) \Delta (g), \hspace{2em} h_2 \in H_2 . \] | (4.3.4) |
Exercise 4.3.1: Fill out the details in the proof of Theorem 4.3.1.
Corollary 4.3.1: The inner product $\left\langle \psi_1^G, \psi_2^G \right\rangle$ equals the dimension of the vector space of functions $\Delta$ on $G$ that satisfy (4.3.2).
Proof. (Click to Expand/Collapse)
In every application, the first step is to compute the double cosets $H_2 \backslash G / H_1$. A double coset is a subset $H_2 x H_1$, and by definition the set of these is $H_2 \backslash G / H_1$. One can think of this as follows: the group $H_2 \times H_1$ acts on $G$ by \[ (h_2, h_1) : x \longmapsto h_2 x h_1^{- 1}, \] and the double cosets are the orbits. In many cases, there aren't very many. Once the double cosets are known, $\left\langle \psi_1^G, \psi_2^G \right\rangle$ can be computed by Mackey's theorem, and conclusions can be deduced from this information. For example if $H_1 = H_2$ and $\psi_1 = \psi_2$ we learn whether the induced representation is irreducible.
We will give typical applications in the following sections.