AN L-FUNCTION OF DEGREE 27 FOR Spin_{9}

by Daniel Bump and David Ginzburg

Stanford University University of Tel-Aviv

Abstract. This paper studies a Rankin-Selberg integral for a degree 27 L-function on Spin(9). It makes use of an Eisenstein series on the exceptional group F_{4}.

In this paper we introduce a Rankin-Selberg integral which represents the Lfunction for the second fundamental representation of PGSp_{8}(ℂ). More precisely, let π be a generic cuspidal representation of Spin_{9}(𝔸). Its L-group is PGSp_{8}(ℂ). Let ρ denote the second fundamental representation of PGSp_{8}(ℂ). Thus dim ρ = 27. The main result of this paper is to introduce a Rankin-Selberg integral for L_{S}(π, ρ, s). To do that we use an Eisenstein series on the exceptional group F_{4}. We will content ourselves with explaining the unfolding of the integral and the unramified computation.

In Bump and Ginzburg [B-G] we introduced an integral representation for the adjoint L-function of GL_{4}. This integral uses exactly the same Eisenstein series we use in this paper. This phenomena fits in the philosophy introduced in Ginzburg and Rallis [G-R]. Briefly, the idea behind [G-R] indicates that finite dimensional representations of L-groups which share the same structure of L-group invariants might have an integral representation which use the same Eisenstein series. In our case, the adjoint representation of GL_{4}(ℂ) and the representation ρ of PGSp_{8}(ℂ) have the same structure of invariants. Both have 3 generators of degrees 2,3 and 4 and both are free (see Kac [K]). These complex Lie groups are the L-groups of GL(4) and Spin(9). The philosophy predicts that the two corresponding L-functions may have Rankin-Selberg integrals involving the same Eisenstein series, and as it turns out this is true.

Finally as indicated in Ginzburg and Jiang [G-J], it is expected that this Lfunction will be related to the lifting from SO_{3}(𝔸) × SO_{3}(𝔸) × SO_{3}(𝔸) × SO_{3}(𝔸) into SO_{9}(𝔸). (See [G-J] for details.) We hope to use the results in this paper to study that lifting.

1. Notations. We start with several notations on the structure of the exceptional group F_{4} and some of its subgroups.

We shall label the positive simple roots of F_{4} as follows

\begin{matrix}α_{1} & α_{2} & & α_{3} & α_{4} \\ ∘ & ∘ & ⇒= & ∘ & ∘ \end{matrix}

Typeset by AMS-TEX

1991 Mathematics Subject Classification. Primary 11F70, Secondary 11F46, 22E46.

Key words and phrases. Rankin-Selberg integral, standard L-function, Spin L-function.

Research supported in part by NSF grants DMS-9622819.

Thus α_{1} and α_{2} are the long simple roots and α_{3} and α_{4} are the short ones. The list of all 24 positive roots of F_{4} can be found in the appendix to Bourbaki [B].

Given a root α, positive or negative, we shall denote by x_{α}(t) the one parameter unipotent subgroup corresponding to α. Here t belongs to the ring in question. If

α = ± ∑_{i=1}^{4} n_{i}α_{i}, where n_{i} ∈ ℕ we shall write x_{±(n_{1}n_{2}n_{3}n_{4})}(t) for x_{α}(t). Given roots

α, β we have

[x_{α}(r), x_{β}(t)] = x_{α+β}(nrt)x_{2α+β}(mr^{2}t)

where n, m ∈ ℤ. Even though it is not important to our construction we shall choose n, m as in [C]. In particular m ≠ 0 only if α and β have different length.

Let W (F_{4}) denote the Weyl group of F_{4}. Let w_{i} denote the simple reflection corresponding to the simple root α_{i}. We shall write w[i_{1} ⋯ i_{r}] for w_{i_{1}}w_{i_{2}} ⋯ w_{i_{r}}.

Recall that the subgroup of F_{4} generated by x_{α_{i}}(t) and x_{−α_{i}}(t) is a homomor-

phic image of SL_{2}. We shall denote by h_{α_{i}}(t) the image of ( \begin{matrix}t \\ & t^{−1} \end{matrix} ) under this

homomorphism. We shall denote the maximal torus of F_{4} by

h(t_{1}, t_{2}, t_{3}, t_{4}) = ∏_{i=1}^{4} h_{α_{i}}(t_{i}) .

For our calculations we will need to know the action of W (F_{4}) on x_{α}(t) and on the maximal torus of F_{4}. These actions can be easily read from the Cartan matrix of F_{4} and also from the decomposition of the unipotent radicals under the action of the Levi part of the various maximal parabolic.

Next we describe the embedding of the group G = Spin_{9} in H = F_{4}. Let G

denote the subgroup of H generated by x_{α}(r) and x_{−α}(t) where α = ∑_{i=1}^{4} n_{i}α_{i} is the

set of all positive roots with n_{4} = 0 or n_{4} = 2. Then G = Spin_{9}. More precisely let us label the simple roots of B_{4} as

\begin{matrix}β_{1} & β_{2} & β_{3} & & β_{4} \\ ∘ & ∘ & ∘ & ⇒= & ∘ \end{matrix}

Then to describe the embedding of Spin_{9} in F_{4} we identify β_{1} with the root α_{2} + 2α_{3} + 2α_{4} in F_{4}, β_{2} with α_{1}, β_{3} with α_{2} and β_{4} with α_{3}.

For a matrix representation of Spin_{9} see [G]. However to our purposes it is enough to study various subgroups of Spin_{9} such as parabolic subgroups. For this it is enough to work with the root system of B_{4} via its embedding in F_{4}. The simple reflections of Spin_{9} as embedded in W (F_{4}), are w_{1}, w_{2}, w_{3} and w[43234] where the last one corresponds to the simple root β_{1} = α_{2} + 2α_{3} + 2α_{4}.

Let P denote the maximal parabolic of H whose Levi part contains the group SL_{2} ×SL_{3} generated by x_{±α_{1}}, x_{±α_{3}} and x_{±α_{4}}. Let R be its unipotent radical. Thus dim R = 20. We shall denote by Q the maximal parabolic subgroup of H whose Levi part in GL_{1} · Spin_{7}. Its unipotent radical V is 15 dimensional.

We shall need the following lemma:

Lemma 1.1. 1) The space P \F_{4}/G contains three elements. We may choose as representatives e, x_{−0111}(1) and x_{−1221}(1).

2) Let γ be one of the three elements above. Denote G^{γ} = γ^{−1}P γ ∩ G. If γ is either e or x_{−0111}(1) then G^{γ} contains as a normal subgroup a unipotent radical of G. For the third γ we have G^{γ} = HU_{1} where H ≃ (GL_{2} × GL_{2})^{∘} and U_{1} is a nine dimensional unipotent subgroup. Here (GL_{2} × GL_{2})^{∘} = {(g_{1}, g_{2}) ∈ GL_{2} × GL_{2} : det g_{1} = det g_{2}}.

Proof. 1) We have F_{4} = ⋃_{w} P wQ = ⋃_{w} P wV (GL_{1} · Spin_{7}). Here GL_{1} · Spin_{7} is viewed as a subgroup of F_{4} via its embedding in Q. It is not hard to check that P \F_{4}/Q contains five elements which we may choose as representatives e, w[234], w[231234], w[234231234] and w[2312343231234]. Since GL_{1} · Spin_{7} ⊂ G and all

x_{α}(r) in V where α = ∑_{i=1}^{4} n_{i}α_{i} with n_{4} = 2 is in G as well we see that a set of

representatives for P \F_{4}/G is contained in elements of the form w. Here w is one of the above Weyl elements and v ∈ V is a product of elements of the form x_{α}(r) with α = n_{1}α_{1} + n_{2}α_{2} + n_{3}α_{3} + α_{4}. Clearly we need to consider those v such that wvw^{−1} ∉ P. Thus we obtain the following list of elements

1) e

2) w[234]x_{0001}(ϵ_{1})x_{0011}(ϵ_{2})

3) w[231234]x_{0001}(ϵ_{1})x_{0011}(ϵ_{2})x_{0111}(ϵ_{3})

4) w[234231234]x_{0001}(ϵ_{1})x_{0011}(ϵ_{2})x_{0111}(ϵ_{3})x_{0121}(ϵ_{4})

5) w[2341234231234]x_{0001}(ϵ_{1})x_{0011}(ϵ_{2})x_{0111}(ϵ_{3})x_{0121}(ϵ_{4})x_{1111}(ϵ_{5})x_{1121}(ϵ_{6}). Here ϵ_{i} are elements in the ring in question. To construct this list we need to find

out which roots go to negative under conjugation by w. For example consider case 3). Clearly w[231234](0001) < 0. Here we use the following notation. We write wα = β if wx_{α}w^{−1} = x_{β}. Since w[23123](0010) is negative then w[231234](0011) will be negative. Now w[2312](0100) < 0. Since w[34]^{−1}(0100) = (0122) we obtain that w[231234](0122) < 0, and so on. To shorten the list above let us write g_{1} ∼ g_{2} if g_{1} = pg_{2}g with p ∈ P and g ∈ G. We start with 2). We have

w[234]x_{0001}(ϵ_{1})x_{0011}(ϵ_{2}) ∼ w[234]x_{0001}(ϵ_{1})x_{0011}(ϵ_{2})x_{0010}(t) = w[234]x_{0010}(t)x_{0001}(ϵ_{1})x_{0011}(ϵ_{1}t_{1} + ϵ_{2}).

Here we used the commutation relation

[x_{0001}(ϵ_{1}), x_{0010}(t)] = x_{0011}(nϵ_{1}t) n ≠ 0

and we denoted t_{1} = nt. Hence if ϵ_{1} ≠ 0 we may choose t_{1} = −ϵ_{2}/ϵ_{1} and get w[234]x_{0010}(t) x_{0001}(ϵ_{1}) ∼ w[234]x_{0001}(ϵ_{1}) where the last equivalence follows from the fact that w[234](0010) = (0001). On the other hand if ϵ_{1} = 0 then w[234]x_{0011}(ϵ_{2}) ∼ w[234]x_{0011}(ϵ_{2})w_{3} = w[2343]x_{0001}(ϵ_{2}) = w[2434]x_{0001}(ϵ_{2}) ∼ w[234]x_{0001}(ϵ_{2}). The first equivalence is using the fact that w_{3} ∈ Spin_{9}. The second equality follows from the relation w_{3}x_{0011}(ϵ_{2})w_{3} = x_{0001}(ϵ_{2}). Then we used the relation w[343] = w[434] and the last equivalence follows from the fact that w_{4} ∈ P. Thus case 2) reduces to w[234]x_{0001}(ϵ). If ϵ = 0 then w[234] ∼ w[234] w[43234] = w[34] ∼ e. If ϵ ≠ 0 then we have x_{0001}(ϵ) = h_{α_{2}}(ϵ^{−1})x_{0001}(1)nh_{α_{2}}(ϵ)^{−1} and hence w[234]x_{0001}(ϵ) ∼ w[234]x_{0001}(1) ∼ w[43234] x_{0001}(1) ∼ x_{−0121}(1). Next we treat case 3). If ϵ_{2} ≠ 0, then using x_{0100}(t) we obtain

w[231234]x_{0001}(ϵ_{1})x_{0011}(ϵ_{2})x_{0111}(ϵ_{3}) ∼ w[231234]x_{0001}(ϵ_{1})x_{0011}(ϵ_{2})x_{0111}(ϵ_{2}t_{1} + ϵ_{3}) = w[231234]x_{0001}(ϵ_{1})x_{0011}(ϵ_{2}).

The first equivalence is obtained by conjugating x_{0100}(t) from right to left. We used the relation w[231234] x_{0100}(t)w[231234]^{−1} = x_{0100}(t) ∈ P. The second equality is obtained by choosing t_{1} = nt = −ϵ_{2}/ϵ_{1} for some n ∈ ℤ. If ϵ_{2} = 0, then

w[231234]x_{0001}(ϵ_{1})x_{0111}(ϵ_{3}) ∼ w[231234]x_{0001}(ϵ_{1})x_{0111}(ϵ_{3})w_{2} = w[2312342]x_{0001}(ϵ_{1})x_{0011}(ϵ_{3}) ∼ w[231234]x_{0001}(ϵ_{1})x_{0011}(ϵ_{3}).

The last equivalence is obtained from the relation w[2312342] = w[2132324] = w[2123234] = w[1213234] ∼ w[213234]. In the second equality we used w[3232] = w[2323]. Thus we need to consider the element w[231234]x_{0001}(ϵ_{1})x_{0011}(ϵ_{2}). If ϵ_{1} ≠ 0 we may use x_{0010}(t) as before to obtain w[231234]x_{0001}(ϵ_{1}). Using the relation h_{α_{3}}(ϵ_{1}^{−1})x_{0001}(ϵ_{1})h_{α_{3}}(ϵ_{1})^{−1} = x_{0001}(1) the last element is equivalent to w[231234]x_{0001}(1) ∼ x_{−1221}(1)w[231234] ∼ x_{−1221}(1) where the last equivalence is obtained from the facts that w[231234] ∼ w[231234]w[43234] = w[214] = w[412] ∼ w[4]. Since the root 1221 commutes with w[4] the result follows. If ϵ_{1} = 0 we obtain w[231234]x_{0011}(ϵ_{2}) = x_{−1110}(ϵ_{2})w[231234] ∼ x_{−1110}(ϵ_{2})w_{4} = w_{4}x_{−1111}(ϵ_{2}) ∼ x_{−1111}(ϵ_{2}). If ϵ_{2} = 0 we get the identity and if ϵ_{2} ≠ 0 then using a suitable torus element x_{−1111}(ϵ_{2}) ∼ x_{−1111}(1) ∼ w[13]x_{−1111}(1) = x_{−0121}(1)w[13] ∼ x_{0121}(1). Overall we get from case 3) a new representative i.e. x_{−1221}(1).

We will now show that cases 4) and 5) produce no new elements. We start with 4). Write γ = w[234231234]. Assume first that ϵ_{1} ≠ 0. Using x_{0010}(t) as before we obtain γx_{0001}(ϵ_{1})x_{0011}(ϵ_{2})x_{0111}(ϵ_{3})x_{0121}(ϵ_{4}) ∼ γx_{0001}(ϵ_{1})x_{0111}(ϵ_{3})x_{0121}(ϵ_{4}). Continue similarly with x_{0110}(t) to obtain γx_{0001}(ϵ_{1})x_{0121}(ϵ_{4}). Finally use x_{0120}(t) to obtain γx_{0001}(ϵ_{1}) ∼ γx_{0001}(1). Similarly we treat the case ϵ_{1} = 0. We get five possibilities, γ, γx_{0001}(1), γx_{0011}(1), γx_{0111}(1), γx_{0121}(1) or by conjugating γ to the right γ, x_{−1231}(1)γ, x_{−1221}(1)γ, x_{−0121}(1)γ, x_{−0111}(1)γ. However γ ∈ Spin_{9}. Thus, γ ∼ e. Also x_{−1231}(1)γ ∼ x_{−1231}(1) ∼ w_{3}x_{−1231}(1) = x_{−1221}(1)w_{3} =∼ x_{−1221}(1). Similarly x_{−0111}(1) ∼ x_{−0121}(1). Case 5) is similar. We obtain the following possibilities

γ, γx_{0001}(1), γx_{0011}(1), γx_{0111}(1), γx_{0121}(1), γx_{1111}(1), γx_{1121}(1)

where γ = w[2341234231234]. Conjugate γ to the right. We have

γ = γw[43234]w[43234] ∼ w[34].

Thus the above cases are equivalent to

e, x_{1232}(1)w[34], x_{−1231}(1)w[34], x_{−1111}(1)w[34], x_{−1110}(1)w[34], x_{−1111}(1)w[34], x_{−0110}(1)w[34] .

Conjugating w[34] to the left we obtain

e, x_{−1231}(1), x_{−1221}(1), x_{−1121}(1), x_{−1111}(1), x_{−0121}(1), x_{−0111}(1).

We have x_{−1231}(1) ∼ x_{−1221}(1), x_{−1121}(1) ∼ x_{−0121}(1) ∼ x_{−1111}(1) ∼ x_{−0111}(1).

To sum up we obtain that |P \F_{4}/G| = 3 and as representatives we may choose e, x_{−0111}(1) and x_{−1221}(1) of the lemma follows.

We proceed to part 2) of the lemma. If γ = e then G^{γ} = G ∩ P. It is easy to see that G ∩ P ⊂ Q_{1} where Q_{1} is the maximal parabolic subgroup of G whose Levi part is GL_{1} · Spin_{7}. Thus the radical of Q_{1} in G, which consists of all positive

roots in F_{4} of the form α = ∑_{i=1}^{3} n_{i}α_{i} + 2α_{4} is the normal subgroup of G^{γ} which is

a unipotent radical of G.

Next assume γ = x_{−0111}(1). We claim that G^{γ} is contained Q_{2}-the maximal parabolic subgroup of Spin_{9} whose Levi part contains GL_{4}. Consider γ^{−1}P γ. Up to a torus element P is generated by all x_{α}(r) where α is positive or α is either one of the negative roots −(1000); −(0010); −(0001); −(0011). Hence conjugation by the root −(0111) cannot give any of the roots −(0010); −(0110); -(1110); −(0120); −(1120); −(1220); −(1232); −(1242); −(1342) and −(2342). Since these are the negative roots of the roots which are in the unipotent radical of Q_{2} we deduce that G^{γ} ⊂ Q_{2}. Moreover one can check that up to element in P, x_{−0111}(1) commutes with the radical of Q_{2}. Indeed we have, [x_{−0111}(1), x_{0110}(ϵ_{1})] = x_{−0001}(n_{1}ϵ_{1}); [x_{0111}(1), x_{1232}(ϵ_{2})] = x_{1121}(n_{2}ϵ_{2}); [x_{−0111}(1), x_{1342}(ϵ_{3})] = x_{1231}(n_{3}ϵ_{3})x_{1120}(n_{4}ϵ_{3}) and [x_{−0111}(1), x_{α}(r)] = 1 where α is either 0010; 1110; 0120; 1120; 1220; 1242 or 2342. Here n_{i} ∈ ℤ. This implies that the unipotent radical is a normal subgroup of G^{γ}. Finally we consider γ = x_{−1221}(1). This is the open orbit representative. Thus dim G^{γ} = dim G − dim(P \F_{4}).

We now describe G^{γ}. We claim that G^{γ} = (GL_{2} × GL_{2})^{∘}U_{1} where the embedding is described as follows. First we define U_{1} to be the unipotent subgroup of Spin_{9} generated by the following one dimensional unipotent subgroups x_{0010}(r); x_{0120}(r); x_{1120}(r); x_{0122}(r); x_{1122}(r); x_{1232}(r); x_{1242}(r); x_{−1100}(r)x_{1342}(r); x_{−0100}(r)x_{2342}(r). To describe (GL_{2} × GL_{2})^{∘} we first define the following two SL_{2} subgroups of Spin_{9}. The first is generated by x_{1000}(r) and the second by x_{1220}(r)x_{−1222}(r) and x_{−1220}(r)x_{1222}(r). Consider the torus T defined by

T = {t : t = h(ab^{−2}c^{2}, ab^{−3}c^{4}, ab^{−3}c^{4}, ab^{−2}c^{3}) such that a, b, c are invertible}

We claim that the subgroup generated by the above two SL_{2} subgroups and T is isomorphic to (GL_{2} × GL_{2})^{∘}. Indeed given t ∈ T we have tx_{1000}(r)t^{−1} equal to

x_{1000}(abr); tx_{1220}(r)x_{−1221}(r)t^{−1} = x_{1220}(a^{−1}b^{−1}c^{−2}r)x_{−1221}(a^{−1}b^{−1}c^{−2}r)

and hence we may identify T with

t → \{ ( \begin{matrix}a \\ & b \end{matrix} ) , ( \begin{matrix}c^{−1} \\ & abc \end{matrix} ) \}

hence the claim follows. Let us show that G^{γ} ⊃ (GL_{2} × GL_{2})^{∘}. To do that we need to show that given g ∈ (GL_{2} × GL_{2})^{∘} there is a p ∈ P such that γ^{−1}pγ = g or p = γgγ^{−1}. Consider the first SL_{2} (as described above). Since, clearly, x_{−1221}(1)x_{±1000}(r)x_{−1221}(1)^{−1} = x_{±0000}(r) ⊂ P then this SL_{2} ⊂ G^{γ}. As for the second we use the commutation relation

[x_{−1221}(1), x_{1220}(r)] = x_{−0001}(−r)x_{−1222}(−r)

and the fact that x_{−1221} commutes with x_{−1222} to deduce that

x_{−1221}(1)x_{1220}(r)x_{−1222}(r)x_{−1221}(−1) =

x_{1220}(r)x_{−1221}(1)x_{−0001}(−r)x_{−1222}(−r)x_{1222}(r)x_{−1221}(−1) =
x_{1220}(r)x_{−0001}(−r) ∈ P.

Similarly for x_{−1220}(r)x_{1222}(r). Using the Cartan matrix of F_{4} we see that T commutes with x_{−1221}(1). Thus (GL_{2} × GL_{2})^{∘} ⊂ G^{γ}. In a similar way, one can check that U_{1} ⊂ G^{γ}. The fact that (GL_{2} × GL_{2})^{∘} normalizes U_{1} can be checked by direct commutations computations. Thus, G^{γ} = (GL_{2} × GL_{2})^{∘}U_{1}.

For latter reference it will be convenient to twist G^{γ} by a Weyl element of Spin_{9}. Also, to make things clearer we shall describe the projection of G^{γ} in the group SO_{9}. To do so we shall choose the form of definition as in [G1]. With this choice, the maximal torus of SO_{9} is parametrized as all 9 × 9 matrices of the form

diag(t_{1}, t_{2}, t_{3}, t_{4}, 1, t_{4}^{−1}, t_{3}^{−1}, t_{2}^{−1}, t_{1}^{−1})

Twist G^{γ} by the Weyl element w[2312] (this twist is made in F_{4}). Denote G_{1}^{γ} = w[2312]G^{γ}w[2312]. Let us describe the projection of G_{1}^{γ} in SO_{9}. First notice that w[2312] conjugates H = (GL_{2} × GL_{2})^{∘} as described above to the following isomorphic group. The first SL_{2} i.e. the one generated by x_{±1000} is conjugated to the SL_{2} generated by x_{±0120}. The other SL_{2} the one generated by x_{1220}(r)x_{−1222}(r) and its negative is conjugated by w[2312] to the SL_{2} generated by the roots x_{0100}(a)x_{0122}(a) and x_{−01000}(b)x_{−0122}(b). Also T is conjugated to all torus elements of the form h(ab^{−3}c^{4}, a^{3}b^{−6}c^{8}, a^{2}b^{−4}c^{5}, ab^{−2}c^{3}). In the projection of G_{1}^{γ} to SO_{9} the first SL_{2} will correspond to

diag\{ ( \begin{matrix}1 \\ & 1 \end{matrix} ) , ( \begin{matrix}a & 0 & & b & 0 \\ 0 & a & & 0 & −b \\ & & 1 \\ c & 0 & & d & 0 \\ 0 & −c & & 0 & d \end{matrix} ) , ( \begin{matrix}1 \\ & 1 \end{matrix} ) \}

and the second one corresponds to

diag{(g, g, 1, g^{∗}, g^{∗})} , g ∈ SL_{2}

where g^{∗} indicates that the above matrix is in SO_{9}. The torus

\{ ( \begin{matrix}a \\ & 1 \end{matrix} ) , ( \begin{matrix}a \\ & 1 \end{matrix} ) \} ⊂ (GL_{2} × GL_{2})^{∘}

is projected on

diag(1, 1, a, 1, 1, 1, a^{−1}, 1, a^{−1}) a ∈ F^{∗}

Next we conjugate the group U_{1} by w[2312]. In terms of the roots of F_{4} we obtain the unipotent group U_{2} generated by

x_{1110}(r); x_{1000}(r); x_{1120}(r); x_{1222}(r); x_{1342}(r) ; x_{1232}(r); x_{2342}(r); x_{1100}(r)x_{1122}(r); x_{1220}(r)x_{1242}(r)

To describe the projection on SO_{9} we denote by Ũ_{2} the unipotent radical of the maximal parabolic of SO_{9} whose Levi part is GL_{2} × SO_{5}. In matrices we choose Ũ_{2} to be all matrices in SO_{9} of the form

( \begin{matrix}I_{2} & ∗ & ∗ \\ & I_{5} & ∗ \\ & & I_{2} \end{matrix} )

Thus Ũ_{2} is two step unipotent such that [Ũ_{2}, Ũ_{2}] is one dimensional and Ũ_{2}/[Ũ_{2}, Ũ_{2}] can be identified with M_{2×5} – the set of all 2 × 5 matrices. U_{2} projects to Ũ_{2} as follows. x_{1000}(r) projects on re_{21}, x_{1222}(r) on re_{12}, x_{1110}(r) on re_{23}, x_{1232}(r) or re_{13}, x_{1120}(r) on re_{24}, x_{1342}(r) on re_{15}, x_{1100}(r) x_{1122}(r) on r(e_{11} + e_{22}) and x_{1220}(r) x_{1242}(r) on r(e_{14} + e_{25}). Here e_{ij} is the matrix with 1 in the (i, j) entry and zero elsewhere. Finally x_{2342}(r) projects on [Ũ_{2}, Ũ_{2}]. This completes the proof of Lemma 1.1. □

2. The Global Integral. In this section we introduce the global integral and show it to be Eulerian.

Let F be a global field and 𝔸 its ring f adeles. Let π be a cusp form on G(𝔸) = Spin_{9}(𝔸). We shall always assume π to be generic. This means the following. Let U be the maximal unipotent radical of Spin_{9}. The roots in U are as given in Section 1. Let ψ be an additive character of F \𝔸. We define a character ψ_{U} of U by

ψ_{U}(u) = ψ_{U}(x_{0122}(r_{1})x_{1000}(r_{2})x_{0100}(r_{3})x_{0010}(r_{4})u_{1}) = ψ(r_{1} + r_{2} + r_{3} + r_{4})

where u_{1} is the product of all other positive roots in a fixed order. We say that π is generic if the space of functions

W_{φ}(g) = ∫_{U_{F} \U_{𝔸}} φ(ug)ψ_{U}(u)du , g ∈ G(𝔸)

is nonzero for some φ ∈ π. (Here we confuse the representation π with its realization as space of functions on G(F)\G(𝔸). Let P be as in Section 1. Denote by δ_{P} its modular function and for s ∈ ℂ let Ind_{P (𝔸)}^{H(𝔸)}δ_{P}^{s} denote the space of all smooth function which satisfies

f_{s}(ph) = δ_{P}^{s} (p)f_{s}(h)

where p ∈ P (𝔸) and g ∈ H(𝔸). Set

E(h, f_{s}) = ∑_{γ∈P (F)\H(F)} f_{s}(γh) .

We introduce our global integral

I(φ, f_{s}, s) = ∫_{G(F)\G(𝔸)} φ(g)E(g, f_{s})dg

Here φ ∈ π. Since φ is a cusp form this integral converges absolutely for all s ∈ ℂ except, possibly, at points where E(g, f_{s}) has a pole. Next we prove that our integral is Eulerian.

Theorem 2.1. For all data as above and for ℜ(s) large we have

I(φ, f_{s}, s) = ∫_{U(𝔸)\G(𝔸)} ∫_{N(𝔸)} W_{φ}(g)f_{s}(γ_{0}ng)ψ_{N}(n)dndg

where γ_{0} = w[23121]x_{−0011}(1) and N and ψ_{N} will be defined later.

Proof. Following the usual unfolding process we obtain for ℜ(s) large,

I(φ, f_{s}, s) = ∑_{δ∈P (F)\F_{4}(F)/G(F)} ∫_{G^{δ}(F)\G(𝔸)} φ(g)f_{s}(δg)dg

From Lemma 1.1 it follows that there are three double cosets to consider. From part two of the lemma the first to contribute zero to I(φ, f_{s}, s). Thus we are left with the third one. As explained in the last part of the proof of Lemma 1.1 it will be convenient to replace the representative x_{−1221}(1) by x_{−1221}(1)w[2312]. We shall denote by γ this last element and G^{γ} = γ^{−1}P γ ∩G. This group was described in Lemma 1.1 and was denoted there by G_{1}^{γ}. Thus

I(φ, f_{s}, s) = ∫_{G^{γ}(F)\G(𝔸)} φ(g)f_{s}(γg)dg

and G^{γ} = H · U_{2}. We recall the definition of the groups (see Lemma 1.1). First H ≃ (GL_{2} × GL_{2})^{∘} which is generated by the roots

x_{0120}(r); x_{−0120}(r); x_{0100}(r)x_{0122}(r); x_{−0100}(r)x_{−0122}(r)

and the torus h(a, a, a, a). Then U_{2} is a unipotent subgroup of G which consists of the roots

x_{1110}(r); x_{1000}(r); x_{1120}(r); x_{1222}(r); x_{1342}(r); x_{1232}(r); x_{2342}(r); x_{1100}(r)x_{1122}(r); x_{1220}(r)x_{1242}(r).

We consider a Fourier expansion along the roots 1122 and 1242. Thus

I(φ, f_{s}, s) = ∫_{H(F)U_{2}(F)\G(𝔸)} ∑_{δ_{i}∈F} ∫_{(F \𝔸)^{2}} φ(x_{1122}(r_{1})x_{1242}(r_{2})g)×

ψ(δ_{1}r_{1} + δ_{2}r_{2}) dr_{i} f_{s}(γg) dg .

The group H acts on the character group of these two roots as follows. Write H = (SL_{2} × SL_{2}) ⋉ T where T = {h(a, a^{3}, a^{2}a)}. The first SL_{2}, which consists of the roots x_{0120}(r) and x_{−0120}(r) acts on x_{1122}(r_{1})x_{1242}(r_{2}) as the standard action of SL_{2}. T commutes with x_{1122}(r_{1}) and acts linearly on x_{1242}(r_{2}). Finally, the SL_{2} which consists of the roots x_{0100}(r)x_{0122}(r) and its negative commutes with x_{1122}(r_{1})x_{1242}(r_{2}) up to elements in U_{2}. Thus H acts with two orbits on x_{1122}(r_{1})x_{1242}(r_{2}). The first orbit, corresponds to the trivial character, contributes zero to the integral. Indeed, in this case we get as an inner integral

∫_{U_{2}(F)\U_{2}(𝔸)} ∫_{(F \𝔸)^{2}} φ(x_{1122}(r_{1})x_{1242}(r_{2})u_{2}g) dr_{i} du_{2} .

However, since the unipotent subgroup generated by U_{2} and the roots 1122 and 1242 form the unipotent radical of the GL_{2} ×Spin_{5} parabolic in Spin_{9}, the above integral is zero by cuspidality of φ. Consider the second orbit. Choose as representative the character ψ′ defined by

ψ′(x_{1122}(r_{1})x_{1242}(r_{2})) = ψ(r_{1})

The stabilizer in H is the group \{\binom{1 r}{1} × GL_{2}\} where the unipotent matrix

corresponds to x_{0120}(r). Thus

I(φ, f_{s}, s) = ∫_{GL_{2}(F)U_{3}(F)\G(𝔸)} ∫_{(F \𝔸)^{2}} φ(x_{1122}(r_{1})x_{1242}(r_{2})g)ψ(r_{1}) dr_{i}f_{s}(γf)dg

where U_{3} consists of the unipotent group generated by U_{2} and x_{0120}(r). Write

∫_{GL_{2}(F)U_{3}(F)\G(𝔸)} =  ∫_{GL_{2}(F)U_{3}(𝔸)\G(𝔸)} ∫_{U_{3}(F)\U_{3}(𝔸)}

Denote by U_{4} the unipotent group generated by U_{3} and the roots x_{1122} and x_{1242}. Define a character ψ_{U_{4}} on U_{4} as

ψ_{U_{4}}(u) = ψ_{U_{4}}(x_{1100}(r_{1})x_{1122}(r_{2})u′) = ψ(r_{1} + r_{2})

where u′ is a product of all other roots in U_{4} in any order. Since x_{1100}(r)x_{1122}(r) is a root in U_{3} a change of variables implies

I(φ, f_{s}, s) =  ∫_{GL_{2}(F)U_{3}(𝔸)\G(𝔸)} ∫_{U_{4}(F)\U_{4}(𝔸)} φ(ug)ψ_{U_{4}}(u)f_{s}(γg)du dg .

Next we consider a Fourier expansion along the roots x_{0010}(r_{1})x_{0110}(r_{2}). As before GL_{2} acts on these two roots with 2 orbits. The trivial orbits will contribute zero by cuspidality. This time using the GL_{4} radical in Spin_{9}. As a representative of the open orbit we choose

ψ′(x_{0010}(r_{1})x_{0110}(r_{2})) = ψ(r_{1}) .

The stabilizer in GL_{2} is ( \begin{matrix}a & r \\ 0 & 1 \end{matrix} ) where ( \begin{matrix}a \\ & 1 \end{matrix} ) is identified with h(a, a^{3}, a^{2}, a)

and \binom{1 r}{1} with x_{0100}(r)x_{0122}(r). Let U_{5} denote the unipotent subgroup of G

with gnerated by U_{3} and x_{0100}(r)x_{0122}(r). Thus

I(φ, f_{s}, s) =  ∫_{GL_{1}(F)U_{5}(𝔸)\G(𝔸)} ∫_{U_{4}(F)\U_{4}(𝔸)} ∫_{(F \𝔸)^{3}} φ(x_{0010}(r_{1})x_{0110}(r_{2})

ux_{0100}(r_{3})x_{0122}(r_{3})g)ψ_{U_{4}}(u)ψ(r_{1})f_{s}(γg) dr_{i} du dg

Denote by U_{6} the unipotent group generated by U_{4} and

x_{0010}(r_{1})x_{0110}(r_{2})x_{0100}(r_{3})x_{0122}(r_{3}).

Define a character ψ_{U_{6}} on U_{6} by

ψ_{U_{6}}(u) = ψ_{U_{6}}(x_{0010}(r_{1})x_{1100}(r_{2})x_{1122}(r_{3})u′) = ψ(r_{1} + r_{2} + r_{3})

where u′ is a product of all other roots in U_{6} in any fixed order. Thus

I(φ, f_{s}, s) =  ∫_{GL_{1}(F)U_{5}(𝔸)\G(𝔸)} ∫_{U_{6}(F)\U_{6}(𝔸)} φ(ug)ψ_{U_{6}}(u)f_{s}(γg)du dg .

In order to obtain the Whittaker functional for φ with respect to the positive roots of G as described in the beginning of this section we need to twist U_{6} by w_{1}. Thus

I(φ, f_{s}, s) =  ∫_{GL_{1}(F)U_{5}(A)\G(𝔸)} ∫_{U_{7}(F)\U_{7}(𝔸)} φ(uw_{1}g)ψ_{U_{7}}(u)f_{s}(γg)du dg

where U_{7} = w_{1}U_{6}w_{1} and ψ_{U_{7}} is defined as follows

ψ_{U_{7}}(u) = ψ_{U_{7}}(x_{0100}(r_{1})x_{0010}(r_{2})x_{0122}(r_{3})u′) = ψ(r_{1} + r_{2} + r_{3}) .

Let us list the roots in U_{7}. They are

x_{−1000}(r); x_{0122}(r); x_{1122}(r)x_{1100}(r); x_{1222}(r); x_{0010}(r); x_{1100}(r); x_{1110}(r); x_{1232}(r); x_{0120}(r); x_{1120}(r); x_{1242}(r)x_{1220}(r); x_{1342}(r); x_{2342}(r) .

Next we consider a Fourier expansion along the root x_{1100}(r). Thus

I(φ, f_{s}, s) = ∫ ∑_{δ∈F} ∫_{U_{7}(F)\U_{7}(𝔸)} ∫_{F \𝔸} φ(x_{1100}(r)uw_{1}g)ψ(δr)ψ_{U_{7}}(u)f_{s}(γg) dr du dg

where g is integrated as before. Write u ∈ U_{7} as u = u_{1}x_{−1000}(r_{1}). Thus

I(φ, f_{s}, s) = ∫ ∑_{δ∈F} ∫_{U_{8}(F)\U_{8}(𝔸)} ∫_{(F \𝔸)^{2}} φ(x_{1100}(r)u_{1}x_{−1000}(r_{1})w_{1}g)×

ψ_{U_{8}}(u_{1})ψ(δr)f_{s}(γg) dr dr_{1} du dg .

Here U_{8} is the unipotent group which consists of all position roots in U_{7}. Since φ is automorphic it is invariant by x_{−1000}(δ) on the right. We have

x_{−1000}(δ)x_{1100}(r)u_{1}x_{−1000}(r_{1}) = x_{1100}(r)x_{0100}(δr)x_{−1000}(δ)u_{1}x_{−1000}(r_{1}) =

x_{1100}(r)x_{0100}(δr)u_{1}^{δ}x_{−1000}(r_{1} + δ) .

Here u_{1}^{δ} = x_{−1000}(δ)u_{1}x_{−1000}(−δ). Plugging this into the above integral we first change variables u_{1}^{δ} → u_{1}. This does not effect ψ_{U_{8}}. Then changing variables in u_{1}, x_{0100}(δr)u_{1} → u_{1} we obtain a factor ψ(−δr). Thus

I(φ, f_{s}, s) = ∫ ∫_{𝔸} ∫_{U_{9}(F)\U_{9}(𝔸)} φ(ux_{−1000}(r)w_{1}g)ψ_{U_{9}}(u)f_{s}(γg) du dr dg .

Here U_{9} is the subgroup of U (the maximal unipotent of G) which consists of all positive roots except x_{1000}. Also

ψ_{U_{9}}(u) = ψ_{U_{9}}(x_{0100}(r_{1})x_{0010}(r_{2})x_{0122}(r_{3})u′) = ψ(r_{1} + r_{2} + r_{3}) .

Now consider a Fourier expansion along x_{1000}. The above GL_{1} acts with two orbits on this root. The trivial contributes zero and the open orbit gives

I(φ, f_{s}, s) = ∫_{U_{5}(𝔸)\G(𝔸)} ∫_{𝔸} W_{φ}(x_{−1000}(r)w_{1}g)f_{s}(γg) dr dg .

This by itself shows that I(φ, f_{s}, s) is Euclidean. We transform this integral to a different form. Conjugate x_{−1000}(r) across w_{1} and collapse the integration over r to obtain

I(φ, f_{s}, s) = ∫_{U′_{5}(𝔸)\G(𝔸)} W_{φ}(w_{1}g)f_{s}(γg)dg .

Here U′_{5} is generated by the roots

x_{1110}(r); x_{1120}(r); x_{1222}(r); x_{1342}(r); x_{1232}(r); x_{2342}(r); x_{0120}(r); x_{1100}(r)x_{1122}(r); x_{1220}(r)x_{1242}(r); x_{0100}(r)x_{0122}(r).

Changing variables w_{1}g → g we obtain

I(φ, f_{s}, s) = ∫_{U′_{5}(𝔸)\G(𝔸)} W_{φ}(g)f_{s}(γw_{1}g)dg

where U′′_{5} = w_{1}U′_{5}w_{1}. Write

∫_{U′′_{5}(𝔸)\G(𝔸)} = ∫_{U(𝔸)\G(𝔸)} ∫_{U′′_{5}(𝔸)\U(𝔸)}

We parametrize the representatives for U′′_{5}(𝔸)\U(𝔸) as

x_{1000}(r_{1})x_{0100}(r_{2})x_{0010}(r_{3})x_{1100}(r_{4})x_{1110}(r_{5})x_{1220}(r_{6})

Thus

I(φ, f_{s}, s) = ∫_{U(𝔸)\G(𝔸)} ∫_{U′′_{5}(𝔸)\U(𝔸)} W_{φ}(ug)f_{s}(γw_{1}ug)dudg .

We have γw_{1} = x_{−1221}(1)w[23121] = w[23121]x_{−0011}(1). Also we have

[x_{−0011}(1), x_{0010}(r)] = x_{−0001}(r)

and

w[23121]x_{0010}(r)w[23121]^{−1} = x_{1110}(r) .

Thus we obtain

I(φ, f_{s}, s) = ∫_{U(𝔸)\G(𝔸)} ∫_{N(𝔸)} W_{φ}(g)f_{s}(γ_{0}ng)ψ_{N}(n)dndg

where N is generated by the roots

x_{−0001}(r); x_{1000}(r); x_{1100}(r); x_{0100}(r); x_{1110}(r); x_{1220}(r)

and ψ_{N} is defined as follows

ψ_{N}(x_{−0001}(r_{1})x_{1000}(r_{2})x_{0001}(r_{3})x_{1100}(r_{4})x_{1110}(r_{5})x_{1220}(r_{6})) = ψ(r_{1} + r_{2} + r_{3}) .

This completes the proof of the theorem. □

In the next section we shall compute the unramified local integrals arising from this global integral.

3. The Unramified Computation. In this section we shall carry out the unramified computation for the local integrals obtained from the factorization of the global integral introduced in the last section.

Let F be a local nonarchimedean field. Let π be an unramified admissible irreducible representation of G(F). We shall assume that π is generic. Given a group H defined over F we shall denote by K(H) the standard maximal compact subgroup of H. Thus, from the definition there exists a unique function W (g) on G(F) satisfying

W (ug) = ψ_{U}(u)W (g), u ∈ U,
W (gk) = W (g), k ∈ K(G)
W (1) = 1.

Here U is the standard maximal unipotent subgroup of G and ψ_{U} is defined in the beginning of Section 2.

Let f_{s} ∈ Ind_{P}^{G}δ_{P}^{s} denote the normalized unramified vector in the induced space. Thus

f_{s}(pg) = δ_{P}^{s} (p)f_{s}(g), p ∈ P g ∈ F_{4},
f_{s}(gk) = f_{s}(g), k ∈ K(F_{4}),
f_{s}(1) = 1.

Since π is unramified, we may assume that π = Ind_{B(G)}^{G}(δ_{B(G)}^{1/2}χ) where χ is an unramified character and B(G) is the Borel subgroup of G.

Next we describe the local L-function we study. Since G = Spin_{9} then ^{L}G = PGSp_{8}(ℂ). Denote by ϖ_{2} the second fundamental representation of PGSp_{8}(ℂ). This is a 27 dimensional representation. Let t_{π} denote a representative of the semi-simple conjugacy class in PGSp_{8}(ℂ) associated with π. We define our local L-function

L(π, Λ^{2}, s) = det [I_{27} − ϖ_{2}(t_{π})q^{−s}]^{−1} .

Here q = |p| where p is a generator of the maximal ideal in the ring of integers of F. We used the notation Λ^{2} since ϖ_{2} ⊕1 is the restriction to Sp_{8}(ℂ) of the exterior square representation of GL_{8}(ℂ).

Finally we shall denote by ζ(s) the local zeta function i.e. ζ(s) = (1 − q^{−s})^{−1}. The local integral we study is

I(W, f_{s}, s) = ∫_{U\G} ∫_{F^{6}} W (g)f_{s}(w[23121]x_{−0011}(1)x_{−0001}(r_{1})

x_{1000}(r_{2})x_{0100}(r_{3})x_{1100}(r_{4})x_{1110}(r_{5})x_{1220}(r_{6})g)ψ(r_{1} + r_{2} + r_{3}) dr_{i}dg

Our main result is

Theorem 3.1. For all unramified data

I(W, f_{s}, s) = \frac{L(π, Λ^{2}, 5s − 2)}{ζ(5s)ζ(5s − 1)ζ(10s − 2)ζ(10s − 3)ζ(10s − 4)ζ(15s − 6)ζ(20s − 8)}

Proof.

We start by writing the Iwasawa decomposition for U\G. Thus, choosing the measure on K(G) to be one we obtain

I(W, f_{s}, s) = ∫_{T} ∫_{F^{6}} W (t)f_{s}(w[23121]x_{−0011}(1)x_{−0001}(r_{1})

x_{1000}(r_{2})x_{0100}(r_{3})x_{1100}(r_{4})x_{1110}(r_{5})x_{1220}(r_{6})t)ψ(r_{1} + r_{2} + r_{3})|t_{1}t_{2}t_{3}t_{4}^{−1}|^{−2} .

Here T is the maximal torus of F_{4} and is defined as all t = h(t_{1}, t_{2}, t_{3}, t_{4}) with t_{i} ∈ F^{∗}. Since the sum of all positive roots in G, as expressed in terms of the F_{4} roots, is 12α_{1} + 22α_{2} + 30α_{3} + 14α_{4} we obtain

δ_{B(G)}(h(t_{1}, t_{2}, t_{3}, t_{4})) = |t_{1}t_{2}t_{3}t_{4}^{−1}|^{2}

From the Cartan matrix of F_{4} we obtain the following identity. Let β = ∑ n_{i}α_{i}, and denote h = h(t_{1}, t_{2}, t_{3}, t_{4}), then

hx_{β}(r)h^{−1} = x_{β}(t_{1}^{2n_{1}−n_{2}}t_{2}^{−n_{1}+2n_{2}−n_{3}}t_{3}^{−2n_{2}+2n_{3}−n_{4}}t_{4}^{−n_{3}+2n_{4}}r)

From this it follows that

hx_{1220}(r)h^{−1} = x_{1220}(t_{2}t_{4}^{−2}r); hx_{1110}(r)h^{−1} = x_{1110}(t_{1}t_{4}^{−1}r);
hx_{1100}(r)h^{−1} = x_{1100}(t_{1}t_{2}t_{3}^{−2}r); hx_{0100}(r)h^{−1} = x_{0100}(t_{1}^{−1}t_{2}^{2}t_{3}^{−2}r);
hx_{1000}(r)h^{−1} = x_{1000}(t_{1}^{2}t_{2}^{−1}r); hx_{−0001}(r)h^{−1} = x_{−0001}(t_{3}t_{4}^{−2}r);
hx_{−0011}(r)h^{−1} = x_{−0011}(t_{2}^{−1}t_{3}t_{4}r).

We also have

w[23121]hw[23121]^{−1} = h(t_{2}^{−1}t_{3}^{2}, t_{1}^{−1}t_{2}^{−1}t_{3}^{2}t_{4}^{2}, t_{1}^{−1}t_{3}, t_{4}, t_{4}) .

Conjugating t to the left and changing variables in the r_{i}’s we obtain

I(W, f_{s}, s) = ∫_{T} ∫_{F^{6}} W (t)f_{s}(h(t_{2}^{−1}t_{3}^{2}, t_{1}^{−1}t_{2}^{−1}t_{3}^{2}t_{4}^{2}, t_{1}^{−1}t_{3}t_{4}, t_{4})w[23121]

x_{−0011}(t_{2}^{−1}t_{3}t_{4})x_{−0001}(r_{1})x_{1000}(r_{2})x_{0100}(r_{3})x_{1100}(r_{4})x_{1110}(r_{5})x_{1220}(r_{6}))

ψ(t_{3}t_{4}^{−2}r_{1} + t_{1}^{2}t_{2}^{−1}r_{2} + t_{1}^{−1}t_{2}^{2}t_{3}^{−2}r_{3})|t_{1}t_{2}t_{3}^{−5}t_{4}^{−3}| dr_{i} dt .

Since the sum of all roots in the unipotent radical of P is 15α_{1} +30α_{2} +40α_{3} +20α_{4} we have δ_{P} (h(t_{1}, t_{2}, t_{3}, t_{4})) = |t_{2}|^{5}. Define K(t) = W (t)δ_{B(G)}^{−1/2}. We have

I(W, f_{s}, s) =

∫_{T} ∫_{F^{6}} f_{s}(w[23121]x_{−0011}(t_{2}^{−1}t_{3}t_{4})x_{−0001}(r_{1})x_{1000}(r_{2})x_{0100}(r_{3})x_{1100}(r_{4})×

x_{1110}(r_{5})x_{1220}(r_{6})) K(t) ψ(t_{3}t_{4}^{−2}r_{1} +t_{1}^{2}t_{2}^{−1}r_{2} +t_{1}^{−1}t_{2}^{2}t_{3}^{−2}r_{3})|t_{1}^{−1}t_{2}^{−1}t_{3}^{2}t_{4}^{2}|^{5s−2} dr_{i} dt .

It will be convenient to parametrize T in a different way. Set

(3.1) T = {t = h(a^{2}b^{3}c^{4}d^{2}, a^{3}b^{6}c^{8}d^{4}, a^{2}b^{4}c^{6}d^{3}, ab^{2}c^{3}d^{2}) where a, b, c, d ∈ F^{∗}}

This parametrization has the advantage that

hx_{α_{1}}(r)h^{−1} = x_{α_{1}}(ar), hx_{α_{2}}(r)h^{−1} = x_{α_{2}}(br),
hx_{α_{3}}(r)h^{−1} = x_{α_{3}}(cr), hx_{α_{4}}(r)h^{−1} = x_{α_{4}}(dr).

Thus

I(W, f_{s}, s) = ∫_{T} ∫_{F^{6}} K(t)f_{s}(w[23121]x_{−0011}(cd)x_{−0001}(r_{1})x_{1000}(r_{2})×

x_{0100}(r_{3})x_{1100}(r_{4})x_{1110}(r_{5})x_{1220}(r_{6}))ψ(d^{−1}r_{1} + ar_{2} + br_{3})|ab^{3}c^{6}d^{4}|^{5s−2} dr_{i} dt

Conjugate x_{1110}(r_{5})x_{1220}(r_{6}) to the left. We obtain

I(W, f_{s}, s) = ∫_{T} ∫_{F^{6}} K(t)f_{s}(w[23]x_{0010}(r_{5})x_{0110}(r_{6})w[121]x_{−0011}(cd)n)

ψ(d^{−1}r_{1} + ar_{2} + br_{3})|ab^{3}c^{6}d^{4}|^{5s−2} dr_{i} dt,

where n is the product of the other four unipotent subgroups i.e. the ones integrating over r_{i}, 1 ≤ i ≤ 4.

Define a character χ_{s} of the Borel subgroup B(F_{4}) of F_{4} as follows. On T, let

χ_{s}(h(t_{1}, t_{2}, t_{3}, t_{4})) = |t_{1}t_{2}|^{5s−1}|t_{3}|^{−10s+4}|t_{4}|^{10s−3}

and extend it trivially to B(F_{4}). In the above integral the r_{5} and r_{6} integration defines an intertwining operator

M_{23} : Ind_{P}^{F_{4}}δ_{P}^{s} ⟶ Ind_{B(F_{4})}^{F_{4}}χ_{s}

That is, given h ∈ F_{4}

(M_{23}f̃_{s})(h) = ∫_{F^{2}} f̃_{s}(w[23]x_{0010}(r_{5})x_{0110}(r_{6})h) dr_{5}dr_{6}

where f̃_{s} ∈ Ind_{P}^{F_{4}}δ_{P}^{s}. Of course, M_{23} carries f_{s}, the unramified vector to a multiple of the unramified standard vector in Ind_{B(F_{4})}^{F_{4}}χ_{s}. To compute this multiple factor we let h = e, and we get

(M_{23}f_{s})(e) = \frac{ζ(5s − 1)}{ζ(5s)} \frac{ζ(10s − 3)}{ζ(10s − 2)}

This computation is a straightforward one and will be omitted. To simplify notations we shall denote by f_{s} the unramified standard vector in Ind_{B(F_{4})}^{F_{4}}χ_{s}.

Thus

I(W, f_{s}, s) =

\frac{ζ(5s − 1)}{ζ(5s)} \frac{ζ(10s − 3)}{ζ(10s − 2)} ∫_{T} ∫_{F^{4}} K(t)f_{s}(w[121]x_{−0011}(cd)x_{−0001}(r_{1})x_{1000}(r_{2})

x_{0100}(r_{3})x_{1100}(r_{4}))ψ(d^{−1}r_{1} + ar_{2} + br_{3})|ab^{3}c^{6}d^{4}|^{5s−2} dr_{i} dt .

We determine the domain of integration on T by using the properties of K(t). Recall that the simple roots of G as embedded in F_{4} are (1000); (0100); (0010) and (1220). We have for h parametrized as in (3.1)

hx_{1000}(r)h^{−1} = x_{1000}(ar); hx_{0100}(r)h^{−1} = x_{0100}(br)

hx_{0010}(r)h^{−1} = x_{0010}(cr); hx_{1220}(r)h^{−1} = x_{1220}(bc^{2}d^{2}r)

Thus K(t) = 0 if |a|, |b|, |c| or |bc^{2}d^{2}| are greater than one. Hence we may assume that the domain of integration on T is |a| ≤ 1; |b| ≤ 1; |c| ≤ 1; |bc^{2}d^{2}| ≤ 1. Going back to our integral write

I(W, f_{s}, s) = ∫_{|cd|≤1} + ∫_{|cd|>1}

We denote the first integral by I_{1} and the second by I_{2}. Thus

I_{1} = ∫_{{|a|,|b|,|c|≤1 \atop |cd|≤1}} ∫_{F^{4}} K(t)f_{s}[w[121]x_{−0001}(r_{1})x_{1000}(r_{2})x_{0100}(r_{3})x_{1100}(r_{4}))

ψ(d^{−1}r_{1} + ar_{2} + br_{3})|ab^{3}c^{6}d^{4}|^{5s−2} dr_{i} dt .

Here we conjugated x_{−0011}(cd) ∈ K(F_{4}) to the right. Also notice that |b| ≤ 1 and |cd| ≤ 1 imply |bc^{2}d^{2}| ≤ 1 hence this condition can be omitted. Next we separate the r_{1} integration to |r_{1}| ≤ 1 and |r_{1}| > 1. When |r_{1}| > 1 we use the SL_{2} identity

( \begin{matrix}1 & 0 \\ z & 1 \end{matrix} ) = \binom{z^{−1} 1}{z} ( \begin{matrix}0 & −1 \\ 1 & z^{−1} \end{matrix} )

16 DANIEL BUMP AND DAVID GINZBURG

when z ≠ 0, to obtain the relation

x_{−0001}(r_{1}) = x_{0001}(r_{1}^{−1})h(1, 1, 1, r_{1}^{−1})k(r_{1})

where k(r_{1}) ∈ K(F_{4}). Since clearly x_{0001}(r_{1}^{−1}) commutes with the roots x_{1000}(r_{2}) and x_{0100}(r_{3}) we obtain

I_{1}(W, f_{s}, s) =

∫_{T′} ∫_{F^{3}} K(t)f_{s}[w[121]x_{1000}(r_{2})x_{0100}(r_{3})x_{1100}(r_{4}))ψ(ar_{2} + br_{3})|ab^{3}c^{6}d^{4}|^{5s−2}×

( ∫_{|r_{1}|≤1} ψ(d^{−1}r_{1}) dr_{1}) dr_{i} dt

+ ∫_{T′} ∫_{F^{3}} ∫_{|r_{1}|>1} K(t)f_{s}[w[121]x_{1000}(r_{2})x_{0100}(r_{3})x_{1100}(r_{4})h(1, 1, 1, r_{1}^{−1}))

ψ(d^{−1}r_{1} + ar_{2} + br_{3})|ab^{3}c^{6}d^{4}|^{5s−2} dr_{i} dt .

Here T′ denotes the domain in T where |a|, |b|, |c| and |bc^{2}d^{2}| are less or equal to one. Conjugating h(1, 1, 1, r_{1}^{−1}) to the left and since f_{s}(h(1, 1, 1, r_{1}^{−1})) = |r_{1}|^{−10s+3} (recall that f_{s} ∈ Ind_{B(F_{4})}^{F_{4}}χ_{s})) we obtain

I_{1}(W, f_{s}, s) =

∫_{T′} ∫_{F^{3}} K(t)f_{s}(w[121]x_{1000}(r_{2})x_{0100}(r_{3})x_{1100}(r_{4}))ψ(ar_{2} + br_{3})|ab^{3}c^{6}d^{4}|^{5s−2}

( ∫_{|r_{1}|≤1} ψ(d^{−1}r_{1}) dr_{1} + ∫_{|r_{1}|>1} |r_{1}|^{−10s+3}ψ(d^{−1}r_{1}) dr_{1}) dr_{i} dt .

Notice that the inner integrals are both zero if |d^{−1}| > 1, and for |d^{−1}| ≤ 1 it equals

1 + ∫_{|r_{1}|>1} |r_{1}|^{−10s+3}ψ(d^{−1}r_{1}) dr_{1} = \frac{ζ(10s − 4)}{ζ(10s − 3)}(1 − |d|^{10s−4}q^{−10s+4}) .

The last equality follows from [G1] p. 776. Thus

I_{1}(W, f_{s}, s) =

\frac{ζ(10s − 4)}{ζ(10s − 3)} ∫_{{|a|,|b|,|c|≤1 \atop |cd|,|d^{−1}|≤1}} ∫_{F^{3}} K(t)f_{s}(w[121]x_{1000}(r_{1})x_{0100}(r_{2})x_{1100}(r_{3}))

ψ(ar_{1} + br_{2})|ab^{3}c^{6}d^{4}|^{5s−2}(1 − |d|^{10s−4}q^{−10s+4}) dr_{i} dt

Next we consider I_{2}(W, f_{s}, s). Since |cd| > 1 we have x_{−0011}(cd) = x_{0011}(c^{−1}d^{−1})h(1, 1, cd, cd)^{−1}k

AN L-FUNCTION OF DEGREE 27 FOR Spin_{9} 17

where k ∈ K(F_{4}). Hence I_{2}(W, f_{s}, s) =

∫_{T′′} ∫_{F^{4}} K(t)f_{s}(w[121]x_{−0001}(r_{1})x_{1000}(r_{2})x_{0100}(r_{3})x_{1100}(r_{4})h(1, 1, cd, cd)^{−1})

ψ(d^{−1}r_{1} + ar_{2} + br_{3})|ab^{3}c^{6}d^{4}|^{5s−2} dr_{i} dt

where T′′ is the domain in T where |a|, |b|, |c|, |bc^{2}d^{2}| ≤ 1 and |cd| > 1. We have, for h = h(1, 1, cd, cd),

hx_{1100}(r_{1})h^{−1} = x_{1100}(c^{−2}d^{−2}r); hx_{0100}(r)h^{−1} = x_{0100}(c^{−2}d^{−2}r)

hx_{−0001}(r)h^{−1} = x_{−0001}(c^{−1}d^{−1}r); hx_{1000}(r)h^{−1} = x_{1000}(r)

and

w[121]h(1, 1, c^{−1}d^{−1}, c^{−1}d^{−1})w[121] = h(c^{−2}d^{−2}, c^{−2}d^{−2}, c^{−1}d^{−1}, c^{−1}d^{−1})

Conjugating h(1, 1, cd, cd)^{−1} to the left, changing variables in r_{i} and using the properties of f_{s} we obtain,

I_{2}(W, f_{s}, s) = ∫_{T′′} ∫_{F^{4}} K(t)f_{s}(w[121]x_{−0001}(r_{1})x_{1000}(r_{2})x_{0100}(r_{3})x_{1100}(r_{4}))

ψ(cr_{1} + ar_{2} + bc^{2}d^{2}r_{3})|ab^{3}c^{2}|^{5s−2} dr_{i} dt .

As in the case of I_{1}(W, f_{s}, s) we compute the r_{1} integration to obtain

I_{2}(W, f_{s}, s) = \frac{ζ(10s − 4)}{ζ(10s − 3)} ∫_{T′′} ∫_{F^{3}} K(t)f_{s}(w[121]x_{1000}(r_{2})x_{0100}(r_{3})x_{1100}(r_{4}))

ψ(ar_{2} + bc^{2}d^{2}r_{3})|ab^{3}c^{2}|^{5s−2}(1 − |c|^{10s−4}q^{−10s+4}) dr_{i} dt .

Given z_{1}, z_{2} ∈ F with |z_{i}| ≤ 1, define

H(z_{1}, z_{2}) = ∫_{F^{3}} f_{s}(w[121]x_{1000}(r_{1})x_{0100}(r_{2})x_{1100}(r_{3}))ψ(z_{1}r_{1} + z_{2}r_{2}) dr_{i}

One can easily check that if z_{1} = p^{k}ϵ and z_{2} = p^{r}η where k, r ∈ ℕ and |ϵ| = |η| = 1 then

H(z_{1}, z_{2}) = H(p^{k}, p^{r})

We have I(W, f_{s}, s) =

\frac{ζ(5s − 1)ζ(10s − 4)}{ζ(5s)ζ(10s − 2)}[ ∫_{T′} K(t)H(a, b)|ab^{3}c^{6}d^{4}|^{5s−2}(1 − |d|^{10s−4}q^{−10s+4}) dt

+ ∫_{T′′} K(t)H(a, bc^{2}d^{2})|ab^{3}c^{2}|^{5s−2}(1 − |c|^{10s−4}q^{−10s+4}) dt],

where

T′ = {t ∈ T : |a|, |b|, |c|, |cd|, |d^{−1}| ≤ 1}

and

T′′ = {t ∈ T : |a|, |b|, |c|, |bc^{2}d^{2}| ≤ 1; |cd| > 1}

Lemma 3.2. Let x = q^{−5s+2}. Then for r, k ∈ ℕ

H(p^{k}, p^{r}) = \frac{ζ(5s − 2)^{2}}{ζ(5s − 1)^{2}} \frac{ζ(10s − 4)}{ζ(10s − 3)}(1 − x^{r+1})(1 − x^{k+1})(1 − x^{r+k+2})

Proof. This is proved in [B-G], Lemma 4.2.

Before going back to our integral we fix some notations. Recall that the L-group of Spin_{9} is PGSp_{8}(ℂ). Let ϖ_{i}, 1 ≤ i ≤ 4 denote the fundamental representations of PGSp_{8}(ℂ). We shall denote by C_{4}[r_{1}, r_{2}, r_{3}, r_{4}] the character of the irreducible

representation ∑_{i=1}^{4} r_{i}ϖ_{i} of PGSp_{8}(ℂ) evaluated at t_{π}. Here r_{i} ∈ ℕ.

Returning to I(W, f_{s}, s) we get a = p^{n_{1}}ϵ_{1}, b = p^{n_{2}}ϵ_{2}, c = p^{n_{3}}ϵ_{3} and d = p^{n_{4}}ϵ_{4}. Using the Casselman-Shalika formula [C-S] we have

K(t) = C_{4}[n_{2} + 2n_{3} + 2n_{4}, n_{1}, n_{2}, n_{3}]

Let x = q^{−5s+2}. For r, k ∈ ℕ set

L(k, r) = (1 − x^{r+1})(1 − x^{k+1})(1 − x^{r+k+2})

Thus

H(p^{k}, p^{r}) = \frac{ζ(5s − 2)^{2}ζ(10s − 4)}{ζ(5s − 1)^{2}ζ(10s − 3)}L(k, r)

Combining all this we obtain

I(W, f_{s}, s) = \frac{ζ(5s − 2)^{2}ζ(10s − 4)^{2}}{ζ(5s)ζ(5s − 1)ζ(10s − 2)ζ(10s − 3)}

[ ∑_{{n_{i}=0 \atop n_{3}≥n_{4}}}^{∞} C_{4}[n_{2} + 2n_{3} − 2n_{4}, n_{1}, n_{2}, n_{3}] L(n_{1}, n_{2}) (1 − x^{2n_{4}+2}) x^{n_{1}+3n_{2}+6n_{3}−4n_{4}}+

∑_{{n_{1},n_{2},n_{3}=0 \atop n_{2}+2n_{3}+2n_{4}=0} ; n_{3}+n_{4}<0}^{∞} C_{4}[n_{2} +2n_{3} +2n_{4}, n_{1}, n_{2}, n_{3}]L(n_{1}, n_{2} +2n_{3} +2n_{4})(1−x^{2n_{3}+2})x^{n_{1}+3n_{2}+2n_{3}}].

Consider the first sum. It equals ∑_{n_{1},n_{2},n_{4}=0}^{∞} ∑_{n_{3}=n_{4}}^{∞}. Changing summation n_{3} →

n_{3} + n_{4} the first sum equals

∑_{n_{i}=0}^{∞} C_{4}[n_{2} + 2n_{3}, n_{1}, n_{2}, n_{3} + n_{4}]L(n_{1}, n_{2})(1 − x^{2n_{4}+2})x^{n_{1}+3n_{2}+6n_{3}+2n_{4}}

In the second sum change index of summation n_{4} → −n_{4} to obtain

∑_{n_{1},n_{3}=0}^{∞} ∑_{n_{4}=n_{3}+1}^{∞} ∑_{n_{2}=2n_{4}−2n_{3}}^{∞} C_{4}[n_{2} + 2n_{3} − 2n_{4}, n_{1}, n_{2}, n_{3}]×

L(n_{1}, n_{2} + 2n_{3} − 2n_{4})(1 − x^{2n_{3}+2})x^{n_{1}+3n_{2}+2n_{3}} .

Change variables n_{4} → n_{4} + n_{3} + 1. We get

∑_{n_{1},n_{3},n_{4}=0}^{∞} ∑_{n_{2}=2n_{4}+2}^{∞} C_{4}[n_{2} − 2n_{4} − 2, n_{1}, n_{2}, n_{3}]×

L(n_{1}, n_{2} − 2n_{4} − 2) (1 − x^{2n_{3}+2})x^{n_{1}+3n_{2}+2n_{3}}

Change variables n_{2} → n_{2} + 2n_{4} + 2 to obtain

∑_{n_{i}=0}^{∞} C_{4}[n_{2}, n_{1}, n_{2} + 2n_{4} + 2, n_{3}] L(n_{1}, n_{2}) (1 − x^{2n_{3}+2}) x^{n_{1}+3n_{2}+2n_{3}+6n_{4}+6}

Thus we have

I(W, f_{s}, s) = \frac{ζ(5s − 2)^{2}ζ(10s − 4)^{2}}{ζ(5s)ζ(5s − 1)ζ(10s − 2)ζ(10s − 3)}

[ ∑_{n_{i}=0}^{∞} C_{4}[n_{2} + 2n_{3}, n_{1}, n_{2}, n_{3} + n_{4}] L(n_{1}, n_{2}) (1 − x^{2n_{4}+2}) x^{n_{1}+3n_{2}+6n_{3}+2n_{4}}

+ x^{6} ∑_{n_{i}=0}^{∞} C_{4}[n_{2}, n_{1}, n_{2} + 2n_{4} + 2, n_{3}]L(n_{1}, n_{2})(1 − x^{2n_{3}+2})x^{n_{1}+3n_{2}+2n_{3}+6n_{4}}].

To prove Theorem 3.1 we need to show

(3.2) (1 − x)^{2}(1 − x^{2})^{3}(1 − x^{3})(1 − x^{4})L(π, Λ^{2}, 5s − 2) =

∑_{n_{i}=0}^{∞} C_{4}[n_{2} + 2n_{3}, n_{1}, n_{2}, n_{3} + n_{4}] L(n_{1}, n_{2}) (1 − x^{2n_{4}+2})x^{n_{1}+3n_{2}+6n_{3}+2n_{4}}

+ x^{6} ∑_{n_{i}=0}^{∞} C_{4}[n_{2}, n_{1}, n_{2} + 2n_{4} + 2, n_{3}] L(n_{1}, n_{2}) (1 − x^{2n_{3}+2}) x^{n_{1}+3n_{2}+2n_{3}+6n_{4}}.

4. The proof of identity (3.2). To complete the proof of Theorem 3.1 we shall now prove (3.2).

Let S^{r}(ϖ_{2}) denote the symmetric r-th operation applied to the second fundamental representation ϖ_{2} of PGSp(4) Using the Poincare identity

L(π, Λ^{2}, 5s − 2) = ∑_{r=0}^{∞} S^{r}(ϖ_{2})x^{r}

to prove (3.2), we need to prove

(4.1) (1 − x)^{2}(1 − x^{2})^{3}(1 − x^{3})(1 − x^{4}) ∑_{r=0}^{∞} S^{r}(ϖ_{2})x^{r} =

∑_{n_{i}=0}^{∞} C_{4}[n_{2} + 2n_{3}, n_{1}, n_{2}, n_{3} + n_{4}] M(n_{1}, n_{2}, n_{4}, n_{3})+

x^{6} ∑_{n_{i}=0}^{∞} C_{4}[n_{2}, n_{1}, n_{2} + 2n_{4} + 2, n_{3}] M(n_{1}, n_{2}, n_{3}, n_{4})

where we set

M(n_{1}, n_{2}, n_{3}, n_{4}) = L(n_{1}, n_{2}) (1 − x^{2n_{3}+2}) x^{n_{1}+3n_{2}+2n_{3}+6n_{4}} .

Let A_{7}[r_{1}, ⋯, r_{7}], where r_{i} ∈ ℕ, denote the irreducible representation of SL_{8}(ℂ)

with highest weight vector ∑_{i=1}^{7} r_{i}ν̃_{i} where ν̃_{i} are the 7 fundamental representa-

tions of SL_{8}(ℂ). Thus A_{7}[1, 0, 0, 0, 0, 0, 0] is the standard representation of SL_{8}(ℂ), A_{7}[0, 1, 0, 0, 0, 0, 0] = Λ^{2} is the exterior square, etc. In Sp_{8}(ℂ) we shall denote by C_{4}[r_{1}, r_{2}, r_{3}, r_{4}] C_{4}[m_{1}, m_{2}, m_{3}, m_{4}] or C_{4}[r_{1}, r_{2}, r_{3}, r_{4}] ⊗ C_{4}[m_{1}, m_{2}, m_{3}, m_{4}] the tensor product of the representations.

Lemma 4.1. We have the identity

(4.2) ∑_{n,k=0}^{∞} A_{7}[0, n, 0, k, 0, 0, 0]x^{n+2k} =

(1 − A_{7}[0, 0, 0, 0, 0, 1, 0]x^{3} + A_{7}[1, 0, 0, 0, 0, 0, 1]x^{4} − A_{7}[2, 0, 0, 0, 0, 0, 0]x^{5}−

A_{7}[0, 0, 0, 0, 0, 0, 2]x^{7} + A_{7}[1, 0, 0, 0, 0, 0, 1]x^{8} − A_{7}[0, 1, 0, 0, 0, 0, 0]x^{9} + x^{12})×

∑_{r=0}^{∞} S^{r}(Λ^{2})x^{r}.

Proof. We will use the notations of Macdonald [M] for Schur functions. If λ is a partition, we will consider the Schur function s_{λ} to be a symmetric polynomial in 8 variables. This means that s_{λ} = 0 if the length l(λ) > 8. Applied to the eigenvalues of a matrix in GL(8, ℂ), the s_{λ} gives the character A_{7}[λ_{1} − λ_{2}, ⋯, λ_{7} − λ_{8}, λ_{8}], (or zero if l(λ) > 8).

Thus we must show that

( ∑ ; λ′ even s_{λ}) (1 − s_{{1^{6}}} + s_{{21^{6}}} − s_{{31^{7}}} − s_{{2^{7}}} + s_{{32^{6}1}} − s{3^{2}2^{6}} + s_{{3^{8}}}) =

∑ ; λ′ even, l(λ) ≤ 4 s_{λ}.

We may rewrite the left side

( ∑ ; λ′ even s_{λ}) (1 − e_{6} + e_{7} e_{1} − e_{8} − h_{2} e_{8} − e_{7}^{2} + e_{8}e_{6} + e_{7}e_{1}e_{8} − e_{8}^{2} − e_{2} e_{8}^{2} + e_{8}^{3}),

where e_{i} and h_{i} are the elementary and complete symmetric polynomials, as in [M]. Distributing the infinite sum on the left over the eight terms in parenthesis and applying Pieri’s formula we see that the coefficient of a Schur function s_{μ} (where μ is assumed to have length ≤ 8) on the left side equals

I − II + III − IV − V − VI + VII + VIII − IX − X + XI, where the individual contributions are as follows. If μ_{8} ≥ k, we denote by μ^{(k)} the partition {μ_{1} − k, μ_{2} − k, ⋯ μ_{8} − k}.

I = 1 if μ′ is even, zero otherwise;

II = the number of ways of removing a vertical strip of length 6 and arriving at a partition λ such that λ′ is even;

III = the number of ways of removing a single node from μ to obtain a partition, then removing a vertical strip of length 7 to obtain a partition λ such that λ′ is even.

IV = 1 if μ_{8} ≥ 1 and μ′ is even, zero otherwise.

V = the number of ways of removing a horizontal strip of length 2 from μ^{(1)} to obtain a partition λ such that λ′ is even if μ_{8} ≥ 1, and zero otherwise;

VI = the number of ways of removing a vertical strip of length 7 to obtain a partition, then removing another vertical strip of length 7 to obtain another partition λ such that λ′ is even.

VII = the number of ways of removing a vertical strip of length 6 from μ^{(1)} to obtain a partition λ such that λ′ is even if μ_{8} ≥ 1, zero otherwise.

VIII = the number of ways of removing a single node from μ^{(1)} to obtain a partition, then removing a vertical strip of length 7 to obtain another partition λ such that λ′ is even provided μ_{8} ≥ 1, zero otherwise;

IX = 1 if μ_{8} ≥ 2 and μ′ is even, zero otherwise;

X = the number of ways of removing a vertical strip of length 2 from μ^{(2)} to obtain a partition λ such that λ′ is even, provided μ_{8} ≥ 2, zero otherwise;

XI = 1 if μ_{8} ≥ 3 and μ′ is even, zero otherwise.

The following table gives the values of these parameters in many cases. In this table ϵ is the number of distinct values in the set {μ_{2}, μ_{4}, μ_{6}, μ_{8}}. First we consider the case where μ′ is even. In this case, the values are given by the following table:

I II III IV V VI VII VIII IX  X XI
l(μ) ≤ 4  1 0 0 0 0 0  0  0 0  0 0
l(μ) = 6  1 1 0 0 0 0  0  0 0  0 0
l(μ) = 8, μ_{8} = 1, μ_{6} = 1 1 ϵ ϵ 1 0 0  0  0 0  0 0
l(μ) = 8, μ_{8} = 1, μ_{6} > 1 1 ϵ ϵ 1 0 1  1  0 0  0 0
l(μ) = 8, μ_{8} = 2  1 ϵ ϵ 1 0 ϵ  ϵ  ϵ 1 ϵ − 1 0
l(μ) = 8, μ_{8} ≥ 3  1 ϵ ϵ 1 0 ϵ  ϵ  ϵ 1  ϵ 1

Next we consider the case where μ′ is not even. In order to analyze these cases, let us define α_{i} to be the number of k ∈ {2, 4, 6, 8} such that μ_{k} − μ_{k−1} = i. Thus ∑_{i=0}^{∞} α_{i} = 4, and the condition that μ′ is not even amounts to α_{0} < 4. This is assumed in the following table.

I II III IV V VI VII VIII IX X XI
l(μ) ≤ 6  0 0 0 0 0 0  0  0 0 0 0
μ_{7} = 1, μ_{8} = 0, α_{0} = α_{2} = 1 0 1 1 0 0 0  0  0 0 0 0
μ_{7} = 2, μ_{8} = 0, α_{0} = 3  0 0 1 0 0 1  0  0 0 0 0
l(μ) = 7, other  0 0 0 0 0 0  0  0 0 0 0
μ_{7} = μ_{8} = 1, α_{0} = α_{1} = 2  0 1 2 0 1 0  0  0 0 0 0
μ_{7} = μ_{8} = α_{2} = 1, α_{0} = 3  0 0 1 0 1 0  0  0 0 0 0
μ_{8} = 1, μ_{7} = α_{0} = α_{1} = 2  0 1 2 0 1 1  1  0 0 0 0
μ_{8} = α_{2} = 1, μ_{7} = α_{0} = 3  0 0 1 0 1 1  0  1 0 0 0
μ_{8} ≥ 2, α_{0} = α_{1} = 2  0 1 2 0 1 2  1  2 0 1 0
μ_{8} ≥ 2, α_{0} = 3, α_{2} = 1  0 0 1 0 1 1  0  1 0 0 0
l(μ) = 8, other  0 0 0 0 0 0  0  0 0 0 0

Therefore the coefficient of s_{μ} equals 1 if μ′ is even and l(μ) ≤ 4, and zero otherwise, proving the Lemma. □

When restricting to Sp_{8}(ℂ) we have

A_{7}[0, 1, 0, 0, 0, 0, 0]|_{Sp_{8}(ℂ)} = A_{7}[0, 0, 0, 0, 0, 1, 0]|_{Sp_{8}(ℂ)} = C_{4}[0, 1, 0, 0] + 1,

A_{7}[2, 0, 0, 0, 0, 0, 0]|_{Sp_{8}(ℂ)} = A_{7}[0, 0, 0, 0, 0, 0, 2]|_{Sp_{8}(ℂ)} = C_{4}[2, 0, 0, 0],
A_{7}[1, 0, 0, 0, 0, 0, 1]|_{Sp_{8}(ℂ)} = C_{4}[2, 0, 0, 0] + C_{4}[0, 1, 0, 0].

Since Λ^{2}|_{Sp_{8}(ℂ)} = ϖ_{2} + 1 we have

∑_{r=0}^{∞} S^{r}(Λ^{2})|_{Sp_{8}(ℂ)}x^{r} = \frac{1}{1 − x} ∑_{r=0}^{∞} S^{r}(ϖ_{2})x^{r}

Hence, restricting (4.2) to Sp_{8}(ℂ) we obtain

∑_{n,k=0}^{∞} A_{7}[0, n, 0, k, 0, 0, 0]|_{Sp_{8}(ℂ)}x^{n+2k} =

\frac{1}{1 − x}(1 − (C_{4}[0, 1, 0, 0] + 1)x^{3} + (C_{4}[2, 0, 0, 0] + C_{4}[0, 1, 0, 0])x^{4} − C_{4}[2, 0, 0, 0]x^{5}−

C_{4}[2, 0, 0, 0]x^{7} + (C_{4}[2, 0, 0, 0]+

C_{4}[0, 1, 0, 0])x^{8} − (C_{4}[0, 1, 0, 0] + 1)x^{9} + x^{12}) ∑_{r=0}^{∞} S^{r}(ϖ_{2})x^{r}

or after cancelling 1 − x,

(4.3) ∑_{n,k=0}^{∞} A_{7}[0, n, 0, k, 0, 0, 0]|_{Sp_{8}(ℂ)}x^{n+2k} =

(1 + x + x^{2} − C_{4}[0, 1, 0, 0]x^{3} + C_{4}[2, 0, 0, 0]x^{4} − C_{4}[2, 0, 0, 0]x^{7}+

C_{4}[0, 1, 0, 0]x^{8} − x^{9} − x^{10} − x^{11}) ∑_{r=0}^{∞} S^{r}(ϖ_{2})x^{r} .

Next we compute the left-hand-side of (4.3). Denote

[n, k] = A_{7}[0, n, 0, k, 0, 0, 0]|_{Sp_{8}(ℂ)} − A_{7}[0, n − 1, 0, k, 0, 0, 0]|_{Sp_{8}(ℂ)}−

A_{7}[0, n, 0, k − 1, 0, 0, 0]|_{Sp_{8}(ℂ)} + A_{7}[0, n − 1, 0, k − 1, 0, 0, 0]|_{Sp_{8}(ℂ)}.

In this definition, A_{7}[0, n, 0, k, 0, 0, 0] is understood to be zero if n or k is negative.

Lemma 4.2. We have

[n, k] = ∑_{r=0}^{k} ∑_{ℓ=max(0,r−n)}^{r} C_{4}[r − ℓ, n + 2ℓ − r, r − ℓ, k − r].

Proof. It follows from the definition that

A_{7}[0, n, 0, k, 0, 0, 0] = ∑_{t=0}^{k} ∑_{u=0}^{n}[n, k].

Thus we need to prove that A_{7}[0, n, 0, k, 0, 0, 0] branches to

(4.4)  ∑_{t=0}^{k} ∑_{u=0}^{n} ∑_{r=0}^{t} ∑_{ℓ=max(0,r−u)}^{r} C_{4}[r − ℓ, u + 2ℓ − r, r − ℓ, t − r].

The branching rule from SL(2n) to Sp(2n) has a simple form due to Littlewood [L] provided we restrict ourselves to representations parametrized by Young diagrams of length ≤ n. In our case, this rule (which we recall below) is sufficient to describe the branching of the left side in (4.4). Let Λ be the Littlewood-Richardson ring, which is the free abelian group generated by all partitions, with multiplication the Littlewood-Richardson rule. It is endowed with an inner product in which the partitions form an orthonormal basis.

If λ is a partition of length ≤ 2n, we will denote by {λ} and by [λ] the representations of SL(2n) and Sp(2n) respectively parametrized by λ; this notation extends to virtual representations by linearity, so that {λ + μ} = {λ} + {μ}.

If λ and μ are partitions such that the diagram of μ is contained in the diagram of λ, then λ\μ is a skew partition. We associate with this the element of Λ which is the image of λ under the map Λ → Λ adjoint to multiplication by μ. Thus [λ\μ] is a representation (not generally irreducible) of Sp(2n). Then Littlewood proved that if the length of λ is ≤ n, then the representation {λ} of SL(2n) branches to

∑ ; μ′ even[λ\μ],

where the sum is over all partitions μ contained in λ whose transpose μ′ is even.

In the case at hand, A_{7}[0, n, 0, k, 0, 0, 0] is represented by λ = {n+k, n+k, k, k}. If μ′ is even and the diagram of μ is contained in that of λ, then μ has the form {m + h, m + h, h, h} where h ≤ k and m + h ≤ n + k. We may compute [λ\μ] by the Littlewood-Richardson rule: we must enumerate the tableaux of shape λ\μ such that the associated word is a lattice permutation, as described by Macdonald [M], I.9. It is easy to see that such a tableau must have the following description. The entries of the first row consist of 1’s, and the second of an equal number of 2’s; the third row can have 1’s and 3’s but no other entries; the fourth row has 2’s and threes below the 1’s of the third row and 4’s below the 3’s of the third row; and if δ is the number of 3’s in the third row then k − δ ≤ m + h.

Let us denote by α the number of 1’s in the first row, and by β, γ and δ the number of 2’s, 3’s and 4’s, respectively in the fourth row. Thus the second row has α 2’s, and the third row has β + γ 1’s and δ 3’s. These parameters must satisfy the inequalities

0 ≤ α, β, γ, δ,
γ + δ ≤ α ≤ n + δ,
β + γ + δ ≤ k.

The contribution of this tableau is C_{4}[γ, α+β −γ −δ, γ, δ]. Thus A_{7}[0, n, 0, k, 0, 0, 0] branches to

∑_{γ=0}^{k} ∑_{δ=0}^{k−γ} ∑_{ϵ=0}^{n+k} N(γ, δ, ϵ) C_{4}[γ, α + β − γ − δ, γ, δ],

where N(γ, δ, ϵ) is the number of pairs α and β satisfying

γ + δ ≤ α ≤ n + δ,
0 ≤ β ≤ k − γ − δ,
α + β = ϵ + γ + δ.

Evidently β must lie in the range

max(0, ϵ + γ − n) ≤ β ≤ min(ϵ, k − γ − δ).

Thus

N(γ, δ, ϵ) = max ( min(ϵ, k − γ − δ) − max(0, ϵ + γ − n) + 1, 0).

Thus what we need to show is that this equals the coefficient of C_{4}[γ, α + β − γ − δ, γ, δ] in (4.4). Indeed, if ℓ is known, then the values of r, t and u are determined by the equations r − ℓ = γ, u + 2ℓ − r = ϵ and t − r = δ. It is easy to see that ℓ must satisfy the same inequality

max(0, ϵ + γ − n) ≤ ℓ ≤ min(ϵ, k − γ − δ)

which was satisfied by β, and the Lemma is proved. □

Clearly,

(1 − x)(1 − x^{2}) ∑_{n,k=0}^{∞} A_{7}[0, n, 0, k, 0, 0, 0]|_{Sp_{8}(ℂ)}x^{n+2k} = ∑_{n,k=0}^{∞}[n, k]x^{n+2k}

Using Lemma 4.2 we prove:

AN L-FUNCTION OF DEGREE 27 FOR Spin_{9} 25

Lemma 4.3. We have

∑_{n,k=0}^{∞}[n, k]x^{n+2k} = \frac{1}{1 − x} ∑_{n_{i}=0}^{∞} C_{4}[n_{2}, n_{1}, n_{2}, n_{3}](1 − x^{n_{1}+1})x^{n_{1}+3n_{2}+2n_{3}} .

□

Proof. We start by some manipulations with the sums involved. Write

∑_{n,k=0}^{∞}[n, k]x^{n+2k} = ∑_{k=0}^{∞} ∑_{n=k}^{∞}[n, k]x^{n+2k} + ∑_{k=0}^{∞} ∑_{n=0}^{k−1}[n, k]x^{n+2k} .

Using Lemma 4.2, the first summand equals

∑_{k=0}^{∞} ∑_{n=k}^{∞} ∑_{r=0}^{k} ∑_{ℓ=0}^{r} C_{4}[r − ℓ, n + 2ℓ − r, r − ℓ, k − r] x^{n+2k} .

Changing the r and k order of summations, this equals

∑_{r=0}^{∞} ∑_{k=r}^{∞} ∑_{n=k}^{∞} ∑_{ℓ=0}^{r} C_{4}[r − ℓ, n + 2ℓ − r, r − ℓ, k − r]x^{n+2k} .

Shifting the indices n → n + k and then k → k + r we obtain

∑_{n,k,r=0}^{∞} ∑_{ℓ=0}^{r} C_{4}[r − ℓ, n + k + 2ℓ, r − ℓ, k] x^{n+3k+3r} =

∑_{n,k,ℓ=0}^{∞} ∑_{r=ℓ}^{∞} C_{4}[r − ℓ, n + k + 2ℓ, r − ℓ, k] x^{n+3k+3r}

where the last equality follows from a change in order of the r and ℓ summation. Shifting r → r + ℓ we obtain

∑_{n,k,ℓ,r=0}^{∞} C_{4}[r, n + k + 2ℓ, r, k] x^{n+3k+3r+3ℓ} .

Next, using Lemma 4.2, we compute

∑_{k=0}^{∞} ∑_{n=0}^{k−1} ∑_{ℓ=0}^{k} ∑_{r=ℓ}^{min{n+ℓ,k}} C_{4}[r − ℓ, n + 2ℓ − r, r − ℓ, k − r] x^{n+2k}

= ∑_{ℓ=0}^{∞} ∑_{k=ℓ}^{∞} ∑_{n=0}^{k−1} ∑_{r=ℓ}^{min{n+ℓ,k}} C_{4}[r − ℓ, n + 2ℓ − r, r − ℓ, k − r] x^{n+2k}

= ∑_{ℓ,k=0}^{∞} ∑_{n=0}^{ℓ+k−1} ∑_{r=0}^{min{n,k}} C_{4}[r, n + ℓ − r, r, k − r] x^{n+2k+2ℓ} .

Here the first equality is obtained by changing the order of the k and ℓ summation. The second equality is obtained by shifting k → k + ℓ and then r → r + ℓ. We have

∑_{n=0}^{ℓ+k−1} ∑_{r=0}^{min{n,k}} = ∑_{r=0}^{k} ∑_{n=r}^{ℓ+k−1}. Using that and a change in the r and k summation

we obtain

∑_{r,ℓ=0}^{∞} ∑_{k=r}^{∞} ∑_{n=r}^{ℓ+k−1} C_{4}[r, n + ℓ − r, r, k − r] x^{n+2k+2ℓ} =

∑_{r,k,ℓ=0}^{∞} ∑_{n=0}^{ℓ+k−1} C_{4}[r, n + ℓ, r, k] x^{n+2k+2ℓ+3r}

where the last summation is obtained by shifting k → k + r and then n → n + r.

Write ∑_{n=0}^{ℓ+k−1} = ∑_{n=0}^{k−1} + ∑_{n=k}^{ℓ+k−1}. Hence, we obtain

∑_{r,k,ℓ=0}^{∞} ∑_{n=0}^{k−1} C_{4}[r, n + ℓ, r, k] x^{n+2k+2ℓ+3r} + ∑_{r,k,ℓ=0}^{∞} ∑_{n=k}^{ℓ+k−1} C_{4}[r, n + ℓ, r, k] x^{n+2k+2ℓ+3r}

In the first summand we may start with k = 1. Shifting in the first summand k → k + 1 and n → n + k in the second summand we obtain

x^{2} ∑_{r,k,ℓ=0}^{∞} ∑_{n=0}^{k} C_{4}[r, n + ℓ, r, k + 1] x^{n+2k+2ℓ+3r}+

∑_{r,k,ℓ=0}^{∞} ∑_{n=0}^{k−1} C_{4}[r, n + k + ℓ, r, k] x^{n+3k+2ℓ+3r}

In the second summand we may start with ℓ = 1. Shifting it by ℓ → ℓ + 1 and then changing order of summation of n and k in the first and n and ℓ in the second, we obtain

x^{2} ∑_{n,r,ℓ=0}^{∞} ∑_{k=n}^{∞} C_{4}[r, n + ℓ, r, k + 1] x^{n+2k+2ℓ+3r}+

x^{2} ∑_{r,k,n=0}^{∞} ∑_{ℓ=n}^{∞} C_{4}[r, n + k + ℓ + 1, r, k] x^{n+3k+2ℓ+3r} =

x^{2} ∑_{r,k,n,ℓ=0}^{∞} C_{4}[r, n + ℓ, r, n + k + 1] x^{3n+2k+2ℓ+3r}+

x^{2} ∑_{r,k,n,ℓ=0}^{∞} C_{4}[r, 2n + k + ℓ + 1, r, k] x^{3n+3k+2ℓ+3r} .

We summarize

(4.5) ∑_{n,k=0}^{∞}[n, k]x^{n+2k} = ∑_{n,k,r,ℓ=0}^{∞} C_{4}[r, n + k + 2ℓ, r, k] x^{n+3k+3r+3ℓ}+

x^{2} ∑_{n,k,r,ℓ=0}^{∞} C_{4}[r, n + k + 2ℓ + 1, r, k] x^{2n+3k+3r+3ℓ}+

x^{2} ∑_{n,k,r,ℓ=0}^{∞} C_{4}[r, n + ℓ, r, n + k + 1] x^{3n+2k+2ℓ+3r} .

We want to compute the coefficient of C_{4}[n_{2}, n_{1}, n_{2}, n_{3}]. Let us start with the third summand. It contributes the coefficient ∑ x^{3n+2k+2ℓ+3r+2} where we sum over all n, k, r, ℓ satisfying r = n_{2}; n + ℓ = n_{1} and n + k = n_{3} − 1. This equals

x^{3n_{2}+2} ∑_{{n+ℓ=n_{1} \atop n+k=n_{3}−1}} x^{3n+2k+2ℓ} = x^{n_{1}+3n_{2}+2n_{3}} ∑_{{n+ℓ=n_{1} \atop n+k=n_{3}−1}} x^{ℓ}

if n_{3} > n_{1} we obtain x^{n_{1}+3n_{2}+2n_{3}} ∑_{ℓ=0}^{n_{1}} x^{ℓ} = \frac{1}{1−x}x^{n_{1}+3n_{2}+2n_{3}}(1 − x^{n_{1}+1}). If n_{1} ≥ n_{3}

we obtain x^{n_{1}+2n_{3}+3n_{2}} ∑_{ℓ=n_{1}−n_{3}+1}^{n_{1}} x^{ℓ} = \frac{1}{1−x}x^{2n_{1}+3n_{2}+n_{4}+1}(1 − x^{n_{3}}). The coefficient

of C_{4}[n_{2}, n_{1}, n_{2}, n_{3}] coming from the third summand equals

\frac{1}{1 − x}\{ \begin{matrix}x^{n_{1}+3n_{2}+2n_{3}}(1 − x^{n_{1}+1}) & if n_{3} > n_{1}, \\ x^{2n_{1}+3n_{2}+n_{3}+1}(1 − x^{n_{3}}) & if n_{1} ≥ n_{3} . \end{matrix}

The first summand of (4.5) contributes to C_{4}[n_{2}, n_{1}, n_{2}, n_{3}] only if n_{1} ≥ n_{3}. We have

∑_{{r=n_{2},k=n_{3} \atop n+k+2ℓ=n_{1}}} x^{n+3k+3r+3ℓ} = x^{n_{1}+3n_{2}+2n_{3}} ∑_{n+2ℓ=n_{1}−n_{3}} x^{ℓ}

= \frac{1}{1 − x}\{ \begin{matrix}x^{n_{1}+3n_{2}+2n_{3}}(1 − x^{(n_{1}−n_{3})/2+1}) & n_{1} − n_{3} even \\ x^{n_{1}+3n_{2}+2n_{3}}(1 − x^{(n_{1}−n_{3}−1)/2+1}) & n_{1} − n_{3} odd. \end{matrix}

The third summand of (4.5) contributes only if n_{1} ≥ n_{3} + 1. We have the contribution

\frac{1}{1 − x}\{ \begin{matrix}x^{2n_{1}+3n_{2}+n_{3}−(n_{1}−n_{3}−1)/2}(1 − x^{(n_{1}−n_{3}−1)/2+1}) & n_{1} − n_{3} odd \\ x^{2n_{1}+3n_{2}+n_{3}−(n_{1}−n_{3})/2+1}(1 − x^{(n_{1}−n_{3})/2}) & n_{1} − n_{3} even . \end{matrix}

One can easily check that in all cases the coefficient of C_{4}[n_{2}, n_{1}, n_{2}, n_{3}] is \frac{1}{1−x}x^{n_{1}+3n_{2}+2n_{3}}(1 − x^{n_{1}+1}). This completes the proof of Lemma 4.3. □

Using the above Lemmas, the identity (4.1) is equivalent to

(4.6) (1 − x^{2})^{2}(1 − x^{3})(1 − x^{4}) ∑_{n_{i}=0}^{∞} C_{4}[n_{2}, n_{1}, n_{2}, n_{3}](1 − x^{n_{1}+1})x^{n_{1}+3n_{2}+2n_{3}}

= (1+x+x^{2}−(x^{3}−x^{8})C_{4}[0, 1, 0, 0]+(x^{4}−x^{7})C_{4}[2, 0, 0, 0]−x^{9}−x^{10}−x^{11})

[ ∑_{n_{i}=0}^{∞} C_{4}[n_{2} + 2n_{3}, n_{1}, n_{2}, n_{3} + n_{4}] M(n_{1}, n_{2}, n_{4}, n_{3})+

x^{6} ∑_{n_{i}=0}^{∞} C_{4}[n_{2}, n_{1}, n_{2} + 2n_{4} + 2, n_{3}] M(n_{1}, n_{2}, n_{3}, n_{4})] .

To prove (4.6) we will need to know how to multiply C_{4}[0, 1, 0, 0] and C_{4}[2, 0, 0, 0] by any representation of Sp_{8}(ℂ). Even though one can produce a formula for the product of C_{4}[2, 0, 0, 0] with an arbitrary representation of Sp_{8}(ℂ), it will be convenient to use the identity C_{4}[2, 0, 0, 0] = C_{4}[1, 0, 0, 0] ⊗ C_{4}[1, 0, 0, 0] − C_{4}[0, 1, 0, 0] − 1 instead, and to produce a formula for the product of C_{4}[1, 0, 0, 0] by any arbitrary representation of Sp_{8}(ℂ). Thus (4.6) is equivalent to

(4.7) (1 − x^{2})^{2}(1 − x^{3})(1 − x^{4}) ∑_{n_{i}=0}^{∞} C_{4}[n_{2}, n_{1}, n_{2}, n_{3}](1 − x^{n_{1}+1})x^{n_{1}+3n_{2}+2n_{3}} =

= (1 + x + x^{2} − x^{4} + x^{7} − x^{9} − x^{10} − x^{11} − (x^{3} + x^{4} − x^{7} − x^{8})C_{4}[0, 1, 0, 0]+

(x^{4}−x^{7})C_{4}[1, 0, 0, 0]C_{4}[1, 0, 0, 0])[ ∑_{n_{i}=0}^{∞} C_{4}[n_{2}+2n_{3}, n_{1}, n_{2}, n_{3}+n_{4}]M(n_{1}, n_{2}, n_{4}, n_{3})+

x^{6} ∑_{n_{i}=0}^{∞} C_{4}[n_{2}, n_{1}, n_{2} + 2n_{4} + 2, n_{3}]M(n_{1}, n_{2}, n_{3}, n_{4})] .

To produce these formulas we need to know the weights of the irreducible representations C_{4}[0, 1, 0, 0] and C_{4}[1, 0, 0, 0]. We will identify the space of weights of Sp(8) with Euclidean space by n_{1}ϖ_{1} + n_{2}ϖ_{2} + n_{3}ϖ_{3} + n_{4}ϖ_{4} → (n_{1}, n_{2}, n_{3}, n_{4}), where the ϖ_{i} are the fundamental dominant weights. Thus a weight (n_{1}, n_{2}, n_{3}, n_{4}) is dominant if and only if each n_{i} ≥ 0. Let ρ = (1, 1, 1, 1), half the sum of the positive roots.

Denote

γ_{1} = {(0, 1, 0, 0); (1, −1, 1, 0); (−1, 0, 1, 0); (1, 0, −1, 1); (−1, 1, −1, 1); (1, 0, 1, −1); (0, −1, 0, 1); (−1, 1, 1 − 1); (1, 1, −1, 0); (0, −1, 2, −1);

(−1, 2, −1, 0); (2, −1, 0, 0); (0, 1, −2, 1); (1, −2, 1, 0); (−2, 1, 0, 0);
(1, −1, −1, 1); (0, 1, 0, −1); (−1, −1, 1, 0); (−1, 0, −1, 1); (1, −1, 1, −1);

(−1, 0, 1, −1); (1, 0, −1, 0); (−1, 1, −1, 0); (0, −1, 0, 0)}.

Thus γ_{1} consists of all nonzero weights of C_{4}[0, 1, 0, 0]. (The weight (0, 0, 0, 0) also occurs in C_{4}[0, 1, 0, 0], with multiplicity three.) Denote also,

γ_{2} = {(1, 0, 0, 0); (−1, 1, 0, 0); (0, −1, 1, 0); (0, 0 − 1, 1), (0, 0, 1 − 1); (0, 1, −1, 0); (1, −1, 0, 0); (−1, 0, 0, 0)}

γ_{2} consists of all weights of C_{4}[1, 0, 0, 0]. We have, Lemma 4.4. 1) For any irreducible representation C_{4}[r_{1}, r_{2}, r_{3}, r_{4}] let ℓ(r_{1}, r_{2}, r_{3}) be the number of 1 ≤ i ≤ 3 such that r_{i} ≠ 0. Then C_{4}[0, 1, 0, 0] ⊗ C_{4}[r_{1}, r_{2}, r_{3}, r_{4}] equals

∑_{(α,β,γ,δ)∈γ_{1}} C_{4}[r_{1} + α, r_{2} + β, r_{3} + γ, r_{4} + δ] + ℓ(r_{1}, r_{2}, r_{3}) C_{4}[r_{1}, r_{2}, r_{3}, r_{4}] .

2) With the same notations as in 1), C_{4}[1, 0, 0, 0] ⊗ C_{4}[r_{1}, r_{2}, r_{3}, r_{4}] equals

∑_{(α,β,γ,δ)∈γ_{2}} C_{4}[r_{1} + α, r_{2} + β, r_{3} + γ, r_{4} + δ] .

In both cases we agree that if a summand on the right hand side has a negative coordinate, then we ignore this summand. □

Proof. These may be proved by a method of decomposing tensor products of representations due to Brauer. It is discussed in [B-G], Proposition 6.1 and the subsequent discussion. We recall the basic principle, referring to [B-G] for the simple proof. If π and σ are irreducible representations of a complex Lie group, and γ is the set of weights in σ, each weight ν ∈ γ occurring with multiplicity m(ν), then the character of π ⊗ σ equals

∑_{ν∈γ} m(ν) c(ν),

where c(ν) may be described as follows. Let μ be the highest weight vector of π. If ν + μ is dominant, then c(ν) = ν + μ. Otherwise, there are two possibilities. First, there may be a Weyl group element w such that w(ν + μ + ρ) − ρ is dominant. In this case, c(ν) = (−1)^{l(w)} (w(ν + μ + ρ) − ρ). The alternative is that ν + μ + ρ lies on the boundary of a Weyl chamber. In the latter case, c(ν) = 0.

Specializing to the case where σ = C_{4}[0, 1, 0, 0] or C_{4}[1, 0, 0, 0] and the representation π = C_{4}[r_{1}, r_{2}, r_{3}, r_{4}], if r_{i} ≥ 1, then both 1) and 2) follow immediately from this rule of Brauer’s, because μ + ν will be dominant for every ν in either γ_{1} or γ_{2}. In case 1) there will be a contribution of 3μ coming from the zero weight, which is of multiplicity three in C_{4}[0, 1, 0, 0]. Since ℓ(r_{1}, r_{2}, r_{3}) = 3 in this case, this is in agreement with the formula.

If some of the r_{i} are zero, and if σ = C_{4}[1, 0, 0, 0], then those ν for which ν + μ is not dominant have ν + μ + ρ lying on the boundary of a Weyl chamber, and for these c(ν) = 0. Thus 2) is proved in any case.

On the other hand if σ = C_{4}[0, 1, 0, 0], there are three weights ν in γ_{1} which could possible give a contribution even if μ + ν is not dominant. These are (−2, 1, 0, 0), (1, −2, 1, 0) and (0, 1, −2, 1). It is easy to see that if r_{1} = 0 then and ν = (−2, 1, 0, 0), then c(ν) = −μ; if r_{2} = 0 and ν = (1, −2, 1, 0), then c(ν) = −μ; and if r_{3} = 0 and ν = (0, 1, −2, 1), then c(ν) = −μ. (All other weights ν ∈ γ_{1} have c(ν) = 0 if μ+ν is not dominant.) Taking these subtractions into account, the coefficient of μ in the weight decomposition of π ⊗ C_{4}[0, 1, 0, 0] is exactly ℓ(r_{1}, r_{2}, r_{3}). □

To prove (4.7) we shall show that given a representation C_{4}[r_{1}, r_{2}, r_{3}, r_{4}], it has the same coefficient on both sides. Notice that a coefficient of a representation is a polynomial in x. First, the only representations which have a possibly nonzero coefficient have the form C_{4}[m_{2}, m_{1}, m_{2}+2m_{4}+2, m_{3}] or C_{4}[m_{2}+2m_{3}, m_{1}, m_{2}, m_{3}+ m_{4}]. It is convenient to verify this in (4.6). The left-hand-side of (4.6) is clearly a sum of representations of the form C_{4}[m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}]. To check that the right hand side of (4.6) is of the above form, we need to verify that if we multiply C_{4}[m_{2}, m_{1}, m_{2} + 2m_{4} + 2, m_{3}] or C_{4}[m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}] by C_{4}[0, 1, 0, 0] or C_{4}[2, 0, 0, 0] we once again obtain representations of the above form. Since the weights of C_{4}[0, 1, 0, 0] are also weights of C_{4}[2, 0, 0, 0] and since C_{4}[2, 0, 0, 0]⊗C_{4}[r_{1}, r_{2}, r_{3}, r_{4}] is a sum of representations of the form C_{4}[r_{1} +α, r_{2} + β, r_{3} + γ, r_{3} + δ] where (α, β, γ, δ) is a weight of C_{4}[2, 0, 0, 0], it is enough to verify our claim for C_{4}[2, 0, 0, 0]. Thus, C_{4}[2, 0, 0, 0]⊗C_{4}[n_{2} +2n_{3}, n_{1}, n_{2}, n_{3} +n_{4}] is a sum terms of the form C_{4}[n_{2} + 2n_{3} + α, n_{1} + β, n_{2} + γ, n_{3} + n_{4} + δ]. This equals (C_{4}[n_{2} + γ) + 2n_{3} + α − γ, n_{1} + β, n_{2} + γ, n_{3} + n_{4} + δ]. If (α, β, γ, δ) is a weight of C_{4}[2, 0, 0, 0] then α − γ is even. This is clear if (α, β, γ, δ) ∈ γ_{1} or if (α, β, γ, δ) = 0. The representation C_{4}[2, 0, 0, 0] has eight more weights which are of the form 2(a, b, c, d) where (a, b, c, d) ∈ γ_{2}. Thus α − γ is even in all cases, and the above representation has the form C_{4}[m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}]. In a similar way we can check that C_{4}[2, 0, 0, 0] ⊗ C_{4}[n_{2}, n_{1}, n_{2} + 2n_{4} + 2, n_{3}] is a sum of representations of the above type. Thus we only need to show that the coefficient of C_{4}[m_{2}, m_{1}, m_{2}+2m_{4}+2, m_{3}] and of C_{4}[m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}] are the same on both sides of (4.7). To do that, using Lemma 4.4, we write

C_{4}[0, 1, 0, 0] ∑_{n_{i}=0}^{∞} C_{4}[n_{2} + 2n_{3}, n_{1}, n_{2}, n_{3} + n_{4}] M(n_{1}, n_{2}, n_{4}, n_{3}) =

= ∑_{(α,β,γ,δ)∈γ_{1}} ∑_{n_{i}=0}^{∞} C_{4}[n_{2} + 2n_{3} + α, n_{1} + β, n_{2} + γ, n_{3} + n_{4} + δ] M(n_{1}, n_{2}, n_{4}, n_{3})

+ ∑_{n_{i}=0}^{∞} ℓ(n_{1}, n_{2}, n_{2} + 2n_{3}) C_{4}[n_{2} + 2n_{3}, n_{1}, n_{2}, n_{3} + n_{4}] M(n_{1}, n_{2}, n_{4}, n_{3}) .

To compute the coefficient of C_{4}[m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}] in this expression we let

(n_{2} + 2n_{3} + α, n_{1} + β, n_{2} + γ, n_{3} + n_{4} + δ) = (m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}) .

This happens if n_{1} = m_{1} − β; n_{2} = m_{2} − γ; n_{3} = m_{3} + 1/2(γ − α) and n_{4} = m_{4} − 1/2(γ − α) − δ. Thus the coefficient of C_{4}[m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}] coming from the above sum, is

(4.8) ∑ M(m_{1} − β, m_{2} − γ, m_{4} − 1/2(γ − α) − δ, m_{3} + 1/2(γ − α))

+ ℓ(m_{1}, m_{2}, m_{2} + 2m_{3}) M(m_{1}, m_{2}, m_{4}, m_{3})

where the summation is over all (α, β, γ, δ) ∈ γ_{1} which satisfy β ≤ m_{1}; γ ≤ m_{2}; 1/2(α − γ) ≤ m_{3} and 1/2(γ − α) + δ ≤ m_{4}. Actually, the last condition is always satisfied. Indeed, as can be checked, 1/2(γ −α)+δ ∈ {0, ±1}. Hence if m_{4} ≥ 1 then 1/2(γ − α) + δ ≤ m_{4} for all (α, β, γ, δ) ∈ γ_{1}. If m_{4} = 0 and 1/2(γ − α) + δ = −1, then the contribution of this term to (4.8) will be M(r_{1}, r_{2}, −1, r_{4}). However, from the definition of M(ℓ_{1}, ℓ_{2}, ℓ_{3}, ℓ_{4}),

(4.9)  M(−1, k_{1}, k_{2}, k_{3}) = M(k_{1}, −1, k_{2}, k_{3}) = M(k_{1}, k_{2}, −1, k_{3}) = 0

for all k_{1}, k_{2}, k_{3} ∈ ℤ. Hence we may drop the condition 1/2(γ − α) + δ ≤ m_{4} from (4.8).

To compute the contribution to the coefficient of C_{4}[m_{2}, m_{1}, m_{2} + 2m_{4} + 2, m_{3}] we let

(n_{2} + 2n_{3} + α, n_{1} + β, n_{2} + γ, n_{3} + n_{4} + δ) = (m_{2}, m_{1}, m_{2} + 2m_{4} + 2, m_{3})

which is satisfied if n_{1} = m_{1} − β; n_{2} = m_{2} + 2m_{4} + 2 − γ; m_{2} = n_{2} + 2n_{3} + α and n_{3}+n_{4}+δ = m_{3}. From the last two equations we get n_{3}+m_{4} = 1/2(γ−α)−1. Since n_{3} ≥ 0 and m_{4} ≥ 0 and since 1/2(γ − α) ∈ {0, ±1} we must have n_{3} = m_{4} = 0 and 1/2(γ − α) = 1. Thus there will be a contribution to the coefficient to the coefficient of C_{4}[m_{2}, m_{1}, m_{2} + 2m_{4} + 2, m_{3}] only if m_{4} = 0 and in this case n_{3} = 0 and γ − α = 2. Thus we get

(4.10)  ∑ M(m_{1} − β, m_{2} − α, m_{3} − δ, 0)

where we sum over all (α, β, γ, δ) ∈ γ_{1} with the conditions m_{1} ≥ β; m_{3} ≥ δ and γ − α = 2. As before, due to (4.9) we may drop the condition m_{3} ≥ δ.

Next we multiply,

C_{4}[0, 1, 0, 0] ∑_{n_{i}=1}^{∞} C_{4}[n_{2}, n_{1}, n_{2} + 2n_{4} + 2, n_{3}] M(n_{1}, n_{2}, n_{3}, n_{4}) =

= ∑_{n_{i}=0}^{∞} ∑_{(α,β,γ,δ)∈γ_{1}} C_{4}[n_{2} + α, n_{1} + β, n_{2} + 2n_{4} + 2 + γ, n_{3} + δ] M(n_{1}, n_{2}, n_{3}, n_{4})

+ ∑_{n_{i}=0}^{∞} ℓ(n_{1}, n_{2}, n_{2} + 2n_{4} + 2) C_{4}[n_{2}, n_{1}, n_{2} + 2n_{4} + 2, n_{3}] M(n_{1}, n_{2}, n_{3}, n_{4}) .

Arguing as before we obtain a contribution to the coefficient of C_{4}[m_{2}, m_{1}, m_{2} + 2m_{4} + 2, m_{3}]

(4.11) ∑ M(m_{1} − β, m_{2} − α, m_{3} − δ, m_{4} + 1/2(α − γ))+

ℓ(m_{1}, m_{2}, m_{2} + 2m_{4} + 2)M(m_{1}, m_{2}, m_{3}, m_{4}) .

Here we sum over all (α, β, γ, δ) ∈ γ_{1} which satisfy m_{1} ≥ β; m_{2} ≥ α and m_{4} ≥ 1/2(γ−α). To the coefficient of C_{4}[m_{2}+2m_{3}, m_{1}, m_{2}, m_{3}+m_{4}] we get a contribution only when m_{3} = 0, which equals

(4.12)  ∑ M(m_{1} − β, m_{2} − α, m_{4} − δ, 0)

where we sum over all (α, β, γδ) ∈ γ_{1} which satisfy m_{1} ≥ β; m_{2} ≥ α and α − γ = 2. Next, using Lemma 4.4 we have

C_{4}[1, 0, 0, 0] C_{4}[1, 0, 0, 0] ∑_{n_{i}=0}^{∞} C_{4}[n_{2}, n_{1}, n_{2} + 2n_{4} + 2, n_{3}] M(n_{1}, n_{2}, n_{3}, n_{4}) =

C_{4}[1, 0, 0, 0] ∑_{(α,β,γ,δ)∈γ_{2}} ∑_{n_{i}=0}^{∞} C_{4}[n_{2} + α, n_{1} + β, n_{2} + 2n_{4} + 2 + γ, n_{3} + δ]×

M(n_{1}, n_{2}, n_{3}, n_{4}) .

Changing variables n_{2} → n_{2} − α; n_{1} → n_{1} − β and n_{3} → n_{3} − δ we get

C_{4}[1, 0, 0, 0] ∑_{γ_{2}} ∑_{{n_{1}=β,n_{2}=α \atop n_{3}=δ,n_{4}=0}}^{∞}C_{4}[n_{2}, n_{1}, n_{2}+2n_{4}+2+γ−α, n_{3}] M(n_{1}−β, n_{2}−α, n_{3}−δ, n_{4}).

Given r ∈ ℤ denote \overline{r} = max\{0, r\}. By Lemma 4.4. if one of the coordinates of the above sum is negative we may ignore its contribution. Hence we may shift the index n_{1} and sum from n_{1} = \overline{β} to ∞. Similarly for n_{2} and n_{3}. Changing variables n_{1} → n_{1} + \overline{β}, n_{2} → n_{2} + \overline{α} and n_{3} → n_{3} + \overline{δ} we obtain

C_{4}[1, 0, 0, 0] ∑_{n_{i}=0}^{∞} ∑_{γ_{2}} C_{4}[n_{2} + \overline{α}, n_{1} + \overline{β}, n_{2} + 2n_{4} + 2 + γ − α + \overline{α}, n_{3} + \overline{δ}]

M(n_{1} − β + \overline{β}, n_{2} − α + \overline{α}, n_{3} − δ + \overline{δ}, n_{4})

= ∑_{n_{i}=0}^{∞} ∑_{γ_{2}×γ_{2}} C_{4}[n_{2} + \overline{α} + a, n_{1} + \overline{β} + b, n_{2} + 2n_{4} + 2 + γ − α + \overline{α} + c, n_{3} + \overline{δ} + d]

M(n_{1} − β + \overline{β}, n_{2} − α + \overline{α}, n_{3} − δ + \overline{δ}, n_{4}) .

To compute the contribution to the coefficient of C_{4}[m_{2}, m_{1}, m_{2} + 2m_{4} + 2, m_{3}] we compare

(n_{2} + \overline{α} + a, n_{1} + \overline{β} + b, n_{2} + 2n_{4} + 2 + γ − α + \overline{α} + c, n_{3} + \overline{δ} + d) = (m_{2}, m_{1}, m_{2} + 2m_{4} + 2, m_{3}) .

We get n_{1} = m_{1} − \overline{β} − b; n_{2} = m_{2} − \overline{α} − a; n_{3} = m_{3} − \overline{δ} − d and n_{4} = m_{4} + 1/2(α − γ + a − c). Thus we obtain the contribution

(4.13) ∑ M(m_{1} − β − b, m_{2} − α − a, m_{3} − δ − d, m_{4} + 1/2(α − γ + a − c))

where we sum over all (α, β, γ, δ), (a, b, c, d) ∈ γ_{2} which satisfy m_{1} ≥ b + \overline{β}; m_{2} ≥ a + \overline{α}; m ≥ d + \overline{δ} and m_{4} ≥ 1/2(γ − α + c − a).

To analyze the contribution to the coefficient of C_{4}[m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}] we let

(n_{2} + \overline{α} + a, n_{1} + \overline{β} + b, n_{2} + 2n_{4} + 2 + γ − α + \overline{α} + \overline{c}, n_{3} + \overline{δ} + d) = (m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}) .

We get n_{1} = m_{1} − b − \overline{β}; n_{2} = m_{2} + 2m_{3} − a − \overline{α}; n_{2} + 2n_{4} + 2 + γ − α + \overline{α} + c = m_{2} and n_{3} + \overline{δ} + d = m_{3} + m_{4}. Hence, m_{3} + n_{4} = 1/2(α − γ + a − c) − 1 and since m_{3}, n_{4} ≥ 0 we get m_{3} = n_{4} = 0 and α − γ + a − c = 2. Thus we obtain a nonzero contribution, if m_{3} = 0, which equals

(4.14)  ∑ M(m_{1} − b − β, m_{2} − a − α, m_{4} − d − δ, 0)

where we sum over all (α, β, γ, δ), (a, b, c, d) ∈ γ_{2} which satisfy m_{1} ≥ b + \overline{β}; m_{2} ≥ a + \overline{α}; m_{4} ≥ d + \overline{δ} and α − γ + a − c = 2.

Next, using Lemma 4.4 again, we have

C_{4}[1, 0, 0, 0] C_{4}[1, 0, 0, 0] ∑_{n_{i}=0}^{∞} C_{4}[n_{2} + 2n_{3}, n_{1}, n_{2}, n_{3} + n_{4}] M(n_{1}, n_{2}, n_{4}, n_{3})

= C_{4}[1, 0, 0, 0] ∑_{n_{i}=0}^{∞} ∑_{(α,β,γ,δ)∈γ_{2}} C_{4}[n_{2} + 2n_{3} + α, n_{1} + β, n_{2} + γ, n_{3} + n_{4} + δ]×

M(n_{1}, n_{2}, n_{4}, n_{3}) .

We break the summation over n_{3} to the case n_{3} = 0 and the sum n_{3} = 1 to ∞. Thus, we obtain

C_{4}[1, 0, 0, 0][ ∑_{{n_{i}=0 \atop n_{3}=1}}^{∞} ∑_{γ_{2}} C_{4}[n_{2} +2n_{3} +α, n_{1} +β, n_{2} +γ, n_{3} +n_{4} +δ] M(n_{1}, n_{2}, n_{4}, n_{3})

+ ∑_{n_{i}=0}^{∞} ∑_{γ_{2}} C_{4}[n_{2} + α, n_{1} + β, n_{2} + γ, n_{4} + δ] M(n_{1}, n_{2}, n_{4}, 0)] .

In the first summand we let n_{3} → n_{3} + 1; n_{1} → n_{1} − β and n_{2} → n_{2} − γ. In the second one we let n_{1} → n_{1} − β and n_{4} → n_{4} − δ. We get

C_{4}[1, 0, 0, 0][ ∑_{γ_{2}} ∑_{{n_{1}=β,n_{2}=γ \atop n_{3},n_{4}=0}}^{∞} C_{4}[n_{2} + 2n_{3} + 2 + α − γ, n_{1}, n_{2}, n_{3} + n_{4} + 1 + δ]×

M(n_{1} − β, n_{2} − γ, n_{4}, n_{3} + 1)

+ ∑_{γ_{2}} ∑_{{n_{1}=β,n_{2}=0 \atop n_{4}=δ}}^{∞} C_{4}[n_{2} + α, n_{1}, n_{2} + γ, n_{4}]M(n_{1} − β, n_{2}, n_{4} − δ, 0)] .

In the first summand we replace the summation index of n_{1} = β and n_{2} = γ by n_{1} = \overline{β} and n_{2} = \overline{γ} and then shift n_{1} → n_{1} + \overline{β} and n_{2} → n_{2} + \overline{γ}. (Recall that \overline{β} = max\{0, β\}). In the second summand we may replace n_{1} = β and n_{4} = δ by n_{1} = \overline{β} and n_{4} = \overline{δ}. Also we may replace n_{2} = 0 by n_{2} = max(α, γ) − α − γ. To explain the last replacement, notice that since (α, β, γ, δ) ∈ γ_{2}, if α ≠ 0 then γ = 0 and if γ ≠ 0 then α = 0. If α = 1 (resp. γ = 1) then γ = 0 (resp. α = 0) and we may start the summation with n_{2} = 0. And, in these cases max(α, γ) − α − γ = 0. If α = −1 (resp. γ = −1) then γ = 0 (resp. α = 0) and then we must start the summation with n_{2} = 1 (otherwise we will obtain a negative coordinate). And, in these cases max(α, γ) − α − γ = 1. Thus we obtain

C_{4}[1, 0, 0, 0][ ∑_{n_{i}=0}^{∞} ∑_{γ_{2}} C_{4}[n_{2} + 2n_{3} + 2 + α − γ + \overline{γ}, n_{1} + \overline{β}, n_{2} + \overline{γ}, n_{3} + n_{4} + 1 + δ]

M(n_{1} − β + \overline{β}, n_{2} − γ + \overline{γ}, n_{4}, n_{3} + 1)+

∑_{n_{i}=0}^{∞} ∑_{γ_{2}}(n_{2} + \{α, γ\} − γ, n_{1} + \overline{β}, n_{2} + \{α, γ\} − α, n_{4} + \overline{δ})

M(n_{1} − β + \overline{β}, n_{2} + \{α, γ\} − α − γ, n_{4} − δ + \overline{δ}, 0)] where we denote {α, γ} = max(α, γ).

Applying Lemma 4.4 again this equals

∑_{n_{i}=0}^{∞} ∑_{γ_{2}×γ_{2}} C_{4}[n_{2} +2n_{3} +2+α−γ +\overline{γ} +a, n_{1} +\overline{β} +b, n_{2} +\overline{γ} +c, n_{3} +n_{4} +1+δ +d]

M(n_{1} − β + \overline{β}, n_{2} − γ + \overline{γ}, n_{4}, n_{3} + 1) + ∑_{n_{i}=0}^{∞} ∑_{γ_{2}×γ_{2}}(n_{2} + \{α, γ\} − γ + a, n_{1} + \overline{β} + b,

n_{2} + \{α, γ\} − α + c, n_{4} + \overline{δ} + d)M(n_{1} − β + \overline{β}, n_{2} + \{α, γ\} − α − γ, n_{4} − δ + \overline{δ}, 0) .

We compute the contribution of the above expression to the coefficient of C_{4}[m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}]. To do that, set

(n_{2} + 2n_{3} + 2 + α − γ + \overline{γ} + a, n_{1} + \overline{β} + b, n_{2} + \overline{γ} + c, n_{3} + n_{4} + 1 + δ + d) = (m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}).

We obtain n_{1} = m_{1} − \overline{β} − b; n_{2} = m_{2} − \overline{γ} − \overline{c}; n_{3} = m_{3} + 1/2(γ − α + c − a) − 1; n_{4} = m_{4} + 1/2(α − γ + a − c) − δ − d. Thus we obtain a contribution (4.15) ∑ M(m_{1} −β −b, m_{2} −γ −c, m_{4} +1/2(α−γ +a−c)−δ −d, m_{3} +1/2(γ −α+c−a))

where we sum over all (α, β, γ, δ) and (a, b, c, d) in γ_{2} which satisfy m_{1} ≥ \overline{β} + b; m_{2} ≥ \overline{γ} + c; m_{3} ≥ 1/2(α − γ + a − c) + 1 and m_{4} ≥ δ + d − 1/2(α − γ + a − c). As before by (4.9) we may ignore the last condition. Next, set

(n_{2} + \{α, γ\} − γ + a, n_{1} + \overline{β} + b, n_{2} + \{α, γ\} − α + c, n_{4} + \overline{δ} + d) = (m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}).

From this we get n_{1} = m_{1} − \overline{β} − b; n_{2} = m_{2} − \{α, γ\} + α − c; n_{4} = m_{4} + 1/2(α − γ + a − c) − \overline{δ} − d and m_{3} = 1/2(α − γ + a − c). This gives us the contribution

(4.16)  ∑ M(m_{1} − β − b, m_{2} − γ − c, m_{4} + 1/2(α − γ + a − c) − δ − d, 0)

where we sum over all (α, β, γ, δ), (a, b, c, d) ∈ γ_{2} which satisfy m_{1} ≥ \overline{β} + b; m_{2} ≥ \{α, γ\} − α + c; m_{3} = 1/2(α − γ + a − c) and m_{4} ≥ \overline{δ} + d − 1/2(α − γ + a − c). In particular, since 1/2(α − γ + a − c) ∈ {0, ±1} there will be a nonzero contribution only if m_{3} = 0, 1.

Now we compute the contribution to the coefficient of C_{4}[m_{2}, m_{1}, m_{2} + 2m_{4} + 2, m_{3}]. We set

(n_{2} + 2n_{3} + 2 + α − γ + \overline{γ} + a, n_{1} + \overline{β} + b, n_{2} + \overline{γ} + c, n_{3} + n_{4} + 1 + \overline{δ} + d) = (m_{2}, m_{1}, m_{2} + 2m_{4} + 2, m_{3}).

This gives us n_{2} = m_{2} + 2m_{4} − \overline{γ} − c + 2 and n_{2} + 2n_{3} + 2 + α − γ + \overline{γ} + a = m_{2}. Combining these two equations we obtain n_{3} + m_{4} = 1/2(γ − α + c − a) − 2. Since 1/2(γ − α + c − a) ∈ {0, ±1}, the right hand side is always negative. Hence there is no contribution from this term. Finally, we set

(n_{2} + \{α, γ\} − γ + a, n_{1} + \overline{β} + b, n_{2} + \{α, γ\} − α + c, n_{4} + \overline{δ} + d) = (m_{2}, m_{1}, m_{2} + 2m_{4} + 2, m_{3}).

Solving the equations we get n_{1} = m_{1} − \overline{β} − b; n_{2} = m_{2} − \{α, γ\} + γ − a; n_{4} = m_{3} − \overline{δ} − d and m_{4} = 1/2(γ − α + c − a) − 1. Since m_{4} ≥ 0 and 1/2(γ − α + c − a) ∈ {0, ±1} we have contribution only if m_{4} = 0 and γ − α + c − a = 2. The only possibilities to satisfy this last equation is when (α, β, γ, δ), (a, b, c, d) ∈ {(−1, 1, 0, 0); (0, −1, 1, 0); (0, 0, 1, −1); (−1, 0, 0, 0)}. Hence {α, γ} = γ, a ∈ {0, −1} and \overline{δ} + d ≤ 0. The contribution we get is

(4.17)  ∑ M(m_{1} − β − b, m_{2} − α − a, m_{3} − δ − d, 0)

where we sum over all (α, β, γ, δ), (a, b, c, d) ∈ γ_{2} satisfying m_{1} ≥ \overline{β} + b and γ − α + c − a = 2. The conditions m_{2} ≥ \{α, γ\} − γ + a and m_{3} ≥ \overline{δ} + d always exists.

We are ready to prove (4.7). We start with the coefficient of C_{4}[m_{2}, m_{1}, m_{2} + 2m_{4} + 2, m_{3}]. From Lemma 4.3 we see that the left hand side of (4.7) has zero coefficient to this representation. The coefficient of this representation coming from the right hand side of (4.7) is

x^{6}(1 + x + x^{2} − x^{4} + x^{7} − x^{9} − x^{10} − x^{11})M(m_{1}, m_{2}, m_{3}, m_{4})

− x^{6}(x^{3} + x^{4} − x^{7} − x^{8})[ ∑_{{m_{1}≥β,m_{2}≥α \atop 2m_{4}≥γ−α}} M(m_{1} − β, m_{2} − α, m_{3} − δ, m_{4} + 1/2(α − γ))

+ ℓ(m_{1}, m_{2}, m_{2} + 2m_{4} + 2)M(m_{1}, m_{2}, m_{3}, m_{4})] + x^{6}(x^{4} − x^{7})×

∑_{{m_{1}≥b+\overline{β},m_{2}≥a+\overline{α} \atop m_{3}≥d+\overline{δ},2m_{4}≥γ−α+c−a}} M(m_{1} − β − b, m_{2} − α − a, m_{3} − δ − d, m_{4} + 1/2(α − γ + a − c))

− (x^{3} + x^{4} − x^{7} − x^{8})ℓ(m_{4}) ∑_{{m_{1}≥β \atop γ−α=2}} M(m_{1} − β, m_{2} − α, m_{3} − δ, 0)

+ (x^{4} − x^{7})ℓ(m_{4}) ∑_{{m_{1}≥\overline{β}+b \atop γ−α+c−a=2}} M(m_{1} − β + b, m_{2} − α − a, m_{3} − δ − d, 0)

where ℓ(m_{4}) = 0 if m_{4} ≥ 1 and equals one if m_{4} = 0. We shall explain where each term comes from. The first summand comes from the factor in (4.7) which does not involve any tensor product. The second summand comes from (4.11), the third summand is from (4.13), the fourth from (4.10) and the last one from (4.17). Notice the polynomials in front of these terms which come from (4.7).

One can check directly or by a computer that (4.18) is indeed zero. Of course (4.18) depends whether the m_{i}’s are zero or not.

Next we compute the coefficient of C_{4}[m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}]. We will separate to the cases m_{3} = 0 and m_{3} > 0. We start with the last one. Combining all contributions, the coefficient of C_{4}[m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}] when m_{3} ≥ 1 is

(1 + x + x^{2} − x^{4} + x^{7} − x^{9} − x^{10} − x^{11})M(m_{1}, m_{2}, m_{3}, m_{4}) − (x^{3} + x^{4} − x^{7} − x^{8})×

[ ∑_{m_{1}≥β,m_{2}≥γ} M(m_{1} − β, m_{2} − γ, m_{4} − 1/2(α − γ) − δ, m_{3} + 1/2(γ − α))

+ ℓ(m_{1}, m_{2}, m_{2} + 2m_{3})M(m_{1}, m_{2}, m_{3}, m_{4})] + (x^{4} − x^{7})×

∑_{{m_{1}≥\overline{β}+b \atop m_{2}≥\overline{γ}+c} ; 2m_{3}≥α−γ+a−c+2} M(m_{1}−β−bm_{2}−γ−c, m_{4}+1/2(α−γ+a−c)−δ−d, m_{3}+1/2(γ−α+c−a))+

ρ(m_{3})(x^{4} − x^{7}) ∑_{{m_{1}≥\overline{β}+b \atop α−γ+a−c=2}} M(m_{1} − β − b, m_{2} − γ − c, m_{4} − δ − d + 1, 0)

where ρ(m_{3}) = 0 if m_{3} > 1 and ρ(m_{3}) = 1 if m_{3} = 1. Here, the first term is the term on (4.7) which does not involve any tensor product. The second term comes from (4.8). The third is (4.15) and the fourth is (4.16). Once again one can check, directly or by using a computer, that this sum is indeed zero.

Finally we discuss the case of C_{4}[m_{2} + 2m_{3}, m_{1}, m_{2}, m_{3} + m_{4}] with m_{3} = 0, i.e. C_{4}[m_{2}, m_{1}, m_{2}, m_{4}]. The coefficient of this representation on the left hand side of (4.7) equals

(1 − x^{2})^{2}(1 − x^{3})(1 − x^{4})(1 − x^{m_{1}+1})x^{m_{1}+3m_{2}+2m_{3}}.

On the other hand the coefficient of this representation on the right hand side of (4.7) equals

(1 + x + x^{2} − x^{4} + x^{7} − x^{9} − x^{10} − x^{11})M(m_{1}, m_{2}, m_{4}, 0)

−(x^{3} +x^{4} −x^{7} −x^{8})[ ∑_{{m_{1}≥β,m_{2}≥γ \atop γ≥α}} M(m_{1} −β, m_{2} −γ, m_{4} −1/2(γ −α)−δ, 1/2(γ −α))+

ℓ(m_{1}, m_{2}, m_{2})M(m_{1}, m_{2}, m_{3}, 0)] + (x^{4} − x^{7})×

∑_{{m_{1}≥\overline{β}+b \atop m_{2}≥\overline{γ}+c} ; γ−α+c−a≥2} M(m_{1} −β −b, m_{2} −γ −c, m_{4} +1/2(α−γ +a−c)−δ−d, 1/2(γ −α+c−a))

+ (x^{4} − x^{7}) ∑ M(m_{1} − β − b, m_{2} − γ − c, m_{4} − δ − d, 0) − x^{6}(x^{3} + x^{4} − x^{7} − x^{8})×

∑_{{m_{1}≥β,m_{2}≥α \atop α−γ=2}} M(m_{1} − β, m_{2} − α, m_{4} − δ, 0)+

x^{6}(x^{4} − x^{7}) ∑_{{m_{1}≥b+\overline{β} \atop m_{2}≥a+\overline{α}} ; α−γ+a−c=2} M(m_{1} − b − β, m_{2} − a − α, m_{4} − d − δ, 0)

where the fourth summand is summed over all (α, β, γ, δ), (a, b, c, d) ∈ γ_{2} which satisfy m_{1} ≥ \overline{β} + b, m_{2} ≥ max(α, γ) − α + c, m_{4} ≥ \overline{δ} + d and α − γ + a − c = 0. The first summand comes from (4.7) corresponding to the term which does not involve tensor products. The second summand is (4.8) and the third is (4.15). The fourth is (4.16). The fifth is (4.12) and the last term is (4.14). Once again a direct calculation or with the use of a computer, one can check that the above two expressions which represent the coefficient of C_{4}[m_{2}, m_{1}, m_{2}, m_{4}] on both sides of (4.7) is equal. This completes the proof of (4.7) and hence the proof of Theorem 3.1. □

References

[B] N. Bourbaki, ’Eléments de mathématique. Groupes et algb̀res de Lie. Chap. IV, V et VI, Hermann, 1968.

[B-G] D. Bump and D. Ginzburg, The Adjoint L-function of GL(4)”, J. Reine Angew. Math., to

appear.

[C] R. Carter, Simple Groups of Lie Type, Wiley-Interscience, 1972. [C-S] W. Casselman and J. Shalika, The unramified principal series of p-adic groups II: the

Whittaker function, Compos. Math. 41 (1980), 207-231.

[G] D. Ginzburg, A Rankin-Selberg integral for the adjoint L-function of Sp(4), Israel J. Math 95 (1996), 301-339.

[G1] D. Ginzburg, On Spin L-functions for Orthogonal Groups, Duke Math. J. 77 (1995), 753-798.

[G-R] D. Ginzburg and S. Rallis, A Tower of Rankin-Selberg Integrals, IMRN 4 (1994), 201-208.

[K] V. Kac, Some remarks on nilpotent orbits, J. Algebra 64 (1980), 190-213.

[L] D. Littlewood, On invariants under restricted groups, Philos. Trans. Roy. Sec. A 239 (1944), 387–417.

[M] I. Macdonald, Symmetric Functions and Hall Polynomials, Second Edition, Oxford, 1995. [G-J] D. Ginzburg and D. Jiang, A Siegel-Weil Identity for G_{2} and on Poles of L-functions”, in

preparation.

Department of Mathematics, Stanford University, Stanford, CA 94305-2125

   E-mail address: bump@gauss.stanford.edu

School of Mathematical Sciences, Sackler Faculty of Exact Sciences, Tel Aviv University, Ramat-Aviv, 69978, Israel

   E-mail address: ginzburg@math.tau.ac.il