A FACTORIZATION THEOREM FOR CLASSICAL

GROUP CHARACTERS, WITH APPLICATIONS TO

PLANE PARTITIONS AND RHOMBUS TILINGS

M. Ciucu and C. Krattenthaler

Department of Mathematics, Indiana University, Bloomington, IN 47405-5701, USA

Fakultät für Mathematik der Universität Wien, Nordbergstraße 15, A-1090 Wien, Austria.

             WWW: http://www.mat.univie.ac.at/~kratt

Dedicated to Georgy P. Egorychev

Abstract. We prove that a Schur function of rectangular shape (M^{n}) whose variables are specialized to x_{1}, x_{1}^{−1}, …, x_{n}, x_{n}^{−1} factorizes into a product of two odd orthogonal characters of rectangular shape, one of which is evaluated at −x_{1}, …, −x_{n}, if M is even, while it factorizes into a product of a symplectic character and an even orthogonal character, both of rectangular shape, if M is odd. It is furthermore shown that the first factorization implies a factorization theorem for rhombus tilings of a hexagon, which has an equivalent formulation in terms of plane partitions. A similar factorization theorem is proven for the sum of two Schur functions of respective rectangular shapes (M^{n}) and (M^{n−1}).

1. Introduction. The purpose of this note is to prove curious factorization properties for Schur functions of rectangular shape, which seem to have escaped the attention of previous authors. (We refer the reader to Section 2 for all definitions.) More precisely, we show that a Schur function of rectangular shape (M^{n}) which is evaluated at x_{1}, x_{2}, …, x_{n} and their

2000 Mathematics Subject Classification. Primary 05E15; Secondary 05A15 05A17 05A19 05B45 11P81 20C15 20G05 52C20.

Key words and phrases. Schur functions, classical group characters, factorization identities, rhombus tilings, lozenge tilings, plane partitions, nonintersecting lattice paths.

The research of the first author was partially supported by NSF grant DMS-0500616. The research of the second author was partially supported by the Austrian Science Foundation FWF, grants Z130-N13 and S9607-N13, the latter in the framework of the National Research Network “Analytic Combinatorics and Probabilistic Number Theory.” This work was done during the authors’ stay at the Erwin Schrödinger Institute for Physics and Mathematics, Vienna, during the programme “Combinatorics and Statistical Physics” in Spring 2008.

Typeset by AMS-TEX

reciprocals x_{1}^{−1}, x_{2}^{−1}, …, x_{n}^{−1} factorizes into two factors, and the same is true for the sum of two Schur functions of respective shapes (M^{n}) and (M^{n−1}).

We begin by describing explicitly the case of one Schur function. If M is even, then both factors are odd orthogonal characters of rectangular shape, one of them evaluated at the variables x_{1}, x_{2}, …, x_{n}, but the other is evaluated at −x_{1}, −x_{2}, …, −x_{n}. If M is odd, then one factor is a symplectic character of rectangular shape, while the other is an even orthogonal character of rectangular shape, both being evaluated at x_{1}, x_{2}, …, x_{n}. The case of even M of this factorization property is presented in the following theorem.

Theorem 1. For any non-negative integers m and n, we have

s_{((2m)^{n})}(x_{1}, x_{1}^{−1}, x_{2}, x_{2}^{−1}, …, x_{n}, x_{n}^{−1})

= (−1)^{mn} so_{(m^{n})}(x_{1}, x_{2}, …, x_{n}) so_{(m^{n})}(−x_{1}, −x_{2}, …, −x_{n}). (1.1)

If M is odd, then the factorization takes the following form.

Theorem 2. For any non-negative integers m and n, we have

s_{((2m+1)^{n})}(x_{1}, x_{1}^{−1}, x_{2}, x_{2}^{−1}, …, x_{n}, x_{n}^{−1})

= sp_{(m^{n})}(x_{1}, x_{2}, …, x_{n}) o_{((m+1)^{n})}^{even}(x_{1}, x_{2}, …, x_{n}). (1.2)

Since our identities involve classical group characters, one might ask whether there are representation-theoretic interpretations of these identities. At first sight, this seems to be a difficult question because of the somewhat “incoherent” right-hand sides of (1.1) and (1.2). However, there is a uniform way of writing the factorization identities of Theorems 1 and 2 that was pointed out to us by Soichi Okada. Namely, by comparing (2.2) and (2.6), and by using (2.3) and (2.7), we see that

(−1)^{∑_{i=1}^{N} λ_{i}} so_{λ}(−x_{1}, −x_{2}, …, −x_{N}) ∏_{i=1}^{N}(x_{i}^{1/2} + x_{i}^{−1/2}) = o_{λ+ \frac{1}{2}}^{even}(x_{1}, x_{2} …, x_{N}), (1.3)

where λ + \frac{1}{2} is short for (λ_{1} + \frac{1}{2}, λ_{2} + \frac{1}{2}, …, λ_{N} + \frac{1}{2}). Furthermore, by comparing (2.2) and (2.4), and by using (2.3) and (2.5), we see that

sp_{λ}(x_{1}, x_{2}, …, x_{N}) ∏_{i=1}^{N}(x_{i}^{1/2} + x_{i}^{−1/2}) = so_{λ+ \frac{1}{2}} (x_{1}, x_{2} …, x_{N}).  (1.4)

Theorem 1 and 2 may therefore be uniformly stated as

( ∏_{i=1}^{n}(x_{i}^{1/2} + x_{i}^{−1/2}))s_{(M^{n})}(x_{1}, x_{1}^{−1}, x_{2}, x_{2}^{−1}, …, x_{n}, x_{n}^{−1})

= so_{(( \frac{M}{2} )^{n})}(x_{1}, x_{2}, …, x_{n}) o_{(( \frac{M+1}{2} )^{n})}^{even}(x_{1}, x_{2}, …, x_{n}). (1.5)

Written in this way, this may, in the end, lead to a representation-theoretic interpretation of these identities, although we must confess that we are not able to offer such an interpretation.

On the other hand, we are able to offer a combinatorial interpretation for Theorem 1. As we show in Section 5, if we specialize x_{1} = x_{2} = ⋯ = x_{n} = 1 in Theorem 1, then one obtains a factorization theorem for rhombus tilings of a hexagon, which has also a natural, equivalent formulation as a factorization theorem for plane partitions (see (5.1)). It is, in fact, this factorization theorem which we observed first, and which formed the starting point of this work. We suspect that a more general factorization theorem for rhombus tilings is lurking behind. If there is a natural combinatorial interpretation of Theorem 2 is less clear. We make an attempt, also in Section 5, but we consider it not entirely satisfactory.

The case of the sum of two Schur functions of rectangular shapes is treated in Section 6. Namely, we show that there are very similar factorization theorems for the sum of two Schur functions of respective rectangular shapes (M^{n}) and (M^{n−1}) (see Theorems 3 and 4). The existence of these was pointed out to us by Ron King.

The proofs of Theorems 1 and 2 are given in Section 4. These proofs are based on an auxiliary identity which is established in Section 3 (see Lemma 1). The proofs of Theorems 3 and 4 are based on two further auxiliary identities of similar type, which are also presented and proved in Section 3 (see Lemmas 2 and 3).

2. Classical group characters. In this section we recall the definitions of the classical group characters involved in the factorizations in Theorems 1 and 2. We also briefly touch upon their significance in representation theory.

Given a partition λ = (λ_{1}, λ_{2}, …, λ_{N}) (i.e., a non-increasing sequence of non-negative integers) the Schur function s_{λ}(x_{1}, x_{2}, …, x_{N}) is defined by (see [4, p. 403, (A.4)], [7, Prop. 1.4.4], or [8, Ch. I, (3.1)])

s_{λ}(x_{1}, x_{2}, …, x_{N}) = \frac{det_{1≤h,t≤N}(x_{h}^{λ_{t}+N−t})}{det_{1≤h,t≤N}(x_{h}^{N−t})}.  (2.1)

It is not difficult to see that the denominator in (2.1) cancels out, so that any Schur function s_{λ}(x_{1}, x_{2}, …, x_{N}) is in fact a polynomial in x_{1}, x_{2}, …, x_{N}, and is thus well-defined for any choice of the variables x_{1}, x_{2}, …, x_{N}. It is well-known (cf. [4, §24.2]) that s_{λ}(x_{1}, x_{2}, …, x_{N}) is an irreducible character of SL_{N}(ℂ) (respectively GL_{N}(ℂ)).

Given a non-increasing sequence λ = (λ_{1}, λ_{2}, …, λ_{n}) of integers or half-integers (the latter being, by definition, positive odd integers divided by 2), the odd orthogonal character so_{λ}(x_{1}, x_{2}, …, x_{N}) is defined by (see [4, (24.28)])

so_{λ}(x_{1}, x_{2}, …, x_{N}) = \frac{det_{1≤h,t≤N}(x_{h}^{λ_{t}+N−t+ \frac{1}{2}} − x_{h}^{−(λ_{t}+N−t+ \frac{1}{2} )})}{det_{1≤h,t≤N}(x_{h}^{N−t+ \frac{1}{2}} − x_{h}^{−(N−t+ \frac{1}{2} )})}.  (2.2)

Again, it is not difficult to see that the denominator in (2.2) cancels out, so that any odd orthogonal character so_{λ}(x_{1}, x_{2}, …, x_{N}) is in fact a Laurent polynomial in x_{1}, x_{2}, …, x_{N} (i.e., a polynomial in x_{1}, x_{1}^{−1}, x_{2}, x_{2}^{−1}, …, x_{N}, x_{N}^{−1}), and is thus well-defined for any choice of the variables x_{1}, x_{2}, …, x_{N} such that all of them are non-zero. By the Weyl denominator formula for type B (cf. [4, Lemma 24.3, Ex. A.62]),

det_{1≤h,t≤n} (x_{h}^{n−t+ \frac{1}{2}} − x_{h}^{−(n−t− \frac{1}{2} )})

= (x_{1}x_{2} ⋯ x_{n})^{−n+ \frac{1}{2}} ∏_{1≤h<t≤n}(x_{h} − x_{t})(x_{h}x_{t} − 1) ∏_{h=1}^{n}(x_{h} − 1), (2.3)

the denominator in (2.2) can be actually evaluated in product form. It is well-known (cf. [4, §24.2]) that so_{λ}(x_{1}, x_{2}, …, x_{N}) is an irreducible character of SO_{2N+1}(ℂ).

Given a partition λ = (λ_{1}, λ_{2}, …, λ_{n}), the symplectic character sp_{λ}(x_{1}, x_{2}, …, x_{N}) is defined by (see [4, (24.18)])

sp_{λ}(x_{1}, x_{2}, …, x_{N}) = \frac{det_{1≤h,t≤N}(x_{h}^{λ_{t}+N−t+1} − x_{h}^{−(λ_{t}+N−t+1)})}{det_{1≤h,t≤N}(x_{h}^{N−t+1} − x_{h}^{−(N−t+1)})}.  (2.4)

Similarly to odd orthogonal characters, sp_{λ}(x_{1}, x_{2}, …, x_{N}) is a Laurent polynomial in x_{1}, x_{2}, …, x_{N}, and is thus well-defined for any choice of the variables x_{1}, x_{2}, …, x_{N} such that all of them are non-zero. By the Weyl denominator formula for type C (cf. [4, Lemma 24.3, Ex. A.52]),

det_{1≤h,t≤n}(x_{h}^{n−t+1} − x_{h}^{n−t+1}) = (x_{1} ⋯ x_{n})^{−n} ∏_{1≤h<t≤n}(x_{h} − x_{t})(x_{h}x_{t} − 1) ∏_{h=1}^{n}(x_{h}^{2} − 1), (2.5)

the denominator in (2.4) can be actually evaluated in product form. Furthermore, sp_{λ}(x_{1}, x_{2}, …, x_{N}) is an irreducible character of Sp_{2N}(ℂ) (cf. [4, §24.2]).

Finally, given a non-increasing sequence λ = (λ_{1}, λ_{2}, …, λ_{N}) of positive integers or halfintegers, the even orthogonal character o_{λ}^{even}(x_{1}, x_{2}, …, x_{N}) is given by (see [4, (24.40) plus the remarks on top of page 411])

o_{λ}^{even}(x_{1}, x_{2}, …, x_{N}) = 2\frac{det_{1≤h,t≤N}(x_{h}^{λ_{t}+N−t} + x_{h}^{−(λ_{t}+N−t)})}{det_{1≤h,t≤N}(x_{h}^{N−t} + x_{h}^{−(N−t)})}.  (2.6)

Here as well, o_{λ}^{even}(x_{1}, x_{2}, …, x_{N}) is a Laurent polynomial in x_{1}, x_{2}, …, x_{N}, and is thus well-defined for any choice of the variables x_{1}, x_{2}, …, x_{N} such that all of them are non-zero. the Weyl denominator formula for type D (cf. [4, Lemma 24.3, Ex. A.66]),

det_{1≤h,t≤n}(x_{h}^{n−t} + x_{h}^{−(n−t)}) = 2 · (x_{1} ⋯ x_{n})^{−n+1} ∏_{1≤h<t≤n}(x_{h} − x_{t})(x_{h}x_{t} − 1),  (2.7)

the denominator in (2.6) can be again evaluated in product form. It is well-known (cf. [4, §24.2]) that o_{λ}^{even}(x_{1}, x_{2}, …, x_{N}) is an irreducible character of O_{2N}(ℂ). When restricted to SO_{2N}(ℂ), it splits into two different irreducible characters of SO_{2N}(ℂ). (The reader should observe that we assumed λ_{N} > 0. If we had allowed λ_{N} = 0, then we would have to divide the right-hand side of (2.6) by 2 in order to obtain an irreducible character of O_{2N}(ℂ) or SO_{2N}(ℂ).)

3. Auxiliary identities. The proofs of Theorems 1 and 2, and those of Theorems 3 and 4 in Section 6, hinge upon certain multivariable identities which we prove in this section. Before we are able to state these identities, we have to introduce some notation.

Throughout, we use the standard notation [n] := {1, 2, …, n}. Let A and B be given subsets of the set of positive integers. Slightly abusing notation for resultants from [7], we define

R (A, B^{−1}) := ∏_{a∈A} ∏_{b∈B}(x_{a} − x_{b}^{−1}).

(In order to avoid any confusion on the part of the reader: the symbol B^{−1} in R(A, B^{−1}) has no meaning by itself, the exponent −1 is just there to indicate that the reciprocals of the variables indexed by B are used on the right-hand side of the definition.) Furthermore, we define

V (A) := _{∏_{a,b∈A} ; a<b}(x_{a} − x_{b})

and

V (A^{−1}) := _{∏_{a,b∈A} ; a<b}(x_{a}^{−1} − x_{b}^{−1}).

In all these definitions, empty products have to be interpreted as 1.

Now we are in the position to state and prove the identity on which the proofs of Theorems 1 and 2 are based.

Lemma 1. For all positive integers N, there holds the identity

∑_{A⊆[2N]} ; |A|=NV (A)V (A^{−1}) V (A^{c})V ((A^{c})^{−1}) R(A, A^{−1}) R(A^{c}, (A^{c})^{−1})

= ∑_{A⊆[2N]} V (A)V (A^{−1}) V (A^{c})V ((A^{c})^{−1}) R(A, (A^{c})^{−1}) R(A^{c}, A^{−1}) , (3.1)

where A^{c} denotes the complement of A in [2N].

Proof. We prove the assertion by induction on N. For N = 1, identity (3.1) reduces to

2(x_{1} − x_{1}^{−1})(x_{2} − x_{2}^{−1}) = 2(x_{1} − x_{2})(x_{1}^{−1} − x_{2}^{−1}) + 2(x_{1} − x_{2}^{−1})(x_{2} − x_{1}^{−1}),

which can be readily verified.

Now let us suppose that we have proved (3.1) with N replaced by N − 1. Below, we shall show that this implies that the identity (3.1) is true if we specialize x_{1} to x_{2}. Let us for the moment suppose that this is already done. Since, as is easy to see, both sides of (3.1) are symmetric in the variables x_{1}, x_{2}, …, x_{2N}, as well as they remain invariant up to an overall multiplicative sign if we replace x_{1} by x_{1}^{−1} (on the left-hand side of (3.1), the latter assertion already applies to each individual summand; on the right-hand side, one has to group the summands in pairs: the summands corresponding to a set A and the symmetric difference A△{1} have to be considered together), this means that identity (3.1) holds for all specializations of x_{1} to one of x_{2}, x_{2}^{−1}, x_{3}, x_{3}^{−1}, …, x_{2N}, x_{2N}^{−1}. These are 4N − 2 specializations. The previously observed fact that the replacement of x_{1} by x_{1}^{−1} changes the sign of both sides of (3.1) implies that both sides vanish for x_{1} = ±1. Thus, we have 4N specializations of x_{1} for which the two sides of (3.1) agree. On the other hand, as Laurent polynomials in x_{1}, the degree of both sides of (3.1) is at most 2N −1. (That is, the maximal exponent e of a power x_{1}^{e} is e = 2N − 1, and the minimal exponent is e = −(2N − 1).) Hence, both sides of (3.1) must be identical.

It remains to prove that, under the induction hypothesis, Equation (3.1) holds for x_{1} = x_{2}. Indeed, under this specialization, terms corresponding to sets A which contain both 1 and 2 and to those which contain neither 1 nor 2 vanish on both sides of (3.1), because of the appearance of the Vandermonde products V (A) respectively V (A^{c}). Therefore, if x_{1} = x_{2}, identity (3.1) reduces to

2(x_{1} − x_{1}^{−1})^{2}( ∏_{j=3}^{2N}(x_{1} − x_{j})(x_{1}^{−1} − x_{j}^{−1})(x_{1} − x_{j}^{−1})(x_{j} − x_{1}^{−1}))

×_{∑_{A⊆{3,4,...,2N}} ; |A|=N−1}V (A)V (A^{−1}) V (A^{c})V ((A^{c})^{−1}) R(A, A^{−1}) R(A^{c}, (A^{c})^{−1})

= 2(x_{1} − x_{1}^{−1})^{2}( ∏_{j=3}^{2N}(x_{1} − x_{j})(x_{1}^{−1} − x_{j}^{−1})(x_{1} − x_{j}^{−1})(x_{j} − x_{1}^{−1}))

× ∑_{A⊆{3,4,...,2N}} V (A)V (A^{−1}) V (A^{c})V ((A^{c})^{−1}) R(A, (A^{c})^{−1}) R(A^{c}, A^{−1}) .

After clearing the product which is common to both sides, we see that the remaining identity is equivalent to (3.1) with N replaced by N − 1, the latter being true due to the induction hypothesis. This finishes the proof of the lemma. □

On the other hand, the proofs of Theorems 3 and 4 are based on the following two lemmas. The reader should observe the subtle difference on the right-hand sides of (3.1) and (3.2) (the left-hand sides being identical).

Lemma 2. For all positive integers N, there holds the identity

∑_{A⊆[2N]} ; |A|=NV (A)V (A^{−1}) V (A^{c})V ((A^{c})^{−1}) R(A, A^{−1}) R(A^{c}, (A^{c})^{−1})

= ∑_{A⊆[2N]} x^{−A}x^{A^{c}}V (A)V (A^{−1}) V (A^{c})V ((A^{c})^{−1}) R(A, (A^{c})^{−1}) R(A^{c}, A^{−1}) , (3.2)

where A^{c} denotes the complement of A in [2N], and where x^{−A} is short for ∏_{a∈A} x_{a}^{−1} and x^{A^{c}} is short for ∏_{a∈A^{c}} x_{a}.

Proof. We proceed as in the proof of Lemma 1. That is, we perform an induction on N. For N = 1, identity (3.2) reduces to

2(x_{1} − x_{1}^{−1})(x_{2} − x_{2}^{−1}) = x_{1}x_{2}(x_{1} − x_{2})(x_{1}^{−1} − x_{2}^{−1}) + x_{1}x_{2}^{−1}(x_{1} − x_{2}^{−1})(x_{2} − x_{1}^{−1})

+ x_{1}^{−1}x_{2}(x_{2} − x_{1}^{−1})(x_{1} − x_{2}^{−1}) + x_{1}^{−1}x_{2}^{−1}(x_{1}^{−1} − x_{2}^{−1})(x_{1} − x_{2}),

which can be readily verified. Almost all the remaining steps are identical with those in the proof of Lemma 1, except that more care is needed to show that, as a Laurent polynomial in x_{1}, the degree of the right-hand side of (3.2) is at most 2N − 1. Indeed, by inspection, this degree is at most 2N, and the coefficient of x_{1}^{2N} is equal to

∑_{A⊆[2N]\{1}}(−1)^{|A^{c}|−1}V (A)V (A^{−1}) V (A^{c})V ((A^{c})^{−1}) R(A, (A^{c})^{−1}) R(A^{c}, A^{−1}) ,

where A^{c} now denotes the complement of A in [2N]\{1}. (Note that only those subsets A of [2N] contribute to the coefficient of x_{1}^{2N} on the right-hand side of (3.2) which do not contain 1, whence the term x^{−A}x^{A^{c}} on the right-hand side of (3.2) got cancelled due to the contributions from the terms R (A, (A^{c})^{−1}) and V ((A^{c})^{−1}), respectively.) However, in this sum, the terms indexed by A respectively A^{c} cancel each other, so that this sum does indeed vanish. □

Lemma 3. For all positive integers N, there holds the identity

∑_{A⊆[2N+1]} ; |A|=NV (A)V (A^{−1}) V (A^{c})V ((A^{c})^{−1}) R(A, A^{−1}) R(A^{c}, (A^{c})^{−1})

= ∑_{A⊆[2N+1]}(−1)^{N+|A|}x^{−A}x^{A^{c}}V (A)V (A^{−1}) V (A^{c})V ((A^{c})^{−1}) R(A, (A^{c})^{−1}) R(A^{c}, A^{−1}) , (3.3)

where A^{c} denotes the complement of A in [2N + 1], while x^{−A} is short for ∏_{a∈A} x_{a}^{−1} and x^{A^{c}} is short for ∏_{a∈A^{c}} x_{a}, as before.

Proof. We proceed again as in the proof of Lemma 1. Here, the induction basis, the case N = 0 of (3.3), reads

(x_{1} − x_{1}^{−1}) = x_{1} − x_{1}^{−1}.

As Laurent polynomials in x_{1}, the degree of the left-hand and right-hand sides of (3.3) are at most 2N + 1. This means that we need 4N + 3 specializations, respectively “informations,” which agree on both sides of (3.3), in order to show that both sides are equal.

In the same way as this is done in Lemma 1, one can show that, if one assumes the truth of (3.3) with N replaced by N − 1, this implies that (3.3) is true if we specialize x_{1} to x_{2}. Similarly to the proof of Lemma 1, it is easy to see that both sides of (3.3) are symmetric in the variables x_{1}, x_{2}, …, x_{2N+1}, and also that they remain invariant up to an overall multiplicative sign if we replace x_{1} by x_{1}^{−1}. This means that identity (3.3) holds for all specializations of x_{1} to one of x_{2}, x_{2}^{−1}, x_{3}, x_{3}^{−1}, …, x_{2N+1}, x_{2N+1}^{−1}. These are 4N specializations. We still need 3 additional specializations, respectively “informations,” which agree on both sides of (3.3).

We get two more specializations by observing that both sides of (3.3) vanish for x_{1} = ±1. For the left-hand side this is obvious because of the factors R(A, A^{−1}) respectively R(A^{c}, (A^{c})^{−1}) appearing in the summand. On the right-hand side, the summands indexed by A and A△{1} cancel each other for x_{1} = ±1.

Finally, the coefficients of x_{1}^{2N+1} on the left-hand and right-hand sides of (3.3) are respectively

∑_{A⊆[2N+1]\{1}} ; |A|=N(−1)^{N}V (A)V (A^{−1}) V (A^{c})V ((A^{c})^{−1}) R(A, A^{−1}) R(A^{c}, (A^{c})^{−1})

and

∑_{A⊆[2N+1]\{1}}(−1)^{N}V (A)V (A^{−1}) V (A^{c})V ((A^{c})^{−1}) R(A, (A^{c})^{−1}) R(A^{c}, A^{−1}) ,

where, in both cases, A^{c} now denotes the complement of A in [2N + 1]\{1}. The equality of these two sums was established in Lemma 1. This completes the proof of the lemma. □

4. Proofs of theorems. This section is devoted to the proofs of Theorems 1 and 2. The idea is to substitute the determinantal definitions (2.1)–(2.6) of the characters into (1.1) respectively (1.2), expand the determinants in the numerators by using Laplace expansion respectively linearity of the determinant in the rows, evaluate the resulting simpler determinants by means of one of the Weyl denominator formulas, and reduce the resulting expressions. By collecting appropriate terms, it is then seen that both identities result from Lemma 1 in the preceding section.

Proof of Theorem 1. We start with the left-hand side of (1.1). By (2.1), we have

s_{((2m)^{n})}(x_{1}, x_{1}^{−1}, x_{2}, x_{2}^{−1}, …, x_{n}, x_{n}^{−1}) = \frac{det_{1≤i,j≤2n} ( \begin{matrix}x_{h}^{2mχ(t≤n)+2n−t} & 1 ≤ h ≤ n \\ x_{h−n}^{−(2mχ(t≤n)+2n−t)} & n + 1 ≤ h ≤ 2n \end{matrix} )}{V ([n])V ([n]^{−1})R([n], [n]^{−1})},

(4.1)

where χ(S) = 1 if S is true and χ(S) = 0 otherwise. Here, we used the evaluation of the Vandermonde determinant (the Weyl denominator formula for type A; cf. [4, p. 400 and Lemma 24.3]) in the denominator. We now do a Laplace expansion of the determinant along the first n columns. Abbreviating the denominator on the right-hand side of (4.1) by D_{1}(n), this leads to

s_{((2m)^{n})}(x_{1}, x_{1}^{−1}, x_{2}, x_{2}^{−1}, …, x_{n}, x_{n}^{−1})

= \frac{1}{D_{1}(n)}_{∑_{A,B⊆[n]} ; |A|+|B|=n}(−1)^{( ∑_{a∈A} a)+( ∑_{b∈B} b)+n|B|−\binom{n+1}{2}} det M_{1}(A, B) · det M_{2}(A^{c}, B^{c}), (4.2)

where M_{1}(A, B) is the n × n matrix

( \begin{matrix}x_{h}^{2m+2n−t} & h ∈ A \\ x_{h}^{−(2m+2n−t)} & h ∈ B \end{matrix} ) ,

and M_{2}(A^{c}, B^{c}) is the n × n matrix

( \begin{matrix}x_{h}^{n−t} & h ∈ A^{c} \\ x_{h}^{−(n−t)} & h ∈ B^{c} \end{matrix} ) ,

with A^{c} denoting the complement of A in [n], and an analogous meaning for B^{c}. All determinants in (4.2) are Vandermonde determinants, except for some trivial factors which can be taken out of the rows of M_{1}(A, B) respectively M_{2}(A, B), and can therefore be evaluated in product form. If we substitute the corresponding results, we obtain

s_{((2m)^{n})}(x_{1}, x_{1}^{−1}, x_{2}, x_{2}^{−1}, …, x_{n}, x_{n}^{−1})

= \frac{1}{D_{1}(n)}_{∑_{A,B⊆[n]} ; |A|+|B|=n}(−1)^{( ∑_{a∈A} a)+( ∑_{b∈B} b)+n|B|−\binom{n+1}{2}}( ∏_{a∈A} x_{a})^{2m+n}( ∏_{b∈B} x_{b}^{−1})^{2m+n}

· V (A)V (B^{−1})R(A, B^{−1})V (A^{c})V ((B^{c})^{−1}) R(A^{c}, (B^{c})^{−1}) . (4.3)

Next we turn to the right-hand side of (1.1). By (2.2), we have

so_{(m^{n})}(x_{1}, x_{2}, …, x_{n}) = \frac{det_{1≤h,t≤n}(x_{h}^{m+n−t+ \frac{1}{2}} − x_{h}^{−(m+n−t+ \frac{1}{2} )})}{det_{1≤h,t≤n}(x_{h}^{n−t+ \frac{1}{2}} − x_{h}^{−(n−t+ \frac{1}{2} )})}

= \frac{det_{1≤h,t≤n}(x_{h}^{m+n−t+ \frac{1}{2}} − x_{h}^{−(m+n−t+ \frac{1}{2} )})}{V ([n]) ∏_{i=1}^{n} x_{i}^{−n+i}(x_{i}^{1/2} − x_{i}^{−1/2}) ∏_{1≤i<j≤n}(x_{i} − x_{j}^{−1})}, (4.4)

where we have used the Weyl denominator formula (2.3). We now use linearity of the determinant in the rows. Abbreviating the denominator on the right-hand side of (4.4) by D_{2}(n), this leads to

so_{(m^{n})}(x_{1}, x_{2}, …, x_{n}) = \frac{1}{D_{2}(n)} ∑_{A⊆[n]}(−1)^{( ∑_{a∈A} a)−\binom{|A|+1}{2}} det M_{3}(A),  (4.5)

where M_{3}(A) is the n × n matrix

( \begin{matrix}x_{h}^{m+n−t+ \frac{1}{2}} & h ∈ A \\ −x_{h}^{−(m+n−t+ \frac{1}{2} )} & h ∈ A^{c} \end{matrix} ) ,

with A^{c} denoting the complement of A in [n], as before. All determinants in (4.5) are Vandermonde determinants, except for some trivial factors which can be taken out of the rows of M_{3}(A), and can therefore be evaluated in product form. If we substitute the corresponding results, we obtain

so_{(m^{n})}(x_{1}, x_{2}, …, x_{n})

= \frac{1}{D_{2}(n)} ∑_{A⊆[n]}(−1)^{( ∑_{a∈A} a)−\binom{|A|}{2}+n}( ∏_{a∈A} x_{a})^{m+ \frac{1}{2}} ( ∏_{a∈A^{c}} x_{a}^{−1})^{m+ \frac{1}{2}}

· V (A)V ((A^{c})^{−1}) R(A, (A^{c})^{−1}) , (4.6)

whence the product of the two characters on the right-hand side of (1.1) can be expanded in the form

so_{(m^{n})}(x_{1}, x_{2}, …, x_{n}) so_{(m^{n})}(−x_{1}, −x_{2}, …, −x_{n})

= \frac{(−1)^{mn}}{D_{1}(n)} ∑_{A,B⊆[n]}(−1)^{( ∑_{a∈A} a)+( ∑_{b∈B} b)−\binom{|A|+1}{2}−\binom{|B|}{2}}

· ( ∏_{a∈A} x_{a})^{m+ \frac{1}{2}} ( ∏_{a∈A^{c}} x_{a}^{−1})^{m+ \frac{1}{2}} ( ∏_{b∈B} x_{b})^{m+ \frac{1}{2}} ( ∏_{b∈B^{c}} x_{b}^{−1})^{m+ \frac{1}{2}}

· V (A)V ((A^{c})^{−1}) R(A, (A^{c})^{−1}) V (B)V ((B^{c})^{−1}) R(B, (B^{c})^{−1}) . (4.7)

We now fix disjoint subsets A′ and B′ of [n] and extract the coefficient of

( ∏_{a∈A′} x_{a})^{2m+n}( ∏_{b∈B′} x_{b}^{−1})^{2m+n}  (4.8)

in (4.3). (When we here say “extract the coefficient of (4.8),” we treat “m” as if it were a formal variable. That is, in the sum on the right-hand side of (4.3), terms must be expanded, from each term which results from the expansion we factor out the product (4.8), and, if whatever remains is independent of m, it contributes to the coefficient.) This coefficient is the subsum of the sum on the right-hand side of (4.3) consisting of the summands corresponding to subsets A and B of [n] with A′ = A\B and B′ = B\A such that |A| + |B| = n. We note at this point that this implies that C := [n]\(A′ ∪ B′) must have even cardinality. We let A′′ be the intersection A′′ = A ∩ B, so that A = A′ ˙∪A′′ and B = B′ ˙∪A′′ (with ˙∪ denoting disjoint union), and we denote the complement of A′′ in C by (A′′)^{c}. Since |A| + |B| = n, we have |A′′| = |(A′′)^{c}|. Using the above notation, then, after some manipulation, the above described subsum can be rewritten as

\frac{1}{D_{1}(n)}(−1)^{n|B′|+ \frac{1}{2} |C|}V (A′)V (B′)V ((A′)^{−1}) V ((B′)^{−1})

× R(A′, (B′)^{−1}) R(B′, (A′)^{−1}) R(C, A′) R(C, (A′)^{−1}) R(C^{−1}, B′) R(C^{−1}, (B′)^{−1})

×_{∑_{A′′⊆C} ; |A′′|= \frac{1}{2} |C|}V (A′′)V ((A′′)^{−1}) R(A′′, (A′′)^{−1}) V ((A′′)^{c}) V (((A′′)^{c})^{−1}) R((A′′)^{c}, ((A′′)^{c})^{−1}) , (4.9)

where we used variations of the resultant notation R(…), namely

R (A, B) := ∏_{a∈A} ∏_{b∈B}(x_{a} − x_{b}),

R (A^{−1}, B) := ∏_{a∈A} ∏_{b∈B}(x_{a}^{−1} − x_{b}),

and

R (A^{−1}, B^{−1}) := ∏_{a∈A} ∏_{b∈B}(x_{a}^{−1} − x_{b}^{−1}).

Now we turn our attention to (4.7). First of all, we observe that by exchanging the roles of A and B in the summand on the right-hand side of (4.7), the summand does not change except for a sign of (−1)^{|A|−|B|}. Thus, all summands corresponding to sets A and B whose cardinalities do not have the same parity cancel each other. We can therefore restrict the sum in (4.7) to subsets A and B of [n] with |A| ≡ |B| (mod 2). Consequently, if we extract the coefficient of (4.8) in (4.7), then, after some manipulation, we obtain

\frac{1}{D_{1}(n)}(−1)^{mn+n|B′|+ \frac{1}{2} |C|}V (A′)V (B′)V ((A′)^{−1}) V ((B′)^{−1})

× R(A′, (B′)^{−1}) R(B′, (A′)^{−1}) R(C, A′) R(C, (A′)^{−1}) R(C^{−1}, B′) R(C^{−1}, (B′)^{−1})

× ∑_{A′′⊆C}V (A′′)V ((A′′)^{−1}) R(A′′, ((A′′)^{c})^{−1}) V ((A′′)^{c}) V (((A′′)^{c})^{−1}) R((A′′)^{c}, (A′′)^{−1}) . (4.10)

Here, with A and B as in (4.7), the meanings of A′, B′, A′′, and C are A′ = A ∩ B, B′ = [n]\(A ∪ B), A′′ = A\B, C = [n]\A′ ∪ B′, and (A′′)^{c} is the complement of A′′ in C, so that A = A′ ˙∪A′′ and B = A′ ˙∪(A′′)^{c}. Because of the restriction |A| ≡ |B| (mod 2), again C must have even cardinality. Clearly, clearing the factors common to (4.9) and (4.10), the equality of (4.9) and (4.10) is a direct consequence of (3.1). This completes the proof of the theorem. □

Proof of Theorem 2. As we have shown when we rewrote Theorems 1 and 2 uniformly as (1.5), Theorem 2 results from the proof of Theorem 1 by “replacing m by m + \frac{1}{2}.” Since, in the proof of Theorem 1, it is nowhere used that m is an integer, and since, in fact, m is treated there as a formal variable, Theorem 2 follows immediately. □

5. Combinatorial interpretations. The purpose of this section is to give combinatorial interpretations of Theorems 1 and 2. As we already said in the introduction, these interpretations were at the origin of this work, which suggested Theorems 1 and 2 in the first place. They involve rhombus tilings, respectively plane partitions. When we speak of a rhombus tiling of some region, we always mean a tiling of the region by unit rhombi with angles of 60^{∘} and 120^{∘}. Examples of such tilings can be found in Figures 1–5. (Dotted lines and shadings should be ignored at this point.) We shall not recall the relevant plane partition definitions here, but instead refer the reader to [2], and to [6] for explanations on the relation between plane partitions and rhombus tilings of hexagons.

We claim that, by specializing x_{1} = x_{2} = ⋯ = x_{n} = 1 in (1.1), we obtain the combinatorial factorization

PP(2m, n, n) = SPP(2m, n, n) · TCPP(2m, n, n),  (5.1)

where PP(2m, n, n) denotes the number of plane partitions contained in the (2m) × n × n box (or, equivalently, the number of rhombus tilings of a hexagon with side lengths 2m, n, n, 2m, n, n; see Figure 1 for an example in which m = 2 and n = 3), SPP(2m, n, n) denotes the number of symmetric plane partitions contained in the (2m) × n × n box (or, equivalently, the number of rhombus tilings of a hexagon with side lengths 2m, n, n, 2m, n, n which are symmetric with respect to the vertical symmetry axis of the hexagon; see Figure 2 for an example in which m = 2 and n = 3), and TCPP(2m, n, n) denotes the number of transpose complementary plane partitions contained in the (2m)×n×n box (or, equivalently, the number of rhombus tilings of a hexagon with side lengths 2m, n, n, 2m, n, n which are symmetric with respect to the horizontal symmetry axis of the hexagon; see Figure 3.a for an example in which m = n = 3; the dotted lines should be ignored for the moment).^{1}

} }^{2m{} n  n

A rhombus tiling of a hexagon

Figure 1

We start with the left-hand side of (5.1). The fact that the specialized Schur function s_{((2m)^{n})}(1, 1, …, 1) (with 2n occurrences of 1 in the argument) is equal to the number of plane partitions in the (2m) × n × n box, and that this is equal to the number of rhombus tilings of a hexagon with side lengths 2m, n, n, 2m, n, n, is well-known (cf. [6] and [10, Sec. 7.21]).

It is also well-known (see [2, Sec. 4.3] or [8, Ch. I, Sec. 5, Ex. 15–17]) that the number of symmetric plane partitions contained in the (2m) × n × n box is equal to so_{(m^{n})}(1, 1, …, 1) (with n occurrences of 1 in the argument).

The argument which explains that so_{(m^{n})}(−1, −1, …, −1) (with n occurrences of −1 in the argument) counts transpose complementary plane partitions contained in the (2m)×n×n

1Clearly, the factorization (5.1) could be readily verified directly by using the known product formulas for PP(2m, n, n), SPP(2m, n, n), and TCPP(2m, n, n) (cf. [2]). However, the point here is that it is a consequence of the more general factorization (1.1) featuring a Schur function and odd orthogonal characters.

} }^{2m{} n  n

A vertically symmetric rhombus tiling of a hexagon

Figure 2

box (up to sign) is more elaborate. We begin by setting x_{i} = −q^{i−1} in (2.2) with N = n and λ = (m^{n}), to obtain

so_{(m^{n})}(−1, −q, −q^{2}, …, −q^{n−1}) = (−1)^{mn} \frac{det_{1≤h,t≤n}(q^{(h−1)(m+n−t+ \frac{1}{2} )} + q^{−(h−1)(m+n−t+ \frac{1}{2} )})}{det_{1≤h,t≤n}(q^{(h−1)(n−t+ \frac{1}{2} )} + q^{−(h−1)(n−t+ \frac{1}{2} )})}.

The determinants on the right-hand side can be evaluated by means of (2.7). (This is seen by replacing h by n + 1 − h and then interchanging the roles of h and t in the determinants above.) If we subsequently let q tend to 1, then we obtain

so_{(m^{n})}(−1, −1, …, −1) = (−1)^{mn} ∏_{1≤h<t≤n} \frac{2m + 2n + 1 − h − t}{2n + 1 − h − t}.

On the other hand, the product on the right-hand side is equal to a specialized symplectic character. Namely, by specializing λ = (m^{N}), x_{i} = q^{i}, i = 1, 2, …, N, in (2.4), using the identity (2.5), and finally letting q tend to 1, it is seen that

sp_{(m^{N})}(1, 1, …, 1) = ∏_{1≤h<t≤N+1} \frac{2m + 2N + 3 − h − t}{2N + 3 − h − t}

(with N occurrences of 1 in the argument). Thus, for N = n − 1, we obtain that

so_{(m^{n})}(−1, −1, …, −1) = (−1)^{mn} sp_{(m^{n−1})}(1, 1, …, 1).

To conclude the argument, it is easy to see using families of m lattice paths starting at the mid-points of the top-most m edges along the left vertical side of length 2m of the hexagon with side lengths 2m, n, n, 2m, n, n and ending at the mid-points of the top-most m edges along the right vertical side of length 2m of the hexagon, the paths “following the rhombi” of the tiling (and, thus, staying necessarily in the upper half of the hexagon; see

} } 2m{ n  n

a. A horizontally symmetric rhombus tiling of a hexagon b. The corresponding path family

[figure: •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; P_{1}; P_{2}; P_{3}; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; P′_{1}; P′_{2}; P′_{3}]

c. Orthogonal lattice paths d. Shorter non-intersecting lattice paths

Figure 3

Figure 3.a,b), that rhombus tilings of a hexagon with side lengths 2m, n, n, 2m, n, n which are symmetric with respect to the horizontal symmetry axis of the hexagon are in bijection with families (P_{1}, P_{2}, …, P_{m}) of non-intersecting lattice paths consisting of horizontal and vertical unit steps, the path P_{i} starting at (−i, i − 1) and ending at (n − i, n + i − 1), i = 1, 2, …, m, all paths never passing below the line y = x + 1 (see Figure 3.c for the path family resulting from the one in Figure 3.b by deforming the paths to orthogonal ones; as usual, the term “non-intersecting” means that no two paths in the family have a common point). Because of the boundary y = x+1 and the condition that paths are non-intersecting, the initial step of any path in such a family must be a vertical step while the final step of any path must be a horizontal step. Thus, our rhombus tilings are in bijection with families (P_{1}, P_{2}, …, P_{m}) of non-intersecting lattice paths consisting of horizontal and vertical unit steps, the path P_{i} starting at (−i, i) and ending at (n − 1 − i, n − 1 + i), i = 1, 2, …, m, all paths never passing below the line y = x+1 (see Figure 3.d). It is known (see [3, Sec. 5]) that these families of non-intersecting lattice paths are counted by sp_{(m^{n−1})}(1, 1, …, 1) (with n occurrences of 1 in the argument). Thus, | so_{(m^{n})}(−1, −1, …, −1)| counts indeed rhombus tilings of a hexagon with side lengths 2m, n, n, 2m, n, n which are symmetric with respect to the horizontal symmetry axis of the hexagon, respectively transpose complementary plane partitions in the (2m) × n × n box.

For Theorem 2 we are also able to provide a combinatorial interpretation in the context of rhombus tilings. However, it may be less convincing.

We specialize x_{1} = x_{2} = ⋯ = x_{n} = 1 in (1.2). Clearly, s_{((2m+1)^{n})}(1, 1, …, 1) (with 2n occurrences of 1 in the argument) counts plane partitions in the (2m + 1) × n × n box, respectively rhombus tilings of a hexagon with side lengths 2m + 1, n, n, 2m + 1, n, n.

} } 2m + 1{ n  n

A horizontally symmetric rhombus tiling of a hexagon with two missing triangles

Figure 4

Moreover, the argument above shows that sp_{(m^{n})}(1, 1, …, 1) (with n occurrences of 1 in the argument) counts rhombus tilings of a hexagon with side lengths 2m+1, n, n, 2m+1, n, n from which two unit triangles have been removed on the left and the right end of the horizontal symmetry axis of the hexagon, the tilings being symmetric with respect to this axis (see Figure 4 for an example in which m = 2 and n = 3). Alternatively, this is also the number of horizontally symmetric rhombus tilings of a (full) hexagon with side lengths 2m, n + 1, n + 1, 2m, n + 1, n + 1 (compare with Figure 3.a).

In order to find a combinatorial interpretation of o_{((m+1)^{n})}^{even}(1, 1, …, 1) (with n occurrences of 1 in the argument), we start with the decomposition

o_{((m+1)^{n})}^{even}(1, 1, …, 1) = so_{((m+1)^{n})}^{even}(1, 1, …, 1) + so_{((m+1)^{n−1},−m−1)}^{even}(1, 1, …, 1),

where

so_{λ}^{even}(x_{1}, x_{2}, …, x_{n}) = \frac{det_{1≤h,t≤n}(x_{h}^{λ_{t}+n−t} + x_{h}^{−(λ_{t}+n−t)}) + det_{1≤h,t≤n}(x_{h}^{λ_{t}+n−t} − x_{h}^{−(λ_{t}+n−t)})}{det_{1≤h,t≤n}(x_{h}^{n−t} + x_{h}^{−(n−t)})}

is an irreducible character of SO_{2n}(ℂ) (and its spin covering group; see [4, (24.40)]). Here, λ = (λ_{1}, λ_{2}, …, λ_{n}) is a non-increasing sequence of (possibly negative) integers or halfintegers with λ_{n−1} ≥ |λ_{n}|. It was proved in [1] (see [5] for a common generalization) that, for all non-negative integers or half-integers c, we have

so_{(c^{n})}^{even}(x_{1}, x_{2}, …, x_{n}) = (x_{1}x_{2} ⋯ x_{n})^{−c} ·_{∑_{ν⊆((2c)^{n})} ; oddcols(((2c)^{n})/ν)=0}s_{ν}(x_{1}, x_{2}, …, x_{n})  (5.2)

and

so_{(c^{n−1},−c)}^{even}(x_{1}, x_{2}, …, x_{n}) = (x_{1}x_{2} ⋯ x_{n})^{−c} ·_{∑_{ν⊆((2c)^{n})} ; oddcols(((2c)^{n})/ν)=2c}s_{ν}(x_{1}, x_{2}, …, x_{n}), (5.3)

where oddcols(((2c)^{n})/ν) denotes the number of odd columns of the skew diagram (cf. [8, p. 4]) ((2c)^{n})/ν. Thus, we have

o_{((m+1)^{n})}^{even}(1, 1, …, 1) = ∑_{ν⊆((2m+2)^{n})} ′ s_{ν}(1, 1, …, 1),

where ∑ ′ is taken over all diagrams ν with the property that ((2m + 2)^{n})/ν consists either of only even columns or of only odd columns. (Both the even orthogonal character on the left-hand side and the Schur functions on the right-hand side contain n occurrences of 1 in their arguments.)

} } 2m + 2{  n  n

• _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} • • • • • •  • • • • • •  ^{P_{1}}^{P_{2}}^{P_{3}}^{P_{4}}^{P_{5}}^{P_{6}}

a. A vertically symmetric rhombus tiling of a hexagon b. Non-intersecting lattice paths

Figure 5

Given a shape ν contained in ((2m+2)^{n}), it is well-known (cf. [3, Sec. 4] or [9, Ch. 4]) that s_{ν}(1, 1, …, 1) counts families (P_{1}, …, P_{2m+2}) of non-intersecting lattice paths consisting of horizontal and vertical unit steps, where the path P_{i} runs from (−i, i) to (ν′_{i} − i, n − ν′_{i} + i), i = 1, 2, …, 2m + 2. (Here, ν′ denotes the partition conjugate to ν; cf. [8, p. 2]). On the other hand, in a similar way as above, such families of non-intersecting lattice paths (if ν is allowed to be any partition) are in bijection with rhombus tilings of a hexagon with side lengths 2m + 2, n, n, 2m + 2, n, n which are symmetric with respect to the vertical symmetry axis of the hexagon (see Figure 5 for an example in which m = 2, n = 4, ν = (5, 5, 2, 2)). The property that ((2m+2)^{n})/ν consists either of only even columns or of only odd columns translates into the property that, in the corresponding rhombus tilings, chains of successive horizontally oriented rhombi along the vertical symmetry axis of the hexagon can appear in the interior of the hexagon only if they have even length. (By definition, a “chain of horizontally oriented rhombi along the vertical symmetry axis of the hexagon” is a set of horizontally oriented rhombi sitting on the vertical symmetry axis which, together, form a topologically connected set. For a chain, to be in the interior, means that none of the rhombi of the chain touches the boundary of the hexagon. In Figure 5.a, there are two such chains. They consist of the two contiguous strings of grey shaded rhombi, respectively.)

In summary, Theorem 2, when specialized to x_{1} = x_{2} = ⋯ = x_{n} = 1, can be interpreted combinatorially as follows: the term s_{((2m+1)^{n})}(1, 1, …, 1) on the left-hand side counts rhombus tilings of a hexagon with side lengths 2m + 1, n, n, 2m + 1, n, n. The term sp_{(m^{n})}(1, 1, …, 1) counts horizontally symmetric rhombus tilings of a hexagon with side lengths 2m + 1, n, n, 2m + 1, n, n from which two unit triangles have been removed on the left and the right end of the horizontal symmetry axis of the hexagon. Finally, the term o_{((m+1)^{n})}^{even}(1, 1, …, 1) counts vertically symmetric rhombus tilings of a hexagon with side lengths 2m + 2, n, n, 2m + 2, n, n with the property that chains of successive horizontally oriented rhombi along the vertical symmetry axis of the hexagon can appear in the interior of the hexagon only if they have even length. We remark that these rhombus tilings could equivalently be seen as symmetric plane partitions in a (2m+2)×n×n box in which “central terraces” (that is, horizontal levels situated along the plane of symmetry) must have even length, except at height 0 and at height 2m + 2. In other words, this specialization yields the identity

PP(2m + 1, n, n) = TCPP(2m, n + 1, n + 1) SPP^{∗}(2m + 2, n, n),  (5.4)

where the ^{∗} indicates that only those symmetric plane partitions are counted which satisfy the “even central terraces in the interior” condition described above. In particular, by specializing x_{h} = q^{h−1} in (2.6), using (2.7) for evaluating the two determinants in (2.6), and then letting q tend to 1, we obtain that

SPP^{∗}(2m, n, n) = 2 ∏_{1≤h<t≤n} \frac{2m + 2n − h − t}{2n − h − t}.  (5.5)

6. More factorization theorems. In this final section, we present two further factorization theorems similar to those of Theorems 1 and 2, which have been hinted at to us by Ron King. Since these theorems can be proved in a manner very similar to the one which led to our proofs of Theorems 1 and 2, we content ourselves with giving only sketches of proofs, leaving the (easily filled in) details to the reader.

Theorem 3. For any non-negative integers m and n, we have

s_{((2m+1)^{n})}(x_{1}, x_{1}^{−1}, x_{2}, x_{2}^{−1}, …, x_{n}, x_{n}^{−1}) + s_{((2m+1)^{n−1})}(x_{1}, x_{1}^{−1}, x_{2}, x_{2}^{−1}, …, x_{n}, x_{n}^{−1}) = (−1)^{mn} so_{((m+1)^{n})}(x_{1}, x_{2}, …, x_{n}) so_{(m^{n})}(−x_{1}, −x_{2}, …, −x_{n}). (6.1)

Sketch of proof. We proceed in the same way as in the proof of Theorem 1. By comparison with (4.3), we see that the left-hand side of (6.1) is equal to

\frac{1}{D_{1}(n)}_{∑_{A,B⊆[n]} ; |A|+|B|=n}(−1)^{( ∑_{a∈A} a)+( ∑_{b∈B} b)+n|B|−\binom{n+1}{2}}( ∏_{a∈A} x_{a})^{2m+n+1}( ∏_{b∈B} x_{b}^{−1})^{2m+n+1}

· V (A)V (B^{−1})R(A, B^{−1})V (A^{c})V ((B^{c})^{−1}) R(A^{c}, (B^{c})^{−1})

+ \frac{1}{D_{1}(n)}_{∑_{A,B⊆[n]} ; |A|+|B|=n−1}(−1)^{( ∑_{a∈A} a)+( ∑_{b∈B} b)+n|B|−\binom{n+1}{2}}( ∏_{a∈A} x_{a})^{2m+n+2}( ∏_{b∈B} x_{b}^{−1})^{2m+n+2}

· V (A)V (B^{−1})R(A, B^{−1})V (A^{c})V ((B^{c})^{−1}) R(A^{c}, (B^{c})^{−1}) , (6.2)

with D_{1}(n) having the same meaning as in the proof of Theorem 1. On the other hand, from (4.6) we see that the right-hand side of (6.1) is equal to

\frac{(−1)^{mn}}{D_{1}(n)} ∑_{A,B⊆[n]}(−1)^{( ∑_{a∈A} a)+( ∑_{b∈B} b)−\binom{|A|+1}{2}−\binom{|B|}{2}}

· ( ∏_{a∈A} x_{a})^{m+ \frac{1}{2}} ( ∏_{a∈A^{c}} x_{a}^{−1})^{m+ \frac{1}{2}} ( ∏_{b∈B} x_{b})^{m+ \frac{3}{2}} ( ∏_{b∈B^{c}} x_{b}^{−1})^{m+ \frac{3}{2}}

· V (A)V ((A^{c})^{−1}) R(A, (A^{c})^{−1}) V (B)V ((B^{c})^{−1}) R(B, (B^{c})^{−1})

= \frac{(−1)^{mn}}{D_{1}(n)} ∑_{A,B⊆[n]}(−1)^{( ∑_{a∈A} a)+( ∑_{b∈B} b)−\binom{|A|+1}{2}−\binom{|B|}{2}}

· ( ∏_{a∈A∩B} x_{a})^{2m+2}( ∏_{a∈A^{c}∩B^{c}} x_{a}^{−1})^{2m+2}( ∏_{a∈A∩B^{c}} x_{b})^{−1}( ∏_{b∈A^{c}∩B} x_{b})

· V (A)V ((A^{c})^{−1}) R(A, (A^{c})^{−1}) V (B)V ((B^{c})^{−1}) R(B, (B^{c})^{−1}) . (6.3)

We now fix disjoint subsets A′ and B′ of [n] and extract the coefficients of

( ∏_{a∈A′} x_{a})^{2m+n}( ∏_{b∈B′} x_{b}^{−1})^{2m+n}

in (6.2) respectively in (6.3) (in the same sense as in Section 4). If |A′| + |B′| has the same parity as n, then in (6.2) only the first sum contributes, while in (6.3) it is only terms where the cardinality of the symmetric difference A△B is even. One can then see in the same way as Theorem 1 followed from Lemma 1, that the equality of corresponding contributions follows from Lemma 2. On the other hand, if |A′| + |B′| has parity different from the parity of n, then in (6.2) only the second sum contributes, while in (6.3) it is only terms where the cardinality of the symmetric difference A△B is odd. In this case, the corresponding equality follows from Lemma 3. □

Theorem 4. For any non-negative integers m and n, we have

s_{((2m)^{n})}(x_{1}, x_{1}^{−1}, x_{2}, x_{2}^{−1}, …, x_{n}, x_{n}^{−1}) + s_{((2m)^{n−1})}(x_{1}, x_{1}^{−1}, x_{2}, x_{2}^{−1}, …, x_{n}, x_{n}^{−1}) = sp_{(m^{n})}(x_{1}, x_{2}, …, x_{n}) o_{(m^{n})}^{even}(x_{1}, x_{2}, …, x_{n}). (6.4)

Sketch of proof. In a similar manner as we saw that the proof of Theorem 2 follows by “replacing m by m + \frac{1}{2}” in the proof of Theorem 1, the proof of Theorem 4 follows by “replacing m by m − \frac{1}{2}” in the proof of Theorem 3. □

Again, using (1.3) and (1.4), Theorems 3 and 4 allow for a uniform statement, namely as

( ∏_{i=1}^{n}(x_{i}^{1/2} + x_{i}^{−1/2}))(s_{(M^{n})}(x_{1}, x_{1}^{−1}, x_{2}, x_{2}^{−1}, …, x_{n}, x_{n}^{−1})

+ s_{(M^{n−1})}(x_{1}, x_{1}^{−1}, x_{2}, x_{2}^{−1}, …, x_{n}, x_{n}^{−1}))

= so_{(( \frac{M+1}{2} )^{n})}(x_{1}, x_{2}, …, x_{n}) o_{(( \frac{M}{2} )^{n})}^{even}(x_{1}, x_{2}, …, x_{n}). (6.5)

In a similar vein as in Section 5, by specializing x_{i} = 1, i = 1, 2, …, n, in Theorems 3 and 4, we are able to derive potentially interesting combinatorial interpretations. If we perform these specializations in (6.1), then, by the combinatorial facts explained in Section 5, we obtain the identity

PP(2m + 1, n, n) + PP(2m + 1, n − 1, n + 1) = SPP(2m + 2, n, n) · TCPP(2m, n, n), (6.6)

while the same specialization in (6.4) yields

PP(2m, n, n) + PP(2m, n − 1, n + 1) = TCPP(2m, n + 1, n + 1) · SPP^{∗}(2m, n, n), (6.7)

where PP(A, B, C) denotes the number of plane partitions contained in the A×B×C box (or, equivalently, the number of rhombus tilings of a hexagon with side lengths A, B, C, A, B, C, SPP(M, N, N) and TCPP(M, N, N) have the same meaning as in Section 5, and where the ∗ in SPP^{∗}(2m, n, n) means that the symmetric plane partitions in consideration satisfy the “even central terraces in the interior” condition explained at the end of Section 5.

Clearly again, the factorizations (6.6) or (6.7) could be readily verified directly by using the product formulas for the combinatorial quantities involved. However, it would have been difficult to see that they exist without having first considered the factorization identities for classical group characters given by Theorems 3 and 4, respectively.

Acknowledgements

We are indebted to Ron King and Soichi Okada for extremely insightful comments on the first version of this manuscript, which helped to improve its contents considerably.

References

1. A. J. Bracken and H. S. Green, Algebraic identities for parafermi statistics of given order, Nuovo Cimento 9A (1972), 349–365.

2. D. M. Bressoud, Proofs and confirmations — The story of the alternating sign matrix conjecture, Cambridge University Press, Cambridge, 1999.

3. M. Fulmek and C. Krattenthaler, Lattice path proofs for determinant formulas for symplectic and orthogonal characters, J. Combin. Theory Ser. A 77 (1997), 3–50.

4. W. Fulton and J. Harris, Representation Theory, Springer–Verlag, New York, 1991.

5. C. Krattenthaler, Identities for classical group characters of nearly rectangular shape, J. Algebra 209 (1998), 1–64.

6. G. Kuperberg, Symmetries of plane partitions and the permanent determinant method, J. Combin. Theory Ser. A 68 (1994), 115–151.

7. A. Lascoux, Symmetric functions and combinatorial operators on polynomials, CBMS Regional Conference Series in Mathematics, vol. 99, Amer. Math. Soc., Providence, RI, 2003.

8. I. G. Macdonald, Symmetric Functions and Hall Polynomials, second edition, Oxford University Press, New York/London, 1995.

9. B. E. Sagan, The symmetric group, 2nd edition, Graduate Texts in Math., vol. 203, Springer–Verlag, New York, 2001.

10. R. P. Stanley, Enumerative Combinatorics, Vol. 2, Cambridge University Press, Cambridge, 1999.

Department of Mathematics, Indiana University, Bloomington, IN 47405-5701, USA

Fakultät für Mathematik der Universität Wien, Nordbergstraße 15, A-1090 Wien, Austria.