The major index generating function of standard Young

tableaux of shapes of the form “staircase minus rectangle”

C. Krattenthaler and M. J. Schlosser

Fakultät für Mathematik, Universität Wien, Oskar-Morgenstern-Platz 1, A-1090 Vienna, Austria.

                 WWW: http://www.mat.univie.ac.at/~kratt
               WWW: http://www.mat.univie.ac.at/~schlosse

Abstract. A specialisation of a transformation formula for multi-dimensional elliptic hypergeometric series is used to provide compact, non-determinantal formulae for the generating function with respect to the major index of standard Young tableaux of skew shapes of the form “staircase minus rectangle”.

1. Introduction. A standard Young tableau of skew shape λ/μ, where λ = (λ_{1}, λ_{2}, …, λ_{n}) and μ = (μ_{1}, μ_{2}, …, μ_{n}) are n-tuples of non-negative integers which are in nonincreasing order and satisfy λ_{i} ≥ μ_{i} for all i, is an arrangement of the numbers 1, 2, …, |λ−μ| (where the last quantity denotes the sum of the respective differences of the integers, ∑_{i=1}^{n}(λ_{i} − μ_{i})) of the form

π_{1,μ_{1}+1} . . . . . . . . . . . . . . . . . π_{1,λ_{1}}
π_{2,μ_{2}+1} … π_{2,μ_{1}+1} . . . . . . . . . . π_{2,λ_{2}}
...  ⋮  ...
π_{n,μ_{n}+1} . . . . . . . . . . . . . . . . . . . . . . . . . . π_{n,λ_{n}}

such that numbers along rows and columns are increasing. The major index of a standard Young tableau T, denoted by maj(T), is defined as the sum over all i such that i+1 appears in a lower row in T than i. It is well-known (see e.g. [19, Prop. 7.19.11 in combination with Theorem 7.16.1 and (7.10) with x_{i} = q^{i−1}, i = 1, 2, …, and y_{i} = 0 for i ≥ 2]) that the generating function ∑_{T} q^{maj(T)}, where the sum runs over all standard Young tableaux of shape λ/μ, equals

[|λ − μ|]_{q}! · det_{1≤i,j≤n} (\frac{1}{[λ_{i} − i − μ_{j} + j]_{q}!}) ,  (1.1)

2010 Mathematics Subject Classification. Primary 05A15; Secondary 05A19 05E05 33C67.

Key words and phrases. standard Young tableaux, basic hypergeometric series associated to root systems, elliptic hypergeometric series associated to root systems.

Research partially supported by the Austrian Science Foundation FWF, grant Z130-N13, grant S9607-N13 (National Research Network “Analytic Combinatorics and Probabilistic Number Theory”), and grant SFB F50 (Special Research Program “Algorithmic and Enumerative Combinatorics”) .

Two skew shapes of the form “staircase minus rectangle”

Figure 1

where [m]_{q}! := [m]_{q} [m − 1]_{q} ⋯ [1]_{q} with [α]_{q} = 1 + q + q^{2} + ⋯ + q^{α−1} = \frac{1−q^{α}}{1−q}.

The purpose of this note is to provide formulae for this major index generating function for standard Young tableaux of shapes λ/μ, where λ is a staircase shape, i.e., λ = (N, N − 1, …, N − n + 1) for some positive integers N and n, and μ is a rectangular shape, i.e., μ = (m, m, …, m, 0, …, 0), for some non-negative integer m and r repetitions of m (in the sequel we denote such partitions μ by (m^{r}), for short), which are (computationally) simpler than the determinantal formula (1.1). Figure 1 shows the Young diagrams of two such shapes according to standard English convention (cf. [18, p. 29]): the diagram on the left represents the shape (6, 5, 4, 3, 2, 1)/(3, 3, 0, 0, 0, 0), and the diagram on the right represents the shape (8, 7, 6, 5, 4, 3)/(2, 2, 2, 2, 0, 0). In particular, for N = n, the above announced formula reduces to a closed form product formula. To be precise, we show that the generating function ∑_{T} q^{maj(T)} for standard Young tableaux T of shape (n, n − 1, …, 1)/(m^{r}) (of which the left shape in Figure 1 is an example) equals

q^{\frac{1}{2} mr(r+m−2n)+\binom{n+1}{3}}(1 + q)^{\binom{n}{2}−mr} [\binom{n+1}{2} − mr]_{q}!

× ∏_{i=1}^{n} \frac{[i − 1]_{q^{2}}!}{[2i − 1]_{q}!} ∏_{i=1}^{r} \frac{[i − 1]_{q^{2}}! [n + m − r + 2i − 1]_{q}!}{[m + i − 1]_{q^{2}}! [n − m − r + 2i − 1]_{q}!}, (1.2)

a result which was originally (implicitly) obtained by DeWitt [4, Theorem V.3] using completely different means. She proves in fact the stronger result that a Schur s-function of a shape of the form (n, n − 1, …, 1)/(m^{r}) is the constant multiple of a particular Schur P -function. If this is combined with Kawanaka’s product formula for the principal specialisation of Schur P -functions (see [10] and [15]), then one obtains the above formula. Furthermore, for N = n+1, we show that the generating function ∑_{T} q^{maj(T)} for standard Young tableaux T of shape (n + 1, n, …, 2)/(m^{r}) equals

(1 + q)^{\binom{n}{2}−(m−1)r}q^{\frac{1}{2} mr(r+m−2n+2)+r(1−n−m)+\binom{n+1}{3}+\binom{n}{2}}

× [\binom{n+2}{2} − mr − 1]_{q}! ∏_{i=1}^{n} \frac{[i − 1]_{q^{2}}!}{[2i]_{q}!} ∏_{i=1}^{r} \frac{[i − 1]_{q^{2}}! [n + m − r + 2i − 1]_{q}!}{[m + i − 1]_{q^{2}}! [n − m − r + 2i]_{q}!}

× ∑_{ℓ=0}^{r} \frac{(−1)^{r}q^{2nl_{1}}}{(1 − q^{2})^{r}} [ \begin{matrix}r \\ ℓ \end{matrix} ]_{q^{2}} \frac{(q^{−2n}; q^{2})_{ℓ} (q^{n+m−r}; q^{2})_{r−ℓ} (q^{n−m−r+1}; q^{2})_{r−ℓ}}{(q^{n+m−r+1}; q^{2})_{r−ℓ}} . (1.3)

Here, the shifted q-factorials are defined by (α; q)_{k} := (1 − α)(1 − αq) ⋯ (1 − αq^{k−1}) for k ≥ 1, and (α; q)_{0} := 1.

In general, if N = n + s, where s is a non-negative integer, we are able to express the major index generating function for standard Young tableaux of shape (n + s, n + s − 1, …, s + 1)/(m^{r}) as an ⌈s/2⌉-fold basic hypergeometric sum, see Theorem 1 in Section 2. If n is large compared to r and s, then this formula is computationally superior to the determinantal formula (1.1).

A notable feature of the proof of Theorem 1 that we give here is that we require a basic hypergeometric specialisation of a transformation formula for multi-dimensional elliptic hypergeometric series due to Rains and, independently, Coskun and Gustafson; see Section 3.

2. The main result. In this section we present our main result, a multi-dimensional basic hypergeometric series which gives the major index generating function for standard Young tableaux of a skew shape that is the difference between a staircase and a rectangle.

Theorem 1. Let N, n, m, r be non-negative integers with N ≥ n and N − r + 1 ≥ m. If N − n is even, the generating function ∑_{T} q^{maj(T)} for standard Young tableaux T of shape (N, N − 1, …, N − n + 1)/(m^{r}) equals

(−1)^{\binom{(N−n)/2}{2}+ \frac{1}{2} r(N−n)}(1 + q)^{\binom{n}{2}−\binom{(N−n)/2}{2}−mr}(1 − q)^{−\binom{(N−n)/2}{2}−r(N−n)}

× q^{\frac{1}{2} mr(r+m−2n)+ \frac{1}{2} r(N−n)(\frac{1}{2} (N−3n)−m+1)+\binom{n+1}{3}+(N−n)(\binom{n}{2}+\binom{(N−n)/2}{2})}

× \frac{[\binom{N+1}{2} − \binom{N−n+1}{2} − mr]_{q}!}{[r + \frac{N−n−2}{2}]_{q^{2}}!^{(N−n)/2} [\frac{N+n−2}{2}]_{q^{2}}!^{(N−n)/2}} \frac{∏_{i=1}^{(N+n)/2}[i − 1]_{q^{2}}!}{∏_{i=1}^{n}[N − n + 2i − 1]_{q}!}

× ∏_{i=1}^{r} \frac{[\frac{N−n}{2} + i − 1]_{q^{2}}! [n + m − r + 2i − 1]_{q}! (q^{n+m−r+2i}; q^{2})_{(N−n)/2}}{[m + i − 1]_{q^{2}}! [N − m − r + 2i − 1]_{q}!}

× ∑_{0≤ℓ_{1}<ℓ_{2}<···<ℓ_{(N−n)/2}≤r+ \frac{N−n−2}{2}} q^{∑_{i=1}^{(N−n)/2}(N+n−2(2i−1))l_{i}} ∏_{1≤i<j≤ \frac{N−n}{2}}[ℓ_{j} − ℓ_{i}]_{q^{2}}^{2}

· ∏_{i=1}^{\frac{N−n}{2}} ( [ \begin{matrix}\frac{N−n−2}{2} + r \\ ℓ_{i} \end{matrix} ]_{q^{2}} (q^{2−N−n}; q^{2})_{ℓ_{i}} (q^{n+m−r−2i+1}; q^{2})_{r+i−ℓ_{i}−1}

· \frac{(q^{N−m−r−2i+2}; q^{2})_{r+i−ℓ_{i}−1}}{(q^{N+m−r−2i+2}; q^{2})_{r+i−ℓ_{i}−1}} ), (2.1)

while, if N − n is odd, it equals

(−1)^{\binom{(N−n+1)/2}{2}+ \frac{1}{2} r(N−n+1)}(1 + q)^{\binom{n}{2}−\binom{(N−n+1)/2}{2}−mr}(1 − q)^{−\binom{(N−n+1)/2}{2}−r(N−n)}

× q^{\frac{1}{2} mr(r+m−2n+2)+ \frac{1}{2} r(N−n+1)(\frac{1}{2} (N−3n+1)−m)+\binom{n+1}{3}+(N−n)\binom{n}{2}+(N−n+1)\binom{(N−n+1)/2}{2}}

× \frac{[\binom{N+1}{2} − \binom{N−n+1}{2} − mr]_{q}!}{[r + \frac{N−n−1}{2}]_{q^{2}}!^{(N−n+1)/2} [\frac{N+n−1}{2}]_{q^{2}}!^{(N−n+1)/2}} \frac{∏_{i=1}^{(N+n+1)/2}[i − 1]_{q^{2}}!}{∏_{i=1}^{n}[N − n + 2i − 1]_{q}!}

× ∏_{i=1}^{r} \frac{[\frac{N−n+1}{2} + i − 1]_{q^{2}}! [n + m − r + 2i − 1]_{q}! (q^{n+m−r+2i+1}; q^{2})_{(N−n−1)/2}}{[m + i − 1]_{q^{2}}! [N − m − r + 2i − 1]_{q}!}

× ∑_{0≤ℓ_{1}<ℓ_{2}<···<ℓ_{(N−n+1)/2}≤r+ \frac{N−n−1}{2}} q^{∑_{i=1}^{(N−n+1)/2}(N+n+1−2(2i−1))l_{i}} ∏_{1≤i<j≤ \frac{N−n+1}{2}}[ℓ_{j} − ℓ_{i}]_{q^{2}}^{2}

· ∏_{i=1}^{\frac{N−n+1}{2}} ( [ \begin{matrix}\frac{N−n−1}{2} + r \\ ℓ_{i} \end{matrix} ]_{q^{2}} (q^{1−N−n}; q^{2})_{ℓ_{i}} (q^{n+m−r−2i+2}; q^{2})_{r+i−ℓ_{i}−1}

· \frac{(q^{N−m−r−2i+2}; q^{2})_{r+i−ℓ_{i}−1}}{(q^{N+m−r−2i+2}; q^{2})_{r+i−ℓ_{i}−1}} ). (2.2)

Proof. According to Formula (1.1), the major index generating function for standard Young tableaux which we want to compute is equal to

[\binom{N+1}{2} − \binom{N−n+1}{2} − mr]_{q}! det_{1≤i,j≤n} (\{ \begin{matrix}\frac{1}{[N + 1 − 2i − m + j]_{q}!} & j ≤ r \\ \frac{1}{[N + 1 − 2i + j]_{q}!} & j > r \end{matrix} ) .

We now do a Laplace expansion with respect to the first r columns. In this way we obtain

[\binom{N+1}{2} − \binom{N−n+1}{2} − mr]_{q}!

× ∑_{1≤k_{1}<···<k_{r}≤n}(−1)^{\binom{r+1}{2}+∑_{i=1}^{r} k_{i}} det_{1≤i,j≤r} (\frac{1}{[N + 1 − 2k_{i} − m + j]_{q}!})

· det_{1≤i≤n, i∉\{k_{1},...,k_{r}\}}_{r+1≤j≤n} (\frac{1}{[N + 1 − 2i + j]_{q}!}) .

Using the simple determinant evaluation

det_{1≤i,j≤s} (\frac{1}{[X_{i} + j]_{q}!}) = q^{2\binom{s+1}{3}+∑_{i=1}^{s}(i−1)X_{i}} ∏_{i=1}^{s} \frac{1}{[X_{i} + s]_{q}!} ∏_{1≤i<j≤s}[X_{i} − X_{j}]

(which is readily established by writing [α]_{q} = (1−q^{α})/(1−q) for every q-integer, factoring out common denominators of rows, and reduction to a Vandermonde determinant), the above determinants can be evaluated, so that we arrive at a multi-dimensional series of basic hypergeometric type:

[\binom{N+1}{2} − \binom{N−n+1}{2} − mr]_{q}!

× ∑_{1≤k_{1}<···<k_{r}≤n}(−1)^{\binom{r+1}{2}+∑_{i=1}^{r} k_{i}}q^{2\binom{r+1}{3}+2\binom{n−r+1}{3}}

· q^{∑_{i=1}^{r}(i−1)(N+1−2k_{i}−m)+∑_{i=1}^{n−r}(i−1)(N+1+r)}

· q^{−2 ∑_{i=1}^{n}(i−1)i+2 ∑_{i=1}^{r}(k_{i}−1)k_{i}+2 ∑_{i=1}^{r} i(\binom{k_{i+1}}{2}−\binom{k_{i}+1}{2})}

· ∏_{i=1}^{n} \frac{1}{[N + n + 1 − 2i]_{q}!} ∏_{i=1}^{r} \frac{[N + n + 1 − 2k_{i}]_{q}!}{[N + 1 − 2k_{i} − m + r]_{q}!}

· ∏_{1≤i<j≤r} [(N + 1 − 2k_{i} − m) − (N + 1 − 2k_{j} − m)]_{q}

·_{∏_{1≤i<j≤n} ; i,j∉{k_{1},...,k_{r}}}[(N + 1 − 2i + r) − (N + 1 − 2j + r)]_{q}

= [\binom{N+1}{2} − \binom{N−n+1}{2} − mr]_{q}! ∏_{i=1}^{n} \frac{1}{[N + n + 1 − 2i]_{q}!} ∏_{1≤i<j≤n}[2j − 2i]_{q}

× ∑_{1≤k_{1}<···<k_{r}≤n}(−1)^{\binom{r+1}{2}+∑_{i=1}^{r} k_{i}}q^{2\binom{r+1}{3}+2\binom{n−r+1}{3}+(N+1−m)\binom{r}{2}+(N+1+r)\binom{n−r}{2}}

· q^{−4\binom{n+1}{3}+2r\binom{n+1}{2}+2 ∑_{i=1}^{r} \binom{k_{i}}{2}−2 ∑_{i=1}^{r}(2i−1)k_{i}}

· ∏_{i=1}^{r} \frac{[N + n + 1 − 2k_{i}]_{q}!}{[N + 1 − 2k_{i} − m + r]_{q}!} ∏_{i=1}^{r} \frac{1}{[2k_{i} − 2]_{q}!! [2n − 2k_{i}]_{q}!!} ∏_{1≤i<j≤r}[2k_{j} − 2k_{i}]_{q}^{2}

= (−1)^{\binom{r}{2}}(1 + q)^{\binom{n}{2}−(n−1)r}(1 − q)^{−r(r−1)}

× q^{2\binom{r+1}{3}+2\binom{n−r+1}{3}+(N+1−m)\binom{r}{2}+(N+1+r)\binom{n−r}{2}−4\binom{n+1}{3}+2r\binom{n+1}{2}−2r^{2}}

× [\binom{N+1}{2} − \binom{N−n+1}{2} − mr]_{q}!

× ∏_{i=1}^{n} \frac{[i − 1]_{q^{2}}!}{[N + n + 1 − 2i]_{q}!} ∏_{i=1}^{r} \frac{[N + n − 1]_{q}!}{[n − 1]_{q^{2}}! [N − m + r − 1]_{q}!}

× ∑_{0≤k_{1}<···<k_{r}≤n−1} q^{−2 ∑_{i=1}^{r}(2i−1)k_{i}} ∏_{1≤i<j≤r}(1 − q^{−2(k_{i}−k_{j})})^{2}

· ∏_{i=1}^{r} \frac{(q^{N−m+r−1}; q^{−2})_{k_{i}} (q^{N−m+r−2}; q^{−2})_{k_{i}} (q^{2n−2}; q^{−2})_{k_{i}}}{(q^{N+n−1}; q^{−2})_{k_{i}} (q^{N+n−2}; q^{−2})_{k_{i}} (q^{−2}; q^{−2})_{k_{i}}} , (2.3)

where the double q-factorials are defined by [2α]_{q}!! = [2α]_{q} [2α − 2]_{q} ⋯ [2]_{q}, and, by convention, k_{r+1} = n + 1.

At this point, we should note that we may change the range of summation to

0 ≤ k_{1} < ⋯ < k_{r} ≤ ⌊\frac{1}{2}(N − m + r − 1)⌋ .  (2.4)

Indeed, the summand in (2.3) vanishes for k_{r} > n − 1 because of the term (q^{2n−2}; q^{−2})_{k_{i}}, unless this incident is “neutralised” by vanishing terms in the denominator. However, since n ≥ r, we have

N − m + r − 1 ≤ N + n − 2,  (2.5)

except if m = 0 and n = r. In the latter case, one sees directly that, due to cancellation of terms, the extension to (2.4) does not change the sum, whereas in the former case the inequality (2.5) guarantees that the summand still vanishes for n−1 < k_{i} ≤ \frac{1}{2}(N−m+r−1).

This last observation makes it possible to apply the transformation formula in Corollary 4 in Section 3. In order to do so, we have to distinguish between N − n being even or odd. If N − n is even, then we have

\frac{(q^{2n−2}; q^{−2})_{k_{i}}}{(q^{N+n−2}; q^{−2})_{k_{i}}} = \begin{matrix}(q^{N+n−2−2k_{i}}; q^{−2}) & (N−n)/2 \\ (q^{N+n−2}; q^{−2}) & (N−n)/2 \end{matrix} .

We then apply (3.5) with q replaced by q^{−2}, s = \frac{N−n}{2}, b = q^{N−m+r−1}, c = q^{N−m+r−2}, d = q^{N+n−2}, f = q^{N+n−1} to the sum in (2.3). After considerable (but routine) manipulation of the arising expression, the result turns out to equal (2.1).

If N − n is odd, then we have

\frac{(q^{2n−2}; q^{−2})_{k_{i}}}{(q^{N+n−1}; q^{−2})_{k_{i}}} = \begin{matrix}(q^{N+n−1−2k_{i}}; q^{−2}) & (N−n+1)/2 \\ (q^{N+n−1}; q^{−2}) & (N−n+1)/2 \end{matrix} .

Here, we apply (3.5) with q replaced by q^{−2}, s = \frac{N−n+1}{2}, b = q^{N−m+r−1}, c = q^{N−m+r−2}, d = q^{N+n−1}, f = q^{N+n−2} to the sum in (2.3). Once again, after lengthy, but routine, manipulation of the arising expression, we arrive at (2.2). □

3. A transformation formula for elliptic hypergeometric series and its consequences. In this section we derive the transformation formula for multi-dimensional basic hypergeometric series which is used crucially in the proof of Theorem 1 in the previous section. It arises by specialising and taking appropriate limits in a transformation formula for multi-dimensional elliptic hypergeometric series. In order to state this formula, we first need to introduce “elliptic notation.” Given a complex number p with |p| < 1, we define a (rescaled) theta function θ(x; p) by

θ(x; p) = ∏_{j=0}^{∞}(1 − p^{j}x)(1 − p^{j+1}/x).

Furthermore, out of these “bricks,” we build elliptic shifted factorials. Namely, fixing another complex parameter, q say, and a non-negative integer m, we set

(a; q, p)_{m} = θ(a; p) θ(aq; p) ⋯ θ(aq^{m−1}; p),

where the right-hand side is understood as 1 if m = 0. For convenience, we also employ the short notation

(a_{1}, a_{2}, …, a_{k}; q, p)_{m} = (a_{1}; q, p)_{m} (a_{2}; q, p)_{m} ⋯ (a_{k}; q, p)_{m}.

The following result is a special case of a multi-dimensional _{12}V_{11} transformation formula conjectured by Warnaar (let x = q in [20, Conj. 6.1]), which has subsequently been proven (in more generality) by Rains [13, Theorem 4.9] and, independently, by Coskun and Gustafson [3]. In the present form the identity has been stated in [16, Theorem 3.2] (with a small typo corrected).

Theorem 2. Let a, b, c, d, e, f be indeterminates, let m be a nonnegative integer, and r ≥ 1. Then we have

∑_{0≤k_{1}<k_{2}<···<k_{r}≤m} q^{∑_{i=1}^{r}(2i−1)k_{i}} ∏_{1≤i<j≤r} θ(q^{k_{i}−k_{j}}; p)^{2} θ(aq^{k_{i}+k_{j}}; p)^{2}

× ∏_{i=1}^{r} \frac{θ(aq^{2k_{i}}; p)(a, b, c, d, e, f, λaq^{2−r+m}/ef, q^{−m}; q, p)_{k_{i}}}{θ(a; p)(q, aq/b, aq/c, aq/d, aq/e, aq/f, efq^{r−1−m}/λ, aq^{1+m}; q, p)_{k_{i}}}

= ∏_{i=1}^{r} \frac{(b, c, d, ef/a; q, p)_{i−1}}{(λb/a, λc/a, λd/a, ef/λ; q, p)_{i−1}}

× ∏_{i=1}^{r} \frac{(aq; q, p)_{m} (aq/ef; q, p)_{m+1−r} (λq/e, λq/f; q, p)_{m−i+1}}{(λq; q, p)_{m} (λq/ef; q, p)_{m+1−r} (aq/e, aq/f; q, p)_{m−i+1}}

× ∑_{0≤k_{1}<k_{2}<···<k_{r}≤m} q^{∑_{i=1}^{r}(2i−1)k_{i}} ∏_{1≤i<j≤r} θ(q^{k_{i}−k_{j}}; p)^{2} θ(λq^{k_{i}+k_{j}}; p)^{2}

× ∏_{i=1}^{r} \frac{θ(λq^{2k_{i}}; p) (λ, λb/a, λc/a, λd/a, e, f, λaq^{2−r+m}/ef, q^{−m}; q, p)_{k_{i}}}{θ(λ; p) (q, aq/b, aq/c, aq/d, λq/e, λq/f, efq^{r−1−m}/a, λq^{1+m}; q, p)_{k_{i}}} , (3.1)

where λ = a^{2}q^{2−r}/bcd.

We now show that, by suitable specialisation, the above elliptic hypergeometric transformation formula reduces to the following transformation formula for multi-dimensional basic hypergeometric series of different dimensions.

Corollary 3. For all non-negative integers m, r and s, we have

∑_{0≤k_{1}<k_{2}<···<k_{r}≤m} q^{∑_{i=1}^{r}(2i−1)k_{i}} ∏_{1≤i<j≤r}(1 − q^{k_{i}−k_{j}})^{2} ∏_{i=1}^{r} \frac{(dq^{k_{i}}; q)_{s} (b; q)_{k_{i}} (q^{−m}; q)_{k_{i}}}{(q; q)_{k_{i}} (f; q)_{k_{i}}}

= \frac{q^{\binom{r+s}{3}+\binom{r+1}{3}+s\binom{r}{2}−m\binom{r+s}{2}}}{f^{\binom{r}{2}} (q; q)_{r+s−1}^{s−1}} ∏_{i=1}^{r} \frac{(b; q)_{i−1} (bq^{s+r+i−m−1}/f; q)_{m−r+1}}{(q^{i−m}/f; q)_{m−i+1}}

× ∏_{i=1}^{r+s−1} \frac{(q; q)_{i−1} (q; q)_{m}}{(q; q)_{m−i}} ∏_{i=r}^{r+s−1} \frac{(dq^{1−r}/b; q)_{i}}{(q; q)_{r+s−i−1} (d; q)_{i−r} (fq^{1−r−s}/b; q)_{i}}

× ∑_{0≤ℓ_{1}<ℓ_{2}<···<ℓ_{s}≤r+s−1} q^{∑_{i=1}^{s}(2i−1)ℓ_{i}} ∏_{1≤i<j≤s}(1 − q^{ℓ_{i}−ℓ_{j}})^{2}

· ∏_{i=1}^{s} \frac{(d; q)_{ℓ_{i}} (fq^{1−r−s}/b; q)_{ℓ_{i}} (q^{1−r−s}; q)_{ℓ_{i}}}{(q; q)_{ℓ_{i}} (dq^{1−r}/b; q)_{ℓ_{i}} (q^{−m}; q)_{ℓ_{i}}} .

(3.2)

Proof. In (3.1), we let p = 0, d → aq/d, f → aq/f, and then a → 0. After having performed these substitutions and limits, we arrive at

∑_{0≤k_{1}<k_{2}<···<k_{r}≤m} q^{∑_{i=1}^{r}(2i−1)k_{i}} ∏_{1≤i<j≤r}(1 − q^{k_{i}−k_{j}})^{2}

· ∏_{i=1}^{r} \frac{(b; q)_{k_{i}} (c; q)_{k_{i}} (e; q)_{k_{i}} (q^{−m}; q)_{k_{i}}}{(q; q)_{k_{i}} (d; q)_{k_{i}} (f; q)_{k_{i}} (bceq^{2r−m−1}/df)_{k_{i}}}

= ∏_{i=1}^{r} \frac{(b; q)_{i−1} (c; q)_{i−1} (eq/f; q)_{i−1}}{(dq^{1−r}/c; q)_{i−1} (dq^{1−r}/b; q)_{i−1} (bceq^{r}/df; q)_{i−1}}

× ∏_{i=1}^{r} \frac{(f/e; q)_{m+1−r} (dfq^{1−r}/bc; q)_{m−i+1}}{(dfq^{1−r}/bce; q)_{m+1−r} (f; q)_{m−i+1}}

× ∑_{0≤k_{1}<k_{2}<···<k_{r}≤m} q^{∑_{i=1}^{r}(2i−1)k_{i}} ∏_{1≤i<j≤r}(1 − q^{k_{i}−k_{j}})^{2}

· ∏_{i=1}^{r} \frac{(dq^{1−r}/c; q)_{k_{i}} (dq^{1−r}/b; q)_{k_{i}} (e; q)_{k_{i}} (q^{−m}; q)_{k_{i}}}{(q; q)_{k_{i}} (d; q)_{k_{i}} (dfq^{1−r}/bc)_{k_{i}} (eq^{r−m}/f)_{k_{i}}} . (3.3)

In this multi-dimensional Whipple transformation we let e → ∞. This yields the identity

∑_{0≤k_{1}<k_{2}<···<k_{r}≤m} q^{∑_{i=1}^{r}(m−2r+2i)k_{i}} (\frac{df}{bc} )^{∑_{i=1}^{r} k_{i}} ∏_{1≤i<j≤r}(1 − q^{k_{i}−k_{j}})^{2}

· ∏_{i=1}^{r} \frac{(b; q)_{k_{i}} (c; q)_{k_{i}} (q^{−m}; q)_{k_{i}}}{(q; q)_{k_{i}} (d; q)_{k_{i}} (f; q)_{k_{i}}}

= (\frac{dq^{1−r}}{bc} )^{\binom{r}{2}} ∏_{i=1}^{r} \frac{(b; q)_{i−1} (c; q)_{i−1}}{(dq^{1−r}/c; q)_{i−1} (dq^{1−r}/b; q)_{i−1}} · \frac{(dfq^{1−r}/bc; q)_{m−i+1}}{(f; q)_{m−i+1}}

× ∑_{0≤k_{1}<k_{2}<···<k_{r}≤m} q^{∑_{i=1}^{r}(m−r+2i−1)k_{i}}f^{∑_{i=1}^{r} k_{i}} ∏_{1≤i<j≤r}(1 − q^{k_{i}−k_{j}})^{2}

· ∏_{i=1}^{r} \frac{(dq^{1−r}/c; q)_{k_{i}} (dq^{1−r}/b; q)_{k_{i}} (q^{−m}; q)_{k_{i}}}{(q; q)_{k_{i}} (d; q)_{k_{i}} (dfq^{1−r}/bc; q)_{k_{i}}} .

This identity becomes more elegant if we replace q by 1/q, b by 1/b, c by 1/c, d by 1/d, and f by 1/f, namely

∑_{0≤k_{1}<k_{2}<···<k_{r}≤m} q^{∑_{i=1}^{r}(2i−1)k_{i}} ∏_{1≤i<j≤r}(1 − q^{k_{i}−k_{j}})^{2} ∏_{i=1}^{r} \frac{(b; q)_{k_{i}} (c; q)_{k_{i}} (q^{−m}; q)_{k_{i}}}{(q; q)_{k_{i}} (d; q)_{k_{i}} (f; q)_{k_{i}}}

= (\frac{bc}{dq^{1−r}} )^{r(m−r+1)} ∏_{i=1}^{r} \frac{(b; q)_{i−1} (c; q)_{i−1}}{(dq^{1−r}/c; q)_{i−1} (dq^{1−r}/b; q)_{i−1}} · \frac{(dfq^{1−r}/bc; q)_{m−i+1}}{(f; q)_{m−i+1}}

× ∑_{0≤k_{1}<k_{2}<···<k_{r}≤m} q^{∑_{i=1}^{r}(2i−1)k_{i}} ∏_{1≤i<j≤r}(1 − q^{k_{i}−k_{j}})^{2}

· ∏_{i=1}^{r} \frac{(dq^{1−r}/c; q)_{k_{i}} (dq^{1−r}/b; q)_{k_{i}} (q^{−m}; q)_{k_{i}}}{(q; q)_{k_{i}} (d; q)_{k_{i}} (dfq^{1−r}/bc; q)_{k_{i}}} . (3.4)

Now we let c = dq^{s} in this transformation formula, where s is a non-negative integer. Due to the term (q^{−r−s+1}; q)_{k_{i}}, on the right-hand side we are now summing over all k_{1}, k_{2}, …, k_{r} with 0 ≤ k_{1} < k_{2} < ⋯ < k_{r} ≤ r + s − 1. Let

{ℓ_{1}, ℓ_{2}, …, ℓ_{s}} = {0, 1, …, r + s − 1}\{k_{1}, k_{2}, …, k_{r}},

with ℓ_{1} < ℓ_{2} < ⋯ < ℓ_{s}. After some simplification, we obtain (3.2). □

Remarks. (1) The multiple Whipple transformation (3.3) also follows from letting t = q^{2} in [2, Eq. (5.10)], followed by the principal specialisation formula for the Macdonald polynomial P_{λ}(q, t).

(2) The transformation formula (3.2) is one of the rare examples of a transformation formula between multi-dimensional (basic) hypergeometric series of different dimension. Other examples that we are aware of are [6, Sec. 8], [7], [8], [9, Thm. 2.2], [12, Conjecture in Sec. 1], [14, Thm. 3.1], and [21, Thm. 4.1].

For our purpose, we shall have need of a more flexible statement of the transformation formula in Corollary 3. More precisely, in our application in the proof of Theorem 1, we have a pair (q^{M}; q^{−2})_{k_{i}} (q^{M+1}; q^{−2})_{k_{i}} of Pochhammer symbols in the numerator of our multi-dimensional sum (see (2.3)), and consequently we would like a statement of (3.2) in which the roles of (b; q)_{k_{i}} and (q^{−m}; q)_{k_{i}} are interchangeable. Indeed, by a standard polynomial argument (see e.g. [1, Sec. 5.1] or [17, Sec. 2.3.4]), one is able to show the following “symmetric” form of Corollary 3.

Corollary 4. Let one of b and c be of the form q^{−m}, where m is a non-negative integer. Then we have

∑_{0≤k_{1}<k_{2}<···<k_{r}} q^{∑_{i=1}^{r}(2i−1)k_{i}} ∏_{1≤i<j≤r}(1 − q^{k_{i}−k_{j}})^{2} ∏_{i=1}^{r} \frac{(dq^{k_{i}}; q)_{s} (b; q)_{k_{i}} (c; q)_{k_{i}}}{(q; q)_{k_{i}} (f; q)_{k_{i}}}

= (−1)^{\binom{r+s}{2}} \frac{q^{\binom{r+1}{3}+s\binom{r}{2}}}{f^{\binom{r}{2}} (q; q)_{r+s−1}^{s−1}} ∏_{i=1}^{r} \frac{(b; q)_{i−1} (q/f; q)_{∞} (bcq^{s+r+i−1}/f; q)_{∞}}{(cq^{i}/f; q)_{∞} (bq^{s+i}/f; q)_{∞}}

× ∏_{i=1}^{r+s−1} (q; q)_{i−1} (c; q)_{i} ∏_{i=r}^{r+s−1} \frac{(dq^{1−r}/b; q)_{i}}{(q; q)_{r+s−i−1} (d; q)_{i−r} (fq^{1−r−s}/b; q)_{i}}

× ∑_{0≤ℓ_{1}<ℓ_{2}<···<ℓ_{s}≤r+s−1} q^{∑_{i=1}^{s}(2i−1)ℓ_{i}} ∏_{1≤i<j≤s}(1 − q^{ℓ_{i}−ℓ_{j}})^{2}

· ∏_{i=1}^{s} \frac{(d; q)_{ℓ_{i}} (fq^{1−r−s}/b; q)_{ℓ_{i}} (q^{1−r−s}; q)_{ℓ_{i}}}{(q; q)_{ℓ_{i}} (dq^{1−r}/b; q)_{ℓ_{i}} (c; q)_{ℓ_{i}}} . (3.5)

Sketch of Proof. The assertion is obvious if c = q^{−m}, where m is a non-negative integer, because in that case the identity in (3.5) is directly equivalent to (3.2).

On the other hand, let us now suppose that b is of the form b = q^{−β}, for some nonnegative integer β. On the right-hand side of (3.2), we write

\frac{(q; q)_{m}}{(q; q)_{m−i}} = (q^{m−i+1}; q)_{i} = (−1)^{i}q^{im−\binom{i}{2}}(q^{−m}; q)_{i}

and

\frac{(bq^{s+r+i−m−1}/f; q)_{m−r+1}}{(q^{i−m}/f; q)_{m−i+1}} = \frac{(bq^{s+r+i−m−1}/f; q)_{∞} (q/f; q)_{∞}}{(bq^{s+i}/f; q)_{∞} (q^{i−m}/f; q)_{∞}} .

Since we are currently assuming that b = q^{−β} for some non-negative integer β, the above relation can be rewritten in the form

\frac{(bq^{s+r+i−m−1}/f; q)_{m−r+1}}{(q^{i−m}/f; q)_{m−i+1}} = \frac{(q/f)_{s+i−β−1}}{(q^{i−m}/f; q)_{s+r−β−1}} .  (3.6)

Consequently, the expression in (3.6) is rational in c = q^{−m}, and therefore as well the expressions on both sides of (3.5). Under a fixed choice of the parameters r, s, b = q^{−β}, one also sees that the degrees in c of numerator and denominator of these rational functions are bounded. Now comes the “polynomial argument”: Identity (3.5) holds for infinitely many c’s (namely for all c’s of the form c = q^{−m}, where m is a non-negative integer), hence it must hold for arbitrary c, that is, for c considered as an indeterminate. This finishes the proof of this corollary. □

For the convenience of the reader, we state the special cases of (3.5) where s = 0 and s = 1 explicitly below. Namely, if we let s = 0, then (3.5) reduces to

∑_{0≤k_{1}<k_{2}<···<k_{r}≤m} q^{∑_{i=1}^{r}(2i−1)k_{i}} ∏_{1≤i<j≤r}(1 − q^{k_{i}−k_{j}})^{2} ∏_{i=1}^{r} \frac{(b; q)_{k_{i}} (q^{−m}; q)_{k_{i}}}{(q; q)_{k_{i}} (f; q)_{k_{i}}}

= (−1)^{\binom{r}{2}}q^{\binom{r}{2}(m−r+1)}b^{r(m−r+1)} ∏_{i=1}^{r} \frac{(b; q)_{i−1} (q; q)_{m} (fq^{1−i}/b; q)_{m−r+1} (q; q)_{i−1}}{(f; q)_{m−i+1} (q; q)_{m−i+1}} , (3.7)

where m is a non-negative integer, while for s = 1 Identity (3.5) reduces to

∑_{0≤k_{1}<k_{2}<···<k_{r}} q^{∑_{i=1}^{r}(2i−1)k_{i}} ∏_{1≤i<j≤r}(1 − q^{k_{i}−k_{j}})^{2} ∏_{i=1}^{r} \frac{(1 − dq^{k_{i}}) (b; q)_{k_{i}} (c; q)_{k_{i}}}{(q; q)_{k_{i}} (f; q)_{k_{i}}}

= (−1)^{\binom{r+1}{2}} \frac{q^{\binom{r+1}{3}+\binom{r}{2}}}{f^{\binom{r}{2}}} ∏_{i=1}^{r} \frac{(q; q)_{i−1} (b; q)_{i−1} (c; q)_{i} (q/f; q)_{∞} (bcq^{r+i}/f; q)_{∞}}{(cq^{i}/f; q)_{∞} (bq^{i+1}/f; q)_{∞}}

× ∑_{ℓ=0}^{r} q^{ℓ} \frac{(d; q)_{ℓ} (dq^{1−r+ℓ}/b; q)_{r−ℓ} (q^{−r}; q)_{ℓ}}{(q; q)_{ℓ} (fq^{−r+ℓ}/b; q)_{r−ℓ} (c; q)_{ℓ}} , (3.8)

where one of b or c is of the form q^{−m}, where m is a non-negative integer. We point out that (3.7) has been explicitly stated earlier in [11, Theorem 6], where two different proofs were provided: one proceeded by showing that this formula comes from the principal specialisation of the obvious expansion of a rectangular Schur function in two sets of variables, while the other derived it by specialising a multi-dimensional q-beta integral formula due to Evans and Kadell.

References

1. W. N. Bailey, Generalized hypergeometric series, Cambridge University Press, Cambridge, 1935.

2. T. H. Baker and P. J. Forrester, Transformation formulas for multivariable basic hypergeometric series, Methods Appl. Anal. 6 (1999), 147–164.

3. H. Coskun and R. A. Gustafson, Well-poised Macdonald functions W_{λ} and Jackson coefficients ω_{λ} on BC_{n}, Proceedings of the workshop on Jack, Hall–Littlewood and Macdonald polynomials (V. B. Kuznetsov and S. Sahi, eds.), Contemp. Math., vol. 417, Amer. Math. Soc., Providence, RI, 2006, pp. 127–155.

4. E. DeWitt, Identities relating Schur s-functions and Q-functions, Ph.D. thesis, University of Michigan,

   Ann Arbor, 2012, available at http://deepblue.lib.umich.edu/handle/2027.42/93841.

5. J. S. Frame, G. B. Robinson and R. M. Thrall, The hook graphs of the symmetric group, Canad. J. Math. 6 (1954), 316–325.

6. I. M. Gessel and C. Krattenthaler, Cylindric partitions, Trans. Amer. Math. Soc. 349 (1997), 429–479.

7. Y. Kajihara, Some remarks on multiple Sears transformations, Contemp. Math. 291 (2001), 139–145.

8. Y. Kajihara, Euler transform formula for multiple basic hypergeometric series of type A and some applications, Adv. Math. 187 (2004), 53–97.

9. Y. Kajihara and M. Noumi, Multiple elliptic hypergeometric series. An approach from the Cauchy determinant, Indag. Math. (N.S.) 14 (2003), 395–421.

10. N. Kawanaka, A q-series identity involving Schur functions and related topics, Osaka J. Math. 36 (1999), 157–176.

11. C. Krattenthaler, Schur function identities and the number of perfect matchings of holey Aztec rectangles, Contemporary Math. 254 (2000), 335–350.

12. C. Krattenthaler, Proof of a summation formula for an Ã_{n} basic hypergeometric series conjectured by Warnaar, “q-Series with Applications to Combinatorics, Number Theory, and Physics,” Urbana– Champaign, Oct. 26–28, 2000 (B. C. Berndt, K. Ono, eds.), Contemporary Math., vol. 291, Amer. Math. Soc., Providence, R.I., 2001, pp. 153–161.

13. E. Rains, BC_{n}-symmetric abelian functions, Duke Math. J. 135 (2006), 99–180.

14. H. Rosengren, New transformations for elliptic hypergeometric series on the root system A_{n}, Ramanujan J. 12 (2006), 155–166.

15. H. Rosengren, Schur Q-polynomials, multiple hypergeometric series and enumeration of marked shifted tableaux, J. Combin. Theory Ser. A 115 (2008), 376–406.

16. M. J. Schlosser, Elliptic enumeration of nonintersecting lattice paths, J. Combin. Theory Ser. A 114 (2007), 505–521.

17. L. J. Slater, Generalized hypergeometric functions, Cambridge University Press, Cambridge, 1966.

18. R. P. Stanley, Enumerative Combinatorics, vol. 1, Wadsworth & Brooks/Cole, Pacific Grove, California, 1986, reprinted by Cambridge University Press, Cambridge, 1998.

19. R. P. Stanley, Enumerative Combinatorics, vol. 2, Cambridge University Press, Cambridge, 1999.

20. S. O. Warnaar, Summation and transformation formulas for elliptic hypergeometric series, Constr. Approx. 18 (2002), 479–502.

21. S. O. Warnaar, The sl_{3} Selberg integral, Adv. Math. 224 (2010), 499–524.

Fakultät für Mathematik, Universität Wien, Oskar-Morgenstern-Platz 1, A-1090 Vienna, Austria.

WWW: http://www.mat.univie.ac.at/~kratt, http://www.mat.univie.ac.at/~schlosse