Cylindric Partitions

Ira M. Gessel^{1} C. Krattenthaler^{2}

Department of Mathematics, Brandeis University, Waltham, MA 02254–9110, USA e-mail: IRA@cs.brandeis.edu

Institut für Mathematik der Universität Wien, Strudlhofgasse 4, A-1090 Wien, Austria. e-mail: KRATT@Pap.Univie.Ac.At

Abstract. A new object is introduced into the theory of partitions that generalizes plane partitions: cylindric partitions. We obtain the generating function for cylindric partitions of a given shape that satisfy certain row bounds as a sum of determinants of q-binomial coefficients. In some special cases these determinants can be evaluated. Extending an idea of Burge (J. Combin. Theory Ser. A 63 (1993), 210–222), we count cylindric partitions in two different ways to obtain several known and new summation and transformation formulas for basic hypergeometric series for the affine root system Ã_{r}. In particular, we provide new and elementary proofs for two Ã_{r} basic hypergeometric summation formulas of Milne (Discrete Math. 99 (1992), 199–246).

1. Introduction. In this paper we introduce a new object into the theory of partitions: cylindric partitions. The most basic objects in the theory of partitions are linear partitions [4]. MacMahon [34, art. 421] extended linear partitions to plane partitions (cf. [4, Sec. 11.2; 34, Sec. IX, X; 17; 18; 24; 40]). Cylindric partitions are the next link in this chain. Before proceeding to the definition of cylindric partitions, we recall the definition of linear partitions and plane partitions.

A linear partition is a linear array π = (π_{1}, π_{2}, …, π_{c}) of integers π_{i} such that π_{1} ≥ π_{2} ≥ ⋯ ≥ π_{c}. Linear partitions are usually called simply partitions.

1991 Mathematics Subject Classification. Primary 05A15; Secondary 05A17, 05A30, 05E05, 11P81, 33D20, 33D45.

Key words and phrases. cylindric partitions, plane partitions, Ã_{r} basic hypergeometric series, symmetric functions, Schur functions.

1Supported in part by NSF grant DMS-9306297

2Supported in part by EC’s Human Capital and Mobility Program, grant CHRX-CT93-0400 and the Austrian Science Foundation FWF, grant P10191-MAT

Let λ = (λ_{1}, λ_{2}, …, λ_{r}) and μ = (μ_{1}, μ_{2}, …, μ_{r}) be r-tuples of integers. A plane partition of shape λ/μ is a planar array π of integers of the form

\begin{aligned}& π_{1,μ_{1}+1} & . . . . . . . . . . . . . . . . & π_{1,λ_{1}} \\ π_{2,μ_{2}+1} … & π_{2,μ_{1}+1} & . . . . . . . . . & π_{2,λ_{2}} \\ ... & ⋮ & ... \\ π_{r,μ_{r}+1} & . . . . . . . . . . . . . . . . . . . . . . . . . . & π_{r,λ_{r}} \end{aligned}  (1.1)

such that the rows and columns are weakly decreasing, i.e.,

π_{i,j} ≥ π_{i,j+1} and π_{i,j} ≥ π_{i+1,j}.  (1.2)

In particular, a linear partition is a one-rowed plane partition. The left-hand array in Figure 1 shows a plane partition of shape (7, 6, 4, 4)/(3, 1, 1, 0).

A cylindric partition is a plane partition with an additional relation between the entries of the first and the last rows. More precisely, let λ and μ be r-tuples of integers, as before, and let d be an integer. A cylindric partition of shape λ/μ/d is a planar array of the form (1.1) such that

π_{i,j} ≥ π_{i,j+1}  (1.3a)
π_{i,j} ≥ π_{i+1,j}  (1.3b)

and

π_{r,j} ≥ π_{1,j+d}.  (1.3c)

Note that (1.3a) and (1.3b) are the same as (1.2), while (1.3c) is the additional relation between the entries of the first and the last row of π. An alternative formulation would be that a cylindric partition is a plane partition that remains a plane partition if the last row, shifted d units to the right, is placed above π. This is illustrated in Figure 1. The left-hand array π_{0} in Figure 1 is a cylindric partition of shape (7, 6, 4, 4)/(3, 1, 1, 0)/4. The right-hand array in Figure 1 shows π_{0} with the last row, shifted by d = 4 units, put on top of it (displayed in italics). This augmented array is a plane partition.

9 7 6 2 11 11 8 7 4 10 8 8 9 8 5 5  9 8 5 5 9 7 6 2 11 11 8 7 4 10 8 8 9 8 5 5 Figure 1

We chose the name cylindric partition for these objects because the entries of such an array π may be viewed as situated on the cylinder obtained by winding up π and gluing together its first and last rows so that the entries π_{r,j} and π_{1,j+d} are adjacent.

As in the theory of plane partitions, we cannot treat completely general shapes. Our results are subject to the restrictions (3.1) on the shape λ/μ/d.

We call the sum ∑ π_{i,j} of all the entries of π the norm of π and denote it by n(π). The norm of the array π_{0} in Figure 1 is n(π_{0}) = 102.

It is well-known [17, 18, 42, 8] that in order to find generating functions for plane partitions it is convenient to convert plane partitions into nonintersecting lattice paths. In section 3 we shall show how cylindric partitions correspond to families of nonintersecting lattice paths subject to the additional condition that a translate of the first path does not intersect the last path, and (in Proposition 1), we derive a generating function for these families of nonintersecting lattice paths. This is the basic result for all that follows. Using this theorem, in section 4 we express the norm generating function ∑ q^{n(π)} for cylindric partitions π of a given shape, in which the entries in each row are restricted by an upper and lower bound, as a sum of determinants whose entries are q-binomial coefficients (Theorem 2). More generally, in Theorem 3 we find the norm generating function for (α, β)-cylindric partitions, which are defined in section 4 as arrays of the form (1.1) such that the entries decrease by at least α_{j} along the rows from column j to column j + 1 and by at least β_{i} along the columns from row i to row i + 1. In particular, (0, 0)-cylindric partitions are the (ordinary) cylindric partitions which we have already defined. (Here 0 denotes the all 0 sequence.)

In some special cases the determinants in Theorems 2 and 3 can be evaluated in closed form. These special cases are described in section 5.

The theorems in sections 4 and 5 contain many results in the literature. First, many of the results about plane partitions (without symmetries) of a given shape (see [17, 18, 24, 34, 40]) are special cases of Theorems 2, 3, 5–7. Next, Burge’s generating function theorem [12] for restricted partition pairs is the special case r = 2, α = 0 of Theorem 3, and the generating function theorem for partitions with prescribed hook differences of Andrews et al. [6] is easily derived from the special case r = 2, α = (1, 1, …, 1) of Theorem 3. Finally, by virtue of Stanley’s theory of (P, ω)-partitions [41], Krattenthaler and Mohanty’s [30] formulas for the major counting of n-dimensional lattice paths in a particular bounded region are corollaries of Theorem 3.

Burge’s paper [12] is a source of inspiration for this paper. The main emphasis in [12] lies on deriving identities by counting “restricted partition pairs” (which are two-rowed (0, β)-cylindric partitions in our language) in two different ways. This idea carries over to (α, β)-cylindric partitions with an arbitrary number of rows. Thus, in sections 6–8 we derive numerous multiple basic hypergeometric series identities. To be more precise, we extend many of Burge’s identities in section 6. In addition, we add many more identities whose specialized forms do not appear in Burge’s paper. The identities of section 6 form the basis for the subsequent sections 7 and 8. There we derive identities for basic hypergeometric series for the affine root system Ã_{r}. Among the identities that we obtain are two identities, (7.5) and (7.11), which were first obtained by Milne [36, Theorems 1.9 and 5.27]. In contrast to Milne’s proof, which used a great deal of machinery (especially for proving (7.5)), our proofs are purely elementary. We also generalize Milne’s identities and derive the Ã_{r} basic hypergeometric summations (7.6) and (7.7), which are new. In section 8 we generalize (7.5) and (7.7) to transformation formulas between Ã_{r} basic hypergeometric series of different dimensions (identities (8.2) and (8.4)), a type of transformation that does not seem to have appeared before. It is our belief that some of our identities (hopefully many) will inspire future research on Ã_{r} basic hypergeometric series and eventually lead to the discovery of more general identities for these series. Of course, we do not claim that our methods for deriving such identities are the “right” ones. They have their restrictions. But they are unusual and show connections between Ã_{r} basic hypergeometric series and combinatorics.

There is a special case in which even more general weights for cylindric partitions can be considered. This special case is discussed in section 9. As a corollary we obtain the new elegant expression (9.6) for the expansion of the monomial symmetric function m_{(r^{c})}(x) in terms of Schur functions (cf. [33] for information about symmetric functions), and the expansion (9.9) of the generating function for partitions with a restricted number of repetitions in terms of Schur functions.

Finally, in section 10 we mention some open problems and areas for further research.

2. Notation. By paths we always mean lattice paths in the plane integer lattice ℤ^{2} consisting of unit horizontal and vertical steps in the positive direction. Given points u and v, we denote the set of all lattice paths from u to v by P(u → v). If u = (u_{1}, …, u_{m}) and v = (v_{1}, …, v_{m}) are vectors of points, we denote the set of all m-tuples (P_{1}, …, P_{m}) of paths in which P_{i} runs from u_{i} to v_{i}, i = 1, …, m, by P(u → v). A set of paths is said to be nonintersecting if no two paths of the set have a point in common. Otherwise it is called intersecting. Let w be a function that to each horizontal edge e in ℤ^{2} assigns a weight w(e). Then the weight w(P) of a path P is defined to be the product of the weights of all its horizontal steps. The weight w(P) of an m-tuple P = (P_{1}, …, P_{m}) of paths is defined to be the product ∏_{i=1}^{m} w(P_{i}) of the weights of all the paths in the m-tuple. For any weight function w defined on a set A, we write

GF(A; w) := ∑_{x∈A} w(x)

for the generating function of the set A with respect to the weight w.

We will use the “q-notations” [α]_{q} = 1 − q^{α}, [n]_{q}! = [1]_{q}[2]_{q} ⋯ [n]_{q}, [0]_{q}! = 1,

(a; q)_{k} = ∏_{j=0}^{k−1}(1 − aq^{j}) , with (a; q)_{0} = 1,

(a; q)_{∞} = ∏_{j=0}^{∞}(1 − aq^{j}) ,

so that in particular [n]_{q}! = (q; q)_{n}, and

[ \begin{matrix}n \\ k \end{matrix} ]_{q} = \{ \begin{matrix}\frac{[n]_{q}[n − 1]_{q} ⋯ [n − k + 1]_{q}}{[k]_{q}!} & k ≥ 0 \\ 0 & k < 0 \end{matrix} .

Occasionally we shall use the Gasper-Rahman condensed notation

(a_{1}, a_{2}, …, a_{r}; q)_{n} := (a_{1}; q)_{n} (a_{2}; q)_{n} ⋯ (a_{r}; q)_{n} .

The base q in [α]_{q}, [n]_{q}!, (a; q)_{k}, (a; q)_{∞}, and [ \begin{matrix}n \\ k \end{matrix} ]_{q} will in most cases be omitted. Only when the base is different from q will it be explicitly stated.

3. Cylindric partitions and nonintersecting lattice paths. As pointed out in the introduction, formulas for plane partition generating functions are most conveniently found by converting plane partitions into nonintersecting lattice paths (cf. [17, 18, 42, 8]). In this section we interpret cylindric plane partitions in terms of nonintersecting paths and count these paths.

Let λ = (λ_{1}, λ_{2}, …, λ_{r}) and μ = (μ_{1}, μ_{2}, …, μ_{r}) be r-tuples of integers and let d be an integer such that

λ_{1} ≥ λ_{2} − 1 ≥ λ_{3} − 2 ≥ ⋯ ≥ λ_{r} − (r − 1) ≥ λ_{1} − d − r  (3.1a)
μ_{1} ≥ μ_{2} − 1 ≥ μ_{3} − 2 ≥ ⋯ ≥ μ_{r} − (r − 1) ≥ μ_{1} − d − r.  (3.1b)

Let a_{1}, a_{2} …, a_{r} and b_{1}, b_{2}, …, b_{r} be integers such that

a_{1} − 1 ≥ a_{2} − 2 ≥ ⋯ ≥ a_{r} − r ≥ a_{1} − r − 1  (3.2a)
b_{1} − 1 ≥ b_{2} − 2 ≥ ⋯ ≥ b_{r} − r ≥ b_{1} − r − 1.  (3.2b)

We claim that cylindric partitions of shape λ/μ/d in which the entries in row i are at most a_{i} and at least b_{i} correspond bijectively to families (P_{1}, P_{2}, …, P_{r}) of nonintersecting lattice paths, in which P_{i} runs from (−λ_{i} +i, b_{i} −i) to (−μ_{i} +i, a_{i} −i), and in addition P_{r}, when shifted by (−r − d, r), does not intersect P_{1}. In certain cases it is necessary to add some additional conditions to the cylindric partition: if λ_{i} = λ_{i+1} − 1 for some i with 1 ≤ i ≤ r − 1, the restriction (1.3b) is also assumed to hold for j = λ_{i} + 1 = λ_{i+1} with π_{i,λ_{i}+1} := b_{i}, and if λ_{r} = λ_{1} − d − 1 the restriction (1.3c) is also assumed to hold for j = λ_{r} + 1 = λ_{1} − d with π_{r,λ_{r}+1} := b_{r}. Similarly, if μ_{i} = μ_{i+1} − 1 for some i with 1 ≤ i ≤ r − 1 then (1.3b) is also assumed to hold for j = μ_{i} + 1 = μ_{i+1} with π_{i+1,μ_{i+1}} := a_{i+1}, and if μ_{r} = μ_{1} − d − 1, the restriction (1.3c) is also assumed to hold for j = μ_{r} + 1 = μ_{1} − d with π_{1,μ_{1}} := a_{1}.

The correspondence between cylindric partitions and paths is as follows. Let π be a cylindric partition of shape λ/μ/d in which the entries in row i are at most a_{i} and at least b_{i}. First we subtract i from the entries of the ith row of π, i = 1, 2, …, r, obtaining the new array π̄. For example, consider the cylindric partition π_{0} in Figure 1. For sake of simplicity let us choose a_{i} = 11 and b_{i} = 2, i = 1, 2, 3, 4, as row bounds. The cylindric partition π_{0} with these row bounds is displayed in the first picture in Figure 2. (Again the italicized numbers indicate the shifted last row.) The second picture of Figure 2 shows the result of subtracting i from the entries of the ith row of π_{0}.

By the operation described above, we obtain from the cylindric partition π the array π̄ that is weakly decreasing along the rows but strictly decreasing along the columns. The relation (1.3c) between the entries of the first and the last row of π translates into the condition

π̄_{r,j} + r > π̄_{1,j+d}.  (3.3)

Let us call such an array a column-strict cylindric partition with first-last relation (3.3). Relation (3.3) is the same as saying that taking the last row of π̄, adding r to each of its entries, shifting it d positions to the right, and putting it on top of π̄, yields an array that still has strictly decreasing columns. Of course, the new row bounds on π̄ are a_{i} − i and b_{i} − i. The italicized numbers on top of the array in the second picture of Figure 2 indicate the last row with r = 4 added to each entry and shifted by d = 4 units.

\begin{aligned}11 ≥ & & & & & 9 & 8 & 5 & 5 & ≥ 2 \\ 11 ≥ & & & & 9 & 7 & 6 & 2 & & ≥ 2 \\ 11 ≥ & & 11 & 11 & 8 & 7 & 4 & & & ≥ 2 \\ 11 ≥ & & 10 & 8 & 8 & & & & & ≥ 2 \\ 11 ≥ & 9 & 8 & 5 & 5 & & & & & ≥ 2 \end{aligned}  \begin{aligned}11 ≥ & & & & & 9 & 8 & 5 & 5 & ≥ 2 \\ 10 ≥ & & & & 8 & 6 & 5 & 1 & & ≥ 1 \\ 9 ≥ & & 9 & 9 & 6 & 5 & 2 & & & ≥ 0 \\ 8 ≥ & & 7 & 5 & 5 & & & & & ≥ −1 \\ 7 ≥ & 5 & 4 & 1 & 1 & & & & & ≥ −2 \end{aligned}

1. cylindric partition π_{0} with row bounds 2. column-strict cylindric partition π̄_{0}

with row bounds

• _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•}_{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•}_{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} . . . . .. . . . .. . . . .^{.....}^{.....}. . . . .. . . . .. . . . .^{.....}. . . . .^{.....}. . . . .. . . . .  • • • • •  • • • • • P_{1} P_{2}  P_{3}  P_{4} 1 1 4 5 5 5 7 2 5 6 9 9 1 5 6 8 5 5 8 9

3. Family of nonintersecting lattice paths with nonintersecting translate

Figure 2

Next, in the usual way, the column-strict cylindric partition π̄ is mapped to a family of lattice paths. We recall that this is done by associating the ith row of π̄ with a path P_{i} from (−λ_{i} + i, b_{i} − i) to (−μ_{i} + i, a_{i} − i) where the entries in the ith row are interpreted as heights of the horizontal steps in the path P_{i}. Thus from π̄ we obtain the family P = (P_{1}, …, P_{r}) of lattice paths. The third picture of Figure 2 (without the dotted path) displays the family of lattice paths that results in this way from the array π̄_{0} displayed in the second picture of Figure 2.

Clearly, the property that the columns of π̄ are strictly decreasing translates into the condition that (P_{1}, P_{2}, …, P_{r}) is nonintersecting. The relation (3.3) translates into the condition that P_{r}, when shifted by (−r − d, r), does not intersect P_{1}. The dotted path in the third picture of Figure 2 indicates this translate of the path P_{4}.

It is clear that this correspondence between cylindric partitions of a given shape with given row bounds and families of nonintersecting lattice paths with given starting and ending points, in which a translate of the last path does not intersect the first path, is a bijection. Thus, to enumerate cylindric partitions, it is sufficient to enumerate the corresponding paths. This is done in the next proposition, which is the basic result of this paper.

In Proposition 1 we use the notation introduced in section 2 for lattice paths and for the extension of a function on horizontal edges of the lattice ℤ^{2} to lattice paths and families of lattice paths.

Proposition 1. Let S be the shift by (−α^{x}, α^{y}) where α^{x} and α^{y} are positive integers. (In particular, S is neither a horizontal nor a vertical shift.) Let w be a function on the horizontal edges of ℤ^{2} with the property that for all horizontal edges e and some fixed indeterminate z

w(Se) = z · w(e).  (3.4)

Let u = (u_{1}, u_{2}, …, u_{r}) and v = (v_{1}, v_{2}, …, v_{r}) be r-tuples of points in ℤ^{2} such that u_{i} lies to the northwest of u_{i+1} and v_{i} lies to the northwest of v_{i+1} for i = 1, 2, …, r−1; Su_{r} lies to the northwest of u_{1} and Sv_{r} lies to the northwest of v_{1}; and u_{i} lies to the southwest of v_{i} for i = 1, 2, …, r. Then the generating function ∑ w(P), where the sum is over all families P = (P_{1}, P_{2}, …, P_{r}) of lattice paths in which P_{i} runs from u_{i} to v_{i} and (SP_{r}, P_{1}, …, P_{r}) is nonintersecting, equals

∑_{k_{1}+···+k_{r}=0} det_{1≤s,t≤r} (z^{α^{x}k_{s}^{2}/2−v_{t}^{x}k_{s}} GF(P(u_{s} → S^{k_{s}}v_{t}); w)),  (3.5)

with v_{t}^{x} denoting the x-coordinate of v_{t}.

Proof. The basic idea for the proof of this Proposition is the same as in the proof that the Gessel-Viennot determinant [18, Cor. 2; 42, Theorem 1.2] is the generating function for “ordinary” nonintersecting lattice paths. However, the choice of the “minimal” intersection point must be made more carefully.

In the case of “ordinary” nonintersecting lattice paths one considers permutations of the end points. Since in our situation a shift is involved, we must combine shifts with permutations of the end points. More precisely, we consider families (P_{1}, P_{2}, …, P_{r}) of lattice paths in which P_{i} runs from u_{i} to S^{k_{i}}v_{σ(i)}, where σ is a permutation in 𝔖_{r}, the symmetric group of order r, and k_{1}, k_{2}, …, k_{r} are integers with k_{1} + k_{2} + ⋯ + k_{r} = 0. Let us write k for (k_{1}, k_{2}, …, k_{r}), v_{σ} for (v_{σ(1)}, v_{σ(2)}, …, v_{σ(r)}) and S^{k}v for (S^{k_{1}}v_{1}, S^{k_{2}}v_{2}, …, S^{k_{r}}v_{r}). Then we are considering families P_{1}, …, P_{r} of lattice paths in the disjoint union

⋃_{σ∈𝔖_{r}} ; k_{1}+···+k_{r}=0P(u → S^{k}v_{σ}).

If P = (P_{1}, …, P_{r}) is an element of P(u → S^{k}v_{σ}) we define a weight w̄(P) by

w̄(P) = ∏_{s=1}^{r} z^{α^{x}k_{s}^{2}/2−v_{σ(s)}^{x}k_{s}}w(P_{s}).  (3.6)

Let us extend our lattice path notation by writing P(u → v)^{+} for the set of all families (P_{1}, …, P_{r}) in P(u → v) for which (SP_{r}, P_{1}, …, P_{r}) is nonintersecting and P(u → v)^{−} for the set of those families in P(u → v) for which (SP_{r}, P_{1}, …, P_{r}) is intersecting. We are interested in GF(P(u → v)^{+}; w). Clearly we have

GF(P(u → v)^{+}; w) = GF(P(u → v)^{+}; w̄)
= GF(P(u → v); w̄) − GF(P(u → v)^{−}; w̄),  (3.7)

since w and w̄ agree on P(u → S^{k}v_{σ}) when k = 0.

We shall construct an involution φ on

⋃_{σ∈𝔖_{r}} ; k_{1}+···+k_{r}=0P(u → S^{k}v_{σ})^{−}  (3.8)

that is sign-reversing and weight-preserving with respect to w̄. To be more precise, if P is in P(u → S^{k}v_{σ})^{−} and φ(P) is in P(u → S^{k′}v_{σ′})^{−} then we shall show that sgn σ′ = − sgn σ and

w̄(φ(P)) = w̄(P).  (3.9)

This will imply that

∑_{σ∈𝔖_{r}} ; k_{1}+···+k_{r}=0 sgn σ GF(P(u → S^{k}v_{σ})^{−}; w̄) = 0.  (3.10)

Now we observe that

P(u → S^{k}v_{σ})^{−} = P(u → S^{k}v_{σ}) if σ ≠ id or k ≠ 0.  (3.11)

This follows from the conditions on the points u_{1}, …, u_{r} and v_{1}, …, v_{r}. Namely, suppose that there is a family (P_{1}, …, P_{r}) in P(u → S^{k}v_{σ}) such that (SP_{r}, P_{1}, …, P_{r}) is nonintersecting. By assumption, Su_{r}, u_{1}, …, u_{r} is a sequence of points, each lying to the northwest of its successor. Similarly,

…, S^{2}v_{r}, Sv_{1}, …, Sv_{r}, v_{1}, …, v_{r}, S^{−1}v_{1}, …, S^{−1}v_{r}, S^{−2}v_{1}, …

is an infinite sequence with the same property. If (SP_{r}, P_{1}, …, P_{r}) is nonintersecting, where P_{i} runs from u_{i} to S^{k_{i}}v_{σ(i)}, i = 1, 2, …, r, and therefore SP_{r} runs from Su_{r} to S^{k_{r}+1}v_{σ(r)}, then the sequence

S^{k_{r}+1}v_{σ(r)}, S^{k_{1}}v_{σ(1)}, …, S^{k_{r}}v_{σ(r)}

must also be a sequence with this property. In particular, S^{k_{r}+1}v_{σ(r)} lies to the northwest of S^{k_{i}}v_{σ(i)}, which in turn lies to the northwest of S^{k_{r}}v_{σ(r)}. If σ(i) < σ(r), we conclude that k_{i} = k_{r}, and if σ(i) > σ(r), we conclude that k_{i} = k_{r} + 1. From ∑_{i=1}^{r} k_{i} = 0, it follows that −rk_{r} = |{i : σ(i) > σ(r)}|. Since k_{r} must be an integer, k_{r} is zero, and hence k_{i} = 0 for all i. Therefore (P_{1}, …, P_{r}) is an element of P(u → v_{σ}). If σ were not the identity, then there would exist i and j, with i < j, such that σ(i) > σ(j). But then v_{σ(i)} would lie to the southeast of v_{σ(j)} and therefore P_{i} and P_{j} would intersect. Thus (3.11) is established.

By consecutive use of (3.7), (3.10), (3.11), and (3.6) we obtain

GF(P(u → v)^{+}; w) = GF(P(u → v); w̄) − GF(P(u → v)^{−}; w̄)

= GF(P(u → v); w̄) +_{∑_{σ≠id} ; k_{1}+···+k_{r}=0} sgn σ GF(P(u → S^{k}v_{σ})^{−}; w̄)

= _{∑_{σ∈𝔖_{r}} ; k_{1}+···+k_{r}=0} sgn σ GF(P(u → S^{k}v_{σ}); w̄)

= _{∑_{σ∈𝔖_{r}} ; k_{1}+···+k_{r}=0} sgn σ ∏_{s=1}^{r} z^{α^{x}k_{s}^{2}/2−v_{σ(s)}^{x}k_{s}} GF(P(u_{s} → S^{k_{s}}v_{σ(s)}); w)

= ∑_{k_{1}+···+k_{r}=0} det_{1≤s,t≤r} (z^{α^{x}k_{s}^{2}/2−v_{t}^{x}k_{s}} GF(P(u_{s} → S^{k_{s}}v_{t}); w)).

Thus the Proposition will be proved once we construct the sign-reversing, w̄-weight-preserving involution φ on the set (3.8).

To construct φ, let P = (P_{1}, …, P_{r}) be an element of P(u → S^{k}v_{σ}) such that (SP_{r}, P_{1}, …, P_{r}) is intersecting. Among all intersection points of (SP_{r}, P_{1}, …, P_{r}) we choose the one that is minimal with respect to a certain order which is induced by the shift S = (−α^{x}, α^{y}). More precisely, among all intersection points (x, y) of (SP_{r}, P_{1}, …, P_{r}) we search for those for which α^{y}x + α^{x}y is minimal, and among all these we choose the right-most. Denote this point by p = (p^{x}, p^{y}). Figure 3 gives a sketch of the typical situation.

[figure: •; p; .....; _{.....}; _{.....}; _{.....}; _{.....}; •; S = (−α^{x}, α^{y}); the line; α^{y}x + α^{x}y = α^{y}p^{x} + α^{x}p^{y}; other intersection points]

Figure 3

Now we have two cases. Either p is the intersection point of two paths P_{i} and P_{j}, with i < j, or it is the intersection point of the translate SP_{r} and some path P_{i}. In the first case we exchange the terminal portions of P_{i} and P_{j} beginning from p, i.e., we replace P_{i} by P′_{i} and P_{j} by P′_{j} where

P′_{i} = [subpath of P_{i} from u_{i} to p joined with subpath of P_{j} from p to its end point]
P′_{j} = [subpath of P_{j} from u_{j} to p joined with subpath of P_{i} from p to its end point]. (3.12)

Thus P = (P_{1}, …, P_{i}, …, P_{j}, …, P_{r}) is mapped to

φ(P) = (P_{1}, …, P′_{i}, …, P′_{j}, …, P_{r}).  (3.13)

Clearly φ(P) is an element of P(u → S^{k^{(i,j)}}v_{σ∘(ij)}) where k^{(i,j)} = (k_{1}, …, k_{j}, …, k_{i}, …, k_{r}) and (i j) denotes the transposition that interchanges i and j.

In the second case, we show that p is actually an intersection point of SP_{r} and P_{1}. For suppose that p is an intersection of SP_{r} and P_{i}, i ≥ 2. Since Su_{r}, the starting point of SP_{r}, lies to the northwest of u_{1}, the starting point of P_{1}, Su_{r} also lies to the northwest of u_{i}, the starting point of P_{i}. Hence P_{1} must intersect either SP_{r} or P_{i} and this intersection “comes before p”, i.e., lies to the southwest of p, contradicting the fact that p was chosen as the left-most and lowest of all intersection points. Now, again, we exchange terminal portions of paths. However, the definition of φ in this case is more delicate, since we are considering a translate of a path. We replace P_{1} by P′_{1} and P_{r} by P′_{r} where

P′_{1} = [subpath of P_{1} from u_{1} to p joined with subpath of SP_{r} from p

to its end point]

P′_{r} = [subpath of P_{r} from u_{r} to S^{−1}p joined with subpath of S^{−1}P_{1} from S^{−1}p
to its end point].  (3.14)

So, what happens is that we shift P_{r} to SP_{r}, exchange terminal portions of P_{1} and SP_{r}, thus obtaining P′_{1} and P′′_{r}, and then shift back P′′_{r} to obtain P′_{r} = S^{−1}P′′_{r} (see Figure 4).

. . . . .^{.....}^{.....}^{.....}^{.....}^{.....}. . . . .. . . . .. . . . .. . . . .. . . . . • • • • • • ∘ Su_{r} u_{1}  u_{r} Sv_{r} v_{1}  v_{r} p SP_{r}  P_{1} P_{r}  ⟶  . . . . .^{.....}^{.....}. . . . .. . . . .^{.....}^{.....}^{.....}^{.....}^{.....}. . . . .. . . . . • • • • •  • ∘ Su_{r} u_{1}  u_{r} Sv_{r} v_{1}  S^{−1}v_{1} p P′′_{r} P′_{1}  P′_{r}

Figure 4

Thus P = (P_{1}, P_{2}, …, P_{r−1}, P_{r}) is mapped to

φ(P) = (P′_{1}, P_{2}, …, P_{r−1}, P′_{r}).  (3.15)

Clearly φ(P) is an element of P(u → S^{k^{(1,r)}}v_{σ∘(1r)}) where k^{(1,r)} = (k_{r} + 1, k_{2}, …, k_{r−1}, k_{1} − 1).

Next we check that φ is an involution. Let P = (P_{1}, …, P_{r}) be an element of (3.8). Let p = (p^{x}, p^{y}) be the “minimal” intersection point of (SP_{r}, P_{1}, …, P_{r}) and let φ(P) = (P̄_{1}, P̄_{2}, …, P̄_{r}). We have to show that p is also the minimal intersection point for (SP̄_{r}, P̄_{1}, …, P̄_{r}) since then applying of φ to φ(P) will give back P.

It should be observed that the paths of P and φ(P) agree with each other up to the line α^{y}x + α^{x}y = α^{y}p^{x} + α^{x}p^{y}. The only changes occur strictly above this line. This follows from our assumption that the shift S is neither horizontal nor vertical, which implies that the line α^{y}x+α^{x}y = α^{y}p^{x} +α^{x}p^{y} is neither horizontal nor vertical. Therefore p is an intersection point of (SP̄_{r}, P̄_{1}, …, P̄_{r}) that minimizes α^{y}x + α^{x}y, and among those intersection points it is the right-most since the intersection points of (SP_{r}, P_{1}, …, P_{r}) and (SP̄_{r}, P̄_{1}, …, P̄_{r}) are the same.

That φ changes the sign of the corresponding permutations is clear, since the permutations are related by a transposition. Therefore φ is sign-reversing.

Finally we have to confirm (3.9), or equivalently, that φ is w̄-weight-preserving. Again let P = (P_{1}, …, P_{r}) be an element of P(u → S^{k}v_{σ}) such that (SP_{r}, P_{1}, …, P_{r}) is intersecting and let p = (p^{x}, p^{y}) be the minimal intersection point.

We consider two cases. First, suppose that p is an intersection of P_{i} and P_{j} for some i < j, which implies that φ(P) is defined by (3.12) and (3.13). In this case φ(P) is an element of P(u → S^{k^{(i,j)}}v_{σ∘(ij)}), where k^{(i,j)} = (k_{1}, …, k_{j}, …, k_{i}, …, k_{r}). We have

w(φ(P)) = w(P)

since no shift is involved. Hence in order to prove (3.9), by the definition (3.6) we have to verify that

∑_{s=1}^{r}(α^{x} \frac{(k_{s}^{(i,j)})^{2}}{2} − v_{(σ∘(ij))(s)}^{x}k_{s}^{(i,j)}) = ∑_{s=1}^{r}(α^{x} \frac{k_{s}^{2}}{2} − v_{σ(s)}^{x}k_{s}).

Clearly, these sums can be transformed into each other by interchange of the ith and jth summand; hence they agree.

Second, suppose that p is an intersection of P_{1} and SP_{r}, so that φ(P) is defined by (3.14) and (3.15). In this case φ(P) = (P′_{1}, P_{2}, …, P_{r−1}, P′_{r}) is an element of P(u → S^{k^{(1,r)}}v_{σ∘(1r)}), where k^{(1,r)} = (k_{r} + 1, k_{2}, …, k_{r−1}, k_{1} − 1). We claim that

w(φ(P)) = z^{v_{σ(r)}^{x}+α^{x}(k_{1}−k_{r}−1)−v_{σ(1)}^{x}}w(P).  (3.16)

In order to prove (3.16), we recall that the weight of a horizontal edge e and the shifted edge Se are related by (3.4). By (3.14), the path P′_{1} is identical with P_{1} up to p, then it follows SP_{r}. We compare the w-weights of the horizontal steps of P′_{1} with those of the corresponding steps in P_{1} and P_{r}. The w-weights of the horizontal steps of P′_{1} up to p are the same as those of P_{1}. The w-weight of each of the horizontal steps of P′_{1} beginning from p by (3.4) equals z times the w-weight of the corresponding step of P_{r}. The end point of SP_{r} is S^{k_{r}+1}v_{σ(r)} = v_{σ(r)} + (k_{r} + 1)(−α^{x}, α^{y}). Hence the portion of P′_{1} beginning from p contains v_{σ(r)} − (k_{r} + 1)α^{x} − p^{x} horizontal steps. Similarly, the weights of the horizontal steps of P′_{r} up to S^{−1}p are the same as those of P_{r}, while the portion of P′_{r} beginning from S^{−1}p contains v_{σ(1)}^{x} − k_{1}α^{x} − p^{x} horizontal steps, the weight of each being equal to z^{−1} times the weight of the corresponding step of P_{1}. This establishes (3.16).

Therefore, by (3.6) and (3.16), relation (3.9) in this second case is equivalent to

∑_{s=1}^{r} (α^{x} \frac{(k_{s}^{(1,r)})^{2}}{2} − v_{(σ∘(1r))(s)}^{x}k_{s}^{(1,r)}) + (v_{σ(r)}^{x} + α^{x}(k_{1} − k_{r} − 1) − v_{σ(1)}^{x})

= ∑_{s=1}^{r} (α^{x} \frac{(k_{s})^{2}}{2} − v_{σ(s)}^{x}k_{s}) . (3.17)

In (3.17) the sums on both sides differ only in the first and last summand. So (3.17), and hence (3.9), is equivalent to

α^{x} \frac{(k_{r} + 1)^{2}}{2} − v_{σ(r)}^{x}(k_{r} + 1) + α^{x} \frac{(k_{1} − 1)^{2}}{2} − v_{σ(1)}^{x}(k_{1} − 1)

+ (v_{σ(r)}^{x} + α^{x}(k_{1} − k_{r} − 1) − v_{σ(1)}^{x}) = α^{x} \frac{k_{1}^{2}}{2} − v_{σ(1)}^{x}k_{1} + α^{x} \frac{k_{r}^{2}}{2} − v_{σ(r)}^{x}k_{r},

which is clearly true. This establishes (3.9) in the second case. Hence φ is w̄-weight-preserving. This completes the proof of the Proposition. □

Remarks. (1) It should be noted that while in the enumeration of (ordinary) nonintersecting lattice paths we have to consider permutations of the end points of the lattice paths, in the previous proof we considered permutations and shifts of the end points. The reader who is familiar with reflection groups (cf. [21, especially chapter 4] for an introduction) will immediately notice the connection with the affine Weyl group of type A, a group that is generated by the permutations of coordinate axes and an additional reflection with respect to a hyperplane that does not contain the origin. In fact, the group that acts on the various configurations of end points of lattice paths that we considered in the proof of Proposition 1 is exactly isomorphic to this Weyl group, since it is generated by permutations of the end points and the additional operation illustrated in Figure 4. This operation corresponds to the additional reflection. The n × n determinants that arise in counting ordinary nonintersecting lattice paths are alternating sums over the symmetric group S_{n}, a Weyl group of type A_{n−1}; the sum of determinants in Proposition 1 is an alternating sum over the corresponding affine Weyl group of type Ã_{n−1}.

(2) The requirement in Proposition 1 that S is neither horizontal nor vertical is crucial. If S is a horizontal or vertical shift, Proposition 1 holds only if the shift S carries the path P_{r} so far that it does not restrict P_{1} at all. More precisely, if S is a horizontal shift, Proposition 1 holds only if Sv_{r} lies strictly to the left of u_{1}, and if S is a vertical shift, Proposition 1 holds only if Sv_{r} lies strictly above u_{1}. However, in these two cases we are reduced to “ordinary” nonintersecting lattice paths. Indeed, (3.5) reduces to the usual Gessel-Viennot determinant [18, Cor. 2; 42, Theorem 1.2] because the determinants in (3.5) in these two cases can be non-zero only for k_{1} = k_{2} = ⋯ = k_{r} = 0.

The proof of Proposition 1 fails for horizontal and vertical shifts because for them it is no longer true that the changes caused by the mapping in this proof happen only strictly above the line α^{y}x + α^{x}y = α^{y}p^{x} + α^{x}p^{y}. So it is impossible to define a unique “minimal” intersection point that remains “minimal” after the application of the mapping (and this is necessary to prove that the mapping is an involution). The problem with horizontal and vertical shifts is that a path can intersect its own shift. It is easy to check by example that Proposition 1 is in fact false for horizontal and vertical shifts (except in trivial cases). See item (1) of the last section for open problems on this subject.

(3) The restriction that α^{x} and α^{y} are positive is no loss of generality. If either α^{x} or α^{y} is negative then Su_{r} could not lie to the northwest of u_{1}.

(4) Proposition 1 could be generalized in several directions. For example, one could consider paths with diagonal steps. One could also consider weights that are induced not only by horizontal steps, but also by vertical steps or diagonal steps, and more generally, weights that distinguish between types of steps. Results similar to Proposition 1 could be derived for these generalizations, but we omit them since we shall not need them. See items (2)–(4) of the last section for some more related remarks.

4. The main results. In this section we apply the results of the previous section to compute the norm generating function for cylindric partitions of a given shape that satisfy given row bounds. We then generalize this result to (α, β)-cylindric partitions, which are defined below.

Theorem 2. Let λ = (λ_{1}, λ_{2}, …, λ_{r}) and μ = (μ_{1}, μ_{2}, …, μ_{r}) be r-tuples of integers and let d be an integer with d > −r such that (3.1) is satisfied. Suppose that a_{1}, a_{2}, …, a_{r} and b_{1}, b_{2}, …, b_{r} are integers satisfying (3.2). Then the generating function ∑ q^{n(π)}, where the sum is over all cylindric partitions π of shape λ/μ/d, and the entries in row i are at most a_{i} and at least b_{i}, equals

∑_{k_{1}+···+k_{r}=0} det_{1≤s,t≤r} (q^{b_{s}(λ_{s}−s−μ_{t}+t−(r+d)k_{s})+(μ_{t}−t)(s−t+rk_{s})+(r+d)(rk_{s}^{2}/2+sk_{s})}

[ \begin{matrix}a_{t} − b_{s} + λ_{s} − μ_{t} − dk_{s} \\ λ_{s} − μ_{t} − s + t − (r + d)k_{s} \end{matrix} ] ). (4.1)

If λ_{i} = λ_{i+1} − 1 for some i with 1 ≤ i ≤ r − 1, the restriction (1.3b) is also assumed to hold for j = λ_{i} + 1 = λ_{i+1} with π_{i,λ_{i}+1} := b_{i}, and if λ_{r} = λ_{1} − d − 1 the restriction (1.3c) is also assumed to hold for j = λ_{r} + 1 = λ_{1} − d with π_{r,λ_{r}+1} := b_{r}. Moreover, if μ_{i} = μ_{i+1} − 1 for some i with 1 ≤ i ≤ r − 1, then (1.3b) is also assumed to hold for j = μ_{i} + 1 = μ_{i+1} with π_{i+1,μ_{i+1}} := a_{i+1}, and if μ_{r} = μ_{1} − d − 1, the restriction (1.3c) is also assumed to hold for j = μ_{r} + 1 = μ_{1} − d with π_{1,μ_{1}} := a_{1}.

Proof. We already saw in section 3 that cylindric partitions π of shape λ/μ/d in which the entries in row i are at most a_{i} and at least b_{i} correspond bijectively to families P = (P_{1}, P_{2}, …, P_{r}) of lattice paths in which P_{i} runs from (−λ_{i} + i, b_{i} − i) to (−μ_{i} + i, a_{i} − i) and (SP_{r}, P_{1}, …, P_{r}) is nonintersecting, where S is the shift by (−r−d, r). If we define the weight w_{q} of a horizontal edge e = (e_{1}, e_{2}) by w_{q}(e) := q^{e_{2}} then this correspondence has the property

q^{n(π)} = q^{∑_{i=1}^{r} i(λ_{i}−μ_{i})}w_{q}(P).  (4.2)

Now we may apply Proposition 1 with u_{i} = (−λ_{i} + i, b_{i} − i), v_{i} = (−μ_{i} + i, a_{i} − i), w = w_{q}, α^{x} = r + d, α^{y} = r. Observe that because of (3.1), the starting points u_{i} and the end points v_{i} satisfy all the properties that are required in Proposition 1. Clearly, α^{y} = r is a positive integer, and since d is greater than −r, so is α^{x} = r + d. Moreover, for the weight function w = w_{q} and the shift S by (−r − d, r), the relation (3.4) holds with z = q^{r}. From Proposition 1 it follows that the generating function ∑ w(P), where the sum is over all families P of lattice paths as described above, is

∑_{k_{1}+···+k_{r}=0} det_{1≤s,t≤r} (q^{r(r+d)k_{s}^{2}/2+r(μ_{t}−t)k_{s}}

GF(P((−λ_{s} + s, b_{s} − s) → (−μ_{t} + t − (r + d)k_{s}, a_{t} − t + rk_{s}); w_{q}))). (4.3)

By a standard partition generating function theorem (see [4, Theorem 3.1]) we have

GF(P((a, b) → (c, d)); w_{q}) = q^{(c−a)b} [ \begin{matrix}(c − a) + (d − b) \\ c − a \end{matrix} ] .  (4.4)

If we use this in (4.3) and, because of (4.2), multiply the expression we obtain by q^{∑_{i=1}^{r} i(λ_{i}−μ_{i})}, then after some manipulation we arrive at (4.1). □

Now we introduce (α, β)-cylindric partitions. Just as (ordinary) cylindric partitions generalize (ordinary) plane partitions, (α, β)-cylindric partitions generalize (α, β)-plane partitions [24, 9].

Let r, d be integers, α = (α_{1}, α_{2}, …, α_{d}) be a d-tuple of integers, and β = (β_{1}, β_{2}, …, β_{r}), λ = (λ_{1}, λ_{2}, …, λ_{r}), μ = (μ_{1}, μ_{2}, …, μ_{r}) be r-tuples of integers. An (α, β)-cylindric partition of shape λ/μ/d is an array π of integers of the form (1.1) with

π_{i,j} ≥ π_{i,j+1} + α_{j},  (4.5a)

where by convention α_{j} ≡ α_{j mod d} for any j,

π_{i,j} ≥ π_{i+1,j} + β_{i}  (4.5b)

and

π_{r,j} ≥ π_{1,j+d} + β_{r}.  (4.5c)

Note that the length of the sequence α equals the length of the “shift” d in the shape, while the length of the sequence β equals the number of rows of the shape. Clearly, in this new terminology “ordinary” cylindric partitions are (0, 0)-cylindric partitions (recall that 0 denotes the all 0 sequence). Moreover, the various types of plane partitions are also special (α, β)-cylindric partitions. For example, columnstrict plane partitions (cf. [40]) are (0, 1)-cylindric partitions (1 denotes the all 1 sequence) of some shape λ/μ/d where d is larger than the difference λ_{1} − μ_{r} (so that (4.5c) becomes void and so there is no relation between the first and the last row for these (0, 1)-cylindric partitions).

Again the norm of an (α, β)-cylindric partition π is defined to be the sum ∑ π_{i,j} of all the entries of π.

The extension of Theorem 2 to (α, β)-cylindric partitions reads as follows.

Theorem 3. Let λ = (λ_{1}, λ_{2}, …, λ_{r}) and μ = (μ_{1}, μ_{2}, …, μ_{r}) be r-tuples of integers and d be an integer with d > −r such that (3.1) is satisfied. Let α = (α_{1}, α_{2}, …, α_{d}) be a d-tuple of integers and let β = (β_{1}, β_{2}, …, β_{r}) be an r-tuple of integers with ∑_{k=1}^{r} β_{k} − ∑_{k=1}^{d} α_{k} < r. Suppose that a_{1}, a_{2}, …, a_{r} and b_{1}, b_{2}, …, b_{r} are integers satisfying

a_{i} + (1 − β_{i}) + ∑_{k=μ_{i+1}+1}^{μ_{i}} α_{k} ≥ a_{i+1}  (4.6a)

b_{i} + (1 − β_{i}) + ∑_{k=λ_{i+1}}^{λ_{i}−1} α_{k} ≥ b_{i+1}  (4.6b)

and

a_{r} + (1 − β_{r}) + ∑_{k=μ_{1}+1}^{μ_{r}+d} α_{k} ≥ a_{1}  (4.6c)

b_{r} + (1 − β_{r}) + ∑_{k=λ_{1}}^{λ_{r}+d−1} α_{k} ≥ b_{1}.  (4.6d)

Then the generating function ∑ q^{n(π)}, where the sum is over all (α, β)-cylindric partitions π of shape λ/μ/d in which the first entry in row i is at most a_{i} and the last entry in row i is at least b_{i}, equals

∑_{k_{1}+···+k_{r}=0} det_{1≤s,t≤r} (q^{T (s,t,k)}

[ \begin{matrix}a_{t} − b_{s} + λ_{s} − μ_{t} − ∑_{k=μ_{t}+1}^{λ_{s}−1} α_{k} + ∑_{k=s}^{t−1} β_{k} − k_{s}(d − ∑_{k=1}^{d} α_{k} + ∑_{k=1}^{r} β_{k}) \\ λ_{s} − μ_{t} − s + t − (r + d)k_{s} \end{matrix} ] ),

(4.7)

where

T (s, t, k) = b_{s}(λ_{s} − s − μ_{t} + t − (r + d)k_{s}) + ∑_{k=μ_{t}}^{λ_{s}−1} α_{k}(k − μ_{t}) − ∑_{k=1}^{λ_{s}−1} α_{k}(s − t + (r + d)k_{s})

+ (μ_{t} − t)( ∑_{k=t}^{s−1}(1 − β_{k}) + k_{s}( ∑_{k=1}^{r}(1 − β_{k}) + ∑_{k=1}^{d} α_{k}))

+ (r + d)( \frac{k_{s}^{2}}{2} ( ∑_{k=1}^{r}(1 − β_{k}) + ∑_{k=1}^{d} α_{k}) + k_{s} ∑_{k=1}^{s−1}(1 − β_{k})).

The sums are interpreted by

∑_{k=m}^{n−1} Expr(k) = \{ \begin{matrix}∑_{k=m}^{n−1} Expr(k) & n > m \\ 0 & n = m \\ − ∑_{k=n}^{m−1} Expr(k) & n < m. \end{matrix}

If λ_{i} = λ_{i+1} − 1 for some i with 1 ≤ i ≤ r − 1, the restriction (4.5b) is also assumed to hold for j = λ_{i}+1 = λ_{i+1} with π_{i,λ_{i}+1} := b_{i}−α_{λ_{i}}, and if λ_{r} = λ_{1}−d−1 the restriction (4.5c) is also assumed to hold for j = λ_{r}+1 = λ_{1}−d with π_{r,λ_{r}+1} := b_{r}−α_{λ_{r}}. Similarly, if μ_{i} = μ_{i+1} − 1 for some i with 1 ≤ i ≤ r − 1, (4.5b) is also assumed to hold for j = μ_{i} +1 = μ_{i+1} with π_{i+1,μ_{i+1}} := a_{i+1} +α_{μ_{i+1}}, and if μ_{r} = μ_{1} −d−1 the restriction (4.5c) is also assumed to hold for j = μ_{r} + 1 = μ_{1} − d with π_{1,μ_{1}} := a_{1} + α_{μ_{1}}.

Proof. Following the proof idea of Theorem 2 we convert (α, β)-cylindric partitions to column-strict cylindric partitions that in turn are converted into a family of nonintersecting lattice paths with nonintersecting translate of the last path. Then we apply Proposition 1.

Let π be an (α, β)-cylindric partition of shape λ/μ/d in which the first entry in row i is at most a_{i} and the last entry in row i is at least b_{i}. Define the new array π̄ by

π̄_{i,j} := π_{i,j} + ∑_{k=1}^{j−1} α_{k} − ∑_{k=1}^{i−1}(1 − β_{k}).  (4.8)

Clearly, (4.5a) translates into π̄_{i,j} ≥ π̄_{i,j+1}, (4.5b) translates into π̄_{i,j} > π̄_{i+1,j}, and (4.5c) translates into

π̄_{r,j} + ∑_{k=1}^{d} α_{k} + ∑_{k=1}^{r}(1 − β_{k}) > π̄_{1,j+d}.  (4.9)

This shows that by (4.8) the (α, β)-cylindric partitions π under consideration correspond bijectively to column-strict cylindric partitions π̄ of shape λ/μ/d with first-last relation (4.9) in which the entries in the ith row are at most ā_{i} with

ā_{i} = a_{i} + ∑_{k=1}^{μ_{i}} α_{k} − ∑_{k=1}^{i−1}(1 − β_{k})

and at least b̄_{i} with

b̄_{i} = b_{i} + ∑_{k=1}^{λ_{i}−1} α_{k} − ∑_{k=1}^{i−1}(1 − β_{k}).

Note that by (4.8) the norms of π and π̄ are related by

n(π) = n(π̄) − ∑_{i=1}^{r} ( ∑_{j=1}^{λ_{i}} α_{j}(λ_{i} − j + 1) − ∑_{j=1}^{μ_{i}} α_{j}(μ_{i} − j + 1))

+ ∑_{i=1}^{r} ((λ_{i} − μ_{i}) ∑_{k=1}^{i−1}(1 − β_{k})). (4.10)

From section 3 we know that these column-strict cylindric partitions correspond bijectively to families (P_{1}, P_{2}, …, P_{r}) of lattice paths in which P_{i} runs from (−λ_{i} +i, b̄_{i}) to (−μ_{i} + i, ā_{i}) and (SP_{r}, P_{1}, …, P_{r}) is nonintersecting, where S is the shift by (−r − d, ∑_{k=1}^{d} α_{k} + ∑_{k=1}^{r}(1 − β_{k})). Moreover, this correspondence transforms the norm of column-strict cylindric partitions into the weight w_{q} (which was introduced in the proof of Theorem 2) of families of lattice paths. Therefore, in order to compute the generating function for the (α, β)-cylindric partitions under consideration, we may apply Proposition 1 with u_{i} = (−λ_{i} + i, b̄_{i}), v_{i} = (−μ_{i} + i, ā_{i}), w = w_{q}, z = q^{∑_{k=1}^{d} α_{k}+∑_{k=1}^{r}(1−β_{k})}, α^{x} = r + d, α^{y} = ∑_{k=1}^{d} α_{k} + ∑_{k=1}^{r}(1 − β_{k}), and multiply the resulting expression (3.5) by the power of q that is determined by relation (4.10).

Then (4.4), with some manipulation, leads to (4.7). Since (3.1) and (4.6) imply that the required conditions for the starting points u_{i} and ending points v_{i} are satisfied, and since d > −r and ∑_{k=1}^{r} β_{k} − ∑_{k=1}^{d} α_{k} < r imply that α^{x} > 0 and α^{y} > 0, we may now apply Proposition 1. □

Remarks. (1) In each of Proposition 1 and Theorems 2 and 3, k_{s} may be replaced by k_{t} in the determinants inside the multisum. This may be seen by reordering terms and renaming the summation indices.

(2) Theorem 2 is the special case α_{i} = 0, β_{i} = 0 of Theorem 3.

(3) By extending the approach to plane partitions in [24] (see also [31]), we could also prove Theorem 3 (and Theorem 2) by working directly with arrays, thus avoiding the transformation to nonintersecting lattice paths.

(4) All of the conditions, (3.1), (4.6), d > −r, and ∑_{i=1}^{r} β_{i} − ∑_{k=1}^{d} α_{k} < r, in Theorem 3, are really necessary. They ensure that after the cylindric partition is transformed into nonintersecting lattice paths, the starting and ending points are in the “right” order (see the formulation of Proposition 1) and that the shift is neither horizontal nor vertical. It is easily seen that if the starting and ending points are not in the “right” order, Proposition 1 does not hold. Also it does not hold for a horizontal or vertical shift except for trivial cases (see Remark (2) after the proof of Proposition 1). Hence, if any of the conditions in Theorem 3 were violated, the theorem would not hold. The same is true for Theorem 2.

(5) Theorem 3 contains all determinantal formulas for norm generating functions for plane partitions (without symmetries) of a given shape [18, Theorems 15, 16; 24, Theorem 6.1; 26, Theorem 1, (3.3); 41, Prop. 21.2] as special cases. This is seen by choosing d ≥ λ_{1} − μ_{r}. This choice causes the restriction (4.5c) to be void and causes all the determinants in the sum (4.7) to vanish except for k_{1} = k_{2} = ⋯ = k_{r} = 0.

(6) In [12] Burge considers restricted partitions pairs. In its most general form [12, p. 220] these are pairs

((p_{N_{1}}, p_{N_{1}−1}, …, p_{1}), (q_{N_{2}}, q_{N_{2}−1}, …, q_{1}))

of partitions with

M_{1} ≥ p_{N_{1}} ≥ p_{N_{1}−1} ≥ ⋯ ≥ p_{1} ≥ 0, M_{2} ≥ q_{N_{2}} ≥ q_{N_{2}−1} ≥ ⋯ ≥ q_{1} ≥ 0

and

p_{i} − q_{i+(1−a)} ≥ 1 − α and q_{i} − p_{i+(1−b)} ≥ 1 − β.

If we define the two-rowed array π by π_{1,i} := p_{N_{1}+1−i} and π_{2,i} := q_{N_{1}+1−i+(1−a)} then these restricted partition pairs in our language are two-rowed (0, (1 − α, 1 − β))- cylindric partitions of shape (N_{1}, N_{1} + 1 − a)/(0, 1 − a + N_{1} − N_{2})/(a + b − 2) in which the entries in row i are at most M_{i} and at least 0, for i = 1, 2. Thus the r = 2, α = 0 special case of Theorem 3 immediately gives for the norm generating function of these

cylindric partitions

∑_{k_{1}+k_{2}=0} q^{k_{1}(α+β)−k_{2}(1−a+N_{1}−N_{2}−2)(α+β)+(a+b)((α+β)(k_{1}^{2}+k_{2}^{2})/2−k_{2}α)}

[ \begin{matrix}M_{1} − k_{1}(α + β) + N_{1} + (a + b)k_{1} \\ N_{1} + (a + b)k_{1} \end{matrix} ] [ \begin{matrix}M_{2} − k_{2}(α + β) + N_{2} + (a + b)k_{2} \\ N_{2} + (a + b)k_{2} \end{matrix} ]

− ∑_{k_{1}+k_{2}=0} q^{−(1−a+N_{1}−N_{2})α+α−k_{2}(1−a+N_{1}−N_{2}−2)(α+β)+k_{2}(α+β)+(a+b)((α+β)(k_{1}^{2}+k_{2}^{2})/2−k_{2}α)}

[ \begin{matrix}M_{2} − α − k_{1}(α + β) + N_{2} + a + (a + b)k_{1} \\ N_{2} + a + (a + b)k_{1} \end{matrix} ]
[ \begin{matrix}M_{1} + α − k_{2}(α + β) + N_{1} − a + (a + b)k_{2} \\ N_{1} − a + (a + b)k_{2} \end{matrix} ] ,  (4.11)

which agrees with Burge’s formula [12, p. 220] for his generating function R(N_{1}, M_{1}, N_{2}, M_{2}, a, b, α, β). (Note that in Burge’s paper the roles of the partitions p and q are as in our paper when he defines his generating function P (N, M, a, b, α, β); however, they are interchanged when he defines his generating function R(N_{1}, M_{1}, N_{2}, M_{2}, a, b, α, β).)

Since the conditions under which formula (4.11) holds are not addressed very clearly in [12], we give the complete set of conditions here: The parameters a and b must be nonnegative integers (this comes from (3.1a)), with at least one of them non-zero (from d > −r). The same has to be true for α and β (from (4.6b, d) and ∑_{i=1}^{r} β_{i} < r). Moreover, we must have a ≥ N_{1} − N_{2} ≥ −b (from (3.1b)) and α ≥ M_{2} − M_{1} ≥ −β (from (4.6a, c)). If a = 0 it is understood that q_{1} ≤ α − 1, if b = 0 it is understood that p_{1} ≤ β − 1, if N_{1} − N_{2} − a = 0 it is understood that p_{N_{1}} ≥ M_{2} + 1 − α, and if N_{2} − N_{1} − b = 0 it is understood that q_{N_{2}} ≥ M_{1} + 1 − β (these four conditions come from the exceptional readings of (4.5b) and (4.5c) that are explained at the end of the formulation of Theorem 3). If any of these conditions is violated the expression (4.11) will not be the correct generating function (see Remark (4)).

We note that a weaker form of Burge’s partition pairs previously was considered by Krattenthaler and Mohanty [31] in lattice path formulation, and is thus also covered by our Theorem 3.

(7) Linear partitions with restrictions on the hook differences were considered in a number of articles [1, 2, 3, 5, 6, 10, 11]. The most general result, which includes all the others, is contained in [6]. There Andrews et al. consider linear partitions λ = (λ_{1}, λ_{2}, …, λ_{M}), N ≥ λ_{1} ≥ λ_{2} ≥ ⋯ ≥ λ_{M} ≥ 0 where the hook differences on diagonal 1−β are at least i+β +1 and on diagonal α−1 are at most K −i−α−1. In [6, Theorem 1] they find the generating function ∑ q^{n(λ)} for these partitions, where again n(λ) is the sum ∑_{i=1}^{M} λ_{i} of all the entries of λ. Andrews et al. give a proof using recurrences. We shall show how their result can be derived by using the r = 2, α = (1, 1, …, 1) case of our Theorem 3 and q-Vandermonde summation.

Let λ be a partition as described above. We write λ in Frobenius notation (cf. [33, p. 3]) as λ = (σ | τ), σ = (σ_{1}, σ_{2}, …, σ_{m}), τ = (τ_{1}, τ_{2}, …, τ_{m}), where m is the (Durfee) rank of the partition λ. We have N − 1 ≥ σ_{1} > σ_{2} > ⋯ > σ_{m} ≥ 0 and M − 1 ≥ τ_{1} > τ_{2} > ⋯ > τ_{m} ≥ 0. The conditions on the hook differences imply that

(σ_{j+1−β} + j + 1 − β) − (τ_{j} + j) ≥ −i + β + 1
(σ_{j+α−1} + j + α − 1) − (τ_{j} + j) ≤ K − i − α − 1.

If we set τ̄_{j} = τ_{j+β−1} then these inequalities can be written as

σ_{j} ≥ τ̄_{j} − i + 2β
τ̄_{j} ≥ σ_{j+α+β−2} − K + i + 2α.

Let us define the two-rowed array π by π_{1,j} := σ_{j}, 1 ≤ j ≤ m, and π_{2,j} := τ̄_{j}, 2 − β ≤ j ≤ m + 1 − β. Thus we see that linear partitions λ, N ≥ λ_{1} ≥ λ_{2} ≥ ⋯ ≥ λ_{M} ≥ 0 in which the hook differences on diagonal 1 − β are ≥ −i + β + 1 and on diagonal α − 1 are ≤ K − i − α − 1 correspond bijectively to (1, (−i + 2β, −K + i + 2α))-cylindric partitions π of shape (m, m + 1 − β)/(0, 1 − β)/(α + β − 2), for some m (where 1 denotes the all 1 sequence) in which the entries in the first row are at most N − 1 and at least 0 and the entries in the second row are at most M − 1 and at least 0. Moreover,

n(λ) = n(π) + m.  (4.12)

For fixed m, the case r = 2, α_{i} ≡ 1 of Theorem 3 gives us the generating function ∑ q^{n(π)} for these cylindric partitions π, namely

∑_{k_{1}=−∞}^{∞} q^{m^{2}−m+k_{1}^{2}(α+β)(K−α−β)−k_{1}i(α+β)+k_{1}Kβ}

[ \begin{matrix}N + k_{1}(K − 2α − 2β) \\ m − (α + β)k_{1} \end{matrix} ] [ \begin{matrix}M − k_{1}(K − 2α − 2β) \\ m + (α + β)k_{1} \end{matrix} ]

− ∑_{k_{1}=−∞}^{∞} q^{m^{2}−m+k_{1}^{2}(α+β)(K−α−β)−k_{1}(α+β)(2β+i)−k_{1}Kβ−β^{2}+βi}

[ \begin{matrix}N + i − 2β − k_{1}(K − 2α − 2β) \\ m − β + (α + β)k_{1} \end{matrix} ] [ \begin{matrix}M − i + 2β + k_{1}(K − 2α − 2β) \\ m + β − (α + β)k_{1} \end{matrix} ] .

Finally, in order to compute the complete generating function ∑ q^{n(λ)} for all linear partitions λ with prescribed hook differences as described above, by virtue of (4.12), we have to multiply the above expression by q^{m} and sum over all m. Using the q-Vandermonde summation (see e.g. [4, (3.3.10)])

∑_{L≥0} q^{(N−L)(H−L)} [ \begin{matrix}N \\ L \end{matrix} ] [ \begin{matrix}M \\ H − L \end{matrix} ] = [ \begin{matrix}N + M \\ H \end{matrix} ] ,

this leads to

∑_{k_{1}=−∞}^{∞} q^{k_{1}^{2}(α+β)K−k_{1}i(α+β)+k_{1}Kβ} [ \begin{matrix}N + M \\ N + k_{1}K \end{matrix} ]

− ∑_{k_{1}=−∞}^{∞} q^{k_{1}^{2}(α+β)K−k_{1}i(α+β)−k_{1}Kβ+βi} [ \begin{matrix}N + M \\ N + i − k_{1}K \end{matrix} ] ,

which is the expression in [6, Theorem 1].

In [6] also, some necessary conditions on the parameters are missing. We give the complete set of conditions below and slightly extend the range of validity of [6, Theorem 1]. The parameters α and β must be nonnegative integers (this comes from (3.1)), with at least one of them non-zero (from d > −r), with α + β < K (from ∑_{i=1}^{r} β_{i} − d < r). Moreover, we must have −i + β ≤ N − M ≤ K − i − α (from (4.6a, c)), i − β ≥ 0 (from (4.6b)), and K − i − α ≥ 0 (from (4.6d)). (This means that in [6, Theorem 1] the condition 1 ≤ i ≤ K/2 has to be dropped and be replaced by the last two conditions.) If α = 0 it is understood that only partitions λ = (σ_{1}, …, σ_{m} | τ_{1}, …, τ_{m}) are considered with σ_{m} ≤ K − i and τ_{1} ≥ N − 1 − K + i, and if β = 0 it is understood that only partitions λ = (σ_{1}, …, σ_{m} | τ_{1}, …, τ_{m}) are considered with σ_{1} ≥ M − 1 − i and τ_{m} ≤ i (these two conventions come from the exceptional readings of (4.5b) and (4.5c) that are explained at the end of the formulation of Theorem 3).

(8) Krattenthaler and Mohanty’s lattice path enumeration results [30, Theorems 1, 2], which are q-analogues of [14], are corollaries of Theorem 3. We concentrate on [30, Theorem 2] since [30, Theorem 1] is a special case of the first. Krattenthaler and Mohanty consider r-dimensional lattice paths consisting of positive unit steps from μ = (μ_{1}, μ_{2}, …, μ_{r}) to λ = (λ_{1}, λ_{2}, …, λ_{r}) in the region

x_{1} ≥ x_{2} ≥ ⋯ ≥ x_{r} ≥ x_{1} − d.  (4.13)

In [30, Theorem 2] they give an expression for the generating function ∑ q^{maj_{τ} Q} for all these lattice paths Q where τ is a fixed permutation in 𝔖_{r} and maj_{τ} is a certain statistic on lattice paths that generalizes MacMahon’s greater index (also called major index) and lesser index. The link between our Theorem 3 and Krattenthaler and Mohanty’s theorem [30, Theorem 2] is Stanley’s (P, ω)-partition theorem [41, Cor. 5.3+7.2]. It says that, given a poset P with labelling ω, the generating function U(P, ω; q) for unbounded (P, ω)-partitions is given by

U(P, ω; q) = \frac{W (P, ω; q)}{[ |P |]_{q}!},  (4.14)

where

W (P, ω; q) = ∑_{σ} q^{maj σ},

the sum being over all permutations σ in the ω-separator [41, p. 17] of P. (|P | denotes the cardinality of P, and maj denotes the (ordinary) major index.)

To apply this theorem, we take P to be the poset P (μ/λ/d) with underlying set

{(i, j) ∈ ℤ^{2} : 1 ≤ i ≤ r and μ_{i} < j ≤ λ_{i}},

ordered by

(i_{1}, j_{1}) ≤ (i_{2}, j_{2}) if and only if \{ \begin{matrix}i_{1} ≤ i_{2} and j_{1} ≤ j_{2} \\ or i_{1} = r, i_{2} = 1, and j_{1} + d ≤ j_{2}, \end{matrix}

and labelled by ω_{τ}, which is defined by

ω_{τ} ( (i, j) ) = ∑_{t=1}^{i−1}(λ_{τ^{−1}(t)} − μ_{τ^{−1}(t)}) + j − μ_{τ^{−1}(i)}.

We may describe ω_{τ} by saying that ω_{τ} numbers the elements of P by 1, 2, …, |P |, beginning with row τ^{−1}(1), continuing with row τ^{−1}(2), etc., with each row numbered from left to right. Then it is not difficult to see that the (P (λ/μ/d), ω_{τ})-partitions are simply (0, c)-cylindric partitions of shape λ/μ/d, where c = (c_{1}, c_{2}, …, c_{r}) with c_{i} = χ(τ(i) > τ(i+1)). (Here χ(A)=1 if A is true and χ(A)=0 otherwise.) Moreover, the ω_{τ} -separator of P (λ/μ/d) corresponds to the lattice paths from μ to λ in the region (4.13), such that the major index of elements of the ω_{τ} -separator equals maj_{τ} of the lattice paths. This is seen as follows. Let |λ − μ| denote the cardinality of P (λ/μ/d) (which is ∑_{i=1}^{r}(λ_{i} − μ_{i})) and let l = (l_{1} ≤ l_{2} ≤ ⋯ ≤ l_{|λ−μ|}) be a linear extension of the poset P (λ/μ/d).

Then l defines an element in the ω_{τ} -separator of P (λ/μ/d), namely

ω_{τ} (l) = ω_{τ} (l_{1})ω_{τ} (l_{2}) … ω_{τ} (l_{|λ−μ|}).

Now we may define a map ρ from P (λ/μ/d) to {1, 2, …, r} by ρ( (i, j) ) := i and form the sequence

ρ(l) = ρ(l_{1})ρ(l_{2}) … ρ(l_{|λ−μ|}).

Then ρ(l) may be viewed as an r-dimensional lattice path from μ to λ in the region (4.13) by reading ρ(l) from left to right and interpreting i as a unit step in the positive x_{i}-direction, i = 1, 2, …, r. By observing that

maj(ω_{τ} (l)) = maj_{τ} (ρ(l))

we see that W (P (λ/μ/d), ω_{τ}; q) is the same as the generating function ∑ q^{maj_{τ} Q} for these lattice paths Q. Hence, by (4.14) this generating function equals [ |λ − μ|]_{q}! times the expression (4.7) with a_{i} → ∞, b_{i} = 0, α_{i} = 0, β_{i} = c_{i} = χ(τ(i) > τ(i + 1)), i.e.,

∑_{k_{1}+···+k_{r}=0}[|λ − μ|]_{q}! det_{1≤s,t≤r} (q^{T_{τ} (s,t,k)}/[λ_{s} − s − μ_{t} + t − (d + r)k_{s}]_{q}!) .

The exponents T_{τ} (s, t, k) are given by

T_{τ} (s, t, k) = (μ_{t} − t) ∑_{k=t}^{s−1}(1 − c_{k}) + k_{s}(μ_{t} − t) ∑_{k=1}^{r}(1 − c_{k})

+ (d + r) (\frac{k_{s}^{2}}{2} ∑_{k=1}^{r}(1 − c_{k}) + k_{s} ∑_{k=1}^{s−1}(1 − c_{k})) .

Thus we recover [30, Theorem 2].

5. Special generating functions and enumeration results. For some particular choices of the parameters in Theorems 2 and 3 the determinants can be evaluated in closed form by means of the following three determinant evaluations (5.1)–(5.3), which are from [24, 25]. All of them result from a single determinant lemma [24, Lemma 2.2]. It was shown in [24, 25] that all the known product formulas for plane partitions of a given shape (without symmetries), such as Stanley’s hook-content and hook formulas and MacMahon’s generating function for plane partitions of rectangular shape, follow from (5.1)–(5.3). In particular, the evaluation (5.3) was used in [25] to obtain new enumeration results for plane partitions of staircase shape.

Lemma 4. There hold

det_{1≤s,t≤r} ([ \begin{matrix}L_{s} + A + t \\ L_{s} + t \end{matrix} ]) = q^{∑_{i=1}^{r}(i−1)(L_{i}+i)} \frac{∏_{1≤i<j≤r}[L_{i} − L_{j}]}{∏_{i=1}^{r}[L_{i} + r]!} \frac{∏_{i=1}^{r}[L_{i} + A + 1]!}{∏_{i=1}^{r}[A + 1 − i]!},

(5.1)

det_{1≤s,t≤r} (q^{tL_{s}} [ \begin{matrix}A \\ L_{s} + t \end{matrix} ]) = q^{∑_{i=1}^{r} iL_{i}} \frac{∏_{1≤i<j≤r}[L_{i} − L_{j}]}{∏_{i=1}^{r}[L_{i} + r]!} \frac{∏_{i=1}^{r}[A + i − 1]!}{∏_{i=1}^{r}[A − L_{i} − 1]!}, (5.2)

and

det_{1≤s,t≤r} ( ( \begin{matrix}BL_{s} + A \\ L_{s} + t \end{matrix} ) )

= \frac{∏_{1≤i<j≤r}(L_{i} − L_{j})}{∏_{i=1}^{r}(L_{i} + r)!} ∏_{i=1}^{r} \frac{(BL_{i} + A)!}{((B − 1)L_{i} + A − 1)!} ∏_{i=1}^{r}(A − Bi + 1)_{i−1}, (5.3)

where (a)_{n} := a(a + 1) ⋯ (a + n − 1) denotes rising factorials.

Proof. For the proof of (5.1) replace λ_{s} − s by L_{s} and a + α − b by A in the computation on p. 188 of [24]. With the same replacements, (5.2) follows from the computation on p. 189 of [24]. Finally, (5.3) is proved by replacing λ_{s} − s by L_{s}, a + α − b by A, and B by 1 − B in the proof of Theorem 5 of [25]. □

Remark. In fact (5.1) is equivalent to (5.2) with A replaced by −A − 1.

Our first theorem in this section concerns cylindric partitions of shape (c^{r})/μ/d. The notation (c^{r}) means the r-tuple (c, c, …, c), as usual. It generalizes Stanley’s hook formula for reverse plane partitions of a given (non-skew) shape with unrestricted part magnitude [40, Proposition 18.3].

Theorem 5. Let μ = (μ_{1}, μ_{2}, …, μ_{r}) be an r-tuple of integers and d be an integer with d > −r such that μ_{1} ≥ μ_{2} ≥ ⋯ ≥ μ_{r} ≥ μ_{1} − d. Then the generating function ∑ q^{n(π)}, where the sum is over all cylindric partitions π of shape (c^{r})/μ/d with nonnegative entries, equals

∑_{k_{1}+···+k_{r}=0} q^{r(r+d) ∑_{i=1}^{r} k_{i}^{2}/2+d ∑_{i=1}^{r} ik_{i}+r ∑_{i=1}^{r} μ_{i}k_{i}}

\frac{∏_{1≤i<j≤r}[μ_{i} + (r + d)k_{i} − i − μ_{j} − (r + d)k_{j} + j]}{∏_{i=1}^{r}[c − μ_{i} − (r + d)k_{i} + i − 1]!}. (5.4)

More generally, let α = (α_{1}, α_{2}, …, α_{d}) and β = (β_{1}, β_{2}, …, β_{r}) be integer sequences such that ∑_{i=1}^{r} β_{k} − ∑_{i=1}^{d} α_{k} < r. Then the generating function ∑ q^{n(π)}, where the sum is over all (α, β)-cylindric partitions π of shape (c^{r})/μ/d with π_{r,c} ≥ 0, equals

∑_{k_{1}+···+k_{r}=0} q^{N_{1}(k)} \frac{∏_{1≤i<j≤r}[μ_{i} + (r + d)k_{i} − i − μ_{j} − (r + d)k_{j} + j]}{∏_{i=1}^{r}[c − μ_{i} − (r + d)k_{i} + i − 1]!},  (5.5)

where

N_{1}(k) = (r + A − B)(r + d) ∑_{i=1}^{r} k_{i}^{2}/2 + (d − A + B) ∑_{i=1}^{r} ik_{i} + (r + A − B) ∑_{i=1}^{r} μ_{i}k_{i}

+ ∑_{i=1}^{r} ∑_{k=μ_{i}}^{c−1} α_{k}(k − μ_{i}) − ∑_{k=1}^{r} β_{k} ∑_{i=k+1}^{r}(c − μ_{i}),

with A := ∑_{k=1}^{d} α_{k} and B := ∑_{k=1}^{r} β_{k}.

Proof. We prove here only (5.4); (5.5) may be established, with some more work, in the same manner.

By Theorem 2 with λ_{i} = c, b_{i} = 0, a_{i} → ∞, and using Remark (1) after the proof of Theorem 3, we see that the generating function for cylindric partitions of shape (c^{r})/μ/d with nonnegative entries is given by

∑_{k_{1}+···+k_{r}=0} q^{r(r+d) ∑_{i=1}^{r} k_{i}^{2}/2+∑_{i=1}^{r}(μ_{i}−i)(rk_{i}−i)}

det_{1≤s,t≤r} (q^{s(μ_{t}−t+(r+d)k_{t})}/[c − μ_{t} − s + t − (r + d)k_{t}]!).

Reversing the order of both rows and columns in the determinant (i.e., replacing s by r + 1 − s and t by r + 1 − t) transforms this expression into

∑_{k_{1}+···+k_{r}=0} q^{r(r+d) ∑_{i=1}^{r} k_{i}^{2}/2+∑_{i=1}^{r}(μ_{i}−i)(rk_{i}−i)+(r+1) ∑_{i=1}^{r}(μ_{i}+(r+d)k_{i})−c\binom{r+1}{2}}

det_{1≤s,t≤r} (q^{s(c−μ_{r+1−t}−(r+d)k_{r+1−t}−t)}/[c − μ_{r+1−t} − (r + d)k_{r+1−t} − t + s]!).

Now the determinant can be evaluated by (5.2) with L_{s} = c−μ_{r+1−s}−(r+d)k_{r+1−s}−s, A → ∞. After a few simplifications we arrive at (5.4).

Of course, for (5.5) we use Theorem 3 instead of Theorem 2. □

Stanley’s formula for reverse plane partitions of a given shape with unrestricted part magnitude [40, Proposition 18.3] results from Theorem 5, (5.4), by setting d = c−μ_{r}. This choice of d causes the condition (4.5c) to become void. Thus we are considering ordinary plane partitions of shape (c^{r})/μ, which by a 180^{∘} rotation become reverse plane partitions of shape (c − μ_{r}, c − μ_{r−1}, …, c − μ_{1})/0. Moreover, for d = c − μ_{r} the sum in (5.4) consists of the single term k_{1} = k_{2} = ⋯ = k_{r} = 0.

The next result concerns cylindric partitions of “infinite rectangular shape” (∞^{r})/0/d.

Theorem 6. Let d > −r. The generating function ∑ q^{n(π)}, where the sum is over all cylindric partitions π of shape (∞^{r})/0/d in which the entries are between 0 and a, equals

∑_{k_{1}+···+k_{r}=0} q^{r(r+d) ∑_{i=1}^{r} k_{i}^{2}/2+d ∑_{i=1}^{r} ik_{i}} \frac{∏_{1≤i<j≤r}[rk_{j} + j − rk_{i} − i]}{∏_{i=1}^{r}[a + rk_{i} + i − 1]!}.  (5.6)

Proof. We apply Theorem 2 with λ = (c^{r}), c → ∞, μ = 0, a_{i} = a, b_{i} = 0 to obtain for the desired generating function the expression

∑_{k_{1}+···+k_{r}=0} det_{1≤s,t≤r} (q^{−t(s−t+rk_{s})+(r+d)(rk_{s}^{2}/2+sk_{s})} \frac{1}{[a + s − t + rk_{s}]!}).

By reversing the order of the rows and columns in the determinant and replacing k_{i} by k_{r+1−i}, we transform this expression into

∑_{k_{1}+···+k_{r}=0} q^{∑_{i=1}^{r}(r+d)(rk_{i}^{2}/2+ik_{i})+∑_{i=1}^{r} i(i−a)}

det_{1≤s,t≤r} (q^{t(a−s+rk_{r+1−s})} \frac{1}{[a + t − s + rk_{r+1−s}]!}).

Now we use (5.2) with the replacements A → ∞, L_{s} → a − s + rk_{r+1−s} and obtain (5.6) after a few simplifications. □

There is an overlap with MacMahon’s result for the generating function for plane partitions of rectangular shape with restricted part magnitude [34, sec. 429, proof in sec. 494]. Namely, the formula for plane partitions of arbitrary shape with restricted part magnitude (that results by letting the width of the rectangle tend to ∞) is also the d → ∞ special case of Theorem 6. (For d → ∞ only the k_{1} = k_{2} = ⋯ = k_{r} = 0 term of the sum remains.)

The last result in this section concerns staircase shapes.

Theorem 7. The number of cylindric partitions π of shape (Λ−L, Λ−2L, …, Λ− rL)/0/Lr in which the entries are between 0 and a equals

∑_{k_{1}+···+k_{r}=0}(L + 1)^{\binom{r}{2}} ∏_{1≤i<j≤r}(rk_{j} + j − rk_{i} − i) ∏_{i=1}^{r}(a + 1 + (Λ − Li)/(L + 1))_{i−1}

∏_{i=1}^{r} \frac{(a + Λ − L(rk_{i} + i))!}{(Λ − (L + 1)(rk_{i} + i) + r)! (a + rk_{i} + i − 1)!}, (5.7)

where again (a)_{n} := a(a + 1) ⋯ (a + n − 1) denotes the rising factorial.

Proof. We apply Theorem 2 with λ_{i} = Λ − iL, μ_{i} = 0, d = Lr, a_{i} = a, b_{i} = 0, q → 1 to obtain for the desired number the expression

∑_{k_{1}+···+k_{r}=0} det_{1≤s,t≤r} ( ( \begin{matrix}a + Λ − L(rk_{s} + s) \\ Λ − (L + 1)(rk_{s} + s) + t \end{matrix} ) ).

Here we use the determinant evaluation (5.3) with the replacements L_{s} → Λ − (L + 1)(rk_{s} + s), B → L/(L + 1), A → a + Λ/(L + 1). Thus, after some simplification we arrive at (5.7). □

We know one more special choice of parameters in which the determinants in Theorem 2 can be evaluated. This is the case of cylindric partitions of shape (c^{r})/0/0 with entries between 0 and a. In this case the generating function can be written down in an even simpler form, since these cylindric partitions are actually in bijection with certain linear partitions. Therefore we do not state the multisum expression that would result from Theorem 2 as a separate theorem. On the other hand, these considerations produce multisum summations that are interesting in their own right; see (6.18) and (7.10).

Another case in which the determinants in Theorem 3 could be evaluated (by means of (5.1)) is μ = 0, a_{i} = a, b_{i} = 0, α_{i} = 0, β_{i} = 1, i.e., the case of “column-strict” cylindric partitions of shape λ/0/d with entries between 0 and a. Unfortunately, in this case the restriction ∑_{i=1}^{r} β_{k} < r is violated, so it is not clear what, if anything, is actually counted by (4.7) with this choice of parameters.

6. Identities I: Sums of determinants. In this section we extend ideas of Burge [12]. He derived a number of summation formulas by considering special cases of his restricted partition pairs (which in our language are two-rowed cylindric partitions; see Remark (6) after the proof of Theorem 3). He chooses special cases in which the restricted partition pairs can be transformed bijectively, by rearranging the parts, into linear partitions. Since he gives the generating function for restricted partition pairs (with certain bounds) in general and since, of course, the generating function for linear partitions (with certain bounds) is well-known, he obtains an identity. We do the same for cylindric partitions with an arbitrary number of rows. We systematically generalize all the identities that Burge derived by this method.

We give a detailed description of the method in the simplest case first, and then generalize. Consider cylindric partitions of shape (c^{r})/0/1 with entries between 0 and a. (As before, the notation (c^{r}) means the r-tuple (c, c, …, c). So we are considering cylindric partitions of rectangular shape, consisting of r rows and c columns, with a shift of 1 relating the first and last rows.) By reading such a cylindric partition along the columns, each column from top to bottom, and the columns in order from left to right, it is seen that π actually is a linear partition with rc parts that are between 0 and a. To be more precise, by (1.3b) and (1.3c) we have for any such cylindric partition π = (π_{ij}),

a ≥ π_{1,1} ≥ π_{2,1} ≥ ⋯ ≥ π_{r,1} ≥ π_{1,2} ≥ π_{2,2} ≥ ⋯ ≥ π_{r,2} ≥ π_{1,3} ≥ ⋯ ≥ π_{r,c} ≥ 0. (6.1)

Conversely, if π satisfies these inequalities, then it is a cylindric partition of shape (c^{r})/0/1 with entries between 0 and a. Since by Theorem 2 (with λ_{i} = c, μ_{i} = 0, d = 1, a_{i} = a, b_{i} = 0) we know the generating function ∑_{π} q^{n(π)} for these cylindric partitions, and since we also know the generating function for the linear partitions with rc parts all of which are between 0 and a (see [4, Theorem 3.1]), we obtain the identity

∑_{k_{1}+···+k_{r}=0} q^{\binom{r+1}{2} ∑_{i=1}^{r} k_{i}^{2}+(r+1) ∑_{i=1}^{r} ik_{i}}

det_{1≤s,t≤r} (q^{t(t−s−rk_{s})} [ \begin{matrix}a + c − k_{s} \\ c − s + t − (r + 1)k_{s} \end{matrix} ]) = [ \begin{matrix}a + rc \\ rc \end{matrix} ] . (6.2)

Burge’s doubly bounded Euler’s Theorem [12, p. 216] is the special case r = 2, a = M, c = N of (6.2).

For the next several paragraphs we fix d = 1. We now work with (α, β)-cylindric partitions instead of just (ordinary) cylindric partitions. The restrictions (3.1) allow more general shapes than just rectangular ones, and the restrictions (4.6) allow more general bounds where the same idea works to produce a q-binomial summation formula. We shall consider 9 such cases. For the “left border” μ of the shape we choose either the skew border μ = (1^{n}, 0^{r−n}) or the “shifted” border μ = (0^{n}, 1^{r−n}). For the “right border” λ of the shape we choose either the skew border λ = ((c + 1)^{m}, c^{r−m}) or the “shifted” border λ = (c^{m}, (c + 1)^{r−m}). We shall combine the choice μ = (1^{n}, 0^{r−n}) (skew border) with two different choices of upper bounds a, and the choice λ = ((c + 1)^{m}, c^{r−m}) (skew border) with two different choices of lower bounds b. Pairing the three choices (μ, a) “on the left” with the three choices (λ, b) “on the right”, we obtain 9 cases.

Since d = 1, the requirement that α_{j} ≡ α_{j mod d} for all j implies that all the α_{j} are identical. Let us therefore write α for α_{j} in what follows.

Case 1. Take λ = ((c + 1)^{m}, c^{r−m}), where 0 < m ≤ r, μ = (1^{n}, 0^{r−n}), where 0 ≤ n < r, d = 1, a_{i} = a − ∑_{k=1}^{i−1} β_{k} − α · χ(i ≤ n), b_{i} = − ∑_{k=1}^{i−1} β_{k} + α · χ(i ≥ m + 1). Here and in the sequel, χ(A)=1 if A is true and χ(A)=0 otherwise. Since we want to apply Theorem 3, we require that ∑_{k=1}^{r} β_{k} < r + α. Note that (3.1) is satisfied for this λ and this μ, and the choice of a_{i} guarantees that (4.6a) is satisfied. On the other hand, (4.6c) is satisfied if and only if ∑_{k=1}^{r} β_{k} ≤ 1 + α. Similarly, with this choice of b_{i} the restriction (4.6b) is satisfied, while (4.6d) holds if and only if ∑_{k=1}^{r} β_{k} ≤ 1 + α. Thus, if ∑_{k=1}^{r} β_{k} ≤ min{1+α, r−1+α} then the generating function ∑_{π} q^{n(π)}, where the sum is over all (α, β)-cylindric partitions π of shape ((c+1)^{m}, c^{r−m})/(1^{n}, 0^{r−n})/1, in which the first entry in row i is at most a_{i} = a − ∑_{k=1}^{i−1} β_{k} − α · χ(i ≤ n) and the last entry in row i is at least b_{i} = − ∑_{k=1}^{i−1} β_{k} + α · χ(i ≥ m + 1), equals

∑_{k_{1}+···+k_{r}=0} q^{− ∑_{k=1}^{r} β_{k}c(r−k)−∑_{k=1}^{m} β_{k}(m−k)+∑_{k=1}^{n} β_{k}(n−k)+αr\binom{r+1}{2}−ncα}

q^{(r+1)(α+∑_{k=1}^{r}(1−β_{k})) ∑_{i=1}^{r} k_{i}^{2}/2+(r+1) ∑_{i=1}^{r} ik_{i}}

det_{1≤s,t≤r} (q^{(χ(t≤n)−t)(s−t+k_{s}(α+∑_{k=1}^{r}(1−β_{k})))}

[ \begin{matrix}a + c + χ(s ≤ m) − χ(t ≤ n) − αc − k_{s}(1 − α + ∑_{k=1}^{r} β_{k}) \\ c + χ(s ≤ m) − χ(t ≤ n) − s + t − (r + 1)k_{s} \end{matrix} ] ). (6.3)

On the other hand, if, here and in the sequel, we make the convention β_{k} ≡ β_{k mod r}, we have by (4.5b) and (4.5c) and by the row bounds for rows n + 1 and m,

a − ∑_{k=1}^{n} β_{k} ≥ π_{n+1,1} ≥ π_{n+2,1} + β_{n+1} ≥ ⋯ ≥ π_{r,1} + ∑_{k=n+1}^{r−1} β_{k} ≥

≥ π_{1,2} + ∑_{k=n+1}^{r} β_{k} ≥ π_{2,2} + ∑_{k=n+1}^{r+1} β_{k} ≥

⋯ ≥ π_{i,j} + ∑_{k=n+1}^{rj−r+i−1} β_{k} ≥ ⋯ ≥ π_{m,c+1} + ∑_{k=n+1}^{rc+m−1} β_{k} ≥ ∑_{k=n+1}^{rc} β_{k}. (6.4)

Conversely, let (π_{i,j}) be an array that satisfies (6.4). Let us also assume that ∑_{k=1}^{r} β_{k} ≥ α. Then it can be seen that π must be an (α, β)-cylindric partition of shape ((c + 1)^{m}, c^{r−m})/(1^{n}, 0^{r−n})/1, in which the first entry in row i is at most a_{i} = a − ∑_{k=1}^{i−1} β_{k} − α · χ(i ≤ n) and the last entry in row i is at least b_{i} = − ∑_{k=1}^{i−1} β_{k} + α · χ(i ≥ m + 1). In fact, (4.5b) and (4.5c) are trivially satisfied. For establishing (4.5a), we observe that by (6.4) we have

π_{i,j} + ∑_{k=n+1}^{rj−r+i−1} β_{k} ≥ π_{i,j+1} + ∑_{k=n+1}^{rj+i−1} β_{k} ,

and since we assume ∑_{k=1}^{r} β_{k} ≥ α, we get

π_{i,j} ≥ π_{i,j+1} + ∑_{k=1}^{r} β_{k} ≥ π_{i,j+1} + α,

which is (4.5a). Thus π is indeed an (α, β)-cylindric partition of shape ((c + 1)^{m}, c^{r−m})/(1^{n}, 0^{r−n})/1. Now for n + 1 ≤ i ≤ r, (6.4) implies that π_{i,1} + ∑_{k=n+1}^{i−1} β_{k} ≤ a − ∑_{k=1}^{n} β_{k}. Equivalently, for n + 1 ≤ i ≤ r the first entry in row i is at most a−∑_{k=1}^{i−1} β_{k}. Since we are assuming that ∑_{k=1}^{r} β_{k} ≥ α, we may also conclude π_{i,2} ≤ a − ∑_{k=1}^{r+i−1} β_{k} ≤ a − ∑_{k=1}^{i−1} β_{k} − α, in particular for 1 ≤ i ≤ n. This establishes that indeed the first entry in row i is at most a_{i} = a−∑_{k=1}^{i−1} β_{k} −α·χ(i ≤ n), for all i. Similarly, we see that the last entry in row i is at least b_{i} = − ∑_{k=1}^{i−1} β_{k}+α·χ(i ≥ m+1).

Therefore, by setting π̄_{i,j} = π_{i,j} + ∑_{k=1}^{rj+i−1} β_{k}, from the array π = (π_{i,j}) we obtain an array π̄ = (π̄_{i,j}) with

a + ∑_{k=1}^{r} β_{k} ≥ π̄_{n+1,1} ≥ π̄_{n+2,1} ≥ ⋯ ≥ π̄_{r,1} ≥

≥ π̄_{1,2} ≥ π̄_{2,2} ≥ ⋯ ≥ π̄_{m,c+1} ≥ (c + 1) ∑_{k=1}^{r} β_{k} .

Thus π̄ = (π̄_{i,j}) is actually a linear partition with rc + m − n entries, all of which are between (c + 1) ∑_{k=1}^{r} β_{k} and a + ∑_{k=1}^{r} β_{k}. Hence, by using the partition theorem [4, Theorem 3.1] again, the generating function ∑_{π̄} q^{n(π̄)} for these arrays π̄ is easily computed. So, by the relation π̄ = π_{i,j} + ∑_{k=1}^{rj+i−1} β_{k} we finally obtain that the generating function ∑_{π} q^{n(π)} for (α, β)-cylindric partitions π of shape ((c + 1)^{m}, c^{r−m})/(1^{n}, 0^{r−n})/1, in which the first entry in row i is at most a_{i} = a−∑_{k=1}^{i−1} β_{k}− α · χ(i ≤ n) and the last entry in row i is at least b_{i} = − ∑_{k=1}^{i−1} β_{k} + α · χ(i ≥ m + 1) also equals

q^{(c+1)(rc+m−n) ∑_{k=1}^{r} β_{k}−∑_{l=r+n}^{rc+r+m−1} ∑_{k=1}^{l} β_{k}} [ \begin{matrix}a − c ∑_{k=1}^{r} β_{k} + rc + m − n \\ rc + m − n \end{matrix} ] . (6.5)

Equating (6.3) and (6.5) we obtain a q-binomial identity. Replacing a by a + αc and ∑_{k=1}^{r} β_{k} by ε+α, and further simplifying, we may write this identity in the following form.

Theorem 8. For 0 < m ≤ r, 0 ≤ n < r, 0 ≤ ε ≤ min{1, r − 1}, we have

∑_{k_{1}+···+k_{r}=0} q^{(r+1)(r−ε) ∑_{i=1}^{r} k_{i}^{2}/2+(r+1) ∑_{i=1}^{r} ik_{i}}

det_{1≤s,t≤r} (q^{(χ(t≤n)−t)(s−t+(r−ε)k_{s})} [ \begin{matrix}a + c + χ(s ≤ m) − χ(t ≤ n) − (ε + 1)k_{s} \\ c + χ(s ≤ m) − χ(t ≤ n) − s + t − (r + 1)k_{s} \end{matrix} ] )

= q^{(r\binom{c+1}{2}−nc)ε} [ \begin{matrix}a + c(r − ε) + m − n \\ rc + m − n \end{matrix} ] . (6.6)

□

Identity (6.2) (with c replaced by c + 1) is the special case ε = 0, m = r, n = 0 of (6.6). So, Burge’s doubly bounded Euler’s Theorem [12, p. 216] for his partition pairs generating function P (N, M, 1, 2, 1, 1) is the special case r = 2, ε = 0, m = 2, n = 0, a = M, c = N − 1 of (6.6). Moreover, his summation at the top of p. 221 of [12] is the special case r = 2, ε = 0, m = 1, n = 0, a = M, c = N of (6.6).

Case 2. Take the same choices for λ, μ, b as in Case 1 (and d = 1 of course), i.e., λ = ((c + 1)^{m}, c^{r−m}), where 0 < m ≤ r, μ = (1^{n}, 0^{r−n}), where 0 ≤ n < r, d = 1, and b_{i} = − ∑_{k=1}^{i−1} β_{k} +α·χ(i ≥ m+1), but take a_{i} = a+∑_{k=1}^{i−1}(1−β_{k})−α·χ(i ≤ n). Then it is easily checked that (3.1) and (4.6) are satisfied. If we assume ∑_{k=1}^{r} β_{k} ≥ α + n it can be seen that our (α, β)-cylindric partitions of shape ((c+1)^{m}, c^{r−m})/(1^{n}, 0^{r−n})/1, in which the first entry in row i is at most a_{i} = a+∑_{k=1}^{i−1}(1−β_{k})−α·χ(i ≤ n) and the last entry in row i is at least b_{i} = − ∑_{k=1}^{i−1} β_{k} + α · χ(i ≥ m + 1) can be characterized by

a + ∑_{k=1}^{n}(1 − β_{k}) ≥ π_{n+1,1} ≥ π_{n+2,1} + β_{n+1} ≥ ⋯ ≥ π_{r,1} + ∑_{k=n+1}^{r−1} β_{k} ≥

≥ π_{1,2} + ∑_{k=n+1}^{r} β_{k} ≥ π_{2,2} + ∑_{k=n+1}^{r+1} β_{k} ≥

⋯ ≥ π_{i,j} + ∑_{k=n+1}^{rj−r+i−1} β_{k} ≥ ⋯ ≥ π_{m,c+1} + ∑_{k=n+1}^{rc+m−1} β_{k} ≥ ∑_{k=n+1}^{rc} β_{k}. (6.7)

In fact, what has to be checked is that an array (π_{i,j}) satisfying (6.7) “obeys the upper row bounds a_{i} = a+∑_{k=1}^{i−1}(1−β_{k})−α·χ(i ≤ n)”, i.e., that π_{i,1} ≤ a_{i} for n+1 ≤ i ≤ r, and that π_{i,2} ≤ a_{i} for 1 ≤ i ≤ n. Everything else was already checked in Case 1. Now, because of (6.7) we have a + ∑_{k=1}^{n}(1 − β_{k}) ≥ π_{i,1} + ∑_{k=n+1}^{i−1} β_{k} for n + 1 ≤ i ≤ r, or equivalently, π_{i,1} ≤ a + ∑_{k=1}^{i−1}(1 − β_{k}) − (i − n − 1). This implies the first assertion. Likewise, because of (6.7) we have a+∑_{k=1}^{n}(1−β_{k}) ≥ π_{i,2}+∑_{k=n+1}^{r+i−1} β_{k} for 1 ≤ i ≤ n. Using our assumption that ∑_{k=1}^{r} β_{k} ≥ α + n, we conclude

π_{i,2} ≤ a + n − ∑_{k=1}^{r+i−1} β_{k} ≤ a + ∑_{k=1}^{i−1}(1 − β_{k}) − α − (i − 1),

which implies the second assertion.

As in Case 1, we may equate the generating function for our (α, β)-cylindric partitions that results from Theorem 3 with the expression that results from the fact that by (6.7) these (α, β)-cylindric partitions are essentially linear partitions. The identity that we obtain, after again replacing a by a + αc and ∑_{k=1}^{r} β_{k} by ε + α, is the following.

Theorem 9. For 0 < m ≤ r, 0 ≤ n ≤ 1, n ≤ ε ≤ min{1, r − 1} we have

∑_{k_{1}+···+k_{r}=0} q^{(r+1)(r−ε) ∑_{i=1}^{r} k_{i}^{2}/2+(r+1) ∑_{i=1}^{r} ik_{i}}

det_{1≤s,t≤r} (q^{(χ(t≤n)−t)(s−t+(r−ε)k_{s})} [ \begin{matrix}a + c + χ(s ≤ m) − χ(t ≤ n) − 1 + t − (ε + 1)k_{s} \\ c + χ(s ≤ m) − χ(t ≤ n) − s + t − (r + 1)k_{s} \end{matrix} ] )

= q^{(r\binom{c+1}{2}−nc)ε} [ \begin{matrix}a + c(r − ε) + m \\ rc + m − n \end{matrix} ] . (6.8)

□

Case 3. Take the same choices for λ, b as in Case 1 (and d = 1), i.e., λ = ((c + 1)^{m}, c^{r−m}), where 0 < m ≤ r, d = 1, b_{i} = − ∑_{k=1}^{i−1} β_{k} + α · χ(i ≥ m + 1), but take μ = (0^{n}, 1^{r−n}), where 0 ≤ n < r and a_{i} = a − ∑_{k=1}^{i−1} β_{k} − α · χ(i ≥ n + 1). Also, we assume here that α ≤ ∑_{k=1}^{r} β_{k} ≤ min{1 + α, r − 1 + α}. This guarantees that (3.1) and (4.6) hold, as is easily checked. Again the (α, β)-cylindric partitions under consideration turn out to be essentially linear partitions. However, there is a little detail that is different from the previous two cases. Namely, our choices force the entries in the first column to equal the upper row bounds. More precisely, since μ_{n} = μ_{n+1} − 1, Theorem 3 (see the last paragraph in the statement of Theorem 3) gives the generating function for (α, β)-cylindric partitions that also satisfy (4.5b) with i = n, j = μ_{n} + 1 = μ_{n+1} = 1, and π_{n+1,1} = a_{n+1} + α = a − ∑_{k=1}^{n} β_{k}, i.e., π_{n,1} ≥ a − ∑_{k=1}^{n} β_{k} + β_{n} = a − ∑_{k=1}^{n−1} β_{k}. From this and (4.5b) we have

a_{1} = a ≥ π_{1,1} ≥ π_{2,1} + β_{1} ≥ ⋯ ≥ π_{n,1} + ∑_{k=1}^{n−1} β_{k} ≥ a.

Hence, π_{i,1} = a − ∑_{k=1}^{i−1} β_{k} for 1 ≤ i ≤ n, as asserted. It can be seen that our (α, β)- cylindric partitions π of shape ((c + 1)^{m}, c^{r−m})/(0^{n}, 1^{r−n})/1, in which the first entry in row i is at most a_{i} = a − ∑_{k=1}^{i−1} β_{k} − α · χ(i ≥ n + 1) and the last entry in row i is at least b_{i} = − ∑_{k=1}^{i−1} β_{k} + α · χ(i ≥ m + 1) can be characterized by

π_{i,1} = a − ∑_{k=1}^{i−1} β_{k},  for 1 ≤ i ≤ n,
a − α ≥ π_{1,2} ≥ π_{2,2} + β_{1} ≥ ⋯ ≥
⋯ ≥ π_{i,j} + ∑_{k=1}^{rj−2r+i−1} β_{k} ≥ ⋯ ≥ π_{m,c+1} + ∑_{k=1}^{rc−r+m−1} β_{k} ≥ ∑_{k=1}^{rc−r} β_{k}. (6.9)

In a manner similar to Case 1 and 2 we finally obtain the following identity.

Theorem 10. For 0 < m ≤ r, 0 ≤ n < r, 0 ≤ ε ≤ min{1, r − 1} we have

∑_{k_{1}+···+k_{r}=0} q^{(r+1)(r−ε) ∑_{i=1}^{r} k_{i}^{2}/2+(r+1) ∑_{i=1}^{r} ik_{i}}

det_{1≤s,t≤r} (q^{(χ(t≥n+1)−t)(s−t+(r−ε)k_{s})} [ \begin{matrix}a + c + χ(s ≤ m) − χ(t ≥ n + 1) − (ε + 1)k_{s} \\ c + χ(s ≤ m) − χ(t ≥ n + 1) − s + t − (r + 1)k_{s} \end{matrix} ] )

= q^{an+rε\binom{c}{2}} [ \begin{matrix}a + (r − ε)(c − 1) + m \\ r(c − 1) + m \end{matrix} ] . (6.10)

□

Case 4. Take λ = ((c + 1)^{m}, c^{r−m}), where 0 < m ≤ r, μ = (1^{n}, 0^{r−n}), where 0 ≤ n < r, d = 1, a_{i} = a−∑_{k=1}^{i−1} β_{k} −α·χ(i ≤ n), b_{i} = ∑_{k=1}^{i−1}(1−β_{k})+α·χ(i ≥ m+1). We assume that r+α−m ≤ ∑_{k=1}^{r} β_{k} ≤ min{1+α, r−1+α}. Then we perform analogous considerations as before. Note that the inequality r + α − m ≤ ∑_{k=1}^{r} β_{k} is needed so that an array of the type (6.4), but with lower bound m − 1 + ∑_{k=n+1}^{rc} β_{k} instead of ∑_{k=n+1}^{rc} β_{k}, automatically satisfies π_{i,c} ≥ ∑_{k=1}^{i−1}(1 − β_{k}) + α for m + 1 ≤ i ≤ r. We finally obtain the following identity.

Theorem 11. For r − 1 ≤ m ≤ r, 0 ≤ n < r, r − m ≤ ε ≤ min{1, r − 1} we have

∑_{k_{1}+···+k_{r}=0} q^{(r+1)(r−ε) ∑_{i=1}^{r} k_{i}^{2}/2}

det_{1≤s,t≤r} (q^{(χ(t≤n)−t)(r−ε)k_{s}} [ \begin{matrix}a + c + χ(s ≤ m) − χ(t ≤ n) − s + 1 − (ε + 1)k_{s} \\ c + χ(s ≤ m) − χ(t ≤ n) − s + t − (r + 1)k_{s} \end{matrix} ] )

= q^{rc(m−1)−c\binom{r}{2}+\binom{m−n}{2}+(r\binom{c+1}{2}−nc)ε} [ \begin{matrix}a + c(r − ε) + 1 − n \\ rc + m − n \end{matrix} ] . (6.11)

□

Case 5. Take the same choices for λ, μ, b as in Case 4 (and d = 1 of course), i.e., λ = ((c + 1)^{m}, c^{r−m}), where 0 < m ≤ r, μ = (1^{n}, 0^{r−n}), where 0 ≤ n < r, d = 1, b_{i} = ∑_{k=1}^{i−1}(1 − β_{k}) + α · χ(i ≥ m + 1), but take a_{i} = a + ∑_{k=1}^{i−1}(1 − β_{k}) − α · χ(i ≤ n). We assume that max{α + n, r + α − m} ≤ ∑_{k=1}^{r} β_{k} < r. Performing analogous considerations as before, we finally arrive at the following identity.

Theorem 12. For 0 < m ≤ r, 0 ≤ n < r, max{n, r − m} ≤ ε < r we have

∑_{k_{1}+···+k_{r}=0} q^{(r+1)(r−ε) ∑_{i=1}^{r} k_{i}^{2}/2}

det_{1≤s,t≤r} (q^{(χ(t≤n)−t)(r−ε)k_{s}} [ \begin{matrix}a + c + χ(s ≤ m) − χ(t ≤ n) − s + t − (ε + 1)k_{s} \\ c + χ(s ≤ m) − χ(t ≤ n) − s + t − (r + 1)k_{s} \end{matrix} ] )

= q^{rc(m−1)−c\binom{r}{2}+\binom{m−n}{2}+(r\binom{c+1}{2}−nc)ε} [ \begin{matrix}a + c(r − ε) + 1 \\ rc + m − n \end{matrix} ] . (6.12)

□

Burge’s summation for P (N, M, 1, 2, 0, 2) [12, p. 216] is the special case r = 2, ε = 0, m = 2, n = 0, a = M, c = N − 1 of (6.12), and his summation for P (N, M, 1, 2, 0, 1) [12, p. 217] is the special case r = 2, ε = 1, m = 2, n = 0, a = M, c = N − 1 of (6.12).

Case 6. Take the same choices for λ, b as in Case 4 (and d = 1), i.e., λ = ((c + 1)^{m}, c^{r−m}), where 0 < m ≤ r, d = 1, b_{i} = ∑_{k=1}^{i−1}(1 − β_{k}) + α · χ(i ≥ m + 1), but take μ = (0^{n}, 1^{r−n}), where 0 ≤ n < r and a_{i} = a − ∑_{k=1}^{i−1} β_{k} − α · χ(i ≥ n + 1). We assume that r + α − m ≤ ∑_{k=1}^{r} β_{k} ≤ min{1 + α, r − 1 + α}. Performing analogous considerations as before we arrive at the following identity.

Theorem 13. For r − 1 ≤ m ≤ r, 0 ≤ n < r, r − m ≤ ε ≤ min{1, r − 1} we have

∑_{k_{1}+···+k_{r}=0} q^{(r+1)(r−ε) ∑_{i=1}^{r} k_{i}^{2}/2}

det_{1≤s,t≤r} (q^{(χ(t≥n+1)−t)(r−ε)k_{s}} [ \begin{matrix}a + c + χ(s ≤ m) − χ(t ≥ n + 1) − s + 1 − (ε + 1)k_{s} \\ c + χ(s ≤ m) − χ(t ≥ n + 1) − s + t − (r + 1)k_{s} \end{matrix} ] )

= q^{an+(c−1)(r(m−1)−\binom{r}{2})+\binom{m}{2}−\binom{n}{2}+r\binom{c}{2}ε} [ \begin{matrix}a + (r − ε)(c − 1) + 1 \\ r(c − 1) + m \end{matrix} ] . (6.13)

□

Case 7. Take λ = (c^{m}, (c + 1)^{r−m}), where 0 < m ≤ r, μ = (1^{n}, 0^{r−n}), where 0 ≤ n < r, d = 1, a_{i} = a − ∑_{k=1}^{i−1} β_{k} − α · χ(i ≤ n), b_{i} = − ∑_{k=1}^{i−1} β_{k} + α · χ(i ≤ m). We assume that α ≤ ∑_{k=1}^{r} β_{k} ≤ min{1 + α, r − 1 + α}. Similar considerations as above give rise to the following identity.

Theorem 14. For 0 < m ≤ r, 0 ≤ n < r, 0 ≤ ε ≤ min{1, r − 1} we have

∑_{k_{1}+···+k_{r}=0} q^{(r+1)(r−ε) ∑_{i=1}^{r} k_{i}^{2}/2+(r+1) ∑_{i=1}^{r} ik_{i}}

det_{1≤s,t≤r} (q^{(χ(t≤n)−t)(s−t+(r−ε)k_{s})} [ \begin{matrix}a + c + χ(s ≥ m + 1) − χ(t ≤ n) − (ε + 1)k_{s} \\ c + χ(s ≥ m + 1) − χ(t ≤ n) − s + t − (r + 1)k_{s} \end{matrix} ] )

= q^{(r\binom{c}{2}−n(c−1))ε} [ \begin{matrix}a + (r − ε)(c − 1) + r − n \\ rc − n \end{matrix} ] . (6.14)

□

Case 8. Take the same choices for λ, μ, b as in Case 7 (and d = 1 of course), i.e., λ = (c^{m}, (c + 1)^{r−m}), where 0 < m ≤ r, μ = (1^{n}, 0^{r−n}), where 0 ≤ n < r, d = 1, b_{i} = − ∑_{k=1}^{i−1} β_{k} + α · χ(i ≤ m), but take a_{i} = a + ∑_{k=1}^{i−1}(1 − β_{k}) − α · χ(i ≤ n). We assume that α + n ≤ ∑_{k=1}^{r} β_{k} ≤ min{1 + α, r − 1 + α}. Similarly to above we obtain the following identity.

Theorem 15. For 0 < m ≤ r, 0 ≤ n ≤ 1, n ≤ ε ≤ min{1, r − 1} we have

∑_{k_{1}+···+k_{r}=0} q^{(r+1)(r−ε) ∑_{i=1}^{r} k_{i}^{2}/2+(r+1) ∑_{i=1}^{r} ik_{i}}

det_{1≤s,t≤r} (q^{(χ(t≤n)−t)(s−t+(r−ε)k_{s})} [ \begin{matrix}a + c + χ(s ≥ m + 1) − χ(t ≤ n) − 1 + t − (ε + 1)k_{s} \\ c + χ(s ≥ m + 1) − χ(t ≤ n) − s + t − (r + 1)k_{s} \end{matrix} ] )

= q^{(r\binom{c}{2}−n(c−1))ε} [ \begin{matrix}a + (r − ε)(c − 1) + r \\ rc − n \end{matrix} ] . (6.15)

□

Case 9. Finally, take the same choices for λ, b as in Case 7 (and d = 1), i.e., λ = (c^{m}, (c + 1)^{r−m}), where 0 < m ≤ r, d = 1, b_{i} = − ∑_{k=1}^{i−1} β_{k} + α · χ(i ≤ m), but take μ = (0^{n}, 1^{r−n}), where 0 ≤ n < r and a_{i} = a − ∑_{k=1}^{i−1} β_{k} − α · χ(i ≥ n + 1). We assume that α ≤ ∑_{k=1}^{r} β_{k} ≤ min{1 + α, r − 1 + α}. The identity that is obtained in this case is the following.

Theorem 16. For 0 < m ≤ r, 0 ≤ n < r, 0 ≤ ε ≤ min{1, r − 1} we have

∑_{k_{1}+···+k_{r}=0} q^{(r+1)(r−ε) ∑_{i=1}^{r} k_{i}^{2}/2+(r+1) ∑_{i=1}^{r} ik_{i}}

det_{1≤s,t≤r} (q^{(χ(t≥n+1)−t)(s−t+(r−ε)k_{s})} [ \begin{matrix}a + c + χ(s ≥ m + 1) − χ(t ≥ n + 1) − (ε + 1)k_{s} \\ c + χ(s ≥ m + 1) − χ(t ≥ n + 1) − s + t − (r + 1)k_{s} \end{matrix} ] )

= q^{an+r\binom{c−1}{2}ε} [ \begin{matrix}a + (r − ε)(c − 2) + r \\ r(c − 1) \end{matrix} ] . (6.16)

□

Burge’s summation for P (N, M, 0, 3, 1, 1) [12, p. 216] is the special case r = 2, ε = 0, m = n = 1, a = M, c = N of (6.16).

Another choice of d that gives rise to identities is d = 0. So let us fix d = 0 for the next several paragraphs. As before, we first explain the method in the simplest case (which is again the rectangular shape), and then generalize. Consider cylindric partitions of shape (c^{r})/0/0 with entries between 0 and a. By (1.3b) and (1.3c) we have

π_{1,j} ≥ π_{2,j} ≥ ⋯ ≥ π_{r,j} ≥ π_{1,j}, for any j,

hence the entries in each column are identical. By (1.3a) we have

a ≥ π_{1,1} ≥ π_{1,2} ≥ ⋯ ≥ π_{1,c} ≥ 0.

So by adding up the entries in each column we obtain (rπ_{1,1}, rπ_{1,2}, …, rπ_{1,c}), which is a linear partition with c parts, all of which are divisible by r and lie between 0 and ra. This correspondence is clearly a bijection. The generating function for these linear partitions is [ \begin{matrix}a+c \\ c \end{matrix} ]_{q^{r}}. On the other hand, Theorem 2 (with λ_{i} = c, μ_{i} = 0,

d = 0, a_{i} = a, b_{i} = 0) tells us the generating function for these cylindric partitions. Since these are in bijection with the corresponding linear partitions, we have proved

∑_{k_{1}+···+k_{r}=0} q^{r^{2} ∑_{i=1}^{r} k_{i}^{2}/2+r ∑_{i=1}^{r} ik_{i}} det_{1≤s,t≤r} (q^{t(t−s−rk_{s})} [ \begin{matrix}a + c \\ c − s + t − rk_{s} \end{matrix} ])

= [ \begin{matrix}a + c \\ c \end{matrix} ]_{q^{r}} . (6.17)

Burge’s summation for P (N, M, 1, 1, 1, 1) [12, p. 217] is the special case r = 2, a = M, c = N of (6.17).

Given the rectangular shape (c^{r})/0/0, the restrictions (4.6) allow more general row bounds for which the same idea works to produce a q-binomial identity. We consider the choices a_{i} = a or a + i and b_{i} = 0 or i. In the theorem below we list the identities that result from these 2 × 2 possibilities. The first comes from a_{i} = a, b_{i} = 0 (and hence repeats (6.17)), the second comes from a_{i} = a + i, b_{i} = 0, the third comes from a_{i} = a, b_{i} = i, and the fourth comes from a_{i} = a + i, b_{i} = i.

Theorem 17. There hold the identities

∑_{k_{1}+···+k_{r}=0} q^{r^{2} ∑_{i=1}^{r} k_{i}^{2}/2+r ∑_{i=1}^{r} ik_{i}} det_{1≤s,t≤r} (q^{t(t−s−rk_{s})} [ \begin{matrix}a + c \\ c − s + t − rk_{s} \end{matrix} ])

= [ \begin{matrix}a + c \\ c \end{matrix} ]_{q^{r}} . (6.18)

∑_{k_{1}+···+k_{r}=0} q^{r^{2} ∑_{i=1}^{r} k_{i}^{2}/2+r ∑_{i=1}^{r} ik_{i}} det_{1≤s,t≤r} (q^{t(t−s−rk_{s})} [ \begin{matrix}a + c + t \\ c − s + t − rk_{s} \end{matrix} ])

= [ \begin{matrix}a + c + 1 \\ c \end{matrix} ]_{q^{r}} . (6.19)

∑_{k_{1}+···+k_{r}=0} q^{r^{2} ∑_{i=1}^{r} k_{i}^{2}/2−r ∑_{i=1}^{r} ik_{i}} det_{1≤s,t≤r} ([ \begin{matrix}a + c − s \\ c − s + t − rk_{t} \end{matrix} ])

= q^{c\binom{r}{2}} [ \begin{matrix}a + c − r \\ c \end{matrix} ]_{q^{r}} . (6.20)

∑_{k_{1}+···+k_{r}=0} q^{r^{2} ∑_{i=1}^{r} k_{i}^{2}/2−r ∑_{i=1}^{r} ik_{i}} det_{1≤s,t≤r} ([ \begin{matrix}a + c − s + t \\ c − s + t − rk_{t} \end{matrix} ])

= q^{c\binom{r}{2}} [ \begin{matrix}a + c + 1 − r \\ c \end{matrix} ]_{q^{r}} . (6.21)

□

As was already remarked, Burge’s summation for P (N, M, 1, 1, 1, 1) [12, p. 217] is the special case r = 2, a = M, c = N of (6.18). Moreover, his summation for P (N, M, 1, 1, 0, 2) [12, p. 217] is the special case r = 2, a = M, c = N of (6.21).

One could also allow more general shapes. Let us consider cylindric partitions of shape ((c + 1)^{m}, c^{r−m−m′}, (c + 1)^{m′})/(1^{n}, 0^{r−n−n′}, 1^{n′})/0 with entries between 0 and a. For convenience we assume that m + m′ < r and n + n′ > 0; i.e., that neither column 1 nor column c + 1 is completely filled. Of course, the considerations above show that for each of these cylindric partitions the entries in column j, 2 ≤ j ≤ c, have to be identical. Moreover, Theorem 2 with λ_{i} = c + 1 − χ(m + 1 ≤ i ≤ r − m′), μ_{i} = 1 − χ(n + 1 ≤ i ≤ r − n), d = 0, a_{i} = a, b_{i} = 0 would not give the generating function for all these cylindric partitions, but only for those where all the r − n − n′ entries in column 1 are equal a and all the m + m′ entries in column c + 1 are equal 0. This is because of the conventions for λ_{i} = λ_{i+1} − 1 and μ_{i} = μ_{i+1} − 1 that are explained in the last paragraph of Theorem 2. Thus (4.1) with the above choices of parameters is equal to q^{(r−n−n′)a} times the generating function for linear partitions with c − 1 parts, all divisible by r and between 0 and ra, and we obtain the following identity.

Theorem 18. Let m, m′, n, n′ be nonnegative integers with 0 ≤ m + m′ < r and 0 < n + n′ ≤ r. Then

∑_{k_{1}+···+k_{r}=0} q^{r^{2} ∑_{i=1}^{r} k_{i}^{2}/2+r ∑_{i=1}^{r} ik_{i}} det_{1≤s,t≤r} (q^{(χ(n+1≤t≤r−n′)+t)(t−s−rk_{s})}

[ \begin{matrix}a + c − χ(m + 1 ≤ s ≤ r − m′) + χ(n + 1 ≤ t ≤ r − n′) \\ c − χ(m + 1 ≤ s ≤ r − m′) + χ(n + 1 ≤ t ≤ r − n′) − s + t − rk_{s} \end{matrix} ] )

= q^{(r−n−n′)a} [ \begin{matrix}a + c − 1 \\ c − 1 \end{matrix} ]_{q^{r}} . (6.22)

□

Note that (6.22) covers (6.18)=(6.17) (set m = m′ = 0, n + n′ = r and replace c by c + 1).

The idea for d = 1 of obtaining identities by merging the entries of appropriate cylindric partitions into linear partitions (or something equivalent) can be extended. Again, we begin with the simplest example, and then generalize.

Consider cylindric partitions of shape ((md)^{r})/0/d with entries between 0 and a, where r, d, and m are arbitrary positive integers. Figure 5.a shows schematically the shape of such a cylindric partition, where m = 4. The arrows indicate the directions of (weak) decrease of entries. The last row is marked with a dotted line, and its shift that dominates the top row is represented as a dotted row on top of the shape. We may cut such a cylindric partition into m rectangular pieces as indicated in Figure 5.a (where m = 4). Then we take the m pieces, which were originally arranged horizontally, and rearrange them vertically, as in Figure 5.b. It is easy to see that again there is decrease along the directions that are indicated by the arrows. Moreover, the last column dominates the first as indicated in Figure 5.b. Finally, we reflect the new array in some antidiagonal line to obtain an array as in Figure 5.c. As suggested by the picture, this array is a cylindric partition of shape ((mr)^{d})/0/r.

..^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..}^{..} ..  ..^{..}^{..}^{..}^{..}^{..} → ↓ \underbrace{d} r\{_{a. cylindric partition of shape ((md)^{r})/0/d}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..} ..  .._{..}_{..}_{..}_{..}_{..}_{..}_{..}_{..} → ↓ \underbrace{r} d\{_{c. after reflection, a cylindric partition of shape ((mr)^{d})/0/r}  ................................................................ ................................................................ ................_{..}^{..} → ↓ \underbrace{d} r\{_{b. pasted together differently}

Figure 5

If we apply the same procedure to the final array we recover our original array. Thus, cylindric partitions of shape ((md)^{r})/0/d with entries between 0 and a are in bijection with cylindric partitions of shape ((mr)^{d})/0/r with entries between 0 and a. (Note that r and d are simply interchanged.) Hence, their generating functions must be the same. So, from this observation and Theorem 2 we obtain the identity

∑_{k_{1}+···+k_{r}=0} q^{(r+d)r ∑_{i=1}^{r} k_{i}^{2}/2+(r+d) ∑_{i=1}^{r} ik_{i}}

det_{1≤s,t≤r} (q^{t(t−s−rk_{s})} [ \begin{matrix}a + md − dk_{s} \\ md − s + t − (r + d)k_{s} \end{matrix} ])

= ∑_{l_{1}+···+l_{d}=0} q^{(r+d)d ∑_{i=1}^{d} l_{i}^{2}/2+(r+d) ∑_{i=1}^{d} il_{i}}

det_{1≤s,t≤d} (q^{t(t−s−dl_{s})} [ \begin{matrix}a + mr − rl_{s} \\ mr − s + t − (r + d)l_{s} \end{matrix} ])  (6.23)

Next we want to apply the same technique to (α, β)-cylindric partitions of “arbitrary” shape. What we shall finally obtain is an extension of (6.23), namely Theorem 19 below. Let λ, μ be shapes contained in the rectangle (d^{r}) (i.e., λ_{i} ≤ d and μ_{i} ≤ d for i = 1, 2, …, r). It is not difficult to see that the above procedure gives a correspondence between (α, β)-cylindric partitions π of shape λ + ((md)^{r})/μ/d and (β, α)-cylindric partitions π̄ of shape λ′ + ((mr)^{d})/μ′/r. (The notation λ + ((md)^{r}) means the vector (λ_{1} + md, λ_{2} + md, …, λ_{r} + md), etc. As usual, λ′ denotes the partition conjugate to λ, cf. [33, p. 2].) Note that here α and β are interchanged, as are r and d, while λ and μ are replaced by their conjugates. In particular, the relation between entries of π and π̄ is

π_{i,j} ⟷ π̄_{j−⌊(j−1)/d⌋d,i+⌊(j−1)/d⌋r} .  (6.24)

Since we are dealing with (α, β)- and (β, α)-cylindric partitions, we also have to consider non-trivial bounds on the entries. This makes the situation a little bit more delicate.

We shall see that (α, β)-cylindric partitions of shape λ + ((md)^{r})/μ/d where the first entry in row i is at most a − ∑_{k=1}^{i−1} β_{k} − ∑_{k=1}^{μ_{i}} α_{k} and the last entry in row i is at least − ∑_{k=1}^{i−1} β_{k} − ∑_{k=1}^{λ_{i}−1} α_{k} are (under suitable conditions) in bijection with (β, α)- cylindric partitions of shape λ′+((mr)^{d})/μ′/r where the first entry in row i is at most a − ∑_{k=1}^{i−1} α_{k} − ∑_{k=1}^{μ′_{i}} β_{k} and the last entry in row i is at least − ∑_{k=1}^{i−1} α_{k} − ∑_{k=1}^{λ′_{i}−1} β_{k}. Observe that also with the bounds, α and β, r and d are interchanged, and λ and μ are replaced by their conjugates.

Since we want to apply Theorem 3 to compute the corresponding generating functions, we have to check the relevant conditions. First, we have to require −d < ∑_{k=1}^{r} β_{k}−∑_{k=1}^{d} α_{k} < r, and because of (4.6c) and (4.6d), −1 ≤ ∑_{k=1}^{r} β_{k}−∑_{k=1}^{d} α_{k} ≤ 1. So, combining these conditions, we assume that

max{−1, −d + 1} ≤ ∑_{k=1}^{r} β_{k} − ∑_{k=1}^{d} α_{k} ≤ min{1, r − 1}.

Next, we investigate when an (α, β)-cylindric partition π of shape λ+((md)^{r})/μ/d with “upper row bound” a − ∑_{k=1}^{i−1} β_{k} − ∑_{k=1}^{μ_{i}} α_{k} does indeed yield by the above procedure a (β, α)-cylindric partition π̄ of shape λ′ + ((mr)^{d})/μ′/r with “upper row bound” a − ∑_{k=1}^{i−1} α_{k} − ∑_{k=1}^{μ′_{i}} β_{k}, and vice versa. So let us take an (α, β)- cylindric partition π of shape λ+((md)^{r})/μ/d where the first part in row i is at most a − ∑_{k=1}^{i−1} β_{k} − ∑_{k=1}^{μ_{i}} α_{k}. The latter assumption means

π_{i,μ_{i}+1} ≤ a − ∑_{k=1}^{i−1} β_{k} − ∑_{k=1}^{μ_{i}} α_{k}.

By (4.5a) we have for j ≥ μ_{i} + 1 that π_{i,μ_{i}+1} ≥ π_{i,j} + ∑_{k=μ_{i}+1}^{j−1} α_{k}, and therefore

π_{i,j} ≤ a − ∑_{k=1}^{i−1} β_{k} − ∑_{k=1}^{j−1} α_{k}.  (6.25)

In particular, for μ_{i} + 1 ≤ j ≤ μ_{i−1} the entry π_{i,j} lies at the (upper) border of the shape, hence after conjugation (as illustrated in Figure 5) it lies at the (left) border of the new shape. By (6.24) π_{i,j} corresponds to π̄_{j,i}, and since this entry is at the left border, π̄_{j,i} = π̄_{j,μ′_{j}+1}. Thus (6.25) reads

π̄_{j,μ′_{j}+1} ≤ a − ∑_{k=1}^{j−1} α_{k} − ∑_{k=1}^{μ′_{j}} β_{k}.  (6.26)

This holds for all j with μ_{i} + 1 ≤ j ≤ μ_{i−1}, for some i. Certainly there exists such an i for μ_{r} + 1 ≤ j ≤ d. Hence (6.26) would hold for all j if μ_{r} = 0. However, if μ_{r} > 0 (which is the same as μ′_{1} = r), then we have to add a further restriction, namely ∑_{k=1}^{r} β_{k} ≤ ∑_{k=1}^{d} α_{k}. For if 1 ≤ j ≤ μ_{r}, then by (6.25) we have the restriction for π_{1,d+j} (which by (6.24) corresponds to π̄_{j,r+1}, which in turn is π̄_{j,μ′_{j}+1}) that

π_{1,d+j} ≤ a − ∑_{k=1}^{d+j−1} α_{k} ≤ a − ∑_{k=1}^{r} β_{k} − ∑_{k=1}^{j−1} α_{k}.

This is equivalent to (6.26) in this case.

Since we must also be able to go in the reverse direction, by symmetry we must assume that ∑_{k=1}^{r} β_{k} ≥ ∑_{k=1}^{d} α_{k} if μ_{1} = d.

Similar considerations show that, given an (α, β)-cylindric partition of shape λ + ((md)^{r})/μ/d with “lower row bound” − ∑_{k=1}^{i−1} β_{k} − ∑_{k=1}^{λ_{i}−1} α_{k}, by the conjugation procedure we obtain a (β, α)-cylindric partition of shape λ′ + ((mr)^{d})/μ′/r with “lower row bound” − ∑_{k=1}^{i−1} α_{k} − ∑_{k=1}^{λ′_{i}−1} β_{k}, and vice versa, provided that ∑_{k=1}^{r} β_{k} ≤ ∑_{k=1}^{d} α_{k} if λ_{1} < d and ∑_{k=1}^{r} β_{k} ≥ ∑_{k=1}^{d} α_{k} if λ′_{1} < r.

Now that we have collected all the necessary conditions, we may substitute into Theorem 3 our two equivalent choices; first r → r, λ → λ + ((md)^{r}), μ → μ, d → d, a_{i} → a − ∑_{k=1}^{i−1} β_{k} − ∑_{k=1}^{μ_{i}} α_{k}, b_{i} → − ∑_{k=1}^{i−1} β_{k} − ∑_{k=1}^{λ_{i}−1} α_{k}; and second, r → d, λ → λ′ + ((mr)^{d}), μ → μ′, d → r, a_{i} → a − ∑_{k=1}^{i−1} α_{k} − ∑_{k=1}^{μ′_{i}} β_{k}, b_{i} → − ∑_{k=1}^{i−1} α_{k} − ∑_{k=1}^{λ′_{i}−1} β_{k}. We may then equate the two resulting expressions for the generating function for these cylindric partitions. After cancellation, and after replacing a by a + m ∑_{k=1}^{d} α_{k}, and then ∑_{k=1}^{r} β_{k} − ∑_{k=1}^{d} α_{k} by ε, we obtain the following identity.

Theorem 19. Suppose that max{−1, −d + 1} ≤ ε ≤ min{1, r − 1} and let λ and μ be shapes contained in the rectangle (d^{r}). If λ_{1} < d or μ′_{1} = r then in addition we assume ε ≤ 0, and if λ′_{1} < r or μ_{1} = d then in addition we assume ε ≥ 0. Then

∑_{k_{1}+···+k_{r}=0} q^{(r+d)(r−ε) ∑_{i=1}^{r} k_{i}^{2}/2+(r+d) ∑_{i=1}^{r} ik_{i}}

det_{1≤s,t≤r} (q^{(μ_{t}−t)(s−t+(r−ε)k_{s})} [ \begin{matrix}a + md + λ_{s} − μ_{t} − (d + ε)k_{s} \\ md + λ_{s} − μ_{t} − s + t − (r + d)k_{s} \end{matrix} ])

= q^{(rd\binom{m+1}{2}−m ∑_{k=1}^{r} μ_{k})ε}

∑_{l_{1}+···+l_{d}=0} q^{(r+d)(d+ε) ∑_{i=1}^{d} l_{i}^{2}/2+(r+d) ∑_{i=1}^{d} il_{i}}

det_{1≤s,t≤d} (q^{(μ′_{t}−t)(s−t+(d+ε)l_{s})} [ \begin{matrix}a + mr − mε + λ′_{s} − μ′_{t} − (r − ε)l_{s} \\ mr + λ′_{s} − μ′_{t} − s + t − (r + d)l_{s} \end{matrix} ])

(6.27)

□

Note that for d = 1 we recover Theorem 8. Moreover, identity (6.23) is the special case λ = 0, μ = 0, ε = 0 of (6.27).

As in the proofs of Theorems 8–16, one could also try other row bounds here. However, one is much more restricted in this more general case. Still, a choice of row bounds different from that in the proof of Theorem 19 will be used in the proof of Theorem 28.

To conclude this section, we apply a different trick to obtain a different type of identity. Consider (0, (0, 0, …, −1))-cylindric partitions of shape (c^{r})/0/0 with entries between 0 and a. Because of our choice of β and because of (4.5b) and (4.5c), each column of such a (0, (0, 0, …, −1))-cylindric partition contains either only one integer all over or two consecutive integers. In the latter case, the larger integer occupies the top entries in this column, the lesser integer the bottom entries. Let k_{j} denote the number of j’s in the first row, j = a, a − 1, …, 1, 0. Then it is not difficult to see that a (0, (0, 0, …, −1))-cylindric partition under consideration can be visualized as in Figure 6.

. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .  . . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .  . . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .  . . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . . . . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .  . . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . .. . . . . a a − 1 a − 1 a − 2  1 0  0 ^{k_{a}}^{k_{a−1}}  ^{k_{1}}^{k_{0}}^{. . . . . . . .} Figure 6

In this figure each of the displayed regions contains only entries equal to the label assigned to that region. Each region is a Ferrers diagram (cf. [39, p. 54]), which is equivalent to a linear partition. Hence it is easy to write down an expression for these cylindric partitions by again appealing to the standard partition theorem [4, Theorem 3.1],

∑_{k_{0}+k_{1}+···+k_{a}=c} q^{∑_{i=1}^{a}(rk_{i}(i−1)+k_{i})} ∏_{i=1}^{r} [ \begin{matrix}k_{i} + r − 1 \\ r − 1 \end{matrix} ] .

Equating this expression with the one obtained from Theorem 3 (with α_{i} = 0, β_{i} = 0 for 1 ≤ i ≤ r − 1, β_{r} = −1, λ_{i} = c, μ_{i} = 0, d = 0, a_{i} = a, b_{i} = 0) we obtain another identity.

Theorem 20. There holds

∑_{k_{1}+···+k_{r}=0} q^{\binom{r+1}{2} ∑_{i=1}^{r} k_{i}^{2}+r ∑_{i=1}^{r} ik_{i}} det_{1≤s,t≤r} (q^{t(t−s−(r+1)k_{s})} [ \begin{matrix}a + c + k_{s} \\ c − s + t − rk_{s} \end{matrix} ])

= ∑_{l_{1}+···+l_{a}≤c} q^{∑_{i=1}^{a}(rl_{i}(i−1)+l_{i})} ∏_{i=1}^{r} [ \begin{matrix}l_{i} + r − 1 \\ r − 1 \end{matrix} ] . (6.28)

□

7. Identities II: Summations for Ã_{r} basic hypergeometric series. In this section we consider special cases of Theorems 8–18 of section 6 in which the determinants can be evaluated by means of (5.1), (5.2), or (5.3). We obtain summations for multiple basic hypergeometric series of a particular type which we now explain. For an introduction to (one-dimensional) basic hypergeometric series the reader should consult the book of Gasper and Rahman [16].

A basic hypergeometric series for the root system A_{r−1} (also called a basic hypergeometric series for U(r)) is a series of the form

∑_{k_{1},...,k_{r}=−∞}^{∞} ( ∏_{1≤i<j≤r} \frac{1 − q^{k_{i}−k_{j}}x_{i}/x_{j}}{1 − x_{i}/x_{j}} · (basic hypergeometric stuff)) .  (7.1)

The reader who is familiar with the root system of type A_{r−1} (cf. [21] for an introduction to root systems) will have no problem in associating the positive roots of A_{r−1} with the terms x_{i}/x_{j}, 1 ≤ i < j ≤ r.

By a basic hypergeometric series for the (affine) root system Ã_{r−1} we mean a series of the form

∑_{k_{1}+···+k_{r}=0} ( ∏_{1≤i<j≤r} \frac{1 − q^{k_{i}−k_{j}}x_{i}/x_{j}}{1 − x_{i}/x_{j}} · (basic hypergeometric stuff)) .  (7.2)

Note that the difference lies in the range of summation. Such a series is usually called a basic hypergeometric series for SU(r). However, in our context (in particular cf. Remark (1) after the proof of Proposition 1) it is more natural to call it a basic hypergeometric series for Ã_{r−1}, or simply an Ã_{r−1} basic hypergeometric series. For work on this subject (mainly undertaken by Milne and Gustafson) see [36, 37, 20] and the references cited therein.

For our first summation we let c → ∞ in (6.2). The resulting determinants on the left-hand side were already evaluated in the proof of Theorem 6 (set d = 1). Thus the left-hand side equals (5.6) with d = 1. This gives the following theorem.

Theorem 21. Let a ≥ 0. Then

∑_{k_{1}+···+k_{r}=0} q^{\binom{r+1}{2} ∑_{i=1}^{r} k_{i}^{2}+∑_{i=1}^{r} ik_{i}} \frac{∏_{1≤i<j≤r}[rk_{j} + j − rk_{i} − i]}{∏_{i=1}^{r}[a + rk_{i} + i − 1]!} = \frac{1}{[a]!}. (7.3) □

Before continuing, we want to point out that this is indeed a series of type (7.2). Namely, in (7.2) replace q by q^{−r} and x_{i} by q^{−i} to obtain

∑_{k_{1}+···+k_{r}=0} \frac{∏_{1≤i<j≤r}[rk_{j} + j − rk_{i} − i]}{∏_{1≤i<j≤r}[j − i]} · (other stuff).

The numerator of the above product is clearly in (7.3). The denominator, which equals ∏_{j=1}^{r}[j − 1]! is absorbed in the denominator of the summand in (7.3). This remark also shows that the expressions (5.4)–(5.6) that we gave for the generating functions of special sets of cylindric partitions are all Ã_{r−1} basic hypergeometric series. Moreover, (5.7) is an Ã_{r−1} (ordinary) hypergeometric series.

If one wants to know if an identity like (7.3) is already known one has to consult the papers of Milne and Gustafson. In [36] one finds Theorem 1.9, which in our notation can be rewritten as

∑_{k_{1}+···+k_{r}=S}(−1)^{(r−1)S}q^{\binom{r+1}{2} ∑_{i=1}^{r} k_{i}^{2}+∑_{i=1}^{r} ik_{i}} \frac{∏_{1≤i<j≤r}[rk_{j} + j − rk_{i} − i]}{∏_{i=1}^{r}[rk_{i} + i − 1]!}

= \frac{q^{(r+1)\binom{S+1}{2}}}{[S]!} , (7.4)

provided S ≥ 0. This identity is strikingly similar to (7.3). In fact, it turns out that (7.3) and (7.4) are equivalent. This is seen by what we call the “rotation trick”. Let S be some fixed integer. Division of S by r gives a unique representation S = Qr + R where Q, R are integers with 0 ≤ R ≤ r. Then in (7.3) replace k_{1} by k_{1+R} − Q, …, k_{r−R} by k_{r} − Q, k_{r−R+1} by k_{1} − Q − 1, …, k_{r} by k_{R} − Q − 1. So the effect is a rotation of the summation indices, combined with a certain shift. If we rewrite (7.3) after these replacements and finally replace a by a + S, we obtain a summation that covers both (7.3) and (7.4).

Theorem 22. Suppose that a + S ≥ 0. Then

∑_{k_{1}+···+k_{r}=S}(−1)^{(r−1)S}q^{\binom{r+1}{2} ∑_{i=1}^{r} k_{i}^{2}+∑_{i=1}^{r} ik_{i}} \frac{∏_{1≤i<j≤r}[rk_{j} + j − rk_{i} − i]}{∏_{i=1}^{r}[a + rk_{i} + i − 1]!}

= \frac{q^{(r+1)\binom{S+1}{2}}}{[a + S]!} , (7.5)

□

Setting S = 0 in (7.5) yields (7.3) and setting a = 0 yields (7.4). Of course, the rotation trick could also be used to derive (7.5) from (7.4). So (7.3) and (7.4) are indeed equivalent. This is remarkable, since Milne used a great deal of machinery to prove (7.4) (in fact a large part of his paper [36] is devoted to the proof of this identity), while our proof of (7.3) is rather elementary. The ingredients are nonintersecting lattice paths, the simple bijection between cylindric partitions and linear partitions, and the determinant lemma (5.2).

In addition, our method also produces new identities. Recall that (6.2) is just a special case of (6.6). Now, in (6.6) set m = r, n = 0, ε = 1, and again let c → ∞. Then a similar computation gives a summation related to (7.3) that appears to be new.

Theorem 23. Let a ≥ 0. Then

∑_{k_{1}+···+k_{r}=0} q^{(r+1)(r−1) ∑_{i=1}^{r} k_{i}^{2}/2+2 ∑_{i=1}^{r} ik_{i}} \frac{∏_{1≤i<j≤r}[(r − 1)k_{j} + j − (r − 1)k_{i} − i]}{∏_{i=1}^{r}[a + (r − 1)k_{i} + i − 1]!} = 0.

(7.6) □

See Theorem 27 for a generalization of this identity.

The next theorem results from Theorem 12 by letting a → ∞.

Theorem 24. Let c ≥ 0, 0 ≤ n < r, and 0 ≤ ε < r. Then

∑_{k_{1}+···+k_{r}=0} q^{(r+1)(r−ε) ∑_{i=1}^{r} k_{i}^{2}/2+(ε+1) ∑_{i=1}^{r} ik_{i}+(r−ε) ∑_{i=1}^{n} k_{i}}

\frac{∏_{1≤i<j≤r}[χ(i ≤ n) + (r + 1)k_{i} − i − χ(j ≤ n) − (r + 1)k_{j} + j]}{∏_{i=1}^{r}[c − χ(i ≤ n) − (r + 1)k_{i} + i − 1]!}

= \frac{q^{ε(r\binom{c}{2}−n(c−1))}}{[rc − n]!} . (7.7)

Proof. To prove (7.7) for n ≤ ε < r we set m = r and replace c by c − 1 in Theorem 12, and then let a → ∞. With these replacements, and by changing k_{s} into k_{t} (see Remark (1) after the proof of Theorem 3), (6.12) reads

∑_{k_{1}+···+k_{r}=0} q^{(r+1)(r−ε) ∑_{i=1}^{r} k_{i}^{2}/2−(r−ε) ∑_{i=1}^{r} ik_{i}+(r−ε) ∑_{i=1}^{n} k_{i}}

det_{1≤s,t≤r} (1/[c−χ(t ≤ n)−s+t−(r+1)k_{t}]!) = q^{ε(r\binom{c}{2}−n(c−1))+c\binom{r}{2}+\binom{n+1}{2}−nr} \frac{1}{[rc − n]!}.

Now in the determinant reverse the order of both rows and columns (i.e., replace s by r + 1 − s and t by r + 1 − t) and then evaluate it by means of (5.1) with L_{i} = c − χ(r + 1 − i ≤ n) − i − (r + 1)k_{r+1−i} and A → ∞. A few simplifications then lead to (7.7) in the case n ≤ ε < r.

To establish (7.7) for 0 ≤ ε < n also, we have to examine where the condition n ≤ ε is needed in the proof of (6.12). Recall that ε actually stands for ∑_{k=1}^{r} β_{k} − α. So n ≤ ε is equivalent to ∑_{k=1}^{r} β_{k} ≥ α+n. It is not elaborated in Case 5, though it is in Case 2 (see the passage around (6.7)), that ∑_{k=1}^{r} β_{k} ≥ α + n is needed so that the linear partitions that are counted by the right-hand side of (6.12) will automatically “obey the upper row bounds a_{i} = a + ∑_{k=1}^{i−1}(1 − β_{k}) − α · χ(i ≤ n)” when interpreted as (α, β)-cylindric partitions. Therefore, if we drop the condition ∑_{k=1}^{r} β_{k} ≥ α + n there might be a difference between the two sides of (6.12). But this difference would be due to linear partitions that (as (α, β)-cylindric partitions) do not obey these row bounds. Therefore, these linear partitions must contain at least one part that violates such an upper row bound. But as a increases to infinity, so does each upper row bound, so in the limit as a → ∞ the difference between the two sides of (6.12) vanishes, for all ε with 0 ≤ ε < r. □

Let us have a closer look at (7.7). For convenience, write y_{i} for χ(i ≤ n) − i. Then, for r = 2, the series in (7.7) reduces to a single sum that can be written in basic hypergeometric notation (cf. section 2) as

\frac{(1 − q^{y_{1}−y_{2}})}{(q; q)_{c−y_{1}−1} (q; q)_{c−y_{2}−1}}

∑_{k=−∞}^{∞} \frac{(q^{3+(y_{1}−y_{2})/2}, −q^{3+(y_{1}−y_{2})/2}, q^{1+y_{1}−c}, q^{2+y_{1}−c}, q^{3+y_{1}−c}; q^{3})_{k}}{(q^{(y_{1}−y_{2})/2}, −q^{(y_{1}−y_{2})/2}, q^{2−y_{2}+c}, q^{1−y_{2}+c}, q^{y_{2}+c}; q^{3})_{k}}

(−1)^{k}q^{(3−6ε)\binom{k}{2}} (q^{−3−3ε−(1+ε)y_{1}+(ε−2)y_{2}+3c})^{k} .

So we see that (7.7) for r = 2 is a special case of Bailey’s very well-poised _{6}ψ_{6}- summation [7, (4.7); 16, (5.3.1)] (cf. [16, (5.1.5)] for notation)

6ψ_{6}[ \begin{matrix}q\sqrt{a}, −q\sqrt{a}, b, c, d, e \\ \sqrt{a}, −\sqrt{a}, aq/b, aq/c, aq/d, aq/e \end{matrix}; q, qa^{2}/bcde]

= \frac{(qa, q, q/a, qa/bc, qa/bd, qa/be, qa/cd, qa/ce, qa/de; q)_{∞}}{(qa/b, qa/c, qa/d, qa/e, q/b, q/c, q/d, q/e, qa^{2}/bcde; q)_{∞}} (7.8)

(take a = q^{y_{1}−y_{2}}, b = q^{1+y_{1}−c}, c = q^{2+y_{1}−c}, d = q^{3+y_{1}−c}, and if ε = 0 let e → ∞, while if ε = 1 let e → 0.) Therefore, for general r, (7.7) should come from Gustafson’s SU(r)-extension of Bailey’s _{6}ψ_{6}-sum [19, Theorem 1.15; 20, Theorem 4.1],

∑_{k_{1}+···+k_{r}=0} q^{∑_{i=1}^{r} ik_{i}} ∏_{1≤i<j≤r} \frac{1 − q^{k_{i}−k_{j}}x_{i}/x_{j}}{1 − x_{i}/x_{j}} ∏_{i,j=1}^{r} \frac{(a_{i}x_{j}; q)_{k_{j}}}{(b_{i}x_{j}; q)_{k_{j}}}

= \frac{(q; q)_{∞}^{r−1} ∏_{i,j=1}^{r}(b_{i}/a_{j}; q)_{∞} ∏_{1≤i<j≤r}(qx_{i}/x_{j}; q)_{∞} (qx_{j}/x_{i}; q)_{∞}}{(q^{1−r} ∏_{i=1}^{r}(b_{i}/a_{i}); q)_{∞} ∏_{i,j=1}^{r}(q/a_{i}x_{j}; q)_{∞} (b_{i}x_{j}; q)_{∞}}

· (q/ ∏_{i=1}^{r} a_{i}x_{i}; q)_{∞} (q^{1−r} ∏_{i=1}^{r} b_{i}x_{i}; q)_{∞}. (7.9)

However, it apparently does not. In basic hypergeometric notation the series in (7.7) would read

\frac{∏_{1≤i<j≤r}(1 − q^{χ(i≤n)−i−χ(j≤n)+j})}{∏_{i=1}^{r}(q; q)_{c−χ(i≤n)+i−1}}

∑_{k_{1}+···+k_{r}=0} q^{−(r+1)(ε+1) ∑_{i=1}^{r} k_{i}^{2}/2+(r+ε+2) ∑_{i=1}^{r} ik_{i}−(ε+1) ∑_{i=1}^{n} k_{i}}

∏_{1≤i<j≤r} \frac{1 − q^{(r+1)k_{i}−(r+1)k_{j}+χ(i≤n)−i−χ(j≤n)+j}}{1 − q^{χ(i≤n)−i−χ(j≤n)+j}} ∏_{i=1}^{r}(q^{1−c+χ(i≤n)−i}; q)_{(r+1)k_{i}}.

In order to generate the summand of the above series from (7.9), one would have to replace q by q^{r+1}, x_{i} by q^{χ(i≤n)−i}, and a_{i} by q^{−c+i}. This is fine with the product ∏_{1≤i<j≤r} in the summand. But it doesn’t quite yield the product ∏_{i=1}^{r}(q^{1−c+χ(i≤n)−i}; q)_{(r+1)k_{i}} in the numerator that is needed, not to mention the powers of q, and at this point one is stuck. So there is some mystery as to the relation between (7.7) and (7.9). It seems that one has to look for an identity involving a sum with “bigger” expressions, very likely for an SU(r)-extension of an _{8}ψ_{8}-identity, such as [16, (5.6.2) or Ex. 5.12], for example. (See also item (8) of the last section.)

Finally we consider (6.17)=(6.18). We may directly apply (5.2) (with A = a + c, L_{i} = c − i − rk_{i}) to evaluate the determinants on the left-hand side. Thus we obtain the identity

∑_{k_{1}+···+k_{r}=0} q^{r^{2} ∑_{i=1}^{r} k_{i}^{2}/2} \frac{∏_{1≤i<j≤r}[rk_{j} + j − rk_{i} − i]}{∏_{i=1}^{r}[c − rk_{i} + r − i]!}

\frac{∏_{i=1}^{r}[a + c + i − 1]!}{∏_{i=1}^{r}[a + rk_{i} + i − 1]!} = [ \begin{matrix}a + c \\ a \end{matrix} ]_{q^{r}} . (7.10)

The rotation trick works here also: replace k_{1} by k_{1+R} − Q, …, k_{r−R} by k_{r} − Q, k_{r−R+1} by k_{1} − Q − 1, …, k_{r} by k_{R} − Q − 1, and then replace a by a + S, and c by c − S, and rewrite. This leads to

∑_{k_{1}+···+k_{r}=S}(−1)^{(r−1)S}q^{r^{2} ∑_{i=1}^{r} k_{i}^{2}/2} \frac{∏_{1≤i<j≤r}[rk_{j} + j − rk_{i} − i]}{∏_{i=1}^{r}[c − rk_{i} + r − i]!}

\frac{∏_{i=1}^{r}[a + c + i − 1]!}{∏_{i=1}^{r}[a + rk_{i} + i − 1]!} = q^{rS^{2}/2} [ \begin{matrix}a + c \\ a + S \end{matrix} ]_{q^{r}} . (7.11)

This identity is less exciting than (7.5) or (7.7) since it is indeed just a special case of Gustafson’s _{6}ψ_{6}-summation (7.9) for SU(r) (replace q by q^{r}, x_{i} by q^{−i−rS·χ(i=r)}, a_{i} by q^{−a+i}, b_{i} by q^{c+r+i}).

8. Identities III: Transformation formulas. In this section we proceed in the spirit of the previous section, but apply determinant evaluations to special cases of Theorems 19 and 20. Here we obtain transformation formulas between Ã_{r} basic hypergeometric series of different dimensions, one more summation formula for Ã_{r} basic hypergeometric series, and a transformation formula between an Ã_{r} basic hypergeometric series and an ordinary multiple basic hypergeometric series. All the identities in this section are new. In particular, transformation formulas between series of different dimensions do not seem to have appeared before.

We begin with setting ε = 0 and μ = 0 in Theorem 19, and then let m → ∞. The resulting determinants (on both sides) were already evaluated in the proof of Theorem 6. We thus obtain the following transformation formula.

Theorem 25. For a ≥ 0 we have

∑_{k_{1}+···+k_{r}=0} q^{r(r+d) ∑_{i=1}^{r} k_{i}^{2}/2+d ∑_{i=1}^{r} ik_{i}} \frac{∏_{1≤i<j≤r}[rk_{j} + j − rk_{i} − i]}{∏_{i=1}^{r}[a + rk_{i} + i − 1]!}

= ∑_{l_{1}+···+l_{d}=0} q^{d(r+d) ∑_{i=1}^{d} l_{i}^{2}/2+r ∑_{i=1}^{d} il_{i}} \frac{∏_{1≤i<j≤d}[dl_{j} + j − dl_{i} − i]}{∏_{i=1}^{d}[a + dl_{i} + i − 1]!}. (8.1)

□

Once more, the rotation trick can be applied (see the paragraph before Theorem 22), this time for both sums simultaneously. This gives rise to the following generalization of Theorem 25.

Theorem 26. For a ≥ 0 we have

∑_{k_{1}+···+k_{r}=S}(−1)^{(r−1)S}q^{r(r+d) ∑_{i=1}^{r} k_{i}^{2}/2+d ∑_{i=1}^{r} ik_{i}−d\binom{S+1}{2}−rS(S+d)/2}

× \frac{∏_{1≤i<j≤r}[rk_{j} + j − rk_{i} − i]}{∏_{i=1}^{r}[a − S + rk_{i} + i − 1]!}

= ∑_{l_{1}+···+l_{d}=T}(−1)^{(d−1)T} q^{d(r+d) ∑_{i=1}^{d} l_{i}^{2}/2+r ∑_{i=1}^{d} il_{i}−r\binom{T +1}{2}−dT (T +r)/2}

× \frac{∏_{1≤i<j≤d}[dl_{j} + j − dl_{i} − i]}{∏_{i=1}^{d}[a − T + dl_{i} + i − 1]!}. (8.2)

□

Note that by setting d = 1 and T = 0 in Theorem 26 we recover Theorem 22 (and similarly, by setting d = 1 in Theorem 25 we recover Theorem 21).

We might also start with setting ε = 1 and μ = 0 in Theorem 19, and then let m → ∞. Note that because of the assumptions of Theorem 19, the choice ε = 1 forces r to be at least 2. In this case the right-hand side vanishes, while the determinants on the left-hand side can be evaluated like those in the proof of Theorem 6. This leads to the following summation.

Theorem 27. For r ≥ 2 and a ≥ 0 we have

∑_{k_{1}+···+k_{r}=0} q^{(r+d)(r−1) ∑_{i=1}^{r} k_{i}^{2}/2+(d+1) ∑_{i=1}^{r} ik_{i}}

× \frac{∏_{1≤i<j≤r}[(r − 1)k_{j} + j − (r − 1)k_{i} − i]}{∏_{i=1}^{r}[a + (r − 1)k_{i} + i − 1]!} = 0. (8.3)

□

This identity generalizes (7.6). The fact that d appears only in the exponent of q implies that in the sum we must actually have term-wise cancellation.

A different type of transformation formula is obtained by replacing m by m − 1 and λ_{i} by d in Theorem 19, and then letting a → ∞. The resulting determinants (on both sides) have already been evaluated in the proof of Theorem 5. This proves the next theorem when max{−1, −d + 1} ≤ ε ≤ min{1, r − 1}. It requires a different argument to show the theorem for −d < ε < r.

Theorem 28. Let −d < ε < r and let μ be a shape contained in the rectangle (d^{r}). Then

∑_{k_{1}+···+k_{r}=0} q^{(r+d)(r−ε) ∑_{i=1}^{r} k_{i}^{2}/2+(d+ε) ∑_{i=1}^{r} ik_{i}+(r−ε) ∑_{i=1}^{r} μ_{i}k_{i}}

× \frac{∏_{1≤i<j≤r}[μ_{i} + (r + d)k_{i} − i − μ_{j} − (r + d)k_{j} + j]}{∏_{i=1}^{r}[md − μ_{i} − (r + d)k_{i} + i − 1]!}

= q^{(rd\binom{m}{2}−(m−1) ∑_{i=1}^{r} μ_{i})ε} ∑_{l_{1}+···+l_{d}=0} q^{(r+d)(d+ε) ∑_{i=1}^{d} l_{i}^{2}/2+(r−ε) ∑_{i=1}^{d} il_{i}+(d+ε) ∑_{i=1}^{d} μ′_{i}l_{i}}

× \frac{∏_{1≤i<j≤d}[μ′_{i} + (r + d)l_{i} − i − μ′_{j} − (r + d)l_{j} + j]}{∏_{i=1}^{d}[mr − μ′_{i} − (r + d)l_{i} + i − 1]!}. (8.4)

□

Proof. Consider (α, β)-cylindric partitions of shape ((md)^{r})/μ/d where the last entry in row i is at least − ∑_{k=i}^{r−1}(1 − β_{k}) and where there is no upper bound on the entries. By the cut-and-paste procedure that led to a proof of Theorem 19 and is illustrated in Figure 5, we see that these (α, β)-cylindric partitions are in bijection with (β, α)-cylindric partitions of shape ((mr)^{d})/μ′/r where the last entry in row i is at least − ∑_{k=i}^{d−1}(1 − α_{k}) and where there is again no upper bound on the entries. Hence, the generating functions for these two sets of cylindric partitions must be the same. By Theorem 3 with the replacements r → r, λ_{i} → md, μ_{i} → μ_{i}, d → d, a_{i} → ∞, b_{i} → − ∑_{k=i}^{r−1}(1 − β_{k}), the generating function for the first set equals

∑_{k_{1}+···+k_{r}=0} q^{N_{2}(k)} det (1/[md − μ_{t} − s + t − (r + d)k_{t}]!),

where

N_{2}(k) = (r + d)(r − B + A) ∑_{i=1}^{r} k_{i}^{2}/2 + (r − B + A) ∑_{i=1}^{r}(μ_{i} − i)k_{i} − (mdr − ∑_{i=1}^{r} μ_{i})(β_{r} + α_{d})

− md\binom{r}{2} + A(rd\binom{m}{2} − (m − 1) ∑_{i=1}^{r} μ_{i}) + ∑_{i=1}^{r}(r − i)μ_{i}

+ md ∑_{k=1}^{r} kβ_{k} + mr ∑_{k=1}^{d} kα_{k} − ∑_{k=1}^{r} β_{k} ∑_{i=1}^{k} μ_{i} − ∑_{k=1}^{d} α_{k} ∑_{i=1}^{k} μ′_{i},

with A := ∑_{k=1}^{d} α_{k} and B := ∑_{k=1}^{r} β_{k}. In the determinant we reverse the order of both row and columns (i.e., we replace s by r + 1 − s and t by r + 1 − t). Then the determinant can be evaluated using (5.1) with L_{i} = md − μ_{r+1−i} − i − (r + d)k_{r+1−i} and A → ∞. After a few manipulations we obtain for the generating function under consideration the expression

∑_{k_{1}+···+k_{r}=0} q^{N_{3}(k)} \frac{∏_{1≤i<j≤r}[μ_{i} + (r + d)k_{i} − i − μ_{j} − (r + d)k_{j} + j]}{∏_{i=1}^{r}[md − μ_{i} − (r + d)k_{i} + i − 1]!},

where

N_{3}(k) = (r + d)(r − B + A) ∑_{i=1}^{r} k_{i}^{2}/2 + (d + B − A) ∑_{i=1}^{r} ik_{i} + (r − B + A) ∑_{i=1}^{r} μ_{i}k_{i}

− (mdr − ∑_{i=1}^{r} μ_{i})(β_{r} + α_{d}) + A(rd\binom{m}{2} − (m − 1) ∑_{i=1}^{r} μ_{i})

+ md ∑_{k=1}^{r} kβ_{k} + mr ∑_{k=1}^{d} kα_{k} − ∑_{k=1}^{r} β_{k} ∑_{i=1}^{k} μ_{i} − ∑_{k=1}^{d} α_{k} ∑_{i=1}^{k} μ′_{i}.

Obviously, the generating function for the second set is the same expression but with r and d, α and β interchanged, and with μ replaced by μ′. If we equate these two expressions and finally replace B − A by ε we obtain (8.4) after some cancellation. □

Note that this transformation formula generalizes the summation in Theorem 24 (set d = 1 in (8.4)). It is safe to speculate that (8.4) must be a special case of a much more general transformation formula between Ã_{r} basic hypergeometric series of different dimensions. See the last section, item (9) for more precise comments on this subject.

Our last computation in this section starts with Theorem 20. If we let c → ∞, the determinants on the left-hand side factor by the determinant evaluation in the proof of Theorem 6 (with r replaced by r + 1). This gives a transformation between an Ã_{r−1} basic hypergeometric series and an ordinary multiple hypergeometric series.

Theorem 29. For a ≥ 0 we have

∑_{k_{1}+···+k_{r}=0} q^{\binom{r+1}{2} ∑_{i=1}^{r} k_{i}^{2}−∑_{i=1}^{r} ik_{i}} \frac{∏_{1≤i<j≤r}[(r + 1)k_{j} + j − (r + 1)k_{i} − i]}{∏_{i=1}^{r}[a + (r + 1)k_{i} + i − 1]!}

= ∑_{l_{1},l_{2},...,l_{a}≥0} q^{∑_{i=1}^{a}(rl_{i}(i−1)+l_{i})} ∏_{i=1}^{r} [ \begin{matrix}l_{i} + r − 1 \\ r − 1 \end{matrix} ] . (8.5)

□

9. Some Schur function expansions. A standard application of nonintersecting lattice paths is to prove the Jacobi-Trudi identities for Schur functions (cf. [18; 39, 154–159; 15, sec. 3]). In this brief section we apply nonintersecting lattice paths with a nonintersecting translate of the last path in a similar way to obtain expansions in Schur functions (see (9.6) and (9.9) below).

Recall that in the proof of the Jacobi-Trudi identity for Schur functions [33, I, (3.4)] by means of nonintersecting lattice paths, the weight assigned to a horizontal edge e from (i − 1, j) to (i, j) is w(e) := x_{j}. Given this weight, the only way to satisfy (3.4) is to choose S to be a horizontal shift. But this is forbidden if we want to apply Proposition 1. Hence, this weight assignment does not make sense in our context.

On the other hand, in the proof of the dual Jacobi-Trudi identity [33, I, (3.5)] (the Nägelsbach-Kostka identity, see (9.2) below) the weight assigned to a horizontal edge e from (i − 1, j) to (i, j) is w(e) := x_{i+j}, so that the weight is invariant along the antidiagonal direction. Therefore we may take this weight and choose S to be a shift in the antidiagonal direction, say S = (−r, r). Then (3.4) is satisfied with z = 1, and, given an appropriate choice of starting points u = (u_{1}, u_{2}, …, u_{r}) and end points v = (v_{1}, v_{2}, …, v_{r}), we may apply Proposition 1. We choose u_{i} = (−i + 1, i − 1) and v_{i} = (c − i + 1, n − c + i − 1). Then by (3.5) the generating function ∑ w(P) for all families P = (P_{1}, P_{2}, …, P_{r}) of lattice paths, where P_{i} runs from u_{i} = (−i + 1, i − 1) to v_{i} = (c − i + 1, n − c + i − 1), and where (SP_{r}, P_{1}, …, P_{r}) is nonintersecting, equals

∑_{k_{1}+···+k_{r}=0} det_{1≤s,t≤r} (e_{c+s−t−rk_{s}}(x_{1}, …, x_{n})),  (9.1)

where e_{m}(x_{1}, …, x_{n}) := ∑_{1≤i_{1}<···<i_{m}≤n} x_{i_{1}} ⋯ x_{i_{m}} denotes the elementary symmetric function of order m in x_{1}, …, x_{n} (cf. [33, p. 12]). By changing again k_{s} into k_{t} (see Remark (1) after the proof of Theorem 3), and by using the dual Jacobi-Trudi identity for Schur functions s_{λ}(x_{1}, …, x_{n}) [33, I, (3.5)],

s_{λ′}(x_{1}, …, x_{n}) = det_{s,t} (e_{λ_{t}−t+s}(x_{1}, …, x_{n})),  (9.2)

we may write (9.1) as

∑_{k_{1}+···+k_{r}=0} s_{(c−rk_{1},c−rk_{2},...,c−rk_{r})′}(x_{1}, …, x_{n}).  (9.3)

The reader should be warned when reading this expression that the subscripts for the Schur functions in (9.3) will not always be partitions (i.e., the components of (c − rk_{1}, …, c − rk_{r}) may be out of order) so one must apply the modification rule

s_{(...,λ_{i},λ_{i+1},...)′}(x_{1}, …, x_{n}) = −s_{(...,λ_{i+1}−1,λ_{i}+1,...)′}(x_{1}, …, x_{n}),  (9.4)

perhaps repeatedly, to obtain a sum of Schur functions that are really indexed by partitions. (See the example below.)

On the other hand, because of our choice of starting points u, end points v, and shift S, all r lattice paths are forced to run in parallel. Hence, the generating function under consideration also equals

e_{c}(x_{1}^{r}, x_{2}^{r}, …, x_{n}^{r}) = m_{(r^{c})}(x_{1}, x_{2}, …, x_{n}),  (9.5)

where

m_{(λ_{1},...,λ_{m})}(x_{1}, …, x_{n}) := ∑_{1≤i_{1},...,i_{m}≤n, i_{j} pairwise distinct} x_{i_{1}}^{λ_{1}} ⋯ x_{i_{m}}^{λ_{m}}

denotes the monomial symmetric function in x_{1}, …, x_{n} (cf. [33, I, (2.1)]). Hence again, by equating (9.3) and (9.5), we have an identity. The most convenient way to write it is after taking the limit n → ∞ and changing the summation indices from k_{i} to −k_{i}.

Theorem 30. Let x = (x_{1}, x_{2}, …). Then

m_{(r^{c})}(x) = ∑_{k_{1}+···+k_{r}=0} s_{(c+rk_{1},c+rk_{2},...,c+rk_{r})′}(x),  (9.6)

where the right-hand side has to be understood according to the modification rule (9.4). □

Let us consider an example. Take c = 3 and r = 2. Then (9.6) and (9.4) give

m_{(2^{3})}(x) = s_{(3,3)′}(x) + s_{(5,1)′}(x) + s_{(1,5)′}(x) + s_{(−1,7)′}(x)
= s_{(3,3)′}(x) + s_{(5,1)′}(x) − s_{(4,2)′}(x) − s_{(6)′}(x)
= s_{(2,2,2)}(x) + s_{(2,1,1,1,1)}(x) − s_{(2,2,1,1)}(x) − s_{(1,1,1,1,1,1)}(x).

We remark that it is not too hard to recover (9.6) from the combinatorial description of the inverse Kostka matrix given by Eğecioğlu and Remmel [13]. Thus in particular, it is also seen that when straightened out by (9.4), the expansion (9.6) is multiplicityfree; i.e., the coefficients on the right-hand side are 1, −1 or 0.

By applying the isomorphism ω (cf. [33, pp. 14–17]) to both sides of (9.6), or by making use of a duality principle in [32, (4.2), (4.3); 38, Theorem 2.6(a)(b)], we obtain the related identity

f_{(r^{c})}(x) = (−1)^{(r−1)c}h_{c}(x_{1}^{r}, x_{2}^{r}, …) = ∑_{k_{1}+···+k_{r}=0} s_{(c+rk_{1},c+rk_{2},...,c+rk_{r})}(x). (9.7)

(Here, f_{m}(x) denotes the “forgotten symmetric function”, cf. [33, p. 15].) What is interesting about this identity is that it suggests what the expression (3.5) means for horizontal or vertical shifts. More precisely, in Proposition 1 take u_{i} = (i, 1), v_{i} = (c + i, ∞), i = 1, 2, …, r, S = (−r, 0) (which is a horizontal shift), and the weight w(e) of a horizontal edge e from (i − 1, j) to (i, j) to be w(e) := x_{j} (which implies z = 1). Then, by the Jacobi-Trudi identity (see [33, I, (3.4)])

s_{λ}(x) = det_{s,t} (h_{λ_{t}−t+s}(x)),  (9.8)

(3.5) reduces exactly to the right-hand side of (9.7) (up to sign). Thus the left-hand side of (9.7) tells us that in this special case the expression (3.5) has a very simple meaning, despite the fact that the shift S is horizontal, so that Proposition 1 does not apply. It would be interesting to figure out what is going on here. (See also item (1) of the last section.)

Another way of looking at what we did in deriving (9.6) is to say that we counted row-strict cylindric partitions (i.e., (1, 0)-cylindric partitions, where 1 denotes the all 1 sequence) of rectangular shape (c^{r})/0/0 with entries between 1 and n. The weight of such a row-strict cylindric partition π is defined by w(π) := ∏ x_{π_{i,j}} with π_{i,j} running over all entries of π. This is seen by the translation of (α, β)-cylindric partitions into families of lattice paths that we gave in the proof of Theorem 3.

So, let us next consider row-strict cylindric partitions of shape (c^{r})/0/1 with entries between 1 and n. Using again the translation into families of lattice paths, Proposition 1, in combination with (9.2), tells us that the generating function for these cylindric partitions equals

∑_{k_{1}+···+k_{r}=0} s_{(c−(r+1)k_{1},c−(r+1)k_{2},...,c−(r+1)k_{r})′}(x_{1}, …, x_{n}).

On the other hand, given any such row-strict cylindric partition π, we may line up the entries of π by reading along the columns as in (6.1). Thus we obtain a linear partition ρ = (ρ_{1}, ρ_{2}, …, ρ_{rc}) with the additional property that ρ_{i} is strictly larger than ρ_{i+r}. This property is the same as saying that no part of ρ may be repeated more than r times. Clearly, the generating function for these linear partitions (with weight defined by w(ρ) := ∏_{i=1}^{rc} x_{ρ_{i}}) is the homogeneous part of degree rc in ∏_{i=1}^{n}(1+x_{i} +x_{i}^{2} +⋯+x_{i}^{r}). Of course, we can also play the same game with row-strict cylindric partitions of shape λ/0/1, provided λ = (λ_{1}, …, λ_{r}) is some partition with λ_{1} − λ_{r} ≤ 1. Taking n → ∞, we obtain the following generating function theorem.

Theorem 31. For the generating function of all linear partitions having the property that no part is repeated more than r times we have the expansion in terms of Schur functions

∏_{i=1}^{∞}(1 + x_{i} + x_{i}^{2} + ⋯ + x_{i}^{r}) = ∑_{k_{1}+···+k_{r}=0} ∑_{λ with λ_{1}−λ_{r}≤1} s_{(λ+(r+1)k)′}(x) , (9.9)

where k = (k_{1}, …, k_{r}), λ = (λ_{1}, λ_{2}, …, λ_{r}), and λ + (r + 1)k is short for componentwise addition and scalar multiplication. Again, the right-hand side of (9.9) must be understood according to the modification rule (9.4). □

As an example, the case r = 2 of Theorem 31 gives

∏_{i=1}^{∞}(1 + x_{i} + x_{i}^{2}) = ∑_{j,k≥0} ξ(k)s_{(2^{j},1^{k})},

where

ξ(k) = \{ \begin{matrix}1, & if k ≡ 0 or 1 (mod 6) \\ −1, & if k ≡ 3 or 4 (mod 6) \\ 0, & if k ≡ 2 or 5 (mod 6) \end{matrix}

By the duality principle in [32, (4.2), (4.3); 38, Theorem 2.6(a)(b)], (9.9) is equivalent to

∏_{i=1}^{∞}(1 + x_{i} + x_{i}^{2} + ⋯ + x_{i}^{r})^{−1} = ∑_{k_{1}+···+k_{r}=0} (−1)^{∑_{i=1}^{r} λ_{i}} ∑_{λ with λ_{1}−λ_{r}≤1} s_{λ+(r+1)k}(x) .

(9.10)

This corrects the expansion (4.8) of Lascoux and Pragacz [32]. However, their argument (specialize the Cauchy identity for Schur functions), if carefully completed, would also give a proof of (9.10), after some work. It should also be remarked, that by a general theorem of Remmel and Yang [38, Theorem 3.1], the expansion (9.9) (and hence also (9.10)) is multiplicity-free.

It should be noted that (9.10) is related to a special case of (3.5) with a horizontal shift S. More precisely, in Proposition 1 take u_{i} = (i, 1), v_{i} = (λ_{i} + i, ∞), i = 1, 2, …, r, where λ is a partition with λ_{1} − λ_{r} ≤ 1, S = (−r − 1, 0) (which is a horizontal shift), and the weight w(e) of a horizontal edge e from (i − 1, j) to (i, j) to be w(e) := x_{j}. Then, again by the Jacobi-Trudi identity (9.8), the expression (3.5) reduces exactly to the homogeneous component of degree λ_{1} + ⋯ + λ_{r} in the righthand side of (9.10) (up to sign). Again, this could be a hint of what the expression (3.5) means for horizontal or vertical shifts S. (See also item (1) of the last section.)

What we did so far in this section was to transfer the ideas that lead to Theorems 8– 18 to counting row-strict cylindric partitions. What about the method that lead to (6.23) and finally to Theorem 19? In particular, what happens if we consider rowstrict cylindric partitions of shape ((md)^{r})/0/d, with arbitrary d? The cut-and-paste method illustrated in Figure 5 should again give us an identity. When applying cutand-paste to such a row-strict cylindric partition we obtain a column-strict cylindric partition, i.e., a (0, 1)-cylindric partition, of shape ((mr)^{d})/0/r. Unfortunately, after translating the latter into lattice paths, we would have to consider a horizontal shift S, and Proposition 1 does not apply in this case. So the problem remains of extending Proposition 1 to horizontal (and also vertical) shifts.

10. Future directions and open problems. (1) Extend Proposition 1 so that it also holds for horizontal and vertical shifts S. Alternatively, is there anything reasonable counted by (3.5) when S is a horizontal or vertical shift? The two cases discussed in section 9 (see the remarks concerning (9.7) and (9.10)) should give hints in this direction.

(2) In [8, 9] Brenti considers what he calls dotted plane partitions. These can be translated into nonintersecting lattice paths consisting of 3 types of steps (horizontal, vertical, and diagonal). He gives a number of very interesting applications, in particular in connection with Hall-Littlewood functions and q-Stirling numbers. So it should be a nice project to explore the implications of considering nonintersecting lattice paths with a nonintersecting translate of the last path, where the lattice paths consist of 3 types of steps, and also of considering weights different from ours (this is for the q-Stirling part).

(3) In [27] it was shown by one of the authors how to enumerate nonintersecting lattice paths with respect to the major index, and, more generally, in [29] with respect to weighted turns. Currently, M. Prohaska is working out the analogous theory for enumerating nonintersecting lattice paths with nonintersecting translate of the last path by major index and weighted turns. In particular, this will give rise to more identities.

(4) In [27, 15, 29, 28] the enumeration of nonintersecting lattice paths that are bounded by a line (diagonal or vertical) was studied. This was shown to have interesting applications to tableau and plane partition counting. Therefore it is natural to ask for enumerating nonintersecting lattice paths with nonintersecting translate of the last path which in addition are bounded by a line. In particular, since the enumeration done in [27, 29] related to C_{r} basic hypergeometric series, this should produce C̃_{r} basic hypergeometric series identities. This project is also currently being undertaken by M. Prohaska.

(5) Is there any representation theoretic meaning of cylindric partitions? A few remarks are in order to motivate this question. As is well-known, column-strict plane partitions of a given shape index the weights of GL- (or SL-) characters. When translated into lattice path language, column-strict plane partitions correspond to nonintersecting lattice paths (see [17, 18]). The Weyl group associated with GL-characters is the symmetric group, the Weyl group of type A. When counting nonintersecting lattice paths it is exactly this group that acts on the end points (or equivalently, starting points) of the lattice paths. Next, symplectic tableaux (introduced by King and El-Sharkaway [23]) of a given shape index the weights of Sp-characters. It was shown in [15] that, when translated into lattice path language, symplectic tableaux correspond to nonintersecting lattice paths that are bounded by a diagonal line. The Weyl group associated with Sp-characters is the hyperoctahedral group, the Weyl group of type C (which is the same as the Weyl group of type B). It is formulated slightly differently in [15], but what happens in essence in the enumeration of nonintersecting lattice paths that are bounded by a diagonal line (cf. [27, sec. 2.2]) is that exactly this group acts on the end points (or on the starting points, as formulated in [27]).

Now, in this paper we considered cylindric partitions that correspond to nonintersecting lattice paths with nonintersecting translate of the last path. As already mentioned in Remark (1) after the proof of Proposition 1, the affine Weyl group of type A acts on the end points of the lattice paths. This Weyl group appears in the representation theory of Kac-Moody algebras of type A_{r}^{(1)} (see [22]). Hence, is there a connection between Kac-Moody algebras of type A_{r}^{(1)} and cylindric partitions? Even more speculatively, is there a connection between Kac-Moody algebras of type C_{r}^{(1)} and nonintersecting lattice paths with nonintersecting translate of the last path that in addition are bounded by a line? This question is motivated by the fact that exactly the Weyl group corresponding to this Kac-Moody algebra (the affine Weyl group of type C) would act on the end points (or starting points) of the lattice paths.

(6) In [12, pp. 217–218] Burge develops sort of a Bailey chain for his partition pair generating functions. The most interesting results of his paper, two doubly bounded summations [12, pp. 219–220] that contain both Rogers’ and Schur’s previously unrelated proofs of the celebrated Rogers-Ramanujan identities, is derived by his Bailey chain argument. It is very unfortunate that we failed to find a Bailey chain for rrowed cylindric partitions, for general r. Once found, it would give rise to many more interesting identities.

(7) Since cylindric partitions proved a special case of Gustafson’s _{6}ψ_{6}-summation (7.9) for SU(r)): Are there more general weights for lattice paths which would lead to a proof of the full formulas (7.9) or at least of some terminating cases such as Milne’s 6ϕ_{5}-summation for SU(r) [35, Theorem 1.49]?

(8) What is the relation between our summation (7.7) and Gustafson’s _{6}ψ_{6}-sum (7.9) for SU(r)? Is there a generalization of (7.9) that would also cover (7.7)? As mentioned in section 7, the answer to these questions would very likely be an SU(r)- extension of some _{8}ψ_{8}-identity, such as [16, (5.6.2) or Ex. 5.12], for example.

(9) The transformation formula (8.4) must be a special case of a more general formula. What is it? In particular, by setting y_{ij} := χ(the cell (i, j) is contained in the shape μ), we could rewrite (8.4) in basic hypergeometric notation as

\frac{∏_{1≤i<j≤r}(1 − q^{∑_{l=1}^{d} y_{il}−i−∑_{l=1}^{d} y_{jl}+j})}{∏_{i=1}^{r}(q; q)_{md−∑_{l=1}^{d} y_{il}+i−1}}

∑_{k_{1}+···+k_{r}=0} q^{−(r+d)(d+ε) ∑_{i=1}^{r} k_{i}^{2}/2+(r+2d+ε) ∑_{i=1}^{r} ik_{i}−(d+ε) ∑_{i=1}^{r} ∑_{j=1}^{d} y_{ij}k_{i}}

∏_{1≤i<j≤r} \frac{1 − q^{(r+d)k_{i}−(r+d)k_{j}+∑_{l=1}^{d} y_{il}−i−∑_{l=1}^{d} y_{jl}+j}}{1 − q^{∑_{l=1}^{d} y_{il}−i−∑_{l=1}^{d} y_{jl}+j}} ∏_{i=1}^{r}(q^{1−md+∑_{l=1}^{d} y_{il}−i}; q)_{(r+d)k_{i}}

= q^{(rd\binom{m}{2}−(m−1) ∑_{i=1}^{r} μ_{i})ε} \frac{∏_{1≤i<j≤d}(1 − q^{∑_{l=1}^{r} y_{li}−i−∑_{l=1}^{r} y_{lj}+j})}{∏_{i=1}^{d}(q; q)_{mr−∑_{l=1}^{r} y_{li}+i−1}}

∑_{l_{1}+···+l_{d}=0} q^{−(r+d)(r−ε) ∑_{i=1}^{d} l_{i}^{2}/2+(d+2r−ε) ∑_{i=1}^{d} il_{i}−(r−ε) ∑_{i=1}^{d} ∑_{j=1}^{r} y_{ji}l_{i}}

∏_{1≤i<j≤d} \frac{1 − q^{(r+d)l_{i}−(r+d)l_{j}+∑_{l=1}^{r} y_{li}−i−∑_{l=1}^{r} y_{lj}+j}}{1 − q^{∑_{l=1}^{r} y_{li}−i−∑_{l=1}^{r} y_{lj}+j}} ∏_{i=1}^{d}(q^{1−mr+∑_{l=1}^{r} y_{li}−i}; q)_{(r+d)l_{i}} (10.1)

Is there a transformation connecting the two series appearing in (10.1) for arbitrary y_{ij}?

(10) Are there more transformation formulas between Ã_{r} basic hypergeometric series and ordinary multiple basic hypergeometric series? In particular, can (8.5) be generalized?

(11) Many identities in sections 6–8 are subject to rather strange restrictions on the parameters (in particular on ε). Special cases show that they are actually wrong outside the range of these restrictions. There must be some more general identities that would explain the “difference” occurring outside this range.

Acknowledgment. We thank Jeff Remmel for helpful conversations regarding the symmetric function expansions in section 9.

References

1. A. K. Agarwal and G. E. Andrews, Hook differences and lattice paths, J. Statist. Plann. Inference 14 (1986), 5–14.

2. G. E. Andrews, Sieves for theorems of Euler, Rogers and Ramanujan, The Theory of Arithmetic Functions (A. A. Gioia, D. L. Goldsmith, eds.), Lecture Notes in Math., No. 251, Springer, Berlin, 1971, pp. 1–20.

3. G. E. Andrews, Sieves in the theory of partitions, Amer. J. Math. 94 (1972), 1214–1230.

4. G. E. Andrews, The Theory of Partitions, Encyclopedia of Mathematics and its Applications, Vol. 2, Addison–Wesley, Reading, 1976.

5. G. E. Andrews, Uses and extensions of Frobenius’ representation of partitions, Enumeration and Design (D. M. Jackson, S. A. Vanstone, ed.), Academic Press, Toronto, 1984, pp. 51–65.

6. G. E. Andrews, R. J. Baxter, D. M. Bressoud, W. H. Burge, P. J. Forrester and G. Viennot, Partitions with prescribed hook differences, Europ. J. Combin. 8 (1987), 341–350.

7. W. N. Bailey, Series of hypergeometric type which are infinite in both directions, Quart. J. Math. 71 (1936), 105–115.

8. F. Brenti, Determinants of super-Schur functions, lattice paths, and dotted plane partitions, Adv. in Math. 98 (1993), 27–64.

9. F. Brenti, Combinatorics and total positivity, J. Combin. Theory Ser. A 71 (1995), 175–218.

10. D. M. Bressoud, Extension of the partition sieve, J. Number Theory 12 (1980), 87–100.

11. W. H. Burge, A correspondence between partitions related to generalizations of the Rogers-Ra-manujan identities, Discrete Math. 34 (1981), 9–15.

12. W. H. Burge, Restricted partition pairs, J. Combin. Theory Ser. A 63 (1993), 210–222.

13. O. Eğecioğlu and J. B. Remmel, A combinatorial interpretation of the inverse Kostka matrix, Linear and Multilinear Algebra 26 (1990), 59–84.

14. M. Filaseta, A new method for solving a class of ballot problems, J. Combin. Theory A 39 (1985), 102–111.

15. M. Fulmek and C. Krattenthaler, Lattice path proofs for determinant formulas for symplectic and orthogonal characters, J. Combin. Theory Ser. A (to appear).

16. G. Gasper and M. Rahman, Basic hypergeometric series, Encyclopedia of Mathematics And Its Applications 35, Cambridge University Press, Cambridge, 1990.

17. I. M. Gessel and G. Viennot, Binomial determinants, paths, and hook length formulae, Adv. in Math. 58 (1985), 300–321.

18. I. M. Gessel and X. Viennot, Determinants, paths, and plane partitions, preprint.

19. R. A. Gustafson, Multilateral summation theorems for ordinary and basic hypergeometric series in U(n), SIAM J. Math. Anal. 18 (1987), 1576–1596.

20. R. A. Gustafson, The Macdonald identities for affine root systems of classical type and hypergeometric series very-well-poised on semisimple Lie algebras, Ramanujan International Symposium on Analysis (Dec. 26th to 28th, 1987, Pune, India) (N. K. Thakare, ed.), 1989, pp. 187–224.

21. J. E. Humphreys, Reflection groups and Coxeter groups, Cambridge University Press, Cambridge, 1990.

22. V. G. Kac, Infinite-dimensional Lie algebras, third edition, Cambridge University Press, Cambridge, 1990.

23. R. C. King and N. G. El-Sharkaway, Standard Young tableaux and weight multiplicities of the classical Lie groups, J. Phys. A: Math. Gen. 16 (1983), 3153–3177.

24. C. Krattenthaler, Generating functions for plane partitions of a given shape, Manuscripta Math. 69 (1990), 173–202.

25. C. Krattenthaler, A determinant evaluation and some enumeration results for plane partitions, Number-Theoretic Analysis (E. Hlawka, R. F. Tichy, eds.), Lect. Notes in Math. 1452, Springer-Verlag, Berlin, New York, 1990, pp. 121–131.

26. C. Krattenthaler, Generating functions for shifted plane partitions, J. Statist. Plann. Inference 34 (1993), 197–208.

27. C. Krattenthaler, The major counting of nonintersecting lattice paths and generating functions for tableaux, Mem. Amer. Math. Soc. 115, no. 552, Providence, R. I., 1995.

28. C. Krattenthaler, Nonintersecting lattice paths and oscillating tableaux, J. Statist. Plann. Inference (to appear).

29. C. Krattenthaler, Non-crossing two-rowed arrays, in preparation.

30. C. Krattenthaler and S. G. Mohanty, q-Generalization of a ballot problem, Discrete Math. (1993).

31. C. Krattenthaler and S. G. Mohanty, On lattice path counting by major and descents, Europ. J. Combin. 14 (1993), 43–51.

32. A. Lascoux and P. Pragacz, S-function series, J. Phys. A: Math. Gen. 21 (1988), 4105–4114.

33. I. G. Macdonald, Symmetric Functions and Hall Polynomials, Oxford University Press, New York/London, 1979.

34. P. A. MacMahon, Combinatory Analysis, vol. 2, Cambridge University Press, 1916; reprinted by Chelsea, New York, 1960.

35. S. C. Milne, An elementary proof of the Macdonald identities for A_{l}^{(1)}, Adv. in Math. 57 (1985), 34–70.

36. S. C. Milne, Classical partition functions and the U(n + 1) Rogers-Selberg identity, Discrete Math. 99 (1992), 199–246.

37. S. C. Milne and G. M. Lilly, Consequences of the A_{l} and C_{l} Bailey Transform and Bailey Lemma, Discrete Math. 139 (1995), 319–346.

38. J. B. Remmel and M. Yang, Special rim hook tabloids and some new multiplicity-free S-series, SIAM J. Disc. Math. 4 (1991), 253–274.

39. B. E. Sagan, The symmetric group, Wadsworth & Brooks/Cole, Pacific Grove, California, 1991.

40. R. P. Stanley, Theory and applications of plane partitions: Part 1,2, Stud. Appl. Math 50 (1971), 167–188, 259–279.

41. R. P. Stanley, Ordered structures and partitions, Mem. Amer. Math. Soc. No. 119, American Mathematical Society, Providence, R. I., 1972.

42. J. R. Stembridge, Nonintersecting paths, pfaffians and plane partitions, Adv. in Math. 83 (1990), 96–131.

Department of Mathematics, Brandeis University, Waltham, MA 02254–9110, USA.

Institut für Mathematik der Universität Wien, Strudlhofgasse 4, A-1090 Wien, Austria.