THE NUMBER OF RHOMBUS TILINGS OF A SYMMETRIC HEXAGON WHICH CONTAIN A FIXED RHOMBUS ON THE SYMMETRY AXIS, II

M. FULMEK AND C. KRATTENTHALER^{†}

Abstract. We compute the number of rhombus tilings of a hexagon with side lengths N, M, N, N, M, N, with N and M having the same parity, which contain a particular rhombus next to the center of the hexagon. The special case N = M of one of our results solves a problem posed by Propp. In the proofs, Hankel determinants featuring Bernoulli numbers play an important role.

1. Introduction

Let a, b and c be positive integers, and consider a hexagon with side lengths a, b, c, a, b, c whose angles are 120^{∘} (see Figure 1.a). The subject of enumerating rhombus tilings of this hexagon (cf. Figure 1.b; here, and in the sequel, by a rhombus we always mean a rhombus with side lengths 1 and angles of 60^{∘} and 120^{∘}) gained a lot of interest recently. This interest comes from two facts. First, it is a rich source of non-trivial enumeration problems which have (or appear to have) beautiful solutions (see e.g. [5, 6, 7, 8, 9, 13, 24, 42, 44]). Second, these problems are very often related to the theory of symmetric functions and/or the representation theory of classical and quantum Lie algebras, and to statistical physics (sometimes in disguise; see e.g. [10, 15, 17, 25, 26, 34, 38, 39, 40]).

As is well-known, the total number of rhombus tilings of a hexagon with side lengths a, b, c, a, b, c equals

∏_{i=1}^{a} ∏_{j=1}^{b} ∏_{k=1}^{c} \frac{i + j + k − 1}{i + j + k − 2}.  (1.1)

(This follows from MacMahon’s enumeration [28, Sec. 429, q → 1; proof in Sec. 494] of all plane partitions contained in an a × b × c box, as these are in bijection with rhombus tilings of a hexagon with side lengths a, b, c, a, b, c, as explained e.g. in [11].)

A natural question to be asked is what the distribution of the rhombi in a random tiling is. On an asymptotic level, this question was answered by Cohn, Larsen and Propp [10]. On the exact (enumerative) level, the significant contributions are [7, 13, 16]. The most general result was obtained in [13] by the authors, where the number of all rhombus tilings of a hexagon with side lengths N, M, N, N, M, N was computed, which contain an arbitrary fixed rhombus on the symmetry axis which cuts through the sides of length M.

2000 Mathematics Subject Classification. Primary 05A15; Secondary 05A16 05A17 05A19 05B45 11B68 30B70 33C20 33C45 52C20.

Key words and phrases. rhombus tilings, lozenge tilings, plane partitions, nonintersecting lattice paths, determinant evaluations, Bernoulli numbers, Bernoulli polynomials, Hankel determinants, continued fractions, orthogonal polynomials, continuous Hahn polynomials.

† Research partially supported by the Austrian Science Foundation FWF, grant P13190-MAT

Copyright \begin{matrix}c \\ ◯ \end{matrix} 2000 by Academic Press.

a. A hexagon with sides a, b, c, a, b, c, where a = 3, b = 4, c = 5  b. A rhombus tiling of a hexagon with sides a, b, c, a, b, c } c } }_{c{}  }  } a  b a b

Figure 1

The purpose of this paper is to add other results in this direction. We compute the number of all rhombus tilings of a hexagon which contain a particular rhombus on the “other” symmetry axis, i.e., the symmetry axis which runs in parallel to the sides of length M. In difference to [13], we are not able to solve this problem for an arbitrary rhombus on this symmetry axis, but only for rhombi which are close to the center. In fact, the case of the central rhombus is already covered by the papers [7, 13, 16]. We provide results for the “next” three cases, i.e., for a rhombus which is by one, two, or three “half units” off the center, see Theorems 1–6 below. In fact, as our proofs show, the computations become increasingly harder, and more elaborate, as we move the rhombus gradually farther away from the center, so that it seems highly unlikely that a uniform formula, similar to the one in [13] for the other symmetry axis, can be found for an arbitrary rhombus. (For further comments on this issue see Section 8, (2).)

Here are our results. (Each of the following formulas has to be interpreted as the appropriate limit if singularities are encountered. For example, if we directly set m = 1 in (1.6) then the term (m − 1) in the second line becomes 0, and, on the other hand, we have a singularity in the sum in the third line caused by the term (3 − n − m)_{h} in the summand for h = n − 1. The correct way to interpret the expression is as the limit as m goes to 1.)

Theorem 1. Let n and m be positive integers. The number of rhombus tilings of a hexagon with side lengths 2n, 2m, 2n, 2n, 2m, 2n, which contain the rhombus above and next to the center of the hexagon (see Figure 2.a; the rhombus which is contained in every

a. A hexagon with sides N, M, N, N, M, N, where N = 4, M = 2.  b. A hexagon with sides N, M, N, N, M, N, where N = 3, M = 3. N  N  M  N  N M

Figure 2

tiling is shaded), equals

\frac{nm\binom{2n}{n}\binom{2n−1}{n}\binom{2m}{m}}{\binom{4n+2m−1}{2n+m}}(−\frac{1}{(n + m)^{2}} + \frac{4n + 2}{(n + 1)(2n − 1)(n + m − 1)(n + m + 1)}

· ∑_{h=0}^{n−1} \frac{(2)_{h} (1 − n)_{h} (\frac{3}{2} + n)_{h} (1 − n − m)_{h} (1 + n + m)_{h}}{(1)_{h} (2 + n)_{h} (\frac{3}{2} − n)_{h} (2 + n + m)_{h} (2 − n − m)_{h}} )

× ∏_{i=1}^{2n} ∏_{j=1}^{2m} ∏_{k=1}^{2n} \frac{i + j + k − 1}{i + j + k − 2}. (1.2)

where the shifted factorial (a)_{k} is defined by (a)_{k} := a(a+1) ⋯ (a+k−1), k ≥ 1, (a)_{0} := 1.

Theorem 2. Let n be a nonnegative integer and m be a positive integer. The number of rhombus tilings of a hexagon with side lengths 2n+1, 2m−1, 2n+1, 2n+1, 2m−1, 2n+1, which contain the rhombus above and next to the center of the hexagon (see Figure 2.b; the rhombus which is contained in every tiling is shaded), equals

\frac{(n + 1)m\binom{2n}{n}\binom{2n+1}{n}\binom{2m−1}{m}}{\binom{4n+2m}{2n+m}}(\frac{1}{(n + m)^{2}} + \frac{4n}{(n + 1)(2n − 1)(n + m − 1)(n + m + 1)}

· ∑_{h=0}^{n−1} \frac{(2)_{h} (1 − n)_{h} (\frac{3}{2} + n)_{h} (1 − n − m)_{h} (1 + n + m)_{h}}{(1)_{h} (2 + n)_{h} (\frac{3}{2} − n)_{h} (2 + n + m)_{h} (2 − n − m)_{h}} )

× ∏_{i=1}^{2n+1} ∏_{j=1}^{2m−1} ∏_{k=1}^{2n+1} \frac{i + j + k − 1}{i + j + k − 2}. (1.3)

Theorem 3. Let n and m be positive integers. The number of rhombus tilings of a hexagon with side lengths 2n, 2m − 1, 2n, 2n, 2m − 1, 2n, which contain the rhombus above and next to the central rhombus (see Figure 3.a; the rhombus which is contained in every tiling is shaded), equals

\frac{(2m − 1)\binom{2m−2}{m−1}\binom{2n−4}{n−2}\binom{2n+2}{n+1}}{(n + m − 1) (n + m) \binom{4n+2m−3}{2n+m−2}}

× (\frac{n (n + 1) (2n − 3) (2n − 1) (m^{2} − m + n + 2m n + n^{2} + 1)}{(n − 1) (n + m − 1) (n + m) (2n + 1)}

+ \frac{6}{(n + m − 2)(n + m + 1)} ∑_{h=0}^{n−2} \frac{(3)_{h} (\frac{5}{2})_{h} (2 − n)_{h} (\frac{3}{2} + n)_{h} (2 − n − m)_{h} (1 + n + m)_{h}}{(1)_{h} (\frac{3}{2})_{h} (2 + n)_{h} (\frac{5}{2} − n)_{h} (2 + n + m)_{h} (3 − n − m)_{h}} )

× ∏_{i=1}^{2n} ∏_{j=1}^{2m−1} ∏_{k=1}^{2n} \frac{i + j + k − 1}{i + j + k − 2}. (1.4)

Theorem 4. Let n and m be positive integers. The number of rhombus tilings of a hexagon with side lengths 2n−1, 2m, 2n−1, 2n−1, 2m, 2n−1, which contain the rhombus above and next to the central rhombus (see Figure 3.b; the rhombus which is contained in every tiling is shaded), equals

\frac{(2m − 1)\binom{2m−2}{m−1}\binom{2n−4}{n−2}\binom{2n+2}{n+1}}{(n + m − 1) (n + m) \binom{4n+2m−3}{2n+m−2}}

× (\frac{n (n + 1) (2n − 3) (2n − 1) (m^{2} − m − 3n + 2m n + n^{2} + 2)}{(n − 1) (n + m − 1) (n + m) (2n + 1)}

+ \frac{6}{(n + m − 2)(n + m + 1)} ∑_{h=0}^{n−2} \frac{(3)_{h} (\frac{5}{2})_{h} (2 − n)_{h} (\frac{3}{2} + n)_{h} (2 − n − m)_{h} (1 + n + m)_{h}}{(1)_{h} (\frac{3}{2})_{h} (2 + n)_{h} (\frac{5}{2} − n)_{h} (2 + n + m)_{h} (3 − n − m)_{h}} )

× ∏_{i=1}^{2n−1} ∏_{j=1}^{2m} ∏_{k=1}^{2n−1} \frac{i + j + k − 1}{i + j + k − 2}. (1.5)

Theorem 5. Let n and m be positive integers. The number of rhombus tilings of a hexagon with side lengths 2n, 2m, 2n, 2n, 2m, 2n, which contain the rhombus above and next to the rhombus which is adjacent to the center of the hexagon (see Figure 4.a; the rhombus which is contained in every tiling is shaded), equals

\frac{\binom{n+m−1}{m}\binom{2n+2}{n−1}\binom{2n+m−1}{n}\binom{2n+m}{2n+1}}{2(2n − 3)(2n − 1)(2n + 2)(n + m − 1)(n + m + 1)\binom{4n−1}{2n}\binom{4n+2m−1}{2m}}

×(\frac{2 (n + 2) (n + 3) (2n − 1) (2n − 3) X(m, n)}{(n + m − 1) (n + m)^{2} (n + m + 1)} − \frac{24(m − 1)(2n + m + 1)(2n + 1)(2n + 3)}{(n + m − 2)(n + m + 2)}

· ∑_{h=0}^{n−1} \frac{(4)_{h} (1 − n)_{h} (\frac{5}{2} + n)_{h} (2 − n − m)_{h} (2 + n + m)_{h}}{(1)_{h} (4 + n)_{h} (\frac{5}{2} − n)_{h} (3 + n + m)_{h} (3 − n − m)_{h}} ) ∏_{i=1}^{2n} ∏_{j=1}^{2m} ∏_{k=1}^{2n} \frac{i + j + k − 1}{i + j + k − 2}, (1.6)

where

a. A hexagon with sides N, M, N, N, M, N, where N = 2, M = 3.  b. A hexagon with sides N, M, N, N, M, N, where N = 3, M = 2. N  N M  N  N M

Figure 3

X(m, n) = −1+4m^{2}−3m^{4}−n+8mn+11m^{2}n−12m^{3}n−6m^{4}n+8n^{2}+22mn^{2}−4m^{2}n^{2}−24m^{3}n^{2}

− 2m^{4}n^{2} + 15n^{3} + 16mn^{3} − 28m^{2}n^{3} − 8m^{3}n^{3} − 5n^{4} − 8mn^{4} − 12m^{2}n^{4} − 14n^{5} − 8mn^{5} − 2n^{6}.

Theorem 6. Let n and m be positive integers. The number of rhombus tilings of a hexagon with side lengths 2n + 1, 2m − 1, 2n + 1, 2n + 1, 2m − 1, 2n + 1, which contain the rhombus above and next to the rhombus which is adjacent to the center of the hexagon (see Figure 4.b; the rhombus which is contained in every tiling is shaded), equals

\frac{\binom{2m−1}{m}\binom{n+m−1}{m−1}\binom{2n+2}{n−1}\binom{2n+m}{n+1}}{(m + 1)(m + 2)(2n − 3)(n + m − 1)(n + m + 1)\binom{2n+m}{m+2}\binom{4n+2m}{2n+m}}

× (\frac{(n + 2) (n + 3) (2n − 1) (2n − 3) Y (m, n)}{(n + m − 1) (n + m)^{2} (n + m + 1)} − \frac{24(m − 1)n(2n + m + 1)(2n + 3)}{(n + m − 2)(n + m + 2)}

· ∑_{h=0}^{n−1} \frac{(4)_{h} (1 − n)_{h} (\frac{5}{2} + n)_{h} (2 − n − m)_{h} (2 + n + m)_{h}}{(1)_{h} (4 + n)_{h} (\frac{5}{2} − n)_{h} (3 + n + m)_{h} (3 − n − m)_{h}} ) ∏_{i=1}^{2n+1} ∏_{j=1}^{2m−1} ∏_{k=1}^{2n+1} \frac{i + j + k − 1}{i + j + k − 2}, (1.7)

where

Y (m, n) = 1 − 2m^{2} + m^{4} + 5n − 4mn − 2m^{2}n + 4m^{3}n − 3m^{4}n + 22n^{2} − 4mn^{2} − 4m^{2}n^{2}

−12m^{3}n^{2}−2m^{4}n^{2}+50n^{3}−16mn^{3}−26m^{2}n^{3}−8m^{3}n^{3}+39n^{4}−28mn^{4}−12m^{2}n^{4}+5n^{5}−8mn^{5}−2n^{6}.

In general, the sum in (1.2) and (1.3) (note that it is indeed exactly the same sum), the sum in (1.4) and (1.5) (it is indeed exactly the same sum), and the sum in (1.6) and (1.7) (again, it is indeed exactly the same sum), does not simplify. However, in the case that n and m are roughly of the same size, the sum does simplify. For the sake of brevity, we give here just a sample of corollaries to Theorems 1 and 2, the first two statements of

Figure 4

a. A hexagon with sides N, M, N, N, M, N, where N = 4, M = 2.  b. A hexagon with sides N, M, N, N, M, N, where N = 3, M = 3. N  N  M  N  N M

which solve the second part of Problem 4 in Propp’s list [41]. We wish to emphasize that there are similar corollaries to Theorems 3–6.

Corollary 7. Let n be a nonnegative integer. The number of rhombus tilings of a hexagon with all sides of length 2n, which contain the rhombus above and next to the center of the hexagon, equals

(\frac{1}{3} − \frac{1}{12} \begin{matrix}\binom{2n}{n} & 3 \\ ( & {6n \atop 3n} & ) \end{matrix} ) ∏_{i=1}^{2n} ∏_{j=1}^{2n} ∏_{k=1}^{2n} \frac{i + j + k − 1}{i + j + k − 2}.  (1.8)

The number of rhombus tilings of a hexagon with all sides of length 2n + 1, which contain the rhombus above and next to the center of the hexagon, equals

(\frac{1}{3} + \frac{1}{3} \begin{matrix}\binom{2n}{n} & 3 \\ ( & {6n+2 \atop 3n+1} & ) \end{matrix}) ∏_{i=1}^{2n+1} ∏_{j=1}^{2n+1} ∏_{k=1}^{2n+1} \frac{i + j + k − 1}{i + j + k − 2}.  (1.9)

The number of rhombus tilings of a hexagon with side lengths 2n, 2n+2, 2n, 2n, 2n+2, 2n, which contain the rhombus above and next to the center of the hexagon, equals

(\frac{1}{3} − \frac{(10n + 2)}{(6n + 3)} \begin{matrix}\binom{2n}{n} & 3 \\ ( & {6n+2 \atop 3n+1} & ) \end{matrix}) ∏_{i=1}^{2n} ∏_{j=1}^{2n+2} ∏_{k=1}^{2n} \frac{i + j + k − 1}{i + j + k − 2}.  (1.10)

The number of rhombus tilings of a hexagon with side lengths 2n + 1, 2n − 1, 2n + 1, 2n + 1, 2n − 1, 2n + 1, which contain the rhombus above and next to the center of the hexagon, equals

(\frac{1}{3} + \frac{(10n + 3)}{24n} \begin{matrix}\binom{2n}{n} & 3 \\ ( & {6n \atop 3n} & ) \end{matrix} ) ∏_{i=1}^{2n+1} ∏_{j=1}^{2n−1} ∏_{k=1}^{2n+1} \frac{i + j + k − 1}{i + j + k − 2}.  (1.11)

The number of rhombus tilings of a hexagon with side lengths 2n + 2, 2n, 2n + 2, 2n + 2, 2n, 2n + 2, which contain the rhombus above and next to the center of the hexagon, equals

(\frac{1}{3} + 4\begin{matrix}\binom{2n}{n} & 3 \\ ( & {6n+4 \atop 3n+2} & ) \end{matrix}) ∏_{i=1}^{2n+2} ∏_{j=1}^{2n} ∏_{k=1}^{2n+2} \frac{i + j + k − 1}{i + j + k − 2}.  (1.12)

The number of rhombus tilings of a hexagon with side lengths 2n + 3, 2n − 1, 2n + 3, 2n + 3, 2n − 1, 2n + 3, which contain the rhombus above and next to the center of the hexagon, equals

(\frac{1}{3} + \frac{2(6n^{2} + 9n + 2)}{(n + 1)^{2}} \begin{matrix}\binom{2n}{n} & 3 \\ ( & {6n+4 \atop 3n+2} & ) \end{matrix}) ∏_{i=1}^{2n+3} ∏_{j=1}^{2n−1} ∏_{k=1}^{2n+3} \frac{i + j + k − 1}{i + j + k − 2}.  (1.13)

If m is not n + 1, n or n − 1, then one may still try to obtain at least estimates for the number of rhombus tilings that contain this particular rhombus. Indeed, from Theorems 1–6, we are able to derive an “arcsine law” for this kind of enumeration, which is analogous to the ones in [7] and [13].

Corollary 8. Let a be any nonnegative real number, and consider a hexagon with side lengths 2n, 2m, 2n, 2n, 2m, 2n. For m ∼ an, the proportion of the rhombus tilings that contain the rhombus above and next to the center of the hexagon in the total number of rhombus tilings is ∼ \frac{2}{π} arcsin(1/(a + 1)) as n tends to infinity. The same is true for a hexagon with side lengths 2n + 1, 2m − 1, 2n + 1, 2n + 1, 2m − 1, 2n + 1. Moreover, for hexagons with side lengths 2n, 2m − 1, 2n, 2n, 2m − 1, 2n or hexagons with side lengths 2n−1, 2m, 2n−1, 2n−1, 2m, 2n−1, the same result holds for the proportion of the rhombus tilings that contain the rhombus above and next to the central rhombus in the total number of rhombus tilings, as well as for the proportion of the rhombus tilings that contain the rhombus above and next to the rhombus which is adjacent to the center of the hexagon in the total number of rhombus tilings of hexagons with side lengths 2n, 2m, 2n, 2n, 2m, 2n or hexagons with side lengths 2n + 1, 2m − 1, 2n + 1, 2n + 1, 2m − 1, 2n + 1.

Also this result is (as well as Corollary 4 in [7] and Theorem 1.3 in [13]; see the respective comments in [13]) in accordance with Conjecture 1 in [10], to which it adds evidence in further special instances.

In the next sections we describe proofs of Theorems 1–6, and of Corollaries 7 and 8. In Section 2 we provide proofs of Corollaries 7 and 8, and we outline the proofs of Theorems 1–6, the latter consisting of two basic steps. In the first step we build on the approach of Helfgott and Gessel in [16], a short summary of which is the contents of Section 3. It allows to write the number that we are interested in in form of a determinant. The evaluation of this determinant is not easy and is carried out in detail in Section 4. For the evaluation we follow a “method” that was first introduced in [22] (see the tutorial description in [23, Sec. 2.4] or [21, Sec. 2]). For accomplishing the required computations, we need to evaluate certain Hankel determinants featuring Bernoulli numbers, which are, in fact, of independent interest. As it turns out, some of the evaluations of these Hankel determinants are already known, provided certain results about orthogonal polynomials, in particular, about continuous Hahn polynomials, and continued fractions are properly combined. For the convenience of the reader, we collect these facts, and their implications, in Section 5. In particular, the evaluation of the relevant Hankel determinants featuring Bernoulli numbers is given in Theorem 23. However, in the proofs of Theorems 5 and 6 (more precisely, in the proof of the subordinate Lemma 14) we encounter a certain Hankel determinant of Bernoulli numbers (see (6.1)), the evaluation of which requires considerable effort. (This is one of the added difficulties mentioned earlier in comparison to the proofs of Theorems 1 and 2.) We evaluate this Hankel determinant by combining the knowledge about continuous Hahn polynomials with a recent theorem on orthogonal polynomials due to Leclerc [27] (restated here as Theorem 24), and applying some integral calculus (see the proof of Lemma 26). Section 6 is devoted to provide the details of these calculations. In Section 7 we make explicit a few unusual evaluations of Hankel determinants of Bernoulli polynomials, which are implicit in the proofs of our enumeration results. Finally, in Section 8, we point to further directions in this research, and propose a few open problems.

2. Outline of proofs

Here we outline the proofs of Theorems 1–6, and we deduce Corollaries 7 and 8. We fill in the details in the subsequent sections.

Proof of Theorems 1–6. Following the approach of Helfgott and Gessel [16] (see Section 3), we may write the number of rhombus tilings of a hexagon with side lengths N, M, N, N, M, N, which contain an arbitrary rhombus on the (N + M)-long vertical symmetry axis (see Figure 5), in form of a determinant. This determinant is given by Proposition 11. That is to say, in order to prove Theorems 1 and 2, we need to evaluate the determinant in (3.2) with l = \frac{N+M}{2} , and in order to prove Theorems 3 and 4, we need to evaluate the same determinant with l = \frac{N+M+1}{2}, and in order to prove Theorems 5 and 6, we need to evaluate the same determinant with l = \frac{N+M+2}{2}. Modulo replacement of parameters, we may thus concentrate on the determinants D(n, n, N), D(n, n − 1, N) and D(n, n − 2, N), where

D(a, b, N) := det_{1≤i,j≤N} ( ∑_{s=−a}^{b−1} s^{i+j}).  (2.1)

The evaluation of determinants D(n, n, N), D(n, n − 1, N) and D(n, n − 2, N) is then carried out in Lemmas 12, 13 and 14, respectively.

Theorem 1 follows upon combining Proposition 11 with N = 2n, M = 2m, l = n + m and Lemma 12 with n replaced by n + m and N = 2n − 1, and some rearrangement of terms. Similarly, Theorem 2 follows upon combining Proposition 11 with N = 2n + 1, M = 2m − 1, l = n + m and Lemma 12 with n replaced by n + m and N = 2n.

Likewise, Theorem 3 follows upon combining Proposition 11 with N = 2n, M = 2m−1, l = n + m and (4.18) with n replaced by n + m and m replaced by n, and Theorem 4 follows upon combining Proposition 11 with N = 2n − 1, M = 2m, l = n + m and (4.19) with n replaced by n + m and m replaced by n − 1.

Likewise, Theorem 5 follows upon combining Proposition 11 with N = 2n, M = 2m, l = n + m + 1 and (4.23) with n replaced by n + m + 1 and m replaced by n − 1, and Theorem 6 follows upon combining Proposition 11 with N = 2n + 1, M = 2m − 1, l = n + m + 1 and (4.24) with n replaced by n + m + 1 and m replaced by n.

Proof of Corollary 7. We have to compute the value of the expressions (1.2) and (1.3) for m = n + 1 (in order to establish (1.10) and (1.9)), m = n (in order to establish (1.8) and (1.11)), and m = n − 1 (in order to establish (1.12) and (1.13)). Clearly, except for trivial manipulations, we will be done once we are able to evaluate the sum in (1.2) and (1.3) (it is indeed the same sum!) for m = n + 1, m = n, respectively m = n − 1.

We treat the case m = n first. We claim that

∑_{h=0}^{n−1} \frac{(2)_{h} (3/2 + n)_{h} (1 − n)_{h} (1 + 2n)_{h} (1 − 2n)_{h}}{(1)_{h} (3/2 − n)_{h} (2 + n)_{h} (2 − 2n)_{h} (2 + 2n)_{h}} = \frac{(n + 1)(2n − 1)^{2}}{2n^{2}} (\frac{1}{6} + \frac{1}{3} \begin{matrix}( & {6n \atop 3n} & ) \\ \binom{2n}{n} & 3 \end{matrix} ) . (2.2)

Let us denote the sum by S(n) and its summand by F (n, i). We use the Gosper–Zeilberger algorithm [36, 50, 51] to obtain the relation

6n^{2}(n + 2)(6n + 1)(6n + 5) F (n, i) − 6(n + 1)(2n − 1)^{2}(3n + 1)(3n + 2) F (n + 1, i) = G(n, i + 1) − G(n, i), (2.3)

with

G(n, i) = \frac{n(n + 2)(2n − 2h − 1)(2n − h − 1)}{(h + 1)(h − n)(2n − h)(2n − h + 1)(2n + h + 2)}

× (144n^{5} − 432h^{2}n^{4} − 432hn^{4} + 312n^{4} − 936h^{2}n^{3} − 936hn^{3} + 236n^{3} + 108h^{4}n^{2}

+ 216h^{3}n^{2} − 588h^{2}n^{2} − 696hn^{2} + 70n^{2} + 117h^{4}n + 234h^{3}n − 83h^{2}n − 200hn

+ 6n + 24h^{4} + 48h^{3} + 6h^{2} − 18h)F (n, i).

Summation of the relation (2.3) from i = 0 to i = n − 1, and little rearrangement, leads to the recurrence

6n^{2}(n + 2)(6n + 1)(6n + 5) S(n) − 6(n + 1)(2n − 1)^{2}(3n + 1)(3n + 2) S(n + 1)

= \frac{(n + 2)(2n − 1)^{2}(36n^{3} + 60n^{2} + 29n + 3)}{2(n + 1)}.

for the sum in (2.2). (Paule and Schorn’s [35] Mathematica implementation of the Gosper– Zeilberger algorithm, which is the one we used, gives this recurrence directly.) Since S(1) = 1, and since the right-hand side of (2.2) satisfies the same recurrence, equation (2.2) is proved.

The procedure in the other two cases is analogous. The respective evaluations that we need to prove are

∑_{h=0}^{n−1} \frac{(2)_{h} (3/2 + n)_{h} (1 − n)_{h} (2 + 2n)_{h} (−2n)_{h}}{(1)_{h} (3/2 − n)_{h} (2 + n)_{h} (1 − 2n)_{h} (3 + 2n)_{h}} = \frac{(n + 1)^{2}(2n − 1)}{(2n + 1)^{2}} (−\frac{2}{3} + \frac{1}{3} \begin{matrix}( & {6n+2 \atop 3n+1} & ) \\ \binom{2n}{n} & 3 \end{matrix}) (2.4)

in the case that m = n + 1, and

∑_{h=0}^{n−1} \frac{(2)_{h} (3/2 + n)_{h} (1 − n)_{h} (2n)_{h} (2 − 2n)_{h}}{(1)_{h} (3/2 − n)_{h} (2 + n)_{h} (3 − 2n)_{h} (1 + 2n)_{h}} = \frac{2n(n + 1)}{2n + 1} (1 + \frac{n}{12(2n − 1)} \begin{matrix}( & {6n−2 \atop 3n−1} & ) \\ \binom{2n−2}{n−1} & 3 \end{matrix} ) (2.5)

in the case that m = n − 1. We leave it to the reader to fill in the details.

Proof of Corollary 8. We concentrate first on the case of a hexagon with side lengths 2n, 2m, 2n, 2n, 2m, 2n and the rhombus above and next to the center of the hexagon.

From MacMahon’s formula (1.1) for the total number of rhombus tilings together with Theorem 1 we infer that the proportion of the rhombus tilings that contain the rhombus above and next to the center of the hexagon with sides lengths 2n, 2m, 2n, 2n, 2m, 2n in the total number of rhombus tilings is given by

\frac{nm\binom{2n}{n}\binom{2n−1}{n}\binom{2m}{m}}{\binom{4n+2m−1}{2n+m}}(−\frac{1}{(n + m)^{2}} + \frac{4n + 2}{(n + 1)(2n − 1)(n + m − 1)(n + m + 1)}

× ∑_{h=0}^{n−1} \frac{(2)_{h} (3/2 + n)_{h} (1 − n)_{h} (1 + n + m)_{h} (1 − n − m)_{h}}{(1)_{h} (3/2 − n)_{h} (2 + n)_{h} (2 − n − m)_{h} (2 + n + m)_{h}} ).

Using the standard hypergeometric notation

rF_{s}[\begin{matrix}a_{1}, …, a_{r} \\ b_{1}, …, b_{s} \end{matrix}; z] = ∑_{k=0}^{∞} \frac{(a_{1})_{k} ⋯ (a_{r})_{k}}{k! (b_{1})_{k} ⋯ (b_{s})_{k}} z^{k},  (2.6)

we may write the above expression as

\frac{nm\binom{2n}{n}\binom{2n−1}{n}\binom{2m}{m}}{\binom{4n+2m−1}{2n+m}}(−\frac{1}{(n + m)^{2}} + \frac{4n + 2}{(n + 1)(2n − 1)(n + m − 1)(n + m + 1)}

×_{7}F_{6}[\begin{matrix}2, 2, 1 + n + m, 1 − n − m, 1, \frac{3}{2} + n, 1 − n \\ 1, 2 − n − m, 2 + n + m, 2, \frac{3}{2} − n, 2 + n \end{matrix}; 1]). (2.7)

Now we observe that the sum in this expression is in fact a very-well-poised hypergeometric series, and therefore we can transform it into a more convenient form by applying Whipple’s transformation (see [46, (2.4.1.1)])

7F_{6}[\begin{matrix}a, 1 + \frac{a}{2}, b, c, d, e, −N \\ \frac{a}{2}, 1 + a − b, 1 + a − c, 1 + a − d, 1 + a − e, 1 + a + N \end{matrix}; 1]

= \frac{(a + 1)_{N} (a − d − e + 1)_{N}}{(a − d + 1)_{N} (a − e + 1)_{N}} _{4}F_{3}[\begin{matrix}a − b − c + 1, d, e, −N \\ a − b + 1, a − c + 1, −a + d + e − N \end{matrix}; 1] (2.8)

(where N is a nonnegative integer) with a = 2, b = 1 + n + m, c = 1 − n − m, d = 1, e = 3/2 + n, and N = n − 1 to it. Thus we obtain the expression

\frac{nm\binom{2n}{n}\binom{2n−1}{n}\binom{2m}{m}}{\binom{4n+2m−1}{2n+m}}(−\frac{1}{(n + m)^{2}} + \frac{2n + 1}{(n + m − 1)(n + m + 1)}

×_{4}F_{3}[\begin{matrix}1, 1, \frac{3}{2} + n, 1 − n \\ 2 − n − m, 2 + n + m, \frac{3}{2} \end{matrix}; 1]). (2.9)

Now we substitute m ∼ an and perform the limit n → ∞. The asymptotics of the binomials appearing in front of the expression (2.9) is easily determined by means of Stirling’s formula. For the _{4}F_{3}-series itself, we may exchange limit and summation by uniform convergence,

lim_{n→∞} _{4}F_{3}[\begin{matrix}1, 1, \frac{3}{2} + n, 1 − n \\ 2 − n − an, 2 + n + an, \frac{3}{2} \end{matrix}; 1] = _{2}F_{1}[1, 1_{\frac{3}{2}}; \frac{1}{(a + 1)^{2}} ] .  (2.10)

Combining all this, and making use of the identity (see [43, p. 463, (133)])

2F_{1}[1, 1_{\frac{3}{2}}; z] = \begin{matrix}arcsin & \sqrt{z} \\ √ & \overline{z(1 − z)} \end{matrix},  (2.11)

we obtain exactly \frac{2}{π} arcsin(1/(a + 1)) as the asymptotic form of (2.9), and, hence, as the asymptotic form of the proportion of rhombus tilings in the statement of the corollary, as desired.

The case of a hexagon with side lengths 2n + 1, 2m − 1, 2n + 1, 2n + 1, 2m − 1, 2n + 1 can be handled in (almost) the same way because the sums in (1.2) and (1.3) are exactly the same.

For the next two cases, i.e., in order to estimate (1.4) and (1.5), we proceed in a similar way. Again we write the sum which appears in (1.4) and (1.5) as a _{7}F_{6}-series,

7F_{6}[\begin{matrix}3, \frac{5}{2}, 1 + n + m, 2 − n − m, 2, \frac{3}{2} + n, 2 − n \\ \frac{3}{2}, 3 − n − m, 2 + n + m, 2, \frac{5}{2} − n, 2 + n \end{matrix}; 1] ,

apply Whipple’s transformation (2.8), and then let n tend to infinity. Here, the _{2}F_{1}-series which is obtained is a slightly different one as before (compare (2.10)),

2F_{1}[1, 2_{\frac{5}{2}}; \frac{1}{(a + 1)^{2}} ] .

In order to be able to use (2.11), we use the relation

2F_{1}[1, 2_{\frac{5}{2}}; \frac{1}{(a + 1)^{2}} ] = −\frac{3(a + 1)^{2}}{2} + \frac{3 (a + 1)^{2}}{2} _{2}F_{1}[1, 1_{\frac{3}{2}}; \frac{1}{(a + 1)^{2}} ] .

The computation is then completed by straightforward use of Stirling’s formula, and subsequent simplification.

The remaining two cases, i.e., the estimations of (1.6) and (1.7), can be dealt with in just the same manner. The _{2}F_{1}-series which is obtained here is again slightly different. It reads

2F_{1}[1, 2_{\frac{3}{2}}; \frac{1}{(a + 1)^{2}} ] = −\frac{(a + 1)^{2}}{2} + \frac{(a + 1)^{2}}{2}_{2}F_{1}[1, 1_{\frac{1}{2}}; \frac{1}{(a + 1)^{2}} ] .

To the _{2}F_{1}-series on the right-hand side we apply the formula (see [43, p. 464, (138)])

2F_{1}[1, 1_{\frac{1}{2}}; z] = \frac{1}{1 − z} + \begin{matrix}\sqrt{z} & arcsin & \sqrt{z} \\ (1 − z) & \frac{3}{2} \end{matrix} .

Again, the computation is then completed by straightforward use of Stirling’s formula, and subsequent simplification.

3. From rhombus tilings to determinants

This section is entirely based on ideas by Helfgott and Gessel [16]. These allow us to find a determinantal expression for the number of rhombus tilings of a hexagon with side lengths N, M, N, N, M, N that contain an arbitrary fixed rhombus on the (N + M)-long symmetry axis (the vertical symmetry axis in Figure 5). We shall state two auxiliary results (Propositions 9 and 10) without proof (the reader can find the details in [16]), and then derive Helfgott and Gessel’s determinant (Proposition 11). It is the specialization l = (N + M)/2 of Proposition 11 (compare the paragraph above (2.1)) which in the long run leads to a proof of our Theorems 1 and 2, it is the specialization l = (N + M + 1)/2

N  N M  M N  N 2 Figure 5

N M  N  6543210 Figure 6

of Proposition 11 which in the long run leads to a proof of our Theorems 3 and 4, and it is the specialization l = (N + M + 2)/2 of Proposition 11 which in the long run leads to a proof of our Theorems 5 and 6. We do want to alert the reader that we use a different convention in our figures of how to draw the hexagons than Helfgott and Gessel. To be precise, our figures turn into those in [16] by a rotation by 90^{∘}.

The first observation is that for any rhombus tiling of the hexagon with side lengths N, M, N, N, M, N, there are exactly N rhombi of the tiling that are cut in two by the (N + M)-long symmetry axis. Removing these rhombi, and cutting the hexagon in two along the symmetry axis, leaves two symmetric halves of trapezoidal shape with N “dents”. The following statement counts rhombus tilings of such “dented trapezoids”. (This is Lemma 2 in [16].)

N  N M  M N  N 54310 Figure 7

Proposition 9. The number of rhombus tilings of a semi-hexagon with side lengths N, M, N (i.e., the “half” of a hexagon with side lengths N, M, N, N, M, N) and N “dents” at positions 0 ≤ r_{0} < ⋯ < r_{N−1} < N + M (see Figure 6, where N = 4, M = 3, and the “dents” are at positions 0, 1, 4 and 5) is

( ∏_{i=1}^{N−1} \frac{1}{i!}) ∏_{0≤i<j≤N−1}(r_{i} − r_{j}) = ( ∏_{i=1}^{N−1} \frac{1}{i!}) det_{0≤i,j≤N−1} (p_{i}(r_{j})) ,

where p_{i}(x) is an arbitrary monic polynomial of degree i in x. (The “standard” case would be p_{i}(x) = x^{i}, which corresponds to the Vandermonde determinant.)

From this proposition, Helfgott and Gessel deduce another enumeration result. (This is Proposition 4 in [16].)

Proposition 10. Let L be a subset of 0, 1, …, N +M −1 of cardinality at least N. Then the number of rhombus tilings of a hexagon with side lengths N, M, N, N, M, N, in which the set of rhombi on the (N + M)-long symmetry axis is a subset of L (given in terms of the numbers of the rhombi, where the rhombi on the symmetry axis are numbered from bottom to top as 0, 1, …, N +M −1; see Figure 7, where N = 4, M = 3, L = {0, 1, 3, 4, 5}; the set L consists of the shaded rhombi), is

( ∏_{i=1}^{N−1} \frac{1}{i!^{2}} ) det_{0≤i,j≤N−1} ( ∑_{s∈L} p_{i}(s)p_{j}(s)),  (3.1)

where, again, p_{i}(x) is an arbitrary monic polynomial of degree i in x.

From this proposition we can derive the following determinantal expression for the number of rhombus tilings of a hexagon with side lengths N, M, N, N, M, N which contain an arbitrary fixed rhombus on the (N + M)-long symmetry axis (see [16]).

Proposition 11. The number of rhombus tilings of a hexagon with side lengths N, M, N, N, M, N which contain the l-th rhombus, 0 ≤ l ≤ N + M − 1, on the (N + M)-long symmetry axis (see Figure 5; the rhombus which is contained in every tiling is shaded, i.e., l = 2), is

( ∏_{i=1}^{N−1} \frac{1}{i!^{2}} ) det_{1≤i,j≤N−1} ( ∑_{s=−l}^{N+M−l−1} s^{i+j}).  (3.2)

Proof. Let us first count the complementary set, i.e., the rhombus tilings which do not contain rhombus l. Obviously, we get this number from Proposition 10 with L = {0, 1, …, N + M − 1} \ {l}. Then, if in addition we choose p_{i}(x) = (x − l)^{i}, by formula (3.1) this number is

( ∏_{i=1}^{N−1} \frac{1}{i!^{2}} ) det_{0≤i,j≤N−1} (_{∑_{s=0}^{N+M−1} ; s≠l}(s − l)^{i+j}) = ( ∏_{i=1}^{N−1} \frac{1}{i!^{2}} ) det_{0≤i,j≤N−1} (_{∑_{s=−l}^{N+M−l−1} ; s≠0}s^{i+j}). (3.3)

We have to subtract this number from the total number of possible rhombus tilings. This can be again expressed by making use of Proposition 10, this time with L = {0, 1, …, N + M − 1}. So, choosing p_{i}(x) = (x − l)^{i} in (3.1) again, we obtain

( ∏_{i=1}^{N−1} \frac{1}{i!^{2}} ) det_{0≤i,j≤N−1} ( ∑_{s=0}^{N+M−1}(s − l)^{i+j}) = ( ∏_{i=1}^{N−1} \frac{1}{i!^{2}} ) det_{0≤i,j≤N−1} ( ∑_{s=−l}^{N+M−l−1}s^{i+j}) (3.4)

for the total number of rhombus tilings.

It should be observed that the determinants in (3.3) and (3.4) are almost the same. The only difference is the (0, 0)-entry, which is by 1 less in (3.3) than in (3.4). Therefore, if we expand the determinant in (3.3) with respect to the first row, we may rewrite (3.3) as

( ∏_{i=1}^{N−1} \frac{1}{i!^{2}} ) ( det_{0≤i,j≤N−1} ( ∑_{s=−l}^{N+M−l−1} s^{i+j}) − det_{1≤i,j≤N−1} ( ∑_{s=−l}^{N+M−l−1} s^{i+j})) .  (3.5)

The number of rhombus tilings that we are interested in is the difference of (3.4) and (3.5), which is exactly (3.2).

4. Determinant evaluations

Lemma 12. Let n and N be positive integers. Then the determinant D(n, n, N), as defined in (2.1), is equal to

n^{N} ∏_{i=1}^{⌊N/2⌋} ((n^{2} − i^{2})^{N−2i+1}(n^{2} − (i − 1/2)^{2})^{N−2i+1})

× \frac{2^{N^{2}}N! (N + 1)! (n − ⌈N/2⌉)_{2⌈N/2⌉+1} ∏_{i=1}^{N} i!^{4}}{n ⌊N/2⌋!^{2} ⌈N/2⌉!^{2} ∏_{i=1}^{2N+1} i!}

× ((−1)^{N} + \frac{4(N + 2)n^{2} ⌊N/2⌋ ⌈N/2⌉}{(N − 1)N(n^{2} − 1)(⌊N/2⌋ + 1)(⌈N/2⌉ + 1)}

× ∑_{h=0}^{⌈N/2⌉−1} \frac{(2)_{h} (1 − ⌈N/2⌉)_{h} (3/2 + ⌈N/2⌉)_{h} (1 − n)_{h} (1 + n)_{h}}{(1)_{h} (2 + ⌈N/2⌉)_{h} (3/2 − ⌈N/2⌉)_{h} (2 + n)_{h} (2 − n)_{h}} ). (4.1)

Proof. We proceed in several steps. An outline is as follows. In the first step we make the obvious observation that D(n, n, N) is actually a polynomial in n, of degree at most N(N + 2). Next, we show that D(n, n, N), as polynomial in n, has a lot of linear factors. More precisely, in the second step, we show that n^{N} is a factor of D(n, n, N). Then, in the third step, we show that ∏_{i=1}^{⌊N/2⌋}(n^{2} − i^{2})^{N−2i+1} is a factor of D(n, n, N). Moreover, in the fourth step, we show that ∏_{i=1}^{⌊N/2⌋}(n^{2} − (i − 1/2)^{2})^{N−2i+1} is a factor of D(n, n, N). From a combination of these four steps we are forced to conclude that

D(n, n, N) = n^{N}( ∏_{i=1}^{⌊N/2⌋}(n^{2} − i^{2})^{N−2i+1}(n^{2} − (i − 1/2)^{2})^{N−2i+1}) · P (n, N),  (4.2)

where P (n, N) is a polynomial in n of degree at most 2 ⌈N/2⌉. Finally, in the fifth step, we evaluate P (n, N) at n = − ⌈N/2⌉ , − ⌈N/2⌉ + 1, …, ⌈N/2⌉ − 1, ⌈N/2⌉. Namely, we show that

P (0, N) = \frac{(−1)^{N(N−1)/2} 3^{−N}}{∏_{i=1}^{⌊N/2⌋} i^{2(N−2i+1)} (i − 1/2)^{2(N−2i+1)}} ∏_{i=1}^{N−1} (\frac{i (i + 1)^{4} (i + 2)}{(2i + 1) (2i + 2)^{2} (2i + 3)})^{N−i} , (4.3)

and, for 1 ≤ e ≤ ⌈N/2⌉, that

P (±e, N) = \frac{(−1)^{(N−2e+1)(N−2e)/2} 2^{N−2e+1} ∏_{i=1}^{2e−1} i!^{2}}{e^{N}_{∏_{i=1}^{⌊N/2⌋} ; i≠e}(e − i)^{N−2i+1} ∏_{i=1}^{⌊N/2⌋}(e + i)^{N−2i+1}(e^{2} − (i − 1/2)^{2})^{N−2i+1}}

× (\frac{(2e)!^{4}}{(4e + 1)!})^{N−2e+1} ∏_{i=1}^{N−2e} (\frac{i (2e + i)^{4} (4e + i)}{(4e + 2i − 1) (4e + 2i)^{2} (4e + 2i + 1)})^{N−2e+1−i} . (4.4)

Clearly, this determines a polynomial of maximal degree 2 ⌈N/2⌉ uniquely. In fact, an explicit expression for P (n, N) can immediately be written down using Lagrange interpolation. As it turns out, the resulting expression for P (n, N) is exactly the expression covering the last three lines of (4.1). In view of (4.2), this would finish the proof of the Lemma.

Step 1. D(n, n, N) is a polynomial in n of degree at most N(N + 2). It is standard that sums of powers, such as the entries of D(n, n, N), can be expressed using Bernoulli numbers. More precisely, we have (cf. [14, p. 269ff]),

∑_{s=−n}^{n−1} s^{m} = ∑_{ℓ=0}^{m} \frac{1}{ℓ + 1}\binom{m}{ℓ}B_{m−ℓ} (n^{ℓ+1} − (−n)^{ℓ+1}) ,  (4.5)

where B_{ℓ} denotes the ℓ-th Bernoulli number. Hence, the (i, j)-entry of D(n, n, N) is a polynomial in n of degree i + j + 1. Thus, by expanding the determinant D(n, n, N) according to the definition of a determinant and determining the degree of each term, it follows that D(n, n, N) is a polynomial in n of degree at most ∑_{1≤i,j≤N}(i + j + 1) = 2\binom{N+1}{2} + N = N(N + 2).

From (4.5) we may read off another property of D(n, n, N), which we record here for later use,

D(−n, −n, N) = (−1)^{N}D(n, n, N).  (4.6)

Step 2. n^{N} is a factor of D(n, n, N). From the definition (2.1) of the entries of D(n, n, N) and (4.5) it is immediate that n divides each entry of D(n, n, N). Hence, n^{N} divides D(n, n, N).

Step 3. ∏_{i=1}^{⌊N/2⌋}(n^{2} − i^{2})^{N−2i+1} is a factor of D(n, n, N). In view of (4.6), it suffices to prove that (n − e)^{N−2e+1} divides D(n, n, N) for e = 1, 2, …, ⌊N/2⌋. In order to do so, we claim that for each such e there are N − 2e + 1 linear combinations of the columns, which are themselves linearly independent, that vanish for n = e. Define the coefficients a_{j} by

∏_{z=−e}^{e−1} ; z≠0(x − z) = (x − e + 1)_{e−1} (x + 1)_{e} = ∑_{j=0}^{2e−1} a_{j}x^{j}.  (4.7)

Then we claim that for k = 2e, 2e + 1, …, N we have

∑_{j=k−2e+1}^{k} a_{j−k+2e−1} · (column j in D(e, e, N)) = 0.  (4.8)

To verify (4.8), for i = 1, 2, …, N we have to prove that

∑_{j=k−2e+1}^{k} a_{j−k+2e−1} ∑_{s=−e}^{e−1} s^{i+j} = 0.  (4.9)

To do so, we interchange sums in (4.9), and then use (4.7), to obtain

∑_{s=−e}^{e−1} (s^{i+k−2e+1} ∑_{j=0}^{2e−1} a_{j}s^{j}) = ∑_{s=−e}^{e−1} (s^{i+k−2e+1}_{∏_{z=−e}^{e−1} ; z≠0}(s − z))

for the left-hand side in (4.9), which clearly vanishes for any integer e > 0.

Step 4. ∏_{i=1}^{⌊N/2⌋}(n^{2} − (i − 1/2)^{2})^{N−2i+1} is a factor of D(n, n, N). In view of (4.6), it suffices to prove that (n − (e − 1/2))^{N−2e+1} divides D(n, n, N) for e = 1, 2, …, ⌊N/2⌋. In order to do so, we claim, in a manner analogous to Step 3, that for each such e there are N −2e+1 linear combinations of the columns, which are themselves linearly independent, that vanish for n = e − 1/2. Define the coefficients ã_{j} by

∏_{z=−e+1}^{e−1}(x − z + 1/2) = (x − e + 3/2)_{2e−1} = ∑_{j=0}^{2e−1} ã_{j}x^{j}.  (4.10)

Then we claim that for k = 2e, 2e + 1, …, N we have

∑_{j=k−2e+1}^{k} ã_{j−k+2e−1} · (column j in D(e − 1/2, e − 1/2, N)) = 0.  (4.11)

To verify (4.11), for i = 1, 2, …, N we have to prove that

∑_{j=k−2e+1}^{k} ã_{j−k+2e−1} ∑_{ℓ=0}^{i+j} \frac{1}{ℓ + 1}\binom{i + j}{ℓ}B_{i+j−ℓ} ((e − 1/2)^{ℓ+1} − (−e + 1/2)^{ℓ+1}) = 0,

where we used the right-hand side of formula (4.5) to express the entries of D(n, n, N). By a variation of (4.5), the last equation can be rewritten as

∑_{j=k−2e+1}^{k} ã_{j−k+2e−1} ∑_{s=−e+1}^{e−1}(s − 1/2)^{i+j} = 0.  (4.12)

Again, to show that (4.12) is true, we interchange sums, and then use (4.10), to obtain

∑_{s=−e+1}^{e−1} ((s − 1/2)^{i+k−2e+1} ∑_{j=0}^{2e−1} ã_{j}(s − 1/2)^{j}) = ∑_{s=−e+1}^{e−1} ((s − 1/2)^{i+k−2e+1} ∏_{z=−e+1}^{e−1}(s − z))

for the left-hand side in (4.12), which clearly vanishes for any integer e > 0.

Step 5. Evaluation of the polynomial P (n, N) at n = − ⌈N/2⌉ , − ⌈N/2⌉ + 1, …, ⌈N/2⌉. We start by observing that the symmetry relation (4.6) for D(n, n, N) is “inherited” by P (n, N). To be precise, we have

P (−n, N) = P (n, N).  (4.13)

In view of (4.13) it suffices to determine the evaluations of P (n, N) at n = 0, 1, …, ⌈N/2⌉.

What we would like to do is, for any e with 0 ≤ e ≤ ⌈N/2⌉, to set n = e in (4.2), compute D(e, e, N), and then express P (e, N) as the ratio of D(e, e, N) and the righthand side product evaluated at n = e. Unfortunately, this is typically a ratio 0/0 and, hence, undetermined. So, we have to first divide both sides of (4.2) by the appropriate power of (n − e), and only then set n = e.

This program is easily carried out for e = 0. As we observed in Step 2, each entry of D(n, n, N) is divisible by n. Hence, division of both sides of (4.2) by n^{N} and then specializing to 0, transforms (4.2) into the equation

P (0, N) = \frac{det_{1≤i,j≤N} (2B_{i+j})}{∏_{i=1}^{⌊N/2⌋} i^{2(N−2i+1)} (i − 1/2)^{2(N−2i+1)}} .

The Hankel determinant of Bernoulli numbers which appears in the right-hand side expression can be evaluated by Theorem 23 with n replaced by N and a = b = c = d = 1. Thus, we obtain (4.3).

The case 1 ≤ e ≤ ⌈N/2⌉ requires more work. First, we have to “preprocess” the determinant D(n, n, N). Define the coefficients a_{j} as before in (4.7),

∏_{z=−e}^{e−1} ; z≠0(x − z) = (x − e + 1)_{e−1} (x + 1)_{e} = ∑_{j=0}^{2e−1} a_{j}x^{j}.

Then, for k = N, N − 1, …, 2e, in this order, add

∑_{j=k−2e+1}^{k−1} a_{j−k+2e−1} · (column j in D(n, n, N))

to column k. Thus, by (4.8), each entry in column k, k = 2e, 2e + 1, …, N, will be divisible by (n − e) after performing these operations. Next, apply the analogous row operations. I.e., for k = N, N − 1, …, 2e, in this order, add

∑_{j=k−2e+1}^{k−1} a_{j−k+2e−1} · (row j in D(n, n, N))

to row k.

Now we divide (n − e)^{N−2e+1} on both sides of (4.2), and only then set n = e. As a result, from equation (4.2) we obtain

P (e, N) = \frac{det_{1≤i,j,≤N} (\begin{matrix}X & ∗ \\ 0 & Y \end{matrix} )}{e^{N}_{∏_{i=1}^{⌊N/2⌋} ; i≠e}(e − i)^{N−2i+1} ∏_{i=1}^{⌊N/2⌋}(e + i)^{N−2i+1}(e^{2} − (i − 1/2)^{2})^{N−2i+1}}

= \frac{det X det Y}{e^{N}_{∏_{i=1}^{⌊N/2⌋} ; i≠e}(e − i)^{N−2i+1} ∏_{i=1}^{⌊N/2⌋}(e + i)^{N−2i+1}(e^{2} − (i − 1/2)^{2})^{N−2i+1}} ,  (4.14)

where X is the (2e − 1) × (2e − 1) matrix X = ( ∑_{s=−e}^{e−1} s^{i+j})_{i,j=1,...,2e−1}, and where Y is the (N − 2e + 1) × (N − 2e + 1) matrix (Y_{ij})_{i,j=1,...,N−2e+1} whose entries are given by

Y_{ij} = (\frac{1}{n − e} ∑_{s=−n}^{n−1} s^{i+j} ((s − e + 1)_{e−1} (s + 1)_{e})^{2} )|_{n=e} .

Alternatively,

Y_{ij} = \frac{d}{dn}( ∑_{s=−n}^{n−1} s^{i+j} ((s − e + 1)_{e−1} (s + 1)_{e})^{2} )|_{n=e} .  (4.15)

The expression on the right-hand side is a certain linear combination of expressions of the form

H_{m} := \frac{d}{dn}( ∑_{s=−n}^{n−1} s^{m})|_{n=e} .

In order to compute H_{m}, we use (4.5) (and variations thereof) to rewrite it as

H_{m} = ∑_{ℓ=0}^{m} \binom{m}{ℓ}B_{m−ℓ}(e^{ℓ} + (−e)^{ℓ})

= 2B_{m} + ∑_{ℓ=1}^{m} m\binom{m − 1}{ℓ − 1}\frac{1}{ℓ}B_{m−ℓ}(e^{ℓ} + (−e)^{ℓ})

= 2B_{m} + ∑_{s=0}^{e−1} ms^{m−1} − ∑_{s=−e}^{−1} ms^{m−1}

= 2B_{m} + ∑_{s=0}^{e−1} (\frac{d}{ds}s^{m}) − ∑_{s=−e}^{−1} (\frac{d}{ds}s^{m}) .  (4.16)

Thus, using the symbolic notation B^{k} ≡ B_{k} for the Bernoulli numbers, a combination of (4.15) and (4.16) leads to

Y_{ij} = 2B^{i+j} ((B − e + 1)_{e−1} (B + 1)_{e})^{2} + ∑_{s=0}^{e−1} \frac{d}{ds} (s^{i+j} ((s − e + 1)_{e−1} (s + 1)_{e})^{2})

− ∑_{s=−e}^{−1} \frac{d}{ds} (s^{i+j} ((s − e + 1)_{e−1} (s + 1)_{e})^{2})

= 2B^{i+j} ((B − e + 1)_{e−1} (B + 1)_{e})^{2} ,

as each summand in either sum on the right-hand side vanishes.

Substituting all this into (4.14), we arrive at

P (e, N) = \frac{D(e, e, 2e − 1) det_{1≤i,j≤N−2e+1} (2B^{i+j} ((B − e + 1)_{e−1} (B + 1)_{e})^{2})}{e^{N}_{∏_{i=1}^{⌊N/2⌋} ; i≠e}(e − i)^{N−2i+1} ∏_{i=1}^{⌊N/2⌋}(e + i)^{N−2i+1}(e^{2} − (i − 1/2)^{2})^{N−2i+1}}, (4.17)

which is valid for 1 ≤ e ≤ ⌈N/2⌉. The determinant D(e, e, 2e − 1) is evaluated separately in Lemma 15. The Hankel determinant of (linear combinations of) Bernoulli numbers which appears in the right-hand side expression can be evaluated by Theorem 23 with n replaced by N − 2e + 1, a = b = e + 1 and c = d = e. Thus, we obtain (4.4).

This completes the proof of the Lemma.

Lemma 13. Let n and m be positive integers. Then the determinant D(n, n−1, 2m−1), as defined in (2.1), is equal to

\frac{2^{−4m+5}}{(n − m + \frac{1}{2})_{2m−1} (∏_{i=1}^{4m−1} i!)} (2m + 1) (2n − 1)^{2} \binom{2m}{m} \binom{2m − 4}{m − 2}

× (n − m)_{m−2} (n + 2)_{m−2} ( ∏_{i=1}^{2m−1} i!^{4}) ( ∏_{i=1}^{2m−2}(2n − i − 2)_{2i+3})

× (\frac{m (2m − 3) (2m − 1) (n − 2) (n + 1) (n^{2} − n + 2m + 1)}{(m − 1) (2m + 1)}

+ \frac{6(n − 1)n}{(m + 1)} ∑_{h=0}^{m−2} \frac{(3)_{h} (\frac{5}{2})_{h} (2 − m)_{h} (\frac{3}{2} + m)_{h} (2 − n)_{h} (1 + n)_{h}}{(1)_{h} (\frac{3}{2})_{h} (2 + m)_{h} (\frac{5}{2} − m)_{h} (2 + n)_{h} (3 − n)_{h}} ). (4.18)

Likewise, the determinant D(n, n − 1, 2m) is equal to

\frac{4}{∏_{i=1}^{4m+1} i!} (2m + 3) (2n − 1)^{2} \binom{2m − 2}{m − 1} \binom{2m + 2}{m + 1}

× (n − m)_{m−2} (n + 2)_{m−2} (n − m − 1)_{2m+2} (∏_{i=1}^{2m} i!^{4}) ( ∏_{i=1}^{2m−2}(2n − i − 2)_{2i+3})

× (\frac{(2m − 1) (m + 1) (2m + 1) (n − 2) (n + 1) (n^{2} − n − 2m)}{m (2m + 3)}

+ \frac{6(n − 1)n}{(m + 2)} ∑_{h=0}^{m−1} \frac{(3)_{h} (\frac{5}{2})_{h} (1 − m)_{h} (\frac{5}{2} + m)_{h} (2 − n)_{h} (1 + n)_{h}}{(1)_{h} (\frac{3}{2})_{h} (3 + m)_{h} (\frac{3}{2} − m)_{h} (2 + n)_{h} (3 − n)_{h}} ) (4.19)

Proof. Basically, the proof proceeds in the same way as above. By considerations which parallel Steps 1–4 of the previous proof, we deduce that

D(n, n − 1, N) = (n − \frac{1}{2})^{N} ((n − 1)n)^{N−1}

× ( ∏_{i=2}^{⌈(N+1)/2⌉} ((n − i + \frac{1}{2}) (n + i − \frac{3}{2}) (n − i)(n + i − 1))^{N−2i+2} ) · Q(n, N), (4.20)

where Q(n, N) is a polynomial in n of degree at most 2 ⌈(N + 1)/2⌉.

Also, Step 5 of the previous proof has a parallel here. Eventually, this yields evaluations of Q(n, N) at n = − ⌈(N + 1)/2⌉+1, − ⌈(N + 1)/2⌉+2, …, ⌈(N + 1)/2⌉−1, ⌈(N + 1)/2⌉. In particular, the symmetry relation which plays the role of (4.13) in this new context is

Q(−n + 1, N) = Q(n, N),  (4.21)

while the evaluation of determinant D(e, e − 1, 2e − 2) which is needed here (and replaces the evaluation of D(e, e, 2e−1) in this new context; compare (4.17)) is evaluated separately in Lemma 16.

Unfortunately, this is not good enough. The polynomial Q(n, N) is a polynomial of maximal degree 2 ⌈(N + 1)/2⌉, but by now we have found only 2 ⌈(N + 1)/2⌉ explicit special evaluations of Q(n, N). Hence, we need one more information about Q(n, N).

We get this missing piece of information by computing the leading coefficient of Q(n, N). This is easily done. By the definition of D(n, n − 1, N), given by (2.1), by the analogue of (4.5),

∑_{s=−n}^{n−2} s^{m} = ∑_{ℓ=0}^{m} \frac{1}{ℓ + 1}\binom{m}{ℓ}B_{m−ℓ} ((n − 1)^{ℓ+1} − (−n)^{ℓ+1}) ,  (4.22)

and by (4.20), the leading coefficient is given by

det_{1≤i,j≤N} (\begin{cases}\frac{2}{i+j+1} & if i + j is even, \\ 0 & otherwise. \end{cases}) .

It is easy to see, by sorting rows and columns with odd indices to the beginning, that this determinant equals the product of the minor consisting of the odd rows and columns times the minor consisting of the even rows and columns, explicitly

det_{1≤i,j≤⌈N/2⌉} (\frac{2}{2i + 2j − 1}) det_{1≤i,j≤⌊N/2⌋} (\frac{2}{2i + 2j + 1}) .

Both of these determinants are easily evaluated by means of the Cauchy determinant evaluation (see [31, vol. III, p. 311])

det_{1≤i,j≤K} (\frac{1}{x_{i} + y_{j}} ) = \frac{∏_{1≤i<j≤K}(x_{i} − x_{j})(y_{i} − y_{j})}{∏_{1≤i,j≤K}(x_{i} + y_{j})}.

For the simplifications of the resulting expression for Q(n, N) it turns out to be convenient to separate the cases N = 2m and N = 2m − 1.

This concludes the proof of the Lemma.

Lemma 14. Let n and m be positive integers. Then the determinant D(n, n−2, 2m+1), as defined in (2.1), is equal to

\frac{(2m − 2)! (2m + 2)! (n − m − 1)_{2m+1} (n − m − 2)_{2m+3}}{(m − 1)! m! (m + 1)! (m + 4)! (4m + 3)! n(n − 2)}

× ∏_{i=0}^{2m} \frac{(i + 1)!^{3} (2n − i − 2)_{2i+1}}{(2m + i + 2)!} (\frac{2 (m + 3) (m + 4) (2m − 1) (2m + 1) x(m, n)}{(n − 2) (n − 1)^{2}n}

− \frac{24(2m + 3)(2m + 5)(n − m − 3)(n + m + 1)}{(n − 3)(n + 1)}

· ∑_{h=0}^{m} \frac{(4)_{h} (−m)_{h} (\frac{7}{2} + m)_{h} (3 − n)_{h} (1 + n)_{h}}{(1)_{h} (5 + m)_{h} (\frac{3}{2} − m)_{h} (2 + n)_{h} (4 − n)_{h}} ), (4.23)

where Likewise, the determinant D(n, n − 2, 2m) is equal to

x(m, n) = −72m − 204m^{2} − 212m^{3} − 96m^{4} − 16m^{5} − 30n − 86mn − 68m^{2}n − 16m^{3}n

− 29n^{2} + 3mn^{2} + 26m^{2}n^{2} + 8m^{3}n^{2} + 44n^{3} + 40mn^{3} + 8m^{2}n^{3} − 11n^{4} − 10mn^{4} − 2m^{2}n^{4}.

\frac{(2m − 2)! (2m + 2)! (n − m − 1)_{2m+1}^{2}}{(2m − 3) (m − 1)! m! (m + 1)! (m + 3)! (4m + 1)! n(n − 2)}

× ∏_{i=0}^{2m−1} \frac{(i + 1)!^{3} (2n − i − 2)_{2i+1}}{(2m + i + 1)!} (\frac{(m + 2) (m + 3) (2m − 3) (2m − 1) y(m, n)}{(n − 2) (n − 1)^{2}n}

− \frac{24m(2m + 3)(n − m − 2)(n + m)}{(n − 3)(n + 1)} ∑_{h=0}^{m−1} \frac{(4)_{h} (1 − m)_{h} (\frac{5}{2} + m)_{h} (3 − n)_{h} (1 + n)_{h}}{(1)_{h} (4 + m)_{h} (\frac{5}{2} − m)_{h} (2 + n)_{h} (4 − n)_{h}} ), (4.24)

where

y(m, n) = 12m^{2} + 44m^{3} + 48m^{4} + 16m^{5} + 16mn + 28m^{2}n + 16m^{3}n + 4n^{2} − 20mn^{2}

− 22m^{2}n^{2} − 8m^{3}n^{2} − 4n^{3} + 12mn^{3} + 8m^{2}n^{3} + n^{4} − 3mn^{4} − 2m^{2}n^{4}.

Proof. Basically, the proof proceeds in the same was as in the preceding lemmas. By considerations which parallel Steps 1–4 of the proof of Lemma 12, we deduce that

D(n, n − 2, N) = (n − 1)^{N}( ∏_{i=1}^{N−1} ((n − 1)^{2} − \frac{i^{2}}{4} )^{N−i} ∏_{i=2}^{⌊N/2⌋} ((n − 1)^{2} − i^{2}) ) · R(n, N), (4.25)

where R(n, N) is a polynomial in n of degree at most 2 ⌈(N)/2⌉ + 2 if N ≥ 2.

Step 5 of the proof of Lemma 12 has a parallel here, too, which yields evaluations of R(n, N) at n = − ⌈N/2⌉+1, …, 0, 2, …, ⌈N/2⌉+1. In particular, the symmetry relation which plays the role of (4.13) in this context is

R(−n + 2, N) = R(n, N).  (4.26)

While the evaluation of the determinant D(e, e − 2, 2e − 3) which is needed here is done separately in Lemma 17 for e > 2, the case e = 2 leads to the 2 × 2 determinant det_{1≤i,j≤2} ((−1)^{i+j} + (−2)^{i+j}) = 4.

The evaluation of R(n, N) at n = 1 requires extra treatment. First, factoring (n − 1) out of each column of D(n, n − 2, N) and then setting n = 1 yields the determinant det_{1≤i,j≤N} (2(B − 1)^{i+j}) (using again the symbolic notation B^{k} ≡ B_{k}). By row and column operations, this determinant can be transformed into the form

2^{N} det_{1≤i,j≤N} (B^{i+j−2}(B − 1)^{2}).

This determinant agrees with the determinant on the left-hand side of (5.13) with n = N, a = b = 0 and c = d = 2 (adopting the usual convention (B+1)_{−1} := 1/B). Unfortunately, formula (5.13) does not hold for this choice of parameters. In fact, the evaluation of this determinant is rather tedious and therefore given separately in Lemma 25.

Altogether, we have 2 ⌈N/2⌉ + 1 evaluations for our polynomial R(n, N) at special values of n so far. So we are short of exactly two informations on R(n, N). We can get these missing informations by computing the leading coefficient of the polynomial, which is done in the same way as in the proof of the preceding lemma, and by exploiting the symmetry (4.26) once again.

For the simplifications of the resulting expression for R(n, N) it turns out to be again convenient to separate the cases N = 2m and N = 2m + 1.

This concludes the proof of the Lemma.

Lemma 15. For e ≥ 1, we have

D(e, e, 2e − 1) = det_{1≤i,j≤2e−1} ( ∑_{s=−e}^{e−1} s^{i+j}) = ∏_{i=1}^{2e−1} i!^{2}.  (4.27)

(The determinant D(a, b, N) was defined in (2.1).)

Proof. For ℓ ≥ 1 define the polynomials p_{ℓ}(x) by p_{ℓ}(x) := (x − ⌈ℓ/2⌉ + 1)_{ℓ}. Furthermore, define coefficients a_{ℓ,k}, k = 1, 2, …, ℓ, by p_{ℓ}(x) = ∑_{k=1}^{ℓ} a_{ℓ,k}x^{k}. Now, for ℓ = 2e − 1, 2e − 2, …, 2, in this order, we replace column c_{ℓ} of the matrix underlying D(e, e, 2e−1) by the linear combination of columns ∑_{k=1}^{ℓ} a_{ℓ,k}c_{k}, and afterwards do the same sort of replacement in the rows. These operations yield a matrix with (i, j)-entry equal to

∑_{s=−e}^{e−1} p_{i}(s)p_{j}(s) = ∑_{s=−e}^{e−1}(s − ⌈i/2⌉ + 1)_{i} (s − ⌈j/2⌉ + 1)_{j}.  (4.28)

The next sequence of operations will turn this matrix into upper triangular form, so that the determinant is easily obtained by forming the product of the diagonal entries.

To begin with, it should be noted that for j = 2e − 1 the sum (4.28) consists of just the term corresponding to s = −e. We subtract p_{j}(−e)/p_{2e−1}(−e) times column 2e − 1 from column j, j = 1, 2, …, 2e − 2. The previous observation tells us that these operations have the effect that the entries that are in one of the columns j = 1, 2, …, 2e − 2 become

∑_{s=−e+1}^{e−1} p_{i}(s)p_{j}(s) = ∑_{s=−e+1}^{e−1}(s − ⌈i/2⌉ + 1)_{i} (s − ⌈j/2⌉ + 1)_{j},  (4.29)

i.e., the summand corresponding to s = −e has been eliminated. In particular, all the entries in the last row, except the rightmost entry, of course, are 0.

Next consider column 2e − 2. After the above column operations, the sum (4.29) with j = 2e − 2 which defines the entries collapses to just the term corresponding to s = e − 1. We subtract p_{j}(e−1)/p_{2e−2}(e−1) times column 2e−2 from column j, j = 1, 2, …, 2e−3. The previous observation tells us that these operations have the effect that now the entries that are in one of the columns j = 1, 2, …, 2e − 3 become

∑_{s=−e+1}^{e−2} p_{i}(s)p_{j}(s) = ∑_{s=−e+1}^{e−2}(s − ⌈i/2⌉ + 1)_{i} (s − ⌈j/2⌉ + 1)_{j},  (4.30)

i.e., the summand corresponding to s = e − 1 has been eliminated as well. In particular, all the entries in the next-to-last row, except the two rightmost entries, of course, are 0.

If we continue in the same manner, then eventually we arrive at an upper triangular matrix whose i-th diagonal entry is i!^{2}. Thus, the result (4.27) follows.

Lemma 16. For e ≥ 2 we have

D(e, e − 1, 2e − 2) = det_{1≤i,j≤2e−2} ( ∑_{s=−e}^{e−2} s^{i+j}) = ∏_{i=1}^{2e−2} i!^{2}.  (4.31)

(The determinant D(a, b, N) was defined in (2.1).)

Proof. This identity can be established in essentially the same manner as the previous lemma. The basic difference is that one has to replace the polynomials p_{ℓ}(x) in the previous proof by the polynomials

q_{ℓ}(x) := \begin{cases}x & if ℓ = 1, \\ (x − ⌊ℓ/2⌋ + 1)_{ℓ} & if ℓ > 1. \end{cases}

Everything else is completely analogous. We leave the details to the reader.

Lemma 17. For e ≥ 3 we have

D(e, e − 2, 2e − 3) = det_{1≤i,j≤2e−3} ( ∑_{s=−e}^{e−3} s^{i+j}) = ∏_{i=1}^{2e−3} i!^{2}.  (4.32)

(The determinant D(a, b, N) was defined in (2.1).)

Proof. This identity can be established in essentially the same manner as the previous lemmas. The basic difference is that one has to replace the polynomials in the previous proofs by the polynomials

r_{ℓ}(x) := \begin{cases}x & if ℓ = 1, \\ x(x + 1) & if ℓ = 2, \\ (x − ⌈ℓ/2⌉ + 2)_{ℓ} & if ℓ > 2. \end{cases}

Everything else is completely analogous. We leave the details to the reader.

5. Orthogonal polynomials, continued fractions, and Hankel determinants of Bernoulli numbers

In this section we review some facts about the interrelations between orthogonal polynomials, continued fractions, and Hankel determinants. Good sources for information about these topics are [18, 20, 37, 47, 48, 49].

To begin with, we recall Favard’s Theorem.

Theorem 18. (Cf. [48, Théorème 9 on p. I-4] or [49, Theorem 50.1]). Let (p_{n}(x))_{n≥0} be a sequence of monic polynomials, the polynomial p_{n}(x) having degree n. Then the sequence (p_{n}(x)) is (formally) orthogonal with respect to a linear functional L, i.e., L(p_{n}(x)p_{m}(x)) = δ_{mn}c_{n} for some sequence (c_{n})_{n≥0} of nonzero numbers, with δ_{m,n} denoting the Kronecker delta (i.e., δ_{m,n} = 1 if m = n and δ_{m,n} = 0 otherwise) if and only if there exist sequences (a_{n})_{n≥1} and (b_{n})_{n≥1}, with b_{n} ≠ 0 for all n ≥ 1, such that the three-term recurrence

p_{n+1}(x) = (a_{n} + x)p_{n}(x) − b_{n}p_{n−1}(x),  for n ≥ 1,  (5.1)

holds, with initial conditions p_{0}(x) = 1 and p_{1}(x) = x + a_{0}.

It is a simple fact that, given a linear functional, the corresponding orthogonal polynomials can be expressed in form of certain determinants.

Theorem 19. (Cf. [49, Theorem 7.15, (7.2.41)] or [48, (36)]). Let L be a linear functional defined on polynomials with moments μ_{n} = L(x^{n}). Then the corresponding set (p_{n}(x))_{n≥0} of orthogonal polynomials is given by

p_{n}(x) = d_{n}^{−1} det (\begin{matrix}μ_{0} & & μ_{1} & & & μ_{2} & & … & & μ_{n} \\ μ_{1} & & μ_{2} & & & … & & μ_{n} & & μ_{n+1} \\ μ_{2} & & … & & & μ_{n} & & μ_{n+1} & & μ_{n+2} \\ . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . \\ μ_{n−1} & & μ_{n} & & & μ_{n+1} & & … & & μ_{2n−1} \\ 1 & & x & & & … & & x^{n−1} & & x^{n} \end{matrix}) ,  (5.2)

where d_{n} = det_{0≤i,j≤n−1}(μ_{i+j}).

The next theorem addresses the relation between orthogonal polynomials and continued fractions, the link being the generating function of the moments.

Theorem 20. (Cf. [49, Theorem 51.1] or [48, Proposition 1, (7), on p. V-5]). Let (p_{n}(x))_{n≥0} be a sequence of monic polynomials, the polynomial p_{n}(x) having degree n, which is orthogonal with respect to some functional L. Let

p_{n+1}(x) = (a_{n} + x)p_{n}(x) − b_{n}p_{n−1}(x)  (5.3)

be the corresponding three-term recurrence which is guaranteed by Favard’s theorem. Then the generating function for the moments μ_{n} = L(x^{n}) satisfies

∑_{n=0}^{∞} μ_{n}x^{n} = \frac{μ_{0}}{1 + a_{0}x − \frac{b_{1}x^{2}}{1 + a_{1}x − \frac{b_{2}x^{2}}{1 + a_{2}x − ⋯}}} .  (5.4)

We remark that a continued fraction of the type as in (5.4) is called a J-fraction.

The next theorem addresses the relation between J-fractions and Hankel determinants.

Theorem 21. (Cf. [49, Theorem 51.1] or [48, Corollaire 6, (19), on p. IV-17]). Let (μ_{n})_{n≥0} be a sequence of numbers whose generating function, when written in terms of a J-fraction, is given by (5.4). Then the Hankel determinant det_{0≤i,j≤n−1}(μ_{i+j}) equals μ_{0}^{n}b_{1}^{n−1}b_{2}^{n−2} ⋯ b_{n−2}^{2}b_{n−1}.

The family of orthogonal polynomials that is of interest to us is the continuous Hahn polynomials, which were first studied by Atakishiyev and Suslov [3] and Askey [2]. The theorem below lists the relevant facts about these polynomials.

Theorem 22. (Cf. [20, Section 1.4]). The continuous Hahn polynomials (p_{n}(a, b, c, d; x))_{n≥0} are the monic polynomials defined as the terminating hypergeometric series

p_{n}(a, b, c, d; x)

= \frac{(\sqrt{−1})^{n} (a + c)_{n} (a + d)_{n}}{(a + b + c + d + n − 1)_{n}} _{3}F_{2}[\begin{matrix}−n, n + a + b + c + d − 1, a + x\sqrt{−1} \\ a + c, a + d \end{matrix}; 1] . (5.5)

These polynomials satisfy the three-term recurrence

p_{n+1}(a, b, c, d; x) = (x − A_{n}(a, b, c, d))p_{n}(a, b, c, d; x) − B_{n}(a, b, c, d)p_{n−1}(a, b, c, d; x), (5.6) where

A_{n}(a, b, c, d) = \sqrt{−1} (a + \frac{n (b + c + n − 1) (b + d + n − 1)}{(a + b + c + d + 2n − 2) (a + b + c + d + 2n − 1)} +

\begin{matrix}(1 − a − b − c − d − n) & (a + c + n) & (a + d + n) \\ (−1 + a + b + c + d + 2n) & (a + b + c + d + 2n) \end{matrix}), (5.7)

and

B_{n}(a, b, c, d) = −\frac{n (a + c + n − 1) (b + c + n − 1) (a + d + n − 1)}{(a + b + c + d + 2n − 3) (a + b + c + d + 2n − 2)^{2}}

× \frac{(b + d + n − 1) (a + b + c + d + n − 2)}{(a + b + c + d + 2n − 1)}. (5.8)

They are orthogonal with respect to the functional L which is given by

L(p(x)) = \frac{1}{2π} ∫_{−∞}^{∞} Γ(a + x\sqrt{−1}) Γ(b + x\sqrt{−1}) Γ(c − x\sqrt{−1}) Γ(d − x\sqrt{−1}) p(x) dx . (5.9)

In this integral, if necessary, the path of integration has to be deformed, so that it separates the increasing sequences ((a+k)\sqrt{−1})_{k≥0} and ((b+k)\sqrt{−1})_{k≥0} of poles from the decreasing sequences ( − (c + k)\sqrt{−1})_{k≥0} and ( − (d + k)\sqrt{−1})_{k≥0} of poles (see [2]).

Explicitly, the orthogonality relation is

L(p_{m}(a, b, c, d; x)p_{n}(a, b, c, d; x))

= \frac{Γ(n + a + c) Γ(n + a + d) Γ(n + b + c) Γ(n + b + d) n!}{(a + b + c + d + n − 1)_{n} (a + b + c + d + n − 1)_{n+1}Γ(n + a + b + c + d − 1)}δ_{m,n}. (5.10)

In particular,

L(1) = \frac{(a + c − 1)! (b + c − 1)! (a + d − 1)! (b + d − 1)!}{(a + b + c + d − 1)!}.  (5.11)

Remark. (1) The reader should be aware that Theorem 22 is formulated for the monic form of the continuous Hahn polynomials, so that our polynomials are not the polynomials which are denoted by p_{n}(x; a, b, c, d) in [20, Section 1.4] but those which are denoted there, slightly confusingly, by p_{n}(x).

(2) Although the definition (5.5) does not show it, the continuous Hahn polynomials are symmetric in a and b, and in c and d, because the orthogonality measure (5.9) has these symmetries. In addition, there is another symmetry when the roles of a, b are interchanged with the roles of c, d. Namely, we have

p_{n}(a, b, c, d; x) = (−1)^{n}p_{n}(c, d, a, b; −x).  (5.12)

This follows from the fact that, when performing the substitution x → −x in the integral (5.9), we obtain almost the same integrand, the only differences being that the roles of a, b and c, d are interchanged, and that p(x) is replaced by p(−x). The path of integration also does not change, at least as long as all of a, b, c, d are positive. This is, however, sufficient to conclude that (5.12) holds for all a, b, c, d because, for fixed n, both sides of (5.12) are rational of bounded degree in a, b, c, d. (The same conclusion could also be less elegantly derived by applying some _{3}F_{2}-transformation formulas.)

Now, by combining Theorems 20, 21, and 22 we are able to derive without difficulty the determinant evaluation that we need in Step 5 of the proof of Lemma 12 and the analogous places in Lemmas 13 and 14 (which, in turn, are essential for the proofs of Theorems 1–6).

Theorem 23. For positive integers a, b and nonnegative integers c, d there holds

det_{1≤i,j≤n} (B^{i+j} (B + 1)_{a−1} (B + 1)_{b−1} (−B + 1)_{c−1} (−B + 1)_{d−1})

= (−1)^{n(n−1)/2} (\frac{(a + c − 1)! (b + c − 1)! (a + d − 1)! (b + d − 1)!}{(a + b + c + d − 1)!})^{n}

× ∏_{i=1}^{n−1}(\frac{i (a + c + i − 1) (b + c + i − 1) (a + d + i − 1)}{(a + b + c + d + 2i − 3) (a + b + c + d + 2i − 2)^{2}}

× \frac{(b + d + i − 1) (a + b + c + d + i − 2)}{(a + b + c + d + 2i − 1)})^{n−i}, (5.13)

where we have again used the symbolic notation B^{k} ≡ B_{k}. (In case that c or d should be zero, we have to interpret (−B + 1)_{−1} as 1/(−B).)

Proof. We claim that the moments for the continuous Hahn polynomials, i.e., for the linear functional as defined in (5.9), are

μ_{n} = (B/\sqrt{−1})^{n}B^{2} (B + 1)_{a−1} (B + 1)_{b−1} (−B + 1)_{c−1} (−B + 1)_{d−1}.  (5.14)

Let us suppose for the moment that this is already established. Then, by Theorem 20 and (5.6) of Theorem 22, we have

∑_{n≥0} (B/\sqrt{−1})^{n}B^{2} (B + 1)_{a−1} (B + 1)_{b−1} (−B + 1)_{c−1} (−B + 1)_{d−1}x^{n}

= \frac{B^{2} (B + 1)_{a−1} (B + 1)_{b−1} (−B + 1)_{c−1} (−B + 1)_{d−1}}{1 + A_{0}(a, b, c, d)x − \frac{B_{1}(a, b, c, d)x^{2}}{1 + A_{1}(a, b, c, d)x − \frac{B_{2}(a, b, c, d)x^{2}}{1 + A_{2}(a, b, c, d)x − ⋯}}} .

But, by Theorem 21, this immediately implies the truth of (5.13).

It remains to verify (5.14). Using (5.9), this is a rather straightforward computation:

μ_{n} = \frac{1}{2π} ∫_{−∞}^{∞} Γ(a + x\sqrt{−1}) Γ(b + x\sqrt{−1}) Γ(c − x\sqrt{−1}) Γ(d − x\sqrt{−1}) x^{n} dx

= \frac{1}{2π} ∫_{−∞\sqrt{−1}}^{∞\sqrt{−1}} Γ(a + z) Γ(b + z) Γ(c − z) Γ(d − z)\frac{z^{n}}{(\sqrt{−1})^{n+1}} dz

= \frac{1}{2π(\sqrt{−1})^{n+1}} ∫_{−∞\sqrt{−1}}^{∞\sqrt{−1}}(z)_{a} (z)_{b} (−z)_{c} (−z)_{d} (Γ(z) Γ(−z))^{2} z^{n} dz

= \frac{1}{2π(\sqrt{−1})^{n+1}} ∫_{−∞\sqrt{−1}}^{∞\sqrt{−1}}(z)_{a} (z)_{b} (−z)_{c} (−z)_{d} (−\frac{π}{z sin πz} )^{2} z^{n} dz

= \frac{1}{2π(\sqrt{−1})^{n+1}} ∫_{−∞\sqrt{−1}}^{∞\sqrt{−1}}(z + 1)_{a−1} (z + 1)_{b−1} (−z + 1)_{c−1} (−z + 1)_{d−1} (\frac{π}{sin πz} )^{2} z^{n+2} dz. (5.15)

In the third line we used the relation Γ(a+z) = (z)_{a} Γ(z) for a ∈ ℕ (see, e.g., [12, 1.2(2)]), and in the fourth line we used the formula Γ(z) Γ(−z) = −\frac{π}{z sin πz} (see [12, 1.2(5)]). The reader should notice that, because of the convention regarding the path of integration in (5.9), in case that c or d are zero the path of integration in the last line of (5.15) is deformed so that it crosses the real axis between the singularities z = −1 and z = 0.

To finish the calculation, we appeal to the following integral representation of Bernoulli numbers (see [33, p. 75], and let α → 0^{+} in the relevant identity in the middle of the page)

B_{ν} = \frac{1}{2π\sqrt{−1}} ∫_{−∞\sqrt{−1}}^{∞\sqrt{−1}} z^{ν} (\frac{π}{sin πz} )^{2} dz.  (5.16)

(If ν = 0 or ν = 1 then the path of integration is indented so that it avoids the singularity z = 0, passing it on the negative side, but to the right of the singularity z = −1.) If we use this formula in (5.15), we obtain (5.14) immediately.

At least two special cases of Theorem 23 have explicitly appeared in the literature before. The case a = b = 1, c = d = 0 appears for example in [1, (3.1)]. The case a = b = c = d = 1 appears in [30, App. A.5, p. 322].

In the proof of Theorems 5 and 6 (to be precise, in the proof of Lemma 25), we make use of a rather recent result on (formal) orthogonal polynomials, due to Leclerc [27, Theorem 1].

Theorem 24. For an arbitrary sequence of numbers (μ_{n})_{n≥0} let (P_{n}(x))_{n≥0} be the sequence of polynomials defined by

P_{n}(x) := det (\begin{matrix}μ_{0} & & μ_{1} & & & μ_{2} & & … & & μ_{n} \\ μ_{1} & & μ_{2} & & & … & & μ_{n} & & μ_{n+1} \\ μ_{2} & & … & & & μ_{n} & & μ_{n+1} & & μ_{n+2} \\ . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . & . \\ μ_{n−1} & & μ_{n} & & & μ_{n+1} & & … & & μ_{2n−1} \\ 1 & & x & & & … & & x^{n−1} & & x^{n} \end{matrix}) .  (5.17)

(These are, up to normalization, the orthogonal polynomials with respect to the linear functional with moments (μ_{n}); compare Theorem 19.) Furthermore, let (Q_{n}(x))_{n≥0} be the sequence of polynomials defined by

Q_{n}(x) := ∑_{k=0}^{n} μ_{k}\binom{n}{k}(−x)^{n−k}.

Then, for arbitrary integers l ≥ 1, m ≥ 1, there holds

det (\begin{matrix}P_{l}(x) & P_{l+1}(x) & … & P_{l+m−1}(x) \\ P′_{l}(x) & P′_{l+1}(x) & … & P′_{l+m−1}(x) \\ ⋮ & ⋮ & & ⋮ \\ P_{l}^{(m−1)}(x) & P_{l+1}^{(m−1)}(x) & … & P_{l+m−1}^{(m−1)}(x) \end{matrix})

= C_{l,m} det (\begin{matrix}Q_{m}(x) & Q_{m+1}(x) & … & Q_{m+l−1}(x) \\ Q_{m+1}(x) & Q_{m+2}(x) & … & Q_{m+l}(x) \\ ⋮ & ⋮ & & ⋮ \\ Q_{m+l−1}(x) & Q_{m+l}(x) & … & Q_{m+2l−2}(x) \end{matrix}) , (5.18)

where

C_{l,m} = (−1)^{lm} ∏_{k=1}^{m−1} k! det (\begin{matrix}μ_{0} & μ_{1} & … & μ_{k+l−1} \\ μ_{1} & μ_{2} & … & μ_{k+l} \\ ⋮ & ⋮ & & ⋮ \\ μ_{k+l−1} & μ_{k+l} & … & μ_{2k+2l−2} \end{matrix}) .  (5.19)

6. Auxiliary lemmas

Lemma 25. For any integer n ≥ 1, we have

det_{1≤i,j≤n} (B^{i+j−2}(B − 1)^{2})

= (−1)^{n(n−1)/2}6^{−n}(1 + 2n + 5n^{2} + 4n^{3} + n^{4}) ∏_{i=1}^{n} (\frac{i(i + 1)^{4}(i + 2)}{(2i + 1)(2i + 2)^{2}(2i + 3)})^{n−i} (6.1)

(again, using the symbolic notation B^{k} ≡ B_{k}).

Proof. The determinant in (6.1) is equal to the determinant in (5.13) with a = b = 0 and c = d = 2 (again, with the convention that (B+1)_{−1} is interpreted as 1/B). Unfortunately, for this choice of parameters, formula (5.13) is not valid. However, the determinant in (6.1) is very close to the determinant in (5.13) with a = b = 2 and c = d = 0. In fact, because of the well-known property of Bernoulli numbers that B_{2k+1} = 0 for all positive integers k, multiplication of all even numbered rows and columns of the determinant in (5.13) by −1 (which does not change the value of the determinant) with this choice of parameters transforms the latter into the determinant Δ_{n} := det_{1≤i,j≤n}(λ_{i+j−2}), where

λ_{0} = B^{0}(B + 1)^{2} = B^{0}(B − 1)^{2} − 2,
λ_{1} = −B^{1}(B + 1)^{2} = B^{1}(B − 1)^{2} + 1,
λ_{n} = (−1)^{n}B^{n}(B + 1)^{2} = B^{n}(B − 1)^{2} for n ≥ 2.  (6.2)

So, because of the deviating definitions of λ_{0} and λ_{1}, the only difference between Δ_{n} and the determinant in (6.1) is in the top-left entry and its right and bottom neighbour. Linearity of the determinant Δ_{n} in the first row and column then implies that

det_{1≤i,j≤n} (B^{i+j−2}(B − 1)^{2}) = Δ_{n} − Δ_{n}^{{1,2};{1,2}} + 2Δ_{n}^{{1};{1}} + 2Δ_{n}^{{1};{2}}.  (6.3)

Here, A^{{i_{1},i_{2},... };{j_{1},j_{2}... }} denotes the minor of A with rows i_{1}, i_{2}, … and columns j_{1}, j_{2} … deleted. (Empty minors are defined to be zero).

Now observe that from Theorem 23 with a = b = 2 and c = d = 0 we obtain immediately that

Δ_{n} = (−1)^{n(n−1)/2} ∏_{i=1}^{n} \frac{(i − 1)! i!^{4} (i + 1)!}{(2i)! (2i + 1)!}.  (6.4)

Likewise, from Theorem 23 with a = b = 1 and c = d = 2 we have

Δ_{n}^{\{1\};\{1\}} = (−1)^{(n−1)(n−2)/2} ∏_{i=1}^{n−1} \frac{(i − 1)! (i + 1)!^{4} (i + 3)!}{(2i + 2)! (2i + 3)!}.  (6.5)

Our next observation is that Δ_{n}^{{1};{2}} is, essentially, the coefficient of x in the continuous Hahn polynomial p_{n−1}(0, 1, 2, 2; x). To make a more precise statement, consider (5.2) with μ_{k} = λ_{k+1}, k = 0, 1, . . .. Then, obviously, Δ_{n}^{{1};{2}} equals

(−1)^{n} det_{1≤i,j≤n−1}(λ_{i+j−1}) · (coefficient of x in p_{n−1}(x)).

On the other hand, Theorem 19 says that, with this choice of the μ_{k}, the polynomials p_{n}(x) are orthogonal with associated moments λ_{k+1}, k = 0, 1, . . .. By comparing (6.2) and (5.14) with a = b = 2, c = 0, and d = 1, we see that we must have

p_{n}(x) = (−\sqrt{−1})^{n} p_{n}(2, 2, 0, 1; −x/\sqrt{−1}).

Therefore, if we remember (5.12), we obtain that Δ_{n}^{{1};{2}} equals

(−1)^{n}(\sqrt{−1})^{n−1} det_{1≤i,j≤n−1}(λ_{i+j−1}) · (coefficient of x in p_{n−1}(0, 1, 2, 2; x/\sqrt{−1})). (6.6)

The Hankel determinant in this expression can be evaluated by using Theorem 23 with a = b = 2, c = 0, d = 1, and n replaced by n − 1. By substituting the result in (6.6) and by using the definition (5.5) of continuous Hahn polynomials with a = 0, b = 1, c = d = 2, we obtain

Δ_{n}^{\{1\};\{2\}} = (−1)^{n(n+1)/2} \frac{n!^{2}}{(n + 3)_{n−1}}

× ( ∏_{i=1}^{n−1} \frac{(i − 1)! i!^{2} (i + 1)!^{2} (i + 2)!}{(2i + 1)! (2i + 2)!}) ∑_{k=1}^{n−1} \frac{(1 − n)_{k} (n + 3)_{k}}{k (k + 1)!^{2}} . (6.7)

The remaining minor Δ_{n}^{{1,2};{1,2}} requires additional work. We employ Theorem 24 with l = n − 2, m = 2, x = 0, and μ_{k} = λ_{k+2}, k = 0, 1, . . .. With this choice of parameters, the determinant on the right-hand side of (5.18) is precisely our remaining minor Δ_{n}^{{1,2};{1,2}}.

The single determinant occurring in the definition (5.19) of C_{n−2,2} can be evaluated by using Theorem 23 with a = b = 1, c = d = 2, and n replaced by n − 2, so that we have

C_{n−2,2} = (−1)^{(n−1)(n−2)/2} ∏_{i=1}^{n−1} \frac{(i − 1)! (i + 1)!^{4} (i + 3)!}{(2i + 2)! (2i + 3)!}.  (6.8)

The determinant on the left-hand side (5.18) is a 2 × 2-determinant whose entries are the constant term and the coefficient of x, respectively, of P_{n−2}(x) and P_{n−1}(x). The polynomials P_{n}(x), n = 0, 1, …, on the other hand, are orthogonal polynomials with associated moments λ_{k+2}, k = 0, 1, … (compare (5.17) and Theorem 19). By comparing the definition (6.2) of the λ_{i}’s with (5.14), it is seen that the polynomials P_{n}(x) must agree, up to normalization, with the continuous Hahn polynomials with parameters a = b = 1 and c = d = 2. To be precise, we have

P_{n}(x) = (\sqrt{−1})^{n} det_{1≤i,j≤n}(λ_{i+j}) p_{n}(1, 1, 2, 2; x/\sqrt{−1}).

Clearly, the Hankel determinant on the right-hand side can be once again evaluated by means of Theorem 23 with a = b = 1 and c = d = 2. In summary, from (5.18) with the above choice of parameters we infer

Δ_{n}^{\{1,2\};\{1,2\}} = (−1)^{n(n−1)/2} \frac{(3)_{n−2}^{2} (3)_{n−1}^{2}}{(n + 3)_{n−2} (n + 4)_{n−1}} (∏_{i=1}^{n−2} \frac{(i − 1)! (i + 1)!^{4} (i + 3)!}{(2i + 2)! (2i + 3)!})

× (c_{0,n−2}c_{1,n−1} − c_{0,n−1}c_{1,n−2}) , (6.9)

where c_{0,n} and c_{1,n} denote the coefficient of x^{0} and x^{1}, respectively, in

∑_{k=0}^{n} \frac{(−n)_{k} (n + 5)_{k} (1 + x)_{k}}{k! (3)_{k}^{2}} .

Using hypergeometric notation (2.6), the first of these two, c_{0,n}, can be expressed as

\frac{4}{(1 + n) (2 + n) (3 + n) (4 + n)} (_{2}F_{1}[\begin{matrix}−2 − n, 3 + n \\ 1 \end{matrix}; 1] − 1 + (n + 2)(n + 3)) .

The _{2}F_{1}-series can be evaluated by means of the Chu–Vandermonde summation formula (see [46, (1.7.7); Appendix (III.4)])

2F_{1}[\begin{matrix}a, −N \\ c \end{matrix}; 1] = \frac{(c − a)_{N}}{(c)_{N}} ,  (6.10)

where N is a nonnegative integer. This yields

c_{0,n} = \begin{cases}\frac{4}{(n+1)(n+4)} & if n is even, \\ \frac{4}{(n+2)(n+3)} & if n odd. \end{cases}  (6.11)

By combining (6.7), (6.9) and (6.11), we get, after a considerable amount of simplification,

2Δ_{n}^{\{1\};\{2\}} − Δ_{n}^{\{1,2\};\{1,2\}} = (−1)^{\binom{n−1}{2}} ∏_{i=1}^{n} \frac{(i − 1)! i!^{4} (i + 1)!}{(2i)! (2i + 1)!}

× (∑_{k=0}^{n−2} \frac{(−1)^{k} (n + k + 3)! (1 + (−1)^{n}(n + 1))}{(k + 1) (k + 2)!^{2} (n − k − 2)!}

+ \frac{(n + 2)!}{(n − 1)!} ∑_{k=0}^{n−2} \frac{(−1)^{n+k−1}(n + k + 3)!}{(k + 2)! (k + 3)! (n − k − 2)!} ∑_{j=0}^{k} \frac{1}{j + 1}).

By Lemma 26, with n replaced by n − 1, this expression reduces to

2Δ_{n}^{\{1\};\{2\}} − Δ_{n}^{\{1,2\};\{1,2\}} = (−1)^{\binom{n+1}{2}}((−1)^{n}(n + 1) + 2)\frac{(n + 2)!}{(n − 1)!} ∏_{i=1}^{n} \frac{(i − 1)! i!^{4} (i + 1)!}{(2i)! (2i + 1)!}. (6.12)

Substituting (6.4), (6.5), and (6.12) in (6.3), and simplifying the resulting expression, we eventually arrive at (6.1).

Lemma 26. For n ≥ 0, we have

\frac{n!}{(n + 3)!} ∑_{k=0}^{n−1} \frac{(−1)^{k} (n + k + 4)! (1 − (−1)^{n}(n + 2))}{(k + 1) (k + 2)!^{2} (n − k − 1)!}

+ ∑_{k=0}^{n−1} \frac{(−1)^{n+k}(n + k + 4)!}{(k + 2)! (k + 3)! (n − k − 1)!} ∑_{j=0}^{k} \frac{1}{j + 1} = (−1)^{n}(n + 2) − 2. (6.13)

Proof. We shall treat the two sums (in the first and the second line in (6.13), respectively) separately.

First we consider the sum in the first line of (6.13). We replace the term 1/(k + 1) by ∫_{0}^{1} x^{k} dx, interchange summation and integration, and write the sum in hypergeometric notation (2.6). This gives

\frac{1 − (−1)^{n}(n + 2)}{(n + 1)_{3}} ∫_{0}^{1} (\frac{n + 2}{x^{2}} _{3}F_{2}[\begin{matrix}n + 3, −n − 1 \\ 1 \end{matrix}; x] − \frac{n + 2}{x^{2}} + \frac{(n + 1)_{3}}{x} ) dx.

Using the transformation formula (see [46, (1.7.1.3), sum reversed at the right-hand side])

2F_{1}[\begin{matrix}a, −N \\ c \end{matrix}; z] = z^{N} \frac{(c − a)_{N}}{(c)_{N}} _{2}F_{1}[\begin{matrix}−N, 1 − c − N \\ 1 + a − c − N \end{matrix}; \frac{z − 1}{z}]  (6.14)

(where N is a nonnegative integer) with a = n + 3, N = n + 1, c = 1, and z = x, this is transformed into

\frac{1 − (−1)^{n}(n + 2)}{(n + 1)_{3}} ∫_{0}^{1}((n + 2)^{2}(−1)^{n+1} ∑_{k=0}^{n+1} \frac{(−1)^{k} (−n − 1)_{k}^{2}}{k! (k + 1)!} (1 − x)^{k} x^{n−k−1}

− \frac{n + 2}{x^{2}} + \frac{(n + 1)_{3}}{x} ) dx. (6.15)

Recall that for ℜ(α) > −1 and ℜ(β) > −1 we have the following integral representation for the Euler beta function,

∫_{0}^{1}(1 − x)^{α}x^{β} dx = \frac{Γ(α + 1)Γ(β + 1)}{Γ(α + β + 2)}.  (6.16)

Use of this identity in (6.15) wherever possible (i.e., it is applied to the summands with k = 0, …, n − 1) yields after some simplification the expression

\frac{1 − (−1)^{n}(n + 2)}{(n + 1)_{3}}((−1)^{n+1} (1 − ∑_{k=0}^{n} \frac{(−n − 2)_{k}^{2}}{(−n)_{k} k!} )

+ ∫_{0}^{1} (\frac{(n + 1)_{2} ((n + 3) − (n + 2)(1 − x)^{n})}{x} + \frac{(n + 2) ((1 − x)^{n+1} − 1)}{x^{2}}) dx). (6.17)

We would like to write the sum in the first line as a hypergeometric series. Unfortunately, this cannot be done by just straighforwardly extending the summation over all nonnegative k because of the term (−n)_{k} in the denominator, which is 0 for k = n + 1. The way to overcome this problem is to rewrite the sum as a limit,

∑_{k=0}^{n} \frac{(−n − 2)_{k}^{2}}{(−n)_{k} k!}

= lim_{ε→0} (_{2}F_{1}[\begin{matrix}−n − 2, n − 2 \\ −n − ε \end{matrix}; 1] − \frac{(−n − 2)_{n+1}^{2}}{(n + 1)! (−n − ε)_{n+1}} − \frac{(−n − 2)_{n+2}^{2}}{(n + 2)! (−n − ε)_{n+2}} ).

The _{2}F_{1}-series can be evaluated by means of the Chu–Vandermonde summation formula (6.10). Substituing the result in (6.17), we obtain

\frac{1 − (−1)^{n}(n + 2)}{(n + 1)_{3}}((−1)^{n+1} (1 − lim_{ε→0} (\frac{(2 − ε)_{n+2} − (n + 2) (n + 2)! (1 − ε) − (n + 2)!}{(−n − ε)_{n+2}}))

+ ∫_{0}^{1} (\frac{(n + 1)_{2} ((n + 3) − (n + 2) (1 − x)^{n})}{x} + \frac{(n + 2) ((1 − x)^{n+1} − 1)}{x^{2}}) dx).

Of course, the limit can be computed by de l’Hospital’s rule, so that we get

\frac{1 − (−1)^{n}(n + 2)}{(n + 1)_{3}}((−1)^{n+1} + \frac{1}{n!} ((n + 3)! ∑_{k=1}^{n+2} \frac{1}{k + 1} − (n + 2)! (n + 2))

+ ∫_{0}^{1} (\frac{(n + 1)_{2} ((n + 3) − (n + 2)(1 − x)^{n})}{x} + \frac{(n + 2) ((1 − x)^{n+1} − 1)}{x^{2}}) dx).

Now, let us turn to the integral. Expanding the integrand by the binomial theorem and simplifying the result, we obtain

\frac{1 − (−1)^{n}(n + 2)}{(n + 1)_{3}} ((−1)^{n+1} − (n + 1) (n + 2)^{2} + (n + 1)_{3} ∑_{k=1}^{n+2} \frac{1}{k + 1})

+ \frac{1 − (−1)^{n}(n + 2)}{(n + 1)_{3}}(n + 2) ∫_{0}^{1} (∑_{k=0}^{n−1} \frac{(−n − 1)_{k+2} − (k + 2)(−n − 2)_{k+3}}{(k + 2)!} x^{k}) dx. (6.18)

Now the integration can be performed without any difficulty. We rewrite the resulting sum over k in the second line as a limit,

∑_{k=0}^{n−1} \frac{(−n − 1)_{k+2} − (k + 2)(−n − 2)_{k+3}}{(k + 1) (k + 2)!}

= lim_{ε→0} (\frac{n + 1}{ε} − \frac{n + 1}{ε}_{2}F_{1}[\begin{matrix}−n, ε \\ 2 \end{matrix}; 1] + \frac{(n + 1)(n + 2)}{ε} − \frac{(n + 1)(n + 2)}{ε}_{2}F_{1}[\begin{matrix}−n, ε \\ 1 \end{matrix}; 1] ).

Again, Chu–Vandermonde summation (6.10) can be applied to evaluate the two_{2}F_{1}-series. If the result is substituted in (6.18), we get

\frac{1 − (−1)^{n}(n + 2)}{(n + 1)_{3}} ((−1)^{n+1} − (n + 1) (n + 2)^{2} + (n + 1)_{3} ∑_{k=1}^{n+2} \frac{1}{k + 1})

+ \frac{1 − (−1)^{n}(n + 2)}{n + 3} lim_{ε→0} (\frac{(n + 1)! − (2 − ε)_{n}}{(n + 1)! ε} + (n + 2)\frac{n! − (1 − ε)_{n}}{n! ε} ) .

Using de l’Hospital’s rule once more, we can compute the limit and obtain after some simplification the expression

(1 − (−1)^{n}(n + 2))(∑_{k=0}^{n} \frac{2}{k + 1} − \frac{(−1)^{n} + 2n^{3} + 11n^{2} + 19n + 11}{(n + 1)(n + 2)(n + 3)}).  (6.19)

Now we turn our attention to the double sum in the second line of (6.13). Analogously to before, we replace the term 1/(j + 1) by ∫_{0}^{1} x^{j} dx. This enables us to evaluate the inner harmonic sum ∑_{j=0}^{k} 1/(j +1) to ∫_{0}^{1}(1−x^{k+1})/(1−x) dx. We substitute this in the double sum in the second line of (6.13). Using hypergeometric notation, the result is

(−1)^{n}(n + 2) ∫_{0}^{1} (\frac{1}{1 − x} (_{2}F_{1}[\begin{matrix}n + 3, −n − 1 \\ 2 \end{matrix}; 1] − \frac{1}{x} _{2}F_{1}[\begin{matrix}n + 3, −n − 1 \\ 2 \end{matrix}; x]) + \frac{1}{x}) dx.

The first hypergeometric series can simply be computed by Chu–Vandermonde summation (6.10). To the second hypergeometric series we apply the transformation formula (see [46, (1.8.10), terminating form])

2F_{1}[\begin{matrix}a, −N \\ c \end{matrix}; z] = \frac{(c − a)_{N}}{(c)_{N}} _{2}F_{1}[\begin{matrix}a, −N \\ 1 + a − c − N \end{matrix}; 1 − z] ,

where N is a nonnegative integer. These operations yield

(−1)^{n} ∫_{0}^{1} (\frac{(−1)^{n+1}}{(1 − x)} − \frac{(−1)^{n+1}}{x(1 − x)} _{2}F_{1}[\begin{matrix}n + 3, −n − 1 \\ 1 \end{matrix}; 1 − x] + \frac{n + 2}{x}) dx.  (6.20)

Now we would like to apply the Euler beta integral formula (6.16) once more. However, this is not possible just straightforwardly, because the beta integral on the left-hand side of (6.16) is not defined for β = −1. In order to overcome this problem, we first rewrite the term 1/x(1 − x) (which appears in the second term of the integrand in (6.20)) as 1/x + 1/(1 − x), and then replace all occurrences of 1/x by lim_{ε→0^{+}} x^{1−ε}, so that (6.20) becomes

∫_{0}^{1} lim_{ε→0^{+}}(∑_{k=0}^{n+1} \frac{(n + 3)_{k} (−n − 1)_{k}}{k!^{2}} x^{ε−1} (1 − x)^{k}

+ ∑_{k=1}^{n+1} \frac{(n + 3)_{k} (−n − 1)_{k}}{k!^{2}} x^{ε} (1 − x)^{k−1} + (−1)^{n}(n + 2)x^{ε−1}) dx.

Next we interchange limit and integration, and apply the Euler beta integral formula (6.16) wherever possible. In the result, the first sum can easily be evaluated by Chu– Vandermonde summation (6.10). Subsequently, we compute the limit by using de l’Hospital’s rule again. This yields the expression

(−1)^{n}(n + 2) (∑_{k=0}^{n} \frac{1}{k + 1} + ∑_{k=1}^{n+1} \frac{1}{k + 1}) + ∑_{k=1}^{n+1} \frac{(n + 3)_{k} (−n − 1)_{k}}{k!^{2} k} .

Replacing once more the term 1/k in the rightmost sum by ∫_{0}^{1} x^{k−1} dx, we obtain the expression

(−1)^{n}(n + 2) (∑_{k=0}^{n} \frac{1}{k + 1} + ∑_{k=1}^{n+1} \frac{1}{k + 1}) + ∫_{0}^{1} \frac{1}{x} (−1 + _{2}F_{1}[\begin{matrix}n + 3, −n − 1 \\ 1 \end{matrix}; x]) dx.

To the _{2}F_{1}-series we apply the transformation formula (6.14). In the result, we replace any occurrence of x^{m} by lim_{ε→0^{+}} x^{m+ε}, so that we arrive at the expression

(−1)^{n}(n + 2) (∑_{k=0}^{n} \frac{1}{k + 1} + ∑_{k=1}^{n+1} \frac{1}{k + 1})

+ ∫_{0}^{1} lim_{ε→0^{+}} (−x^{ε−1} + (−1)^{n+1}(n + 2) ∑_{k=0}^{n+1} \frac{(−n − 1)_{k}^{2}}{k! (k + 1)!}(x − 1)^{k}x^{n−k+ε}) dx.

Again, we interchange limit and integration and apply the Euler beta integral formula (6.16) once more. Writing the result in hypergeometric notation, we obtain

(−1)^{n}(n + 2) (∑_{k=0}^{n} \frac{1}{k + 1} + ∑_{k=1}^{n+1} \frac{1}{k + 1})

+ lim_{ε→0^{+}} (−\frac{1}{ε} + (−1)^{n+1} \frac{1}{n + 2} (1 − _{2}F_{1}[\begin{matrix}−n − 2, −n − 2 \\ −n − 1 − ε \end{matrix}; 1])) .

The hypergeometric series can be computed by Chu–Vandermonde summation (6.10). Once more we make use of de l’Hospital’s rule for the limit and obtain after some simplifications the expression

−\frac{1 + (−1)^{n} (n^{2} + 3n + 3)}{n + 2} − (1 − (−1)^{n}(n + 2)) ∑_{k=0}^{n} \frac{2}{k + 1}.  (6.21)

When adding together (6.19), the result of our computation for the sum in the first line of (6.13), and (6.21), the result of our computation for the double sum in the second line of (6.13), the harmonic sums cancel, and it is easy to verify that, magically, the remaining terms simplify to the right-hand side of (6.13).

Remark. Peter Paule demonstrated to us, that the identity (6.13) can also be proved algorithmically. Clearly, the Gosper–Zeilberger algorithm [36, 50, 51] finds a recurrence for the sum in the first line of (6.13). Carsten Schneider’s extension of Karr’s algorithm [19], implemented by Schneider, finds a recurrence for the double sum in the second line of (6.13). Finally, Mallinger’s Mathematica package GeneratingFunctions [29] or Salvy and Zimmermann’s Maple package gfun [45] can be used to combine these two recurrences into one, a recurrence of order 10. It is then routine to check (preferably on the computer) that the right-hand side of (6.13) satisfies this same recurrence. However, in the present implementation, these algorithms are not able to find the explicit evaluations, in terms of harmonic numbers, of the sums in the first and second line of (6.13), given in (6.19) and (6.21), respectively.

7. Evaluations of Hankel determinants featuring Bernoulli polynomials

There are several theorems hidden in the body of this paper. Among these are evaluations of Hankel determinants of Bernoulli polynomials evaluated at special values. Recall that the l-th Bernoulli polynomial is defined by

B_{l}(x) := ∑_{k=0}^{l} \binom{l}{k}B_{l−k}x^{k}.

The Hankel determinants of Bernoulli polynomials of which we are talking are special cases of the determinant B(N; x) given by

B(N; x) := det_{1≤i,j≤N} (B_{i+j}(x)) .  (7.1)

This is in a fundamental way different from the Hankel determinant

det_{0≤i,j≤N}(B_{i+j}(x)),

which has been considered earlier (see [1, Sec. 5]). (Note that the difference is that, in the latter determinant, indices start already with 0.) As is not difficult to see (cf. [32, p. 419] or [23, Lemma 15]), the latter determinant does in fact not depend on x (i.e., the powers of x cancel in the expansion of the determinant), so that its value is equal to its value at x = 0, which, in turn, is given by Theorem 23 with a = b = 1, c = d = 0. This is in sharp contrast to the Hankel determinant (7.1), where the powers of x do not cancel, so that (7.1) is a nontrivial polynomial in x. As such, the evaluation of the determinant (7.1) is much more difficult. Below, we provide evaluations of (7.1) for x = −1, x = −1/2 and x = 1/2. Needless to say that the evaluation in the special case x = 0 (and as well in the special case x = 1) is given by Theorem 23 with a = b = c = d = 1.

First of all, in the proof of Lemma 14, we observed that (in symbolic notation B^{k} ≡ B_{k})

det_{1≤i,j≤N} ((B − 1)^{i+j}) = det_{1≤i,j≤N} (B^{i+j−2}(B − 1)^{2}).

The determinant on the right-hand side was then evaluated in Lemma 25. The linear combination (B − 1)^{l} of Bernoulli numbers is nothing else but B_{l}(−1), the l-th Bernoulli polynomial evaluated at −1. Thus, we obtain the following corollary.

Corollary 27. Let N be a positive integer. Then there holds

B(N; −1) = det_{1≤i,j≤N} (B_{i+j} + (−1)^{i+j}(i + j))

= (−1)^{N(N−1)/2}6^{−N}(1 + 2N + 5N^{2} + 4N^{3} + N^{4}) ∏_{i=1}^{N} (\frac{i(i + 1)^{4}(i + 2)}{(2i + 1)(2i + 2)^{2}(2i + 3)})^{N−i} . (7.2)

Our next evaluation results from the determinant evaluation in Lemma 13.

Theorem 28. Let m be a positive integer. Then for

B(N; −1/2) = det_{1≤i,j≤N} ((2^{1−i−j} − 1)B_{i+j} − (−1/2)^{i+j−1}(i + j))

there hold

B(2m − 1; −1/2) = \frac{(−1)^{m−1} (2m − 1)!^{2} (∏_{i=1}^{2m−1} i!)^{4} (∏_{i=1}^{m} (2i − 1)!)^{4}}{2^{6(m−1)} (m − 1)!^{6} (∏_{i=1}^{4m−1} i!)}

× (3 + 8m + \frac{8 (m − 1) (2m + 1)}{3m (m + 1) (2m − 3) (2m − 1)} _{4}F_{3}[\begin{matrix}3, \frac{3}{2}, 2 − m, \frac{3}{2} + m \\ \frac{5}{2}, 2 + m, \frac{5}{2} − m \end{matrix}; 1]) (7.3)

and

B(2m; −1/2) = \frac{(−1)^{m} (∏_{i=1}^{2m} i!)^{4} (∏_{i=1}^{m+1} (2i − 1)!)^{4}}{2^{6m} m!^{6} (∏_{i=1}^{4m+1} i!)}

× (1 + 8m − \frac{8m (2m + 3)}{3 (m + 1) (m + 2) (2m − 1) (2m + 1)} _{4}F_{3}[\begin{matrix}3, \frac{3}{2}, 1 − m, \frac{5}{2} + m \\ \frac{5}{2}, 3 + m, \frac{3}{2} − m \end{matrix}; 1]) . (7.4)

Proof. Consider the determinant D(n, n−1, N) (see (2.1) for definition). Factor 2(n−1/2) out of each column of D(n, n − 1, N), and then set n = 1/2. By the appropriate variant of (4.5) and de l’Hospital’s rule, this yields the Hankel determinant B(N; −1/2). On the other hand, we have evaluated D(n, n − 1, N) in Lemma 13. Then by dividing the results in (4.18) and (4.19) by 2^{N}(n − 1/2)^{N} and then setting n = 1/2, we obtain the expressions on the right-hand sides of (7.3) and (7.4).

When solving the enumeration of rhombus tilings of a hexagon which contain the central rhombus, the determinant D(n, n + 1, N) was explicitly [16] (see the proof of Lemma 10 and Proposition 14 in [16]) or implicitly [7, 13] evaluated (compare Proposition 11). If one adapts the preceding proof to this case, one obtains the following result.

Theorem 29. Let m be a positive integer. Then for

B(N; 1/2) = det_{1≤i,j≤N} ((2^{1−i−j} − 1)B_{i+j})

there hold

B(2m−1; 1/2) = \begin{matrix}(2m)! & 2 \\ 2^{4m−2} & (m − 1)!^{2} & m!^{2} \end{matrix} ( ∏_{i=1}^{2m} \frac{(i − 1)!^{5}}{(2m + i − 1)!}) ∑_{i=0}^{m−1} \frac{(−1)^{m−i}}{(2m − 2i − 1)} \frac{(\frac{1}{2} − i)_{2i}}{i!^{2}} (7.5)

and

B(2m; 1/2) = \begin{matrix}(2m + 2)! & 2 \\ 2^{4m+2} & m!^{2} & (m + 1)!^{2} \end{matrix} ( ∏_{i=1}^{2m+1} \frac{(i − 1)!^{5}}{(2m + i)!}) ∑_{i=0}^{m} \frac{(−1)^{m−i}}{(2m − 2i + 1)} \frac{(\frac{1}{2} − i)_{2i}}{i!^{2}} . (7.6)

Proof. Consider the determinant D(n, n + 1, N). Factor 2(n + 1/2) out of each column of D(n, n + 1, N), and then set n = −1/2. By the appropriate variant of (4.5) and de l’Hospital’s rule, this yields the Hankel determinant B(N; 1/2). On the other hand, by Proposition 11 (with N replaced by N + 1, l = n, and M = 2n − N) we know that D(n, n + 1, N) counts, up to a multiplicative constant, the number of rhombus tilings of a hexagon with sides N + 1, 2n − N, N + 1, N + 1, 2n − N, N + 1, which contain the central rhombus. This enumeration problem was solved in [7, 13, 16]. If in the result we perform the according manipulations and then set n = −1/2, we obtain the expressions on the right-hand sides of (7.5) and (7.6).

8. Concluding comments and open problems

We conclude this article by pointing to open questions which are raised by this work.

(1) In Corollary 7 it was demonstrated that, for M close to N, the number of rhombus tilings of a hexagon with side lengths N, M, N, N, M, N, which contain the rhombus above and next to the center of the hexagon, equals (\frac{1}{3}+r(N, M)) T (N, M), where T (N, M) is the total number of rhombus tilings of the hexagon, and where r(N, M) is a “closed form” expression. (Magically, the value of 1/3 which appears here is, according to [7, Corollary 3], the exact proportion of the rhombus tilings that contain the central rhombus in the total number of rhombus tilings of a hexagon with side lengths 2n−1, 2n, 2n−1, 2n−1, 2n, 2n−1 or with side lengths 2n, 2n − 1, 2n, 2n, 2n − 1, 2n.) As we mentioned in the Introduction, it is easy to derive many more such results, also for the central rhombus and the other two cases that were considered in Theorems 3–6. Our proof, given in Section 2, consisted of starting with the expressions (1.2) and (1.3) and applying Zeilberger’s algorithm to establish the simplification of the sum in these expressions when m and n are close. This is, unfortunately, not conceptual (as it just verifies, but does not derive the result), and therefore does not explain why these simplifications take place. The fact that apparently many more such results exist indicates that there must be a hypergeometric transformation formula lurking in the background, which we were, however, unable to discover. (It is obvious that the sums in (1.2)–(1.7) can be written as very-well-poised _{7}F_{6}-series — see e.g. (2.7) — and, by means of Whipple’s transformation formula (2.8), can therefore be transformed into balanced _{4}F_{3}-series, to which, in turn, we could apply Sears’ _{4}F_{3} transformation formulas. However, it seems that this does not suffice to find the desired identity which would “explain” Corollary 7.)

(2) Is it possible to find a uniform formula for the number of rhombus tilings of a hexagon with side lengths N, M, N, N, M, N, which contain an arbitrary (but fixed) rhombus on the “vertical” symmetry axis (i.e., the symmetry axis which runs in parallel to the sides of length M)? Recall that (as we mentioned already in the Introduction) in [13] such a formula was found for the “horizontal” symmetry axis (i.e., the symmetry axis which cuts through the sides of length M). In contrast, here we encountered increasing difficulties in the proofs of our enumerations the farther we moved the rhombus which is contained in every tiling from the center. Recall that for solving our enumeration problems we needed to compute the determinants D(n, n − t, N) (see (2.1) for definition) for t = 0, 1, 2. For the case of a rhombus which is even farther away from the center, we would have to evaluate this determinant for even larger values of t. The increasing difficulties in doing this arise in Step 5 (compare the proof of Lemma 12) of the computation. The previous steps, Steps 1–4, would prove that

D(n, n − t, N) = (product of linear factors in n) · S(n, N, t),

where S(n, N, t) is a polynomial of degree 2 ⌈(N + t)/2⌉ (compare (4.2), (4.20), and (4.25)). Thus, in order to determine S(n, N, t), the larger t becomes, the more evaluations of S(n, N, t) at special values of n (or other informations about S(n, N, t)) we need. (The computations in [13] have exactly the opposite behaviour: The farther the rhombus which is contained in every tiling is moved away from the center, the smaller in degree becomes the irreducible polynomial in the result.) Even worse, the larger t becomes, the more difficult it becomes to obtain these special values. (Remember, for example, the difficulty of evaluation of R(n, N) at n = 1 via Lemmas 25 and 26.)

That the problem that we considered here is at a different level of complexity than the problem in [13] is also indicated by the (partially conjectural) form of the asymptotic behaviour of the proportion of the rhombus tilings that contain this particular rhombus in the total number of rhombus tilings. While the asymptotic behaviour is totally smooth when the rhombus which is contained in every tiling is moved along the “horizontal” symmetry axis (see [13, Theorem 1.3]), the conjectured form [10, Conjecture 1] of the asymptotics when the rhombus which is contained in every tiling is moved along the “vertical” symmetry axis behaves nonsmoothly. It is increasing for some time when the rhombus is moved away from the center, but at some point, when the rhombus enters the “arctic region” near the (top or bottom) corner, it becomes 1 and stays 1 from thereon. Thus, a formula for exact enumeration must, somehow, reflect this nonsmooth asymptotic behaviour.

Is there a way to overcome these difficulties?

(3) In Theorem 23 only c or d may be 0, but not a or b. In fact, Theorem 23 is wrong if a = 0 or b = 0. But, apparently, not terribly wrong. Lemma 25 shows the evaluation of the determinant in (5.13) with a = b = 0, c = d = 2. Remarkably, the result is almost identical with the right-hand side in (5.13), the only difference being the polynomial in n of fourth degree in (6.1). In fact, computer experiments suggest that a much more general result holds.

Conjecture. For positive integers integers c, d there holds

det_{1≤i,j≤n} (B^{i+j−2} (−B + 1)_{c−1} (−B + 1)_{d−1})

= (−1)^{n(n−1)/2} (\frac{(c − 1)! (c − 1)! (d − 1)! (d − 1)!}{(c + d − 1)!})^{n}

× ∏_{i=1}^{n−1}(\frac{i (c + i − 1) (c + i − 1) (d + i − 1) (d + i − 1) (c + d + i − 2)}{(c + d + 2i − 3) (c + d + 2i − 2)^{2} (c + d + 2i − 1)})^{n−i}P (n; c, d), (8.1)

where P (n; c, d) is a certain polynomial in n of degree 2(c + d − 2).

Furthermore, for positive integers integers b, c, d there holds

det_{1≤i,j≤n} (B^{i+j−1} (B + 1)_{b−1} (−B + 1)_{c−1} (−B + 1)_{d−1})

= (−1)^{n(n−1)/2} (\frac{(c − 1)! (b + c − 1)! (d − 1)! (b + d − 1)!}{(b + c + d − 1)!})^{n}

× ∏_{i=1}^{n−1}(\frac{i (c + i − 1) (b + c + i − 1) (d + i − 1)}{(b + c + d + 2i − 3) (b + c + d + 2i − 2)^{2}}

× \frac{(b + d + i − 1) (b + c + d + i − 2)}{(b + c + d + 2i − 1)})^{n−i}R(n; b, c, d), (8.2)

where R(n; b, c, d) is a certain rational function in n, which can be written with a numerator of degree c + d − 2 and a denominator of degree b − 1.

In principle, our approach of proving Lemma 25 (the special case c = d = 2 of (8.1)), which consisted of using linearity of the determinant in order to break it into several pieces, to each of which we could either apply Theorem 23 or Theorem 24, should make a proof of the above conjecture possible. However, serious difficulties have to be expected in actually doing the calculations, in particular, when working through a generalized form of Lemma 26. We believe that, in view of the simplicity of the result (6.1) and of the conjectured results (8.1) and (8.2), there must be a more elegant way to attack these Hankel determinant evaluations, in particular, if one also desires to obtain explicit forms for the polynomial P (n; c, d) and the rational function R(n; b, c, d).

Note. Since first versions of this article were distributed, Ilse Fischer (“Enumeration of rhombus tilings of a hexagon which contain a fixed rhombus in the centre”, preprint, math/9906102) generalized Theorems 1 and 2 to arbitrary semiregular hexagons.

References

[1] W. A. Al-Salam and L. Carlitz, Some determinants of Bernoulli, Euler and related numbers, Portugaliae Math. 18 (1959), 91–99.

[2] R. Askey, Continuous Hahn polynomials, J. Phys. A – Math. Gen. 18 (1985), L1017–L1019.

[3] N. M. Atakishiyev and S. K. Suslov, The Hahn and Meixner polynomials of an imaginary argument and some of their applications, J. Phys. A – Math. Gen. 18 (1985), 1583–1596.

[4] T. J. Bromwich, An Introduction to the Theory of Infinite Series, MacMillan, London, 1949.

[5] M. Ciucu, Plane partitions I: A generalization of MacMahon’s formula, preprint, math/9808017.

[6] M. Ciucu, T. Eisenkölbl, C. Krattenthaler and D. Zare, Enumeration of lozenge tilings of hexagons with a central triangular hole, preprint.

[7] M. Ciucu and C. Krattenthaler, The number of centered lozenge tilings of a symmetric hexagon, J. Combin. Theory Ser. A 86 (1999), 103–126.

[8] M. Ciucu and C. Krattenthaler, Plane partitions II: 5\frac{1}{2} symmetry classes, in: Combinatorial Methods in Representation Theory, M. Kashiwara, K. Koike, S. Okada, I. Terada, H. Yamada, eds., Advanced Study in Pure Mathematics, RIMS, Kyoto (to appear).

[9] M. Ciucu and C. Krattenthaler, Enumeration of lozenge tilings of hexagons with cut off corners, in preparation.

[10] H. Cohn, M. Larsen and J. Propp, The shape of a typical boxed plane partition, New York J. Math. 4 (1998), 137–166.

[11] G. David and C. Tomei, The problem of the calissons, Amer. Math. Monthly. 96 (1989), 429–431.

[12] A. Erdélyi, V. Magnus, F. Oberhettinger and F. Tricomi, Higher transcendental functions, McGraw– Hill, New York, 1953.

[13] M. Fulmek and C. Krattenthaler, The number of rhombus tilings of a symmetric hexagon which contain a fixed rhombus on the symmetry axis, I, Ann. Combin. 2 (1998), 19–40.

[14] R. L. Graham, D. E. Knuth and O. Patashnik, Concrete Mathematics, Addison Wesley, Reading, Massachusetts, 1988.

[15] A. J. Guttmann, A. L. Owczarek and X. G. Viennot, Vicious walkers and Young tableaux I: Without walls, J. Phys. A – Math. Gen. 31 (1998), 8123–8135.

[16] H. Helfgott and I. M. Gessel, Exact enumeration of certain tilings of diamonds and hexagons with defects, Elect. J. Combin. 6 (1) (1999), #R16, 26 pp.

[17] K. Johansson, Discrete orthogonal polynomial ensembles and the Plancherel measure, preprint,

math/9906120.

[18] W. B. Jones and W. J. Thron, Continued Fractions, Addison–Wesley, Reading, Massachusetts, 1980.

[19] M. Karr, Summation in finite terms, J. Assoc. Comput. Machinery 28 (1981), 305–350.

[20] R. Koekoek and R. F. Swarttouw, The Askey–scheme of hypergeometric orthogonal polynomials and its q-analogue, TU Delft, The Netherlands, 1998; on the WWW:

ftp://ftp.twi.tudelft.nl/TWI/publications/tech-reports/1998/DUT-TWI-98-17.ps.gz.

[21] C. Krattenthaler, An alternative evaluation of the Andrews–Burge determinant, in: Mathematical Essays in Honor of Gian-Carlo Rota, B. E. Sagan, R. P. Stanley, eds., Progress in Math., vol. 161, Birkhäuser, Boston, 1998, pp. 263–270.

[22] C. Krattenthaler, Determinant identities and a generalization of the number of totally symmetric self-complementary plane partitions, Electron. J. Combin. 4(1) (1997), #R27, 62 pp.

[23] C. Krattenthaler, Advanced determinant calculus, Séminaire Lotharingien Combin. 42 (1999) (“The Andrews Festschrift”), paper B42q, 67 pp.

[24] G. Kuperberg, Symmetries of plane partitions and the permanent determinant method, J. Combin. Theory Ser. A 68 (1994), 115–151.

[25] G. Kuperberg, Four symmetry classes of plane partitions under one roof, J. Combin. Theory Ser. A 75 (1996), 295–315.

[26] G. Kuperberg, An exploration of the permanent-determinant method, Electron. J. Combin. 5 (1998), #R46, 34 pp.

[27] B. Leclerc, On certain formulas of Karlin and Szegő, Séminaire Lotharingien Combin. 41 (1998), paper B41d, 21 pp.

[28] P. A. MacMahon, Combinatory Analysis, vol. 2, Cambridge University Press, 1916; reprinted by Chelsea, New York, 1960.

[29] C. Mallinger, Algorithmic manipulations and transformations of univariate holonomic functions and sequences, diploma thesis, RISC, J. Kepler University, Linz, 1996.

[30] M. L. Mehta, Matrix theory, Editions de Physique, Orsay, France, 1989.

[31] T. Muir, The theory of determinants in the historical order of development, 4 vols., Macmillan, London, 1906–1923.

[32] T. Muir, A treatise on the theory of determinants, Longmans, Green & Co., London, 1933.

[33] N. E. Nörlund, Vorlesungen über Differenzenrechnung, Springer, Berlin, 1924.

[34] S. Okada and C. Krattenthaler, The number of rhombus tilings of a “punctured” hexagon and the minor summation formula, Adv. Appl. Math. 21 (1998), 381–404.

[35] P. Paule and M. Schorn, A Mathematica version of Zeilberger’s algorithm for proving binomial coefficient identities, J. Symbol. Comp. 20 (1995), 673–698.

[36] M. Petkovšek, H. Wilf and D. Zeilberger, A=B, A. K. Peters, Wellesley, 1996.

[37] O. Perron, Die Lehre von den Kettenbrüchen, B. G. Teubner, Stuttgart, 1977.

[38] R. A. Proctor, Shifted plane partitions of trapezoidal shape, Proc. Amer. Math. Soc. 89 (1983), 553–559.

[39] R. A. Proctor, Bruhat lattices, plane partition generating functions, and minuscule representations, Europ. J. Combin. 5 (1984), 331–350.

[40] R. A. Proctor, New symmetric plane partition identities from invariant theory work of DeConcini and Procesi, Europ. J. Combin. 11 (1990), 289–300.

[41] J. Propp, Twenty open problems on enumeration of matchings, manuscript, 1996, math.CO/9801060.

[42] J. Propp, Enumeration of matchings: Problems and progress, in: ”New Perspectives in Geometric Combinatorics”, L. Billera, A. Björner, C. Greene, R. Simion, and R. P. Stanley, eds., Mathematical Sciences Research Institute publication #37, Cambridge University Press, 1999 (to appear).

[43] A. P. Prudnikov, Yu. A. Brychkov, O. I. Marichev, Integrals and Series, vol. 3: More Special Functions, Gordon and Breach, New York, London, 1989.

[44] D. P. Robbins. The story of 1, 2, 7, 42, 429, 7436, …, The Math. Intelligencer 13 (1991), 12–19.

[45] B. Salvy and P. Zimmermann, Gfun: a Maple package for the manipulation of generating and holonomic functions in one variable, ACM Trans. Math. Software 20 (1994).

[46] L. J. Slater, Generalized Hypergeometric Functions, Cambridge University Press, Cambridge, 1966.

[47] G. Szegő, Orthogonal Polynomials, American Mathematical Society Colloquium Publications Vol. XXIII, New York, 1959.

[48] X. Viennot, Une théorie combinatoire des polynômes orthogonaux généraux, UQAM, Montreal, Quebec, 1983.

[49] H. S. Wall, Analytic Theory of Continued Fractions, Van Nostrand, New York, 1948.

[50] D. Zeilberger, A fast algorithm for proving terminating hypergeometric identities, Discrete Math. 80 (1990), 207–211.

[51] D. Zeilberger, The method of creative telescoping, J. Symbolic Comput. 11 (1991), 195–204.

Institut für Mathematik der Universität Wien, Strudlhofgasse 4, A-1090 Wien, Austria

  E-mail address: Mfulmek@Mat.Univie.Ac.At, Kratt@Pap.Univie.Ac.At
  WWW: http://radon.mat.univie.ac.at/~mfulmek
          http://radon.mat.univie.ac.at/People/kratt