CHAPTER 1 Introduction

Let ℤ denote the set of integers. Consider the d-dimensional integer lattice ℤ^{d}. A lattice path (path for short) P is a sequence P = (P_{0}, P_{1}, …, P_{l}) of points P_{i} in ℤ^{d}. The left-hand picture in Figure 1 shows the lattice path ((0, 0), (1, 1), (2, 1), (3, 1), (3, 2), (4, 3)). The point P_{0} is called the starting point and P_{l} is called the end point of P. The vectors \overrightarrow{P_{0}P_{1}}, \overrightarrow{P_{1}P_{2}}, …, \overrightarrow{P_{l−1}P_{l}} are called the steps of P. The number of steps, l, is called the length of P. Alternatively, the lattice path P could be considered as a pair (P_{0}, s_{1}s_{2} … s_{l}) of a point P_{0} in ℤ^{d}, the starting point of P, and a sequence s_{1}s_{2} … s_{l} of vectors in ℤ^{d}, the steps of P. Clearly, the relation between these two notations is given by s_{i} = \overrightarrow{P_{i−1}P_{i}}, or P_{i} = P_{0} + ∑_{j=1}^{i} s_{j}. We shall very often switch between these two different ways of writing down lattice paths. Also, as shown by Figure 1, pictorially a lattice path is always viewed with its points being connected so that the steps become visible.

[figure: •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •] Figure 1

The standard problem is to count all lattice paths from a given starting point A ∈ ℤ^{d} to a given end point E ∈ ℤ^{d} with steps from a given set 𝕊 ∈ ℤ^{d} subject to some restrictions R. In order to have a uniform notation, we shall denote this set of paths by L(A → E; 𝕊 | R). For instance, the set of all lattice paths in the plane ℤ^{2} from (0, 0) to (4, 3) which consist of positive unit steps and diagonal steps (1, 1) and stay below the main diagonal will be denoted by

L((0, 0) → (4, 3); {(1, 0), (0, 1), (1, 1)} | x ≥ y).

An example of such a lattice path is depicted in the left-hand picture of Figure 1.

There could be a misunderstanding of how to interpret the restriction x ≥ y in this notation. Therefore, for convenience, let us make the convention that, if a restriction is an affirmative statement then it is meant to hold for all points of a lattice path, while if a restriction is a negative statement then it is meant to hold only for at least one point of a lattice path. Thus the set

L((0, 0) → (4, 3); {(1, 0), (0, 1), (1, 1), (0, −1)} | x ≱ y)

would be the set of all lattice paths from (0, 0) to (4, 3) with steps from {(1, 0), (0, 1), (1, 1), (0, −1)} which do not always stay below the line x = y. An example of a lattice path from this set is depicted in the right-hand picture of Figure 1.

Sometimes, if we are given a function f on lattice paths, we may write L(A →

E; 𝕊 | f(.) = ℓ) meaning the set of all paths P from A to E with steps from 𝕊 which satisfy f(P) = ℓ.

If there is no restriction on the paths then we simply write L(A → E; 𝕊). Also, if we are considering lattice paths consisting of only positive unit steps parallel to the coordinate axes, then we shall drop the step set and write L(A → E | R), or briefly L(A → E) if there is also no restriction on the lattice paths. Lattice paths consisting of only positive unit steps parallel to the coordinate axes are sometimes called minimal lattice paths or simple lattice paths.

In many places, we shall consider weighted counting. Also here, we shall use a uniform notation. Given a set M and a weight function w on M we denote by GF (M; w) the generating function for M with respect to w, i.e.

GF (M; w) := ∑_{x∈M} w(x).  (1.1)

Binomial coefficients will appear at many places. Since we deal with lattice paths, the binomial coefficient in this book will have a definition slightly deviating from the usual one. Namely, for integers a and b we define the binomial coefficient \binom{a}{b} by ??(everywhere true)??

\binom{a}{b} := \begin{cases}\frac{a!}{b! (a − b)!} & if a ≥ b ≥ 0, \\ 0 & otherwise. \end{cases}

Note that this makes a difference if a < 0.

Similarly, we define the q-binomial coefficient [ \begin{matrix}a \\ b \end{matrix}]_{q} by

[\begin{matrix}a \\ b \end{matrix}]_{q} := \begin{cases}\frac{[a]_{q}!}{[b]_{q}! [a − b]_{q}!} & if a ≥ b ≥ 0, \\ 0 & otherwise, \end{cases}

where the q-factorial [m]_{q}! is defined by

[m]_{q}! := 1 · (1 + q) · (1 + q + q^{2}) ⋯ (1 + q + ⋯ + q^{m−1}).

Furthermore, we use the usual multidimensional notation. If λ = (λ_{1}, λ_{2}, …, λ_{n}) and μ = (μ_{1}, μ_{2}, …, μ_{n}), then |λ| = λ_{1} + λ_{2} + ⋯ + λ_{n}, and λ − μ = (λ_{1} − μ_{1}, λ_{2} − μ_{2}, …, λ_{n} − μ_{n}). If λ_{i} ≥ μ_{i} for all i = 1, 2, …, n, we write λ ≥ μ. If λ = (λ_{1}, λ_{2}, …, λ_{n}) is a vector and if σ is a permutation in 𝔖_{n}, the set of permutations of {1, 2, …, n}, then we write λ_{σ} for the “permuted” vector (v_{σ(1)}, v_{σ(2)}, …, v_{σ(n)}).

Given a vector λ = (λ_{1}, λ_{2}, …, λ_{n}) of integers we define the multinomial coefficient \binom{|λ|}{λ} = \binom{λ_{1}+λ_{2}+···+λ_{n}}{λ_{1},λ_{2},...,λ_{n}} by

(\begin{matrix}| & λ & | \\ λ \end{matrix}) = \begin{cases}\frac{(λ_{1} + λ_{2} + ⋯ + λ_{n})!}{λ_{1}! λ_{2}! ⋯ λ_{n}!} & if λ_{i} ≥ 0 for all i, \\ 0 & otherwise. \end{cases}

When n = 2 the above expression gives the binomial coefficient as defined before,

(\begin{matrix}λ_{1} + λ_{2} \\ λ_{1}, λ_{2} \end{matrix}) = (\begin{matrix}λ_{1} + λ_{2} \\ λ_{1} \end{matrix}) = (\begin{matrix}λ_{1} + λ_{2} \\ λ_{2} \end{matrix}) .

Similarly, given a vector λ = (λ_{1}, λ_{2}, …, λ_{n}) of integers we define the q-multi-

nomial coefficient [ \begin{matrix}| & λ & | \\ λ \end{matrix} ]_{q} = [ \begin{matrix}λ_{1}+λ_{2}+···+λ_{n} \\ λ_{1},λ_{2},...,λ_{n} \end{matrix} ]_{q} by

[\begin{matrix}| & λ & | \\ λ \end{matrix}]_{q} = \begin{cases}\frac{[λ_{1} + λ_{2} + ⋯ + λ_{n}]_{q}!}{[λ_{1}]_{q}! [λ_{2}]_{q}! ⋯ [λ_{n}]_{q}!} & if λ_{i} ≥ 0 for all i, \\ 0 & otherwise. \end{cases}

When n = 2 the above expression gives the q-binomial coefficient as defined before,

[\begin{matrix}λ_{1} + λ_{2} \\ λ_{1}, λ_{2} \end{matrix}]_{q} = [\begin{matrix}λ_{1} + λ_{2} \\ λ_{1} \end{matrix}]_{q} = [\begin{matrix}λ_{1} + λ_{2} \\ λ_{2} \end{matrix}]_{q} .

??upper factorials ??self-avoiding walk

??appendix: partitions and q-binomials, better: chapter on lattice paths partitions, q-binomials, q-multinomials, independently developing. Also here areacounting-determinant formula. ??appendix: species and generating functions ??appendix: Lagrange inversion (also multivariable) ??appendix: hypergeometric and basic hypergeometric series + Gosper + Zeilberger ??appendix: Tree of connections ??appendix: Glossary of all counting problems with references to the book or literature

??global: left-hand — right-hand picture → label Figure 1.a), b) + description: a. nonintersecting lattice path, b. CSPP …

CHAPTER 2 Simple lattice paths in two dimensions

In this chapter we consider lattice paths in the plane integer lattice ℤ^{2} consisting of positive unit steps. Clearly, the number of all such paths from (a, b) → (c, d) is

|L((a, b) → (c, d))| = \binom{c + d − a − b}{c − a},  (2.1)

since each path from (a, b) to (c, d) can be identified with a sequence of (c − a) horizontal steps and (d − b) vertical steps, the number of those sequences being given by the binomial coefficient in (2.1).

??Survey of this chapter

1. Simple paths with boundaries of type x = y

Next we want to count paths from (a, b) to (c, d), where a ≥ b and c ≥ d, which stay below the main diagonal y = x. Always, when we say ‘below the line l’ we mean that the lattice paths are allowed to touch the line l. When we want to exclude touches, we shall say ‘strictly below the line l’. So, what we want to know is the number |L((a, b) → (c, d) | x ≥ y)|. This problem is most conveniently solved by the so-called reflection principle introduced by André [?].

Theorem 2.1. Let a ≥ b and c ≥ d. The number of all paths from (a, b) to (c, d) staying below the line y = x is given by

|L((a, b) → (c, d) | x ≥ y)| = \binom{c + d − a − b}{c − a} − \binom{c + d − a − b}{c − b + 1}. (2.2)

Proof. First we observe that the number in question is the number of all paths from (a, b) to (c, d) minus the number of those paths which cross the line y = x

|L((a, b) → (c, d) | x ≥ y)| = |L((a, b) → (c, d))| − |L((a, b) → (c, d) | x ≱ y)| . (2.3)

By (2.1) we already know |L((a, b) → (c, d))|. The reflection principle shows that paths from (a, b) to (c, d) which cross y = x are in bijection with paths from (b − 1, a + 1) to (c, d). This implies

|L((a, b) → (c, d) | x ≱ y)| = |L((b − 1, a + 1) → (c, d))| .

Hence, using (2.1) again we establish (2.2).

[figure: •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; ^{.....}; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; _{.....}; .....; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; •; •; •; •; (a, b); (b − 1, a + 1); (c, d); P; y = x; S; P′; y = x + 1]^{Figure 2}

The claimed bijection is obtained as follows. Consider a path P from (a, b) to (c, d) crossing the line y = x. See Figure 2 for an example. Then P must meet the line y = x + 1. Among all the meeting points of P and y = x + 1 choose the right-most. Denote this point by S. Now reflect the portion of P from (a, b) to S with respect to the line y = x + 1, leaving the portion from S to (c, d) invariant. Thus we obtain a new path P′ from (b−1, a+1) to (c, d). To construct the reverse mapping we only have to observe that any path from (b − 1, a + 1) to (c, d) must meet y = x+1 since (b−1, a+1) and (c, d) lie on different sides of y = x+1. Again we choose the right-most meeting point, denote it by S, and reflect the portion from (b − 1, a + 1) to S with respect to the line y = x + 1, thus obtaining a path from (a, b) to (c, d) that meets the line y = x + 1, or equivalently, crosses the line y = x.

In particular, for a = b = 0 we obtain

1. SIMPLE PATHS WITH BOUNDARIES OF TYPE x = y 7

Corollary 2.2. If c ≥ d we have

|L((0, 0) → (c, d) | x ≥ y)| = \frac{c + 1 − d}{c + d + 1}\binom{c + d + 1}{d}  (2.4)

and

|L((0, 0) → (n, n) | x ≥ y)| = \frac{1}{n + 1}\binom{2n}{n}.  (2.5)

The numbers \frac{c+1−d}{c+d+1} \binom{c+d+1}{d} are called ballot numbers since they give the answer to the classical ballot problem of Bertrand [?]. The problem is stated as follows: In an election candidate A received c votes and candidate B received d votes; how many ways of counting the votes are there such that at each stage during the counting candidate A has at least as many votes as candidate B? Representing a vote for A by a horizontal step and a vote for B by a vertical step is seen that the number in question is the same as the number of lattice paths from (0, 0) to (c, d) staying below y = x. This number is given in (2.4). More about ballot problems appears in Section ??.

The numbers \frac{1}{n+1} \binom{2n}{n} are called Catalan numbers [?]. They appear at numerous places (see [?]). For example, Catalan numbers count ordered trees on n vertices, parenthizations?? of n elements of a free noncommutative group, … ??. refer to q-analogues: Hofbauer, Fürlinger, … ??

Interestingly, an iterated reflection argument will give the number of paths between two diagonal lines.

Theorem 2.3. Let a + t ≥ b ≥ a + s and c + t ≥ d ≥ c + s. The number of all paths from (a, b) to (c, d) staying below the line y = x + t and above the line y = x + s is given by

|L((a, b) → (c, d) | x + t ≥ y ≥ x + s)|

= ∑_{k∈ℤ}(\binom{c + d − a − b}{c − a − k(t − s + 2)} − \binom{c + d − a − b}{c − b − k(t − s + 2) + t + 1}) . (2.6)

Proof. The common way to obtain (2.6) is to argue by inclusion-exclusion (see e.g. Mohanty [?, pp. 6,7]). The number of all paths from (a, b) to (c, d) which do not cross either of the lines y = x + t and y = x + s equals the number of all paths from (a, b) to (c, d), minus those which cross y = x + t, minus those which cross y = x + s, plus those which cross y = x + t and then y = x + s, plus those which cross y = x + s and then y = x + t, and so on. Then, by iterated reflection, one would show that each of these set of paths crossing our two lines in some particular way is in bijection with some other set of paths without restrictions. And these can be easily counted. Once having guessed the result it is more convenient to give a proof by a cancelling bijection. In fact, this method works well in many situations and is therefore of particular importance. Later on, we shall see more of these cancelling bijections.

In view of (2.1) we may write the right-hand side of (2.6) as

| ⋃_{k∈ℤ}L((a + k(t − s + 2), b − k(t − s + 2)) → (c, d))|  (2.7a)

− | ⋃_{k∈ℤ}L((b − t − 1 + k(t − s + 2), a + t + 1 − k(t − s + 2)) → (c, d))|.(2.7b)

We shall construct a bijection between paths from the set in (2.7a) that touch y = x + t + 1 or y = x + s − 1 and paths from the set in (2.7b) that also touch one of these two lines. Suppose that this had been done. Then in (2.7) all contributions of paths that touch one of these two lines would cancel. But it is easily checked that because of our assumptions a + t ≥ b ≥ a + s and c + t ≥ d ≥ c + s all the paths in (2.7a) for k ≠ 0 and all the paths in (2.7b) have to touch one of these two lines. Hence, the difference (2.7), and thus the right-hand side in (2.6), counts what remains, i.e. all the paths from (a, b) to (c, d) staying between the lines y = x + t and y = x + s, which is the left-hand side of (2.6).

To construct the bijection, consider a path P in L((a + k(t − s + 2), b − k(t −

s + 2)) → (c, d)) that touches either y = x + t + 1 or y = x + s − 1. Choose the last meeting point between P and any of these two lines. Denote this point by S. Now reflect the path’s initial portion up to S with respect to the line that P is meeting in S. Thus we obtain a new path P′, either from (b − k(t − s + 2) − t − 1, a + k(t − s + 2) + t + 1) to (c, d) and touching y = x + t + 1, or from (b−k(t−s+2)−s−1, a+k(t−s+2)+s+1) to (c, d) and touching y = x+s−1. In both cases P′ belongs to the set in (2.7b). The reverse mapping clearly is defined in the same way.

More on counting paths between two lines with slope 1 and q-analogues will be discussed in Chapter ??, Section ??.

2. Simple paths with boundaries of type x = μy

2. SIMPLE PATHS WITH BOUNDARIES OF TYPE x = μy 9

When we want to count lattice paths bounded by an arbitrary line y = kx + d, k, d ∈ ℝ, the reflection principle obviously does not help, since the reflection of a lattice path in a generic line does not necessarily give a lattice path. In fact, a solution in form of a determinant can be given when the boundary is viewed as a special case of a set of general boundaries (see Section 5, Theorem 2.12). However, there are cases where simpler expressions can be obtained, and these are discussed in this section. ??refer to Sato??

The first case, where a nicer answer can be obtained, is when counting lattice paths from the (0, 0) to (c, d) which stay below the line x = μy, where μ is a positive integer. Of course we have to assume c ≥ μd. Since, as already remarked, the reflection principle will not help (except for μ = 1), we have to use another method. The most straight-forward method is by decomposition of paths and use of generating functions.

Theorem 2.4. Let μ be a nonnegative integer and c ≥ μd. The number of all lattice paths from the origin to (c, d) which lie below x = μy is given by

|L((0, 0) → (c, d) | x ≥ μy)| = \frac{c − μd + 1}{c + d + 1}\binom{c + d + 1}{d}.  (2.8)

Proof. We start with the simpler case that the end point also lies on x = μy. Let P be a path in L((0, 0) → (μd, d) | x ≥ μy) (see Figure 3).

[figure: •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; s_{h}; s_{h}; s_{v}; x = μy; S_{0}; S_{1}; ^{. . . . .}; ^{. . . . .}; \overbrace{P_{0}}; \overbrace{P_{1}}; \overbrace{P_{2}}]^{Figure 3}

For l = 0, 1, …, μ − 1 the path P will meet the line x = μy + l (which is parallel to our boundary x = μy) for the last time. Denote this point by S_{l}. Clearly, the path P must leave S_{l} by a horizontal step, which we denote by s_{h} for short. This gives us a unique decomposition of P of the form

P = P_{0}s_{h}P_{1}s_{h} … s_{h}P_{μ}s_{v}

where P_{0} is P ’s portion from the origin up to S_{0}, P_{1} is P ’s portion from the point immediately following S_{0} up to S_{1}, etc. By s_{v} we denote a vertical step. All the portions P_{i} (when shifted appropriately) belong to L((0, 0) → (μn, n) | x ≥ μy) for some n. Let

L_{0} = ⋃_{n≥0} L((0, 0) → (μn, n) | x ≥ μy).

Then we have the following decomposition:

L_{0} = {ε} ∪ ((L_{0}s_{h})^{μ}L_{0}s_{v}).  (2.9)

Here, as always in the sequel, ε denotes the empty path.

??Maybe: P : (0, 0) → (0, n) ⟶ z^{n}!!

By (??) this immediately translates into a functional equation for the generating function

F_{0}(z) := ∑_{n≥0} |L((0, 0) → (μn, n) | x ≥ μy)| z^{n}

for L_{0},

F_{0}(z) = 1 + zF_{0}(z)^{μ+1}.  (2.10)

The coefficients of F_{0}(z), which we are interested in, are easily extracted by use of the Lagrange inversion. To be able to apply the Lagrange inversion, we only have to write F_{0}(z) = 1 + G_{0}(z). Equation (2.10) then reads

\frac{G_{0}(z)}{(1 + G_{0}(z))^{μ+1}} = z,  (2.11)

which simply says that G_{0}(z) is the compositional inverse of z/(1+z)^{μ+1}. For convenience, let us denote the coefficient of z^{n} in some power series f(z) by ⟨z^{n}⟩f(z). Then, an application of (??) gives

L((0, 0) → (μn, n) | x ≥ μy) = ⟨z^{n}⟩ G_{0}(z)

= \frac{1}{n}⟨z^{−1}⟩\frac{(1 + z)^{n(μ+1)}}{z^{n}}

= \frac{1}{n}\binom{n(μ + 1)}{n − 1} = \frac{1}{1 + n(μ + 1)}\binom{1 + n(μ + 1)}{n}.

Turning to the more general problem, consider a lattice path P in |L((0, 0) →

(μd + k, d) | x ≥ μy)| (see Figure 4).

[figure: •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; s_{h}; s_{h}; s_{h}; x = μy; S_{0}; S_{1}; S_{2}; (c, d); ^{. . . . .}; ^{. . . . .}; \overbrace{P_{0}}; \overbrace{P_{1}}; \overbrace{P_{2}}; \overbrace{P_{3}}]^{Figure 4}

For l = 0, 1, …, k − 1 the path will meet the line x = μy + l for the last time. Denote this point by S_{l}. Clearly, the path P must leave S_{l} by a horizontal step, for which we again write s_{h}. This gives us a decomposition of P of the form

P = P_{0}s_{h}P_{1}s_{h} … s_{h}P_{k},

where P_{0} is the portion of P from the origin up to S_{0}, P_{1} is the portion of P from the point immediately following S_{0} up to S_{1}, and so on. Observe that again the portions P_{l} belong L_{0} (being defined immediately before (2.9)). Let

L_{k} = ⋃_{n≥0} L((0, 0) → (μn + k, n) | x ≥ μy).

Then we have the following decomposition:

L_{k} = (L_{0}s_{h})^{k}L_{0}.

By (??) this translates into an equation for the generating function

F_{k}(z) := ∑_{n≥0} |L((0, 0) → (μn + k, n) | x ≥ μy)| z^{n}

for L_{k}, namely

F_{k}(z) = F_{0}(z)^{k+1} = (1 + G_{0}(z))^{k+1},  (2.12)

or equivalently

∑_{n≥0} |L((0, 0) → (μn + k, n) | x ≥ μy)| \frac{z^{n}}{(1 + z)^{n(μ+1)}} = (1 + z)^{k+1}. (2.13)

Again, use of the Lagrange formula (??) with g(z) = (1+z)^{k+1}, f(z) = z/(1+z)^{μ+1} from (2.13) yields

L((0, 0) → (μn + k, n) | x ≥ μy) = \frac{1}{n}⟨z^{−1}⟩(k + 1) (1 + z)^{k} \frac{(1 + z)^{n(μ+1)}}{z^{n}}

= \frac{k + 1}{n}\binom{μn + k + n}{n − 1}

= \frac{k + 1}{μn + k + n + 1}\binom{μn + k + n + 1}{n},

which turns into (2.8) once we replace μn + k by c and n by d.

In particular, for μ = 1 the generating function F_{0}(z) can be explicitly evaluated from solving the quadratic equation (2.10). This gives the familiar generating function for the Catalan numbers (compare with the second paragraph after Corollary 2.2)

∑_{n≥0} C_{n}z^{n} = \frac{1 − \sqrt{1 − 4z}}{2z} .  (2.14)

The reason that we are allowed to forget about the second solution of the quadratic equation is that it does not give a proper formal power series. So in this case we do not actually need the Lagrange inversion to extract the coefficients of F_{0}(z), since the binomial theorem applied to (1 − 4z)^{1/2} also leads to the solution.

Identity (2.13) written in the equivalent form

∑_{n≥0} |L((0, 0) → (μn + k, n) | x ≥ μy)| \frac{z^{n}}{(1 + z)^{n(μ+1)+k+1}} = 1  (2.15)

will be generalized to weighted counting in Proposition 4.3 and ?? by using different methods.

Since the answer (2.8) to our last counting problem is so nice, it suggests itself to look for a combinatorial proof. Indeed, there is. It is sometimes know under the name “penetrating analysis”. It bases on a lemma due to Dvoretzky and Motzkin [?], called the “cycle lemma”, which has been rediscovered a lot of times (see Dershovitz?? and Zaks?? [?] for a list of references.)

Lemma 2.5 (Cycle Lemma). Let μ be a nonnegative integer. For any sequence p_{1}p_{2} … p_{m+n} of m 1’s and n 2’s, m ≥ μn, there exist exactly m − μn cyclic permutations p_{i}p_{i+1} … p_{m+n}p_{1} … p_{i−1}, 1 ≤ i ≤ m + n, that have the property that for all j = 1, 2, …, m + n the first j letters of this permutation contain more 1’s than μ times the number of 2’s.

Proof. A sequence p_{1}p_{2} … p_{m+n} of m 1’s and n 2’s can be seen as a lattice path from (0, 0) to (m, n) by interpreting the 1’s as horizontal steps and the 2’s as vertical steps. Cyclically permuting p_{1}p_{2} … p_{m+n} means to cut the corresponding lattice path into two pieces and put them together in exchanged order, thus obtaining a new lattice path from (0, 0) to (m, n). Finally, the property that in each initial string of a sequence the number of 1’s dominates (i.e., is larger than) μ times the number of 2’s means that the corresponding lattice path stays strictly below the line x = μy, with the exception of the starting point (0, 0).

For the proof of the lemma interpret p_{1}p_{2} … p_{m+n} as a path, as described before, and join a shifted copy of this path at the end point (m, n), another shifted copy at (2m, 2n), etc. Figure 5 shows an example with μ = 2, m = 9, n = 3. The path P corresponds to the sequence 12111211.

• _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{.....}. . . . .^{.....} •  •  • . . . . . ^{. . . . .} ^{. . . . .}  ^{. . . . .} ^{. . . . .} ^{. . . . .}  ^{. . . . .} (m, n) \overbrace{P}^{Figure 5}

Cyclic permutations of p_{1}p_{2} … p_{m+n} correspond to cutting a piece of m + n successive steps out of this lattice path structure. Then imagine a sun to be located in direction (μ, 1) illuminating the lattice path structure. A cyclic permutation will satisfy the dominance property for each initial string if and only if the first step of the corresponding lattice path is illuminated. In Figure 5 the illuminated steps are indicated by thick lines. It is an easy matter of fact that of any m + n successive steps there are exactly m − μn illuminated steps. Therefore out of the m + n cyclic permutations of p_{1}p_{2} … p_{m+n} there are exactly m − μn cyclic permutations having the dominance property for each initial string.

Combinatorial Proof of Theorem 2.4. We want to count paths from (0, 0) to (c, d) staying (weakly) below x = μy. To fit with the cycle lemma we adjoin a horizontal step at the beginning and shift everything by one unit to the right. Thus we are now asking for the number of paths from (0, 0) to (c + 1, d) staying strictly below x = μy, except for the starting point (0, 0). To find this number, N say, we count in two different ways the set of all pairs (P, Q), where

(1) P is any path from (0, 0) to (c + 1, d),

(2) Q is a path from (0, 0) to (c + 1, d) staying strictly below x = μy except for (0, 0),

(3) P and Q are related by a cyclic permutation of their steps. Figure 6 displays an example of such a pair (P, Q), with μ = 2, c = 6, d = 2, where Q is the cyclic permutation of P which starts with the third step of P. The bold steps in the Figure indicate the possible beginnings of a path Q.

[figure: •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; P; (c + 1, d); x = μy; •; •]

a. Arbitrary path

[figure: •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; Q; (c + 1, d); x = μy; •; •]

b. Path staying strictly below x=μy

Figure 6

First of all, there are \binom{c+d+1}{d} possible paths P, and, by the cycle lemma 2.5, for each P there are exactly c+1−μd cyclic permutations Q staying strictly below x = μy except for (0, 0). So our set of pairs (P, Q) has cardinality (c + 1 − μd)\binom{c+d+1}{d}. On the other hand, there are N possible paths Q, and for each Q there are c+d+1 cyclic permutations P. Therefore the cardinality of our set of pairs (P, Q) is also equal to (c + d + 1)N. Equating the two expressions for the cardinality gives the desired result for the number N that we are looking for.

??Put Krat/Mohanty multi-one correspondence proof

??give Chapter 3-Takacs/Narayana cyclic things here?

So far we only counted paths bounded by x = μy where the starting point lies on the boundary. If we drop this latter assumption and now want to enumerate all paths from (a, b) to (c, d) staying below x = μy, the problem becomes more difficult. Yet there is an answer, although only in terms of a sum. In fact, we can offer two different expressions. Which of these two is preferable depends on the particular situation, to be more precise, on which of the numbers (a/μ − b) or (d−a/μ) being larger (see Figure 7 for the pictorial significance of these numbers). While the proof for the first expression is rather straight-forward, the proof for the second expression is trickier. ??cite Niederhausen, Sato ??Prove, at least for μ = 2 or 3 that there cannot be a “NICE” solution by Zeilberger-Petkovšek

[figure: •; •; (c, d); (a, b); . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; .....; .....; .....; .....; .....; d − a/μ; a/μ − b; x = μy; }; }]^{Figure 7}

Theorem 2.6. Let μ be a nonnegative integer, a ≥ μb and c ≥ μd. The number of all lattice paths from (a, b) to (c, d) staying below x = μy is given by

|L((a, b) → (c, d) | x ≥ μy)| = \binom{c + d − a − b}{c − a}

− ∑_{i=⌊a/μ⌋+1}^{c} \binom{i(μ + 1) − a − b − 1}{i − b}\frac{c − μi + 1}{c + d − i(μ + 1) + 1}\binom{c + d − i(μ + 1) + 1}{d − i},

(2.16)

and also

|L((a, b) → (c, d) | x ≥ μy)| = ∑_{i=0}^{⌊a/μ⌋−b}(−1)^{i}\binom{a − μ(b + i)}{i}

× \frac{c − μd + 1}{c + d − (μ + 1)(b + i) + 1}\binom{c + d − (μ + 1)(b + i) + 1}{d − b − i}. (2.17)

Proof of (2.16). The number of paths in question equals the number of all paths from (a, b) to (c, d) minus those paths which cross x = μy. To count the latter observe that any path crossing x = μy must meet the line x = μy − 1, and for the last time in some point (μi − 1, i) where ⌊a/μ⌋ + 1 ≤ i ≤ c. Fix any i, then the number of all these paths is

|L((a, b) → (μi − 1, i))| · |L((μi, i) → (c, d) | x ≥ μy)| .

We already know the first number thanks to (2.1), and we also know the second number thanks to (2.8), since a shift in direction (−μi, −i) shows that the second number equals |L((0, 0) → (c − μi, d − i) | x ≥ μy)|.

Proof of (2.17). Formula (2.17) is an inclusion-exclusion formula (observe the alternating sign). As in Theorem 2.3 we prefer to prove (2.17) by means of a cancelling bijection, more precisely by a cancelling involution, i.e. by a map φ with φ ∘ φ = id.

First we give a combinatorial interpretation of the right-hand side of (2.17). For convenience, given a set M, let us denote the collection of all k-element subsets of M by S_{k}(M). Then (2.17) can be written as

∑_{(S,P)}(−1)^{|S|},  (2.18)

where the sum is over all pairs (S, P) in

⋃_{i=0}^{⌊a/μ⌋−b} S_{i}({μ(b + i), μ(b + i) + 1, …, a − 1})

× L((μ(b + i), b + i) → (c, d) | x ≥ μy), (2.19)

i.e. S is an i-element subset of {μ(b+i), μ(b+i)+1, …, a−1}, and P is a path from (μ(b + i), b + i) to (c, d) staying below x = μy, for some i = 0, 1, …, ⌊a/μ⌋ − b. Figure 8 illustrates the situation. Both, P_{1} and P_{2} are examples of P. It is clear that by (2.18) we have written the right-hand side of (2.17) as a generating function for the set (2.19). What we shall do is to construct an involution φ on the set of all pairs (S, P) in (2.19) where either S is nonempty or P contains at least one vertical step strictly to the left of the vertical line x = a. This involution will be shown to be sign-reversing, in the sense that if φ((S, P)) = (S′, P′) then (−1)^{|S|} = −(−1)^{|S′|}.

[figure: •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; •; •; •; •; (a, b); (c, d); (μb, b); (μ(b + i), b + i); x = μy; x = a; P_{1}; P_{2}] Figure 8

Suppose that this had been done. Then in the sum (2.18) all contributions of pairs (S, P) in (2.19) where either S is nonempty or P contains at least one vertical step strictly to the left of x = a will cancel. Thus only the contributions (∅, P) in S_{0}(∅) × L((μb, b) → (c, d) | x ≥ μy) will survive where P does not contain a vertical step strictly to the left of x = a, i.e. starts with a − μb horizontal steps. The path P_{2} in Figure 8 is an example of a “surviving” path. Clearly the set of pairs (∅, P) as described above is equinumerous with L((a, b) → (c, d) | x ≥ μy) which is what we need.

Now we construct the involution φ. Let (S, P) be a pair in S_{i}({μ(b+i), μ(b+i)+ 1, …, a−1})×L((μ(b+i), b+i) → (c, d) | x ≥ μy) with either S being nonempty or P containing a vertical step strictly left of x = a. Let s be the smallest element in S and t be the x-coordinate of a left-most vertical step in P. Then, introducing notations which will be explained below, we define φ by

φ((S, P)) = \begin{cases}(S \ \{s\}, P ⊕ \{s\}) & s ≤ t, \\ (S ∪ \{t\}, P ⊖ \{t\}) & s > t. \end{cases}

This has to be understood in the following sense. If s ≤ t then take s out of S, insert a vertical step with x-coordinate s into P leaving the end point invariant, and add μ horizontal steps at the beginning of P, again leaving the end point invariant, thus obtaining P ⊕ {s}. It is easy to see that (S \ {s}, P ⊕ {s}) belongs to S_{i−1}({μ(b + i − 1), μ(b + i − 1) + 1, …, a − 1}) × L((μ(b + i − 1), b + i − 1) →

(c, d) | x ≥ μy). Besides, P ⊕ {s} contains a vertical step strictly to the left of x = a since s ≤ a − 1. On the other hand, if s > t we add t to S, delete a vertical step with x-coordinate t from P leaving the end point invariant, and delete μ horizontal steps at the beginning of P (which must be there because P starts at the line x = μy and is staying below x = μy) again leaving the end point invariant, thus obtaining P ⊖ {t}. It is easy to see that (S ∪ {t}, P ⊖ {t}) belongs to S_{i+1}({μ(b + i + 1), μ(b + i + 1) + 1, …, a − 1}) × L((μ(b + i + 1), b + i + 1) →

(c, d) | x ≥ μy) with S ∪ {t} being nonempty.

It is not difficult to show that φ is an involution, i.e. when applying φ again one arrives back at the original pair. Since exactly one element is added to S or deleted from S the mapping φ is sign-reversing.

3. Simple paths with a piecewise linear boundary

In this section we generalize the one-sided linear boundary results Theorems 2.2, 2.4, 2.6 to piecewise linear boundaries. To be more precise, we want to count lattice paths from the origin (0, 0) to (c, d) staying below the line segments {(x, y) : x = μ_{1}y + ν_{1}, 0 = y_{0} ≤ y ≤ y_{1}}, {(x, y) : x = μ_{2}y + ν_{2}, y_{1} ≤ y ≤ y_{2}}, …, {(x, y) : x = μ_{m}y + ν_{m}, y_{m−1} ≤ y ≤ y_{m} = d} for some sequence 0 = y_{0} < y_{1} < …, y_{m} = d, nonnegative integers μ_{1}, μ_{2}, …, μ_{m}, and integers ν_{1}, ν_{2}, …, ν_{m} such that the bound is monotone?? and to the right of the y-axis. Let us denote this piecewise linear restriction by R.?? See Figure 9 for an example. By an iteration argument it will be seen that the solution to this problem can be given in form of an (m − 1)-fold sum. In fact, suppose that μ_{i}y_{i−2} + ν_{i} ≥ 0 for all i.

[figure: •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; ..........; ^{..........}; ^{..........}; .....; ^{.....}; ^{.....}; ^{.....}; ^{.....}; •; •; •; •; •; ⋆; ⋆; P; x = μ_{1}y + ν_{1}; x = μ_{2}y + ν_{2}; x = μ_{3}y + ν_{3}; y_{0}; y_{1}; y_{2}; y_{3} = d; (c, d)]^{Figure 9}

4. SIMPLE PATHS WITH LINEAR BOUNDARIES WITH RATIONAL SLOPE 19

Evidently, any path from (0, 0) to (c, d) has to touch x = μ_{m}y + ν_{m} for the last time, say in (μ_{m}i + ν_{m}, i). In Figure 9 we have m = 3, the last touching point of the path P with x = μ_{m}y + ν_{m} is (10, 2), it is marked by a star. Then we could utilize the same idea which led to the formula (2.16) to obtain the number in question being equal to

|L((0, 0) → (c, d) | R)|

= ∑_{i=0}^{y_{m−1}} |L((0, 0) → (μ_{m}i + ν_{m} − 1, i) | R)|

|L((μ_{m}i + ν_{m}, i) → (c, d) | x ≥ μ_{m}y + ν_{m})|

= ∑_{i=0}^{y_{m−1}} |L((0, 0) → (μ_{m}i + ν_{m} − 1, i) | R)|

\frac{c − μ_{m}(i + d) − ν_{m} + 1}{c + d − μ_{m}(i + 1) − ν_{m} + 1}\binom{c + d − μ_{m}(i + 1) − ν_{m} + 1}{d − i},

(2.20)

by virtue of Theorem 2.4. Now we can iterate this argument which involves the last touching points with the other lines. The last touching point of the path P with x = μ_{1}y + ν is marked again by a star. Thus we obtain a certain sum for |L((0, 0) → (μ_{m}i + ν_{m} − 1, i) | R)|, etc., until we get an (m−1)-fold sum involving binomial expressions.

Unfortunately, in case our favourable assumption μ_{i}y_{i−2} + μ_{i} ≥ 0 is violated for some i, then the argument breaks down. However, the situation can be rescued by “extending the path numbers”. This bases on the observation that in our situation the numbers of paths to be counted will always be polynomials, hence can be extended to negative values ...

4. Simple paths with linear boundaries with rational slope

??Bizley!! In Section 3 we considered lattice paths bounded by a line x = μy, with μ a nonnegative integer. Now we want to consider a more general linear boundary of the form νx = μy, where ν, μ are nonnegative integers. We describe a generating function approach, due to Sato [?], which works for a large class of cases. Alternative solutions, which work in all cases, in the form of a determinant can be given as a special case of a set of general boundaries. These are discussed in Section 5, see Theorems 2.12, 2.14.

??Figure!! what does it mean? It means from (0,0) to ”very close” to the boundary!! The problem that we want to attack here is to enumerate all lattice paths from an arbitrary starting point to an arbitrary end point staying below the line νx = μy. A simple shift of the plane shows that this is equivalent to enumerating paths from the origin to an arbitrary end point staying below νx = μy − ρ, for an appropriate ρ. Without loss of generality we may assume in the sequel that ν < μ. For the approach of Sato this is the more convenient formulation of the problem. The idea is to introduce ν × ν matrices which contain the path numbers that we are looking for. More precisely, define the ν ×ν matrix W (z; c, ρ) by

W (z; c, ρ) := (w(z; c + g, ρ + h))_{0≤g,h≤ν−1},  (2.21)

where

w(z; c, ρ) = ∑_{μn+c≡ρ (mod ν)} w(n; c, ρ) z^{n},  (2.22)

with

w(n; c, ρ) = \begin{cases}|L((0, 0) → (\frac{μn+c−ρ}{ν}, n) | νx ≥ μy − ρ)| & μn + c ≡ ρ (mod ν) \\ & and μn + c ≥ ρ \\ \binom{n+ \frac{μn+c−ρ}{ν}}{n} & μn + c ≡ ρ (mod ν) \\ & and μn + c < ρ. \end{cases}

(2.23)

So, what the matrix W (z; c, ρ) contains is generating functions of the path numbers w(n; c, ρ) that we want to know. The definition of w(n; c, ρ) for μn + c < ρ (in which case there cannot be any paths from (0, 0) to (\frac{μn+c−ρ}{ν}, n)) is just for technical convenience. Basically, the matrix W (z; c, ρ) is

W (z; c, ρ) = ( ∑_{μn+c+g−h≡ρ (mod ν)} |L((0, 0) → (\frac{μn + c + g − ρ − h}{ν}, n) |

νx ≥ μy − ρ − h)| z^{n})_{0≤g,h≤ν−1}. (2.24)

The following theorem of Sato [?, Theorem 1] tells how to compute W (z; c, ρ).

Theorem 2.7. Let

M = ((−1)^{ν−h−1}s_{(c+g+1,1^{ν−h−1})}(u_{0}(z), …, u_{ν−1}(z)))_{0≤g,h≤ν−1} , (2.25)

where

s_{(α,1^{β})}(u_{0}, …, u_{ν−1})

= ∑_{ν−1≥i_{α}≥i_{α−1}≥...i_{1}<j_{1}<···<j_{β}≤ν−1} u_{i_{α}}(z)u_{i_{α−1}}(z) ⋯ u_{i_{1}}(z)u_{j_{1}}(z) ⋯ u_{j_{β}}(z), (2.26)

u_{l}(z) being defined by

u_{l}(z) = e^{(2πil/ν)} ∑_{n≥0} \frac{1}{1 + (ν + μ)n}\binom{\frac{1}{ν} + (1 + \frac{μ}{ν})n}{n} (ze^{2πilμ/ν})^{n} ,

l = 0, 1, …, ν − 1. (2.27)

Furthermore, let

Φ(z; ρ) = (_{∑_{μl≡ρ−g+h (mod ν)} ; μl≤ρ−g+h}(−1)^{l}\binom{(ρ − g + h − μl)/ν}{l}z^{l})_{0≤g,h≤ν−1}. (2.28)

Then, for any nonnegative integers c, ρ, ν, μ, with ν < μ, we have

W (z; c, ρ) = M(z; c, ρ) Φ(z; ρ).  (2.29)

Note 2.8. Note that s_{(α,1^{β})}(u_{0}, …, u_{ν−1}) is a Schur function of hook shape, which is to be explained in Chapter 3, Section 2.

Before we proceed to the proof of the theorem, it might be useful first to discuss an example, in order to illustrate what it is all about.

??remark that everything works also for ν = 2, μ arbitrary

Example 2.9. We take ν = 2, μ = 3. So, by (2.23), the quantity w(n; c, ρ) represents the number of all lattice paths from (0, 0) to ((3n + c − ρ)/2, n) which stay below the line 2x = 3y−ρ, where c ≡ 3n−ρ (mod 2), i.e., c ≡ n+ρ (mod 2). ??Figure

By definition (2.21), we have

W (z; c, ρ) = (w(z; c + g, ρ + h))_{0≤g,h≤1}

= (\begin{matrix}w(z; c, ρ) & w(z; c, ρ + 1) \\ w(z; c + 1, ρ) & w(z; c + 1, ρ + 1) \end{matrix}) .

Using (2.29), it can be written as

W (z; c, ρ) = M(z; c, 2) Φ(z; ρ),

where

Φ(z; ρ) = (\begin{matrix}_{\begin{matrix}∑ & μl≡ρ (mod ν) \\ μl≤ρ \end{matrix}}(−1)^{l}\binom{(ρ−μl)/ν}{l}z^{l} & _{\begin{matrix}∑ & μl≡ρ+1 (mod ν) \\ μl≤ρ+1 \end{matrix}}(−1)^{l}\binom{(ρ+1−μl)/ν}{l}z^{l} \\ _{\begin{matrix}∑ & μl≡ρ−1 (mod ν) \\ μl≤ρ−1 \end{matrix}}(−1)^{l}\binom{(ρ−1−μl)/ν}{l}z^{l} & _{\begin{matrix}∑ & μl≡ρ (mod ν) \\ μl≤ρ \end{matrix}}(−1)^{l}\binom{(ρ−μl)/ν}{l}z^{l} \end{matrix})

by (2.28), and

M(z; c, 2) = (\begin{matrix}−s_{(c+1,1)}(u_{0}(z), u_{1}(z)) & s_{(c+1)}(u_{0}(z), u_{1}(z)) \\ −s_{(c+2,1)}(u_{0}(z), u_{1}(z)) & s_{(c+2)}(u_{0}(z), u_{1}(z)) \end{matrix})

by (2.25), with s_{(α,1^{β})}(u_{0}(z), u_{1}(z)) being defined in (2.26), and

u_{l}(z) = (−1)^{l} ∑_{n≥0} \frac{(−1)^{ln}}{1 + 5n}\binom{\frac{1}{2} + \frac{5}{2}n}{n}z^{n},

as given in (2.27).

So, in particular, in case that c = ρ = 0 the matrix Φ(z; 0) is the 2 × 2 identity matrix, and so we have

W (z; 0, 0) = (\begin{matrix}w(z; 0, 0) & w(z; 0, 1) \\ w(z; 1, 0) & w(z; 1, 1) \end{matrix})

= M(z; 0, 2) = (\begin{matrix}−u_{0}(z)u_{1}(z) & u_{0}(z) + u_{1}(z) \\ −u_{0}(z)u_{1}(z)(u_{0}(z) + u_{1}(z)) & u_{0}^{2}(z) + u_{0}(z)u_{1}(z) + u_{1}^{2}(z) \end{matrix}) .

Whence, for even n the number of all lattice paths from (0, 0) to (3n/2, n) which stay below the line 2x ≥ 3y equals

|L((0, 0) → (\frac{3n}{2}, n) | 2x ≥ 3y)| = ⟨z^{n}⟩w(z; 0, 0)

= ∑_{l=0}^{n}(−1)^{l} \frac{1}{1 + 5l}\binom{\frac{1}{2} + \frac{5}{2}l}{l} · \frac{1}{1 + 5(n − l)}\binom{\frac{1}{2} + \frac{5}{2}(n − l)}{n − l},

and for odd n the number of all lattice paths from (0, 0) to ((3n − 1)/2, n) which stay below the line 2x ≥ 3y − 1 equals

|L((0, 0) → (\frac{3n−1}{2}, n) | 2x ≥ 3y − 1)| = ⟨z^{n}⟩w(z; 0, 1) = \frac{2}{1 + 5n}\binom{\frac{1}{2} + \frac{5}{2}n}{n}.

Clearly, as ρ and c increase (the geometrical meaning being that the starting and end point, respectively, move away from the boundary) the (explicit) expressions become more complicated.

??mention that this can also be derived using the cycle lemma, also mention that this nice expression can always obtained for arbitrary μ, ν in one particular case, w(z; 0, ν − 1)??

??ν = 1: compare with reflection principle results.

Proof of Theorem 2.7. The proof is divided into four steps.

First step. Let μn + c ≡ ρ and μn + c ≥ ρ. Consider paths from (0, 0) to ((μn + c − ρ)/ν, n). There are \binom{n+(μn+c−ρ)/ν}{n} such paths. Among them there are some which stay below the line νx = μy − ρ. These are counted by

|L((0, 0) → (\frac{μn + c − ρ}{ν}, n) | νx ≥ μy − ρ)| = w(n; c, ρ).

The rest consists of paths which cross the line νx = μy − ρ somewhere. In such a lattice path consider the last step which connects a lattice point outside the region νx ≥ μy−ρ to a lattice point inside the region νx ≥ μy−ρ, say (⌈(μl − ρ)/ν⌉−1, l) to (⌈(μl − ρ)/ν⌉ , l). This defines a unique decomposition of such a path into an arbitrary path from (0, 0) to (⌈(μl − ρ)/ν⌉ − 1, l), followed by a horizontal step, and then by a path in

L((⌈(μl − ρ)/ν⌉ , l) → ((μn + c − ρ)/ν, n) | νx ≥ μy − ρ).

Putting everything together, we obtain ??Figure!!

\binom{n + \frac{μn+c−ρ}{ν}}{n} = |L((0, 0) → ((μn + c − ρ)/ν, n) | νx ≥ μy − ρ)|

+ ∑_{ρ<μl≤μn+c} \binom{l + ⌈\frac{μl−ρ}{ν}⌉ − 1}{l}

· |L((⌈(μl − ρ)/ν⌉ , l) → ((μn + c − ρ)/ν, n) | νx ≥ μy − ρ)|  (2.33)
= |L((0, 0) → ((μn + c − ρ)/ν, n) | νx ≥ μy − ρ)|

+ ∑_{r=0}^{ν−1}_{∑_{μl+r≡ρ (mod ν)} ; ρ+ν≤μl+r≤μn+c}\binom{l + \frac{μl−ρ+r}{ν} − 1}{l}

· |L(((μl − ρ + r)/ν, l) → ((μn + c − ρ)/ν, n) | νx ≥ μy − ρ)|  (2.34)

= w(n; c, ρ) + ∑_{r=0}^{ν−1} ( ∑_{μl+r≡ρ (mod ν)} w(n − l; c, r)\binom{l + \frac{μl+r−ρ}{ν} − 1}{l}

−_{∑_{μl+r≡ρ (mod ν)} ; μl+r≤ρ}w(n − l; c, r)\binom{l + \frac{μl+r−ρ}{ν} − 1}{l}). (2.35)

Note that the equation (2.35) also holds if μn + c < ρ because of our definition of w(n; c, ρ) for the case μn + c < ρ.

If we define the matrix F (z; a) by

F (z; a) := ( ∑_{μn+a≡0 (mod ν)} \binom{n + \frac{μn+a+g−h}{n}}{n}z^{n})_{0≤g,h≤ν−1} ,  (2.36)

equation (2.35) translates into the matrix identity

F (z; c − ρ) = W (z; c, ρ) + W (z; c, 0)(F (z; −ρ − ν) − Φ(z; ρ)). (2.37)

If in this identity we put ρ = 0 we get

F (z; c) = W (z; c, 0) + W (z; c, 0)(F (z; −ν) − Φ(z; 0)).

Now, because of our assumption ν < μ we have from the definition (2.28) of Φ(z; ρ) that Φ(z; 0) = I, the ν × ν identity matrix. Hence our last equation gives

F (z; c) = W (z; c, 0)F (z; −ν).  (2.38)

Second step. We define four matrices V_{1}, V_{2}, Λ, H as follows:

V_{1} = (u_{h}^{g}(z))_{0≤g,h≤ν−1} , V_{2} := (u_{g}^{−h}(z))_{0≤g,h≤ν−1} , Λ = (χ(g = h) u_{g}(z))_{1≤g,h≤ν−1} , (2.39)

H := (χ(g = h)\frac{1}{ν − (ν + μ)zu_{g}(z)^{μ}} )_{0≤g,h≤ν−1} ,  (2.40)

where u_{g}(z) is given by (2.27), as before, and χ(A)=1 if A is true and χ(A)=0 otherwise.

Then we claim that

F (z; c) = V_{1}Λ^{c}HV_{2}.  (2.41)

For the proof of the claim consider the equation

z = \frac{v(z)}{(1 + v(z))^{1+ \frac{μ}{ν}}} ,  (2.42)

to be solved for some formal power series v(z). By the Lagrange inversion ?? with g(z) = (1 + z)^{1/ν} and f(z) = z/(1 + z)^{1+μ/ν}, the unique power series solution of

4. SIMPLE PATHS WITH LINEAR BOUNDARIES WITH RATIONAL SLOPE 25

this equation satisfies

(1 + v(z))^{\frac{1}{ν}} = ∑_{n≥0} z^{n} (\frac{1}{n}⟨t^{−1}⟩\frac{(1 + t)^{\frac{1}{ν} +(1+ \frac{μ}{ν})n−1}}{t^{n}} )

= ∑_{n≥0} z^{n} \frac{1}{νn}\binom{\frac{1}{ν} + (1 + \frac{μ}{ν})n − 1}{n − 1}

= ∑_{n≥0} z^{n} \frac{1}{1 + (μ + ν)n}\binom{\frac{1}{ν} + (1 + \frac{μ}{ν})n}{n}.

So, comparing this with u_{0}(z) in (??), we have

u_{0}(z) = (1 + v(z))^{1/ν}.  (2.43)

Now, by (2.42) we obtain

\frac{(1 + v(z))^{\frac{c}{ν}}}{1 − (1 + \frac{μ}{ν})z(1 + v(z))^{\frac{μ}{ν}}} = \frac{(1 + v(z))^{\frac{c}{ν}}}{1 − (1 + \frac{μ}{ν})\frac{v(z)}{(1+v(z))}} ,

and therefore by the second form ?? of the Lagrange inversion with ??, we get

\frac{(1 + v(z))^{\frac{C}{ν}}}{1 − (1 + \frac{μ}{ν})z(1 + v(z))^{\frac{μ}{ν}}}

= ∑_{n≥0} z^{n}⟨t^{−1}⟩\frac{(1 + z)^{\frac{C}{ν}}}{(1 − (1 + \frac{μ}{ν})\frac{z}{1+z} )} · \frac{(1 − (1 + \frac{μ}{ν})\frac{z}{1+z} )}{z^{n+1}}(1 + z)^{(1+ \frac{μ}{ν})n}

= ∑_{n≥0} z^{n}⟨t^{n}⟩(1 + z)^{\frac{C}{ν} +(1+ \frac{μ}{ν})n}

= ∑_{n≥0} \binom{\frac{C}{ν} + (1 + \frac{μ}{ν})n}{n}z^{n}.  (2.44)

Let us write f(z; C) for the last line in the above computation,

f(z; C) := ∑_{n≥0} \binom{\frac{C}{ν} + (1 + \frac{μ}{ν})n}{n}z^{n}.

It should be noted that the matrix F (z; a), defined in (2.36), can be written as

F (z; a) = (f(z; a + g − h))_{0≤g,h≤ν−1}.  (2.45)

Now, if we combine (2.44) with (2.43), we obtain

f(z; C) = \frac{u_{0}(z)^{C}}{1 − (1 + \frac{μ}{ν})zu_{0}(z)^{μ}} .  (2.46)

On the other hand, we have

f(z; C) = ∑_{νn+C≡0 (mod ν)} \binom{\frac{C}{ν} + (1 + \frac{μ}{ν}n)}{n}z^{n}

= ∑_{νn+C≡0 (mod ν)} \binom{\frac{C}{ν} + (1 + \frac{μ}{ν}n)}{n}z^{n} \frac{1}{ν} ∑_{l=0}^{ν−1} e^{2πil(μn+C)/ν}

= \frac{1}{ν} ∑_{l=0}^{ν−1} e^{2πilC/ν} ∑_{νn+C≡0 (mod ν)} \binom{\frac{C}{ν} + (1 + \frac{μ}{ν}n)}{n} (ze^{2πilμ/ν})^{n}

= \frac{1}{ν} ∑_{l=0}^{ν−1} (e^{2πi/ν})^{lc} f (z (e^{2πi/ν})^{lμ}; C) .

Combined with (2.46) this gives, writing ω for e^{2πi/ν},

f(z; C) = ∑_{l=0}^{ν−1} ω^{lC} \frac{u_{0} (zω^{μl})^{C}}{ν − (μ + ν)zω^{μl}u_{0}^{μ}(z)ω^{μl}}

= ∑_{l=0}^{ν−1} \frac{u_{l}(z)^{C}}{ν − (μ + ν)zu_{l}^{μ}(z)},  (2.47)

the last line because of u_{l}(z) = ω^{l}u_{0}(zω^{μl}), see (2.27). Recalling our remark (2.45), we see that this last equation implies our claim (2.41).

Third step. Now we are able to finish the proof of the theorem. If we substitute (2.41) into (2.38), and rearrange terms a little, we get

W (z; c, 0) = V_{1}Λ^{c+ν}V_{1}^{−1}.  (2.48)

On the other hand, a combination of (2.41) and (2.48) implies

F (z; c − ρ) = V_{1}Λ^{c+ν}V_{1}^{−1}F (z; −ρ − ν).

Use of this identity reduces equation (2.37) to

W (z; c, ρ) = V_{1}Λ^{c+ν}V_{1}^{−1}Φ(z; ρ).  (2.49)

Fourth step. If we compare (2.49) with (2.29) we see that M(z; c, ρ) = V_{1}Λ^{c+ν}V_{1}^{−1}. So it remains to establish the claim (2.25) about the explicit form of the coefficients of V_{1}Λ^{c+ν}V_{1}^{−1}.

The matrices V_{1} and Λ are defined in (2.39). So the (g, h)-entry of V_{1}Λ^{c+ν}V_{1}^{−1} is given by

∑_{k=0}^{ν−1} u_{k}(z)^{c+ν+g} · \frac{(−1)^{k+h} (h, k)-minor of V_{1}}{det(V_{1})} = \frac{det Ṽ_{1}}{det V_{1}} ,

where Ṽ_{1} arises from V_{1} by replacing the h-th row by (u_{0}^{c+ν+g}(z), …, u_{ν−1}^{c+ν+g}(z)). In the matrix Ṽ_{1} we rearrange the rows so that the h-th row of Ṽ_{1} becomes the last row of the new matrix and the other rows keep their relative position. Thus we see that the (g, h)-entry of V_{1}Λ^{c+ν}V_{1}^{−1} equals

(−1)^{ν−h−1} \begin{matrix}det & 0≤g,h≤ν−1 & (u_{h}^{λ_{g}+g}(z)) \\ det & 0≤g,h≤ν−1 & (u_{h}^{g}(z)) \end{matrix},  (2.50)

where

λ = (λ_{0}, …, λ_{ν−1}) = (0^{h−1}, 1^{ν−h−1}, c + g + 1),

the notation a^{r} denoting a sequence of r a’s. The quotient of determinants in (2.50) is exactly the Schur function s_{(c+g+1,1^{ν−h−1})}(u_{0}(z), …, u_{ν−1}(z)), see e.g. [?].

It should be observed that equation (??) is exactly the matrix formulation of (??) for large C??connection to Niederhausen??

Sato [?] also derived a result of similar type for two parallel linear boundaries with rational slope. To be precise, we want to enumerate all lattice paths from an arbitrary starting point to an arbitrary end point staying below a given line νx = μy − ρ and above another given line νx = μy + σ, where μ, ν, ρ, σ are nonnegative integers. Again, without loss of generality we may assume that ν < μ and that the starting point is the origin.

Following the approach we have taken earlier, we define the ν × ν matrix T (z; c, ρ, σ) by

T (z; c, ρ, σ) := (t(z; c + g, ρ + h, σ − h))_{0≤g,h≤ν−1},  (2.51)

where

t(z; c, ρ, σ) = ∑_{μn+c≡ρ (mod ν)} t(n; c, ρ, σ) z^{n},  (2.52)

with

t(n; c, ρ, σ) = \begin{cases}|L((0, 0) → (\frac{μn+c−ρ}{ν}, n) | μy + σ ≥ νx ≥ μy − ρ)| & μn + c ≡ ρ (mod ν) \\ & and μn + c ≥ ρ \\ \binom{n+ \frac{μn+c−ρ}{ν}}{n} & μn + c ≡ ρ (mod ν) \\ & and μn + c < ρ. \end{cases}

(2.53)

Similar to the one boundary case, what the matrix T (z; c, ρ, σ) contains is generating functions of the path numbers t(n; c, ρ, σ) that we want to evaluate. The definition of t(n; c, ρ, σ) for μn + c < ρ (in which case there cannot be any paths from (0, 0) to (\frac{μn+c−ρ}{ν}, n)) is just for technical convenience. Basically, the matrix T (z; c, ρ, σ) is

T (z; c, ρ, σ) = ( ∑_{μn+c+g−h≡ρ (mod ν)} |L((0, 0) → (\frac{μn + c + g − ρ − h}{ν}, n) |

μy + σ − h ≥ νx ≥ μy − ρ − g)| z^{n})_{0≤g,h≤ν−1}. (2.54)

The following theorem of Sato [?, Theorem 4] tells how to compute T (z; c, ρ, σ).

Theorem 2.10. For any nonnegative integers c, ρ, σ, ν, μ, with ν < μ, we have

T (z; c, ρ, σ) = Φ(z; ρ + σ + 1 − c − μ) Φ^{−1}(z; ρ + σ + 1) Φ(z; ρ), (2.55)

where Φ(z; θ) is given by (2.28).

Proof. We proceed in a similar way as in the proof of Theorem 2.7. Namely, first we derive a recurrence from combinatorial considerations, which is subsequently trasformed into a matrix equation from which finally (2.55) can be derived.

Let μn + c ≡ ρ (mod ν) and μn + c > ρ + σ. Consider paths from (0, 0) to ((μn + c − ρ)/ν, n) which stay below νx = μy − ρ. There are w(z; c, ρ) such paths (see (2.23) for notation). Among them there are some which also stay above the line νx = μy + σ. The number of these is

|L((0, 0) → (\frac{μn + c − ρ}{ν}, n) | μy + σ ≥ νx ≥ μy − ρ)| = t(n; c, ρ, σ).

The rest consists of paths which cross the line νx = μy + σ somewhere. In such a lattice path consider the first step which connects a lattice point inside the region νx ≤ μy +σ to a lattice point outside the region νx ≤ μy +σ, say (⌊(μl + σ)/ν⌋ , l) to (⌊(μl + σ)/ν⌋ + 1, l). This defines a unique decomposition of such a path into a path in

L((0, 0) → (⌊(μl + σ)/ν⌋ , l) | μy + σ ≥ νx ≥ μy − ρ),

followed by a horizontal step, and then by a path in

L((⌊(μl + σ)/ν⌋ + 1, l) → ((μn + c − ρ)/ν, n) | νx ≥ μy − ρ).

Putting everything together, we obtain ??Figure!!

w(n; c, ρ) = |L((0, 0) → ((μn + c − ρ)/ν, n) | νx ≥ μy − ρ)|  (2.56)

= |L((0, 0) → ((μn + c − ρ)/ν, n) | μy + σ ≥ νx ≥ μy − ρ)|

+ ∑_{0≤μl+σ<μn+c−ρ} |L((0, 0) → (⌊(μl + σ)/ν⌋ , l) | μy + σ ≥ νx ≥ μy − ρ)|

· |L((⌊(μl + σ)/ν⌋ + 1, l) → ((μn + c − ρ)/ν, n) | νx ≥ μy − ρ)|  (2.57)

= |L((0, 0) → ((μn + c − ρ)/ν, n) | μy + σ ≥ νx ≥ μy − ρ)|

+ ∑_{r=0}^{ν−1}_{∑_{μl+σ≡r (mod ν)} ; 0≤μl+σ−r≤μn+c−ρ−ν} |L((0, 0) → ((μl + σ − r)/ν, l) | μy + σ ≥ νx ≥ μy − ρ)|

· |L(((μl + σ − r)/ν + 1, l) → ((μn + c − ρ)/ν, n) | νx ≥ μy − ρ)| (2.58)

= t(n; c, ρ, σ) + ∑_{r=0}^{ν−1} ( ∑_{μl+σ≡r (mod ν)} t(l; σ + ρ − r, ρ, σ) w(n − l; c, σ + ρ + ν − r)

−_{∑_{μl+σ≡r (mod ν)} ; μl+σ−r≥μn+c−ρ}t(l; σ + ρ − r, ρ, σ) w(n − l; c, σ + ρ + ν − r)), (2.59)

or, after the index transformation r → ν − 1 − r,

w(n; c, ρ) = t(n; c, ρ, σ)

+ ∑_{r=0}^{ν−1} ( ∑_{μl+σ+r+1≡0 (mod ν)} t(l; σ + ρ + r + 1 − ν, ρ, σ) w(n − l; c, σ + ρ + r + 1)

−_{∑_{μl+σ+r+1≡0 (mod ν)} ; μl+σ+r+1−ν≥μn+c−ρ}t(l; σ + ρ + r + 1 − ν, ρ, σ) w(n − l; c, σ + ρ + r + 1)). (2.60)

The equation (2.60) also holds if μn+c ≤ ρ+σ. For, if ρ+σ ≥ μn+c ≥ ρ then no path from the origin to the end point ((μn + c − ρ)/ν, n) can have a point below the line νx = μy + σ, since any point (a, b) of such a path satisfies

νa ≤ ν \frac{μn + c − ρ}{ν} ≤ σ ≤ μy + σ.

Therefore the sums in (2.60) are empty, and so (2.60) is true since t(n; c, ρ, σ) must equal w(n; c, ρ) since the boundary νx = μy + σ can never be crossed in this case. On the other hand, if μn + c < ρ then the definitions of t(n; c, ρ, σ) and w(n; c, ρ) in this exceptional case guarantee the validity of (2.60), the sums in (2.60) being again empty.

Equation (2.60) implies the following identity for the associated matrices,

W (z; c, ρ) = T (z; c, ρ, σ)

+ (W (z; c, ρ + σ + 1) − Φ(z; ρ + σ + 1 − c − ν))T (z; ρ + σ + 1 − ν, ρ, σ). (2.61)

If in this identity we put c = ρ + σ + 1 − ν we get

W (z; ρ + σ + 1 − ν, ρ) = W (z; ρ + σ + 1 − ν, ρ + σ + 1)T (z; ρ + σ + 1 − ν, ρ, σ),

again because our assumption ν < μ implies that Φ(z; 0) equals the ν × ν identity matrix. If we use (2.29) in this equation, we find

T (z; ρ + σ + 1 − ν, ρ, σ) = Φ^{−1}(z; ρ + σ + 1) Φ(z; ρ).  (2.62)

Now, if we substitute (2.62) and (2.29) into (2.61) then we obtain (2.55) after simplification.

Example 2.11. As an illustration, let us again consider the case ν = 2, μ = 3. By (2.53), the quantity t(n; c, ρ, σ) represents the number of all lattice paths from (0, 0) to ((3n + c − ρ)/2, n) which stay below the line 2x = 3y − ρ and above the line 2x = 3y + σ, where c ≡ 3n − ρ (mod 2), i.e., c ≡ n + ρ (mod 2). ??Figure

By definition (2.21), we have

T (z; c, ρ) = (t(z; c + g, ρ + h, σ − h))_{0≤g,h≤1}

= (\begin{matrix}t(z; c, ρ, σ) & t(z; c, ρ + 1, σ − 1) \\ t(z; c + 1, ρ, σ) & t(z; c + 1, ρ + 1, σ − 1) \end{matrix}) .

By Theorem 2.10, it can be written as

T (z; c, ρ, σ) = (\begin{matrix}ϕ(z; ρ + σ − c − 1) & ϕ(z; ρ + σ − c) \\ ϕ(z; ρ + σ − c − 2) & ϕ(z; ρ + σ − c − 1) \end{matrix})

× (\begin{matrix}ϕ(z; ρ + σ + 1) & ϕ(z; ρ + σ + 2) \\ ϕ(z; ρ + σ) & ϕ(z; ρ + σ + 1) \end{matrix})^{−1}  (2.63)
× (\begin{matrix}ϕ(z; ρ) & ϕ(z; ρ + 1) \\ ϕ(z; ρ − 1) & ϕ(z; ρ) \end{matrix}) ,  (2.64)

where

ϕ(z; a) = _{∑_{μl≡a (mod ν)} ; μl≤a}(−1)^{l}\binom{(a − μl)/ν}{l}z^{l}.

In particular, if c = σ and ρ = 0, so that we are interested e.g.?? in the number of all lattice paths from (0, 0) to ((3n + c)/2, n) which stay below the line 2x = 3y and above the line 2x = 3y + c (the reader should observe that this means that the starting point is on the first line whereas the end point is on the second line), then our formula (2.64) reduces to

T (z; c, ρ) = (\begin{matrix}t(z; c, 0, c) & t(z; c, 1, c − 1) \\ t(z; c + 1, 0, c) & t(z; c + 1, 1, c − 1) \end{matrix})
= \frac{1}{ϕ^{2}(z; c + 1) − ϕ(z; c)ϕ(z; c + 2)} (\begin{matrix}ϕ(z; c) & ϕ(z; c + 1) \\ 0 & 0 \end{matrix}) . (2.65)

Thus we obtain

t(z; c, 0, c) = ∑_{n≡c (mod 2)} |L((0, 0) → (\frac{3n + c}{2}, n) | 3y + c ≥ 2x ≥ 3y)| z^{n}

= \frac{ϕ(z; c)}{ϕ^{2}(z; c + 1) − ϕ(z; c)ϕ(z; c + 2)},  (2.66)

with

ϕ(z; a) = _{∑_{3l≡a (mod 2)} ; 3l≤a}(−1)^{l}\binom{(a − 3l)/2}{l}z^{l}.

5. Simple paths with general boundaries

The most general problem to encounter is to count paths in a region that is bounded by nonlinear upper and lower boundaries as exemplified in Figure 10.

• _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{.....}. . . . .. . . . .^{.....}. . . . .. . . . .^{.....}. . . . .^{.....}^{.....}^{.....} ...... . . . .^{.....}^{.....}. . . . .^{.....}. . . . .. . . . .. . . . .^{.....}^{.....}. . . . .. . . . .. . . . . a_{1} a_{2} a_{3} a_{4}  (n, a_{n}) (0, b_{1}) b_{1} b^{2} b^{3} b_{4} y_{1} y_{2} y_{3} y_{4} •  •^{Figure 10}

To have a convenient notation, let a_{1} ≤ a_{2} ≤ ⋯ ≤ a_{n} and b_{1} ≤ b_{2} ≤ ⋯ ≤ b_{n} be integers with a_{i} ≥ b_{i}. We abbreviate a = (a_{1}, a_{2}, …, a_{n}) and b = (b_{1}, b_{2}, …, b_{n}). By L((0, b_{1}) → (n, a_{n}) | a ≥ y ≥ b) we denote the set of all lattice paths from (0, b_{1}) to (n, a_{n}) that satisfy the property that for all i = 1, 2, …, n the height y_{i} of the i-th horizontal step is in the interval [b_{i}, a_{i}]. If we also write y(P) = (y_{1}, y_{2}, …, y_{n}) for the sequence of horizontal steps of a path P, then the notation just introduced explains itself. Pictorially (see Figure 10) the described restriction means that we consider paths in a ladder-shaped region, the upper ladder being determined by a, the lower ladder being determined by b.

Curiously enough, the most conceptual and most elegant way to attack this problem is by seemingly complicating it, namely by replacing our restricted (single) lattice paths by whole families of paths, more precisely by families of nonintersecting (by which we actually mean non-touching; see Chapter 3) lattice paths. This transformation works as follows. First, given any lattice path P from (0, b_{1}) to (n, a_{n}), we encode P by its sequence y(P) = (y_{1}, y_{2}, …, y_{n}) of heights of its horizontal steps, see Figures 10 and 11.a. Clearly, this sets up a one-to-one correspondence between L((0, b_{1}) → (n, a_{n}) | a ≥ y ≥ b) and sequences (y_{1}, y_{2}, …, y_{n}) that satisfy a_{i} ≥ y_{i} ≥ b_{i}, i = 1, 2, …, n. Such a sequence is now replaced by a set (P_{1}, P_{2}, …, P_{n}) of lattice paths P_{i}, where for i = 1, 2, …, n the path P_{i} runs from (n − i, b_{i} + i) to (n + 1 − i, a_{i} + i), having its unique horizontal step at height y_{i} + i. This can be achieved by a series of transformations, as is illustrated in Figure 11, which displays an example with n = 6, a = (3, 5, 7, 8, 8, 8), b = (0, 1, 1, 2, 5, 5), y(P_{0}) = (2, 2, 2, 4, 6, 8).

• _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} P_{0}^{.....}. . . . .. . . . .^{.....}. . . . .. . . . .^{.....}. . . . .^{.....}^{.....}^{.....} ...... . . . .^{.....}^{.....}. . . . .^{.....}. . . . .. . . . .. . . . .^{.....}^{.....}. . . . .. . . . .. . . . . 3 5 7 8 8 8 0 1 1 2 5 5 2 2 2 4 6 •  •

a. A bounded lattice path

⟷

• _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} • • • • • • • • • • • •^{.....}. . . . .. . . . .^{.....}. . . . .. . . . .^{.....}. . . . .^{.....}^{.....}^{.....} ...... . . . .^{.....}^{.....}. . . . .^{.....}. . . . .. . . . .. . . . .^{.....}^{.....}. . . . .. . . . .. . . . .

b. Family of lattice paths

↕

• _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} • • • • • • • • • • • •

⟷

• _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} • • • • • • • • • • • •

d. Family of nonintersecting lattice paths

c. Lattice paths shifted by one unit relative to each other Figure 11

What becomes immediately evident from the Figure 11 is, that the construction guarantees that no two of the lattice paths P_{1}, P_{2}, …, P_{n} have a point in common, and its converse, namely that any family (P_{1}, P_{2}, …, P_{n}) of lattice paths, where P_{i} runs from (n−i, b_{i} +i) to (n+1−i, a_{i} +i) and no two of the paths have a point in common, corresponds to a sequence (y_{1}, y_{2}, …, y_{n}) with a_{i} ≥ y_{i} ≥ b_{i}. Thus we have set up a one-to-one correspondence between L((0, b_{1}) → (n, a_{n}) | a ≥ y ≥ b) and the set of all families (P_{1}, P_{2}, …, P_{n}) of nonintersecting lattice paths, where P_{i} runs from (n−i, b_{i}+i) to (n+1−i, a_{i}+i). As we shall learn from Chapter 3, the main theorem on nonintersecting lattice paths (Theorem 3.1) immediately gives the answer to our counting problem.

Theorem 2.12. Let a = (a_{1}, a_{2}, …, a_{n}) and b = (b_{1}, b_{2}, …, b_{n}) be integer sequences with a_{1} ≤ a_{2} ≤ ⋯ ≤ a_{n}, b_{1} ≤ b_{2} ≤ ⋯ ≤ b_{n}, and a_{i} ≥ b_{i}, i = 1, 2, …, n. The number of all paths from (0, b_{1}) to (n, a_{n}) satisfying the property that for all i = 1, 2, …, n the height of the i-th horizontal step is between b_{i} and a_{i} is given by

|L((0, b_{1}) → (n, a_{n}) | a ≥ y ≥ b)| = det_{1≤i,j≤n} (\binom{a_{i} − b_{j} + 1}{j − i + 1}) . (2.67)

Proof. Apply Theorem 3.1 with A_{i} = (n − i, b_{i} + i), E_{i} = (n + 1 − i, a_{i} + i).

Originally, the above result was derived by Kreweras [?] using recurrence relations. In the one-sided boundary case, the result was given by Narayana [?]. He used properties of the determinant and induction to derive the result (see [?, p. ??]). The above proof is due to Sulanke [?].

Of course, with increasing n this formula will become less tractable. An alternative formula can be obtained by rotating the whole picture by 90^{∘} and applying formula (2.67) to the new situation. Now the size of the determinant is a_{n}, which is smaller than before if a_{n} < n, i.e., if the difference between the y-coordinates of end and starting point is less than the difference between the respective x-coordinates.

In some cases, a different type of formula might be preferrable, which we are going to describe next. This different type of formula is derived by a method which we call dummy path technique, as proposed in Krattenthaler and Mohanty [?]. Again, it comes from nonintersecting lattice paths. It is based on the following observation (see Stanley [?, ??]).

Lemma 2.13. Let C_{1}, C_{2}, …, C_{n} be pairwise distinct points in ℤ^{2}. Then the number of lattice paths from (a, b) to (c, d) which avoid C_{1}, C_{2}, …, C_{n} is given by

det_{1≤i,j≤n+1} ( |L(A_{j} → E_{i})| ),  (2.68)

where A_{1} = (a, b), A_{2} = C_{1}, …, A_{n+1} = C_{n+1}, E_{1} = (c, d), E_{2} = C_{1}, …, E_{n+1} = C_{n}.

Proof. We reformulate our counting problem in the weird way that we want to determine the number of families (P_{1}, P_{2}, …, P_{n+1}) of nonintersecting lattice paths, where P_{1} runs from A_{1} = (a, b) to E_{1} = (c, d), and for i = 1, 2, …, n the “dummy path” P_{i+1} runs from A_{i+1} = C_{i} to E_{i+1} = C_{i}. By Theorem 3.1, with 𝕊 = {(1, 0), (0, 1)}, A_{i} and E_{i} as above, this number would equal the determinant in (2.68), provided the (σ = id)-condition is true. Indeed, suppose that (P_{1}, P_{2}, …, P_{n+1}) is a family of nonintersecting lattice paths, where for i = 1, 2, …, n + 1 the path P_{i} runs from A_{σ(i)} to E_{i}, for some permutation σ ∈ 𝔖_{n+1}. Suppose that σ(i) ≠ i for some i ≥ 2. Then the path P_{i} starts at A_{σ(i)} = C_{σ(i)−1} and terminates at E_{i} = C_{i−1}. Since C_{σ(i)−1} ≠ C_{i−1}, there must be another path, P_{j} say, starting in C_{i−1}. But then P_{i} and P_{j} would intersect in C_{i−1}, which is absurd. Therefore σ(i) = i for all i = 2, 3, …, n + 1, hence also for i = 1.

(??refer to a Figure??)

Now, in order to count paths from (0, b_{1}) to (n, a_{n}) which stay between the ladder-shaped boundaries induced by a and b, we would proceed by counting all paths from (0, b_{1}) to (n, a_{n}) which avoid the points (0, a_{1} + 1), (1, a_{2} + 1), …, (n − 2, a_{n−1} + 1), (2, b_{2} − 1), …, (n, b_{n} − 1). By Lemma 2.13 we would obtain a (2n − 1) × (2n − 1)-determinant for the desired number. In general, this is of course much worse than the n × n determinant in Theorem 2.12. However, this determinant would reduce in size if some of the a_{i}’s would equal a_{n} or some of the b_{i}’s would equal b_{1}, which is reflected by the fact that one could forget about dummy paths at points (i, a_{n} + 1) or (i, b_{1} − 1). For concreteness, we state what is obtained by the dummy path technique for the one-boundary case b_{i} ≡ 0. The reader should be reminded, that also this determinant would reduce in size if some of the a_{i}’s would equal a_{n}. The result was first derived in Mohanty [?, ??] by use of recurrence, similar in type to (??). The method in [?] also led to a formula for the two-boundary case which however is not as elegant as the one suggested above (and also not as the one in Theorem 2.12). (The reader should be warned that there are misprints in [?, ??].)

Theorem 2.14. Let a = (a_{1}, a_{2}, …, a_{n}) be a sequence of integers satisfying a_{1} ≤ a_{2} ≤ ⋯ ≤ a_{n}. The number of all lattice paths from (0, 0) to (n, a_{n}) satisfying the property that for i = 1, 2, …, n the height of the i-th horizontal step is at most a_{i} equals

|L((0, 0) → (n, a_{n}) | a ≥ y)| = det_{1≤i,j≤n}(d_{ij}),  (2.69)

where

d_{ij} = \begin{cases}\binom{a_{n}+n}{n} & i = j = 1 \\ \binom{a_{n−i+1}+n−i+1}{n−i} & i ≠ 1, j = 1 \\ \binom{a_{n}−a_{n−j+1}+j−1}{j} & i = 1, j ≠ 1 \\ \binom{a_{n−i+1}−a_{n−j+1}+j−i}{j−i} & i ≠ 1, j ≠ 1. \end{cases}

Proof. Apply Lemma 2.13 with (a, b) = (0, 0), (c, d) = (n, a_{n}), C_{i} = (n − i − 1, a_{n−i} + 1), i = 1, 2, …, n − 1.

6. A first excursion to weighted counting

. In this section we lay the ground work for weighted counting of lattice paths. More precisely, given a function w acting on lattice paths, called a weight function, we are interested in determining the generating function GF (M; w) = ∑_{P} w(P), where the sum is over all lattice paths P in some particular set M of lattice paths. In most cases w will map P to a monomial q^{s(P)} (or more generally q_{1}^{s_{1}(P)}q_{2}^{s_{2}(P)} ⋯ q_{m}^{s_{m}(P)}), where s(P) (respectively s_{1}(P), s_{2}(P), …, s_{m}(P)) is some “lattice path statistics”, that is an integer-valued (very rarely real-valued) function on lattice paths. Once an expression for GF (M; q^{s(.)}), say, is obtained, extraction of the coefficient of q^{n} gives the number of paths P in M for which the value s(P) is exactly n. Before reading this section the reader should recall the q-notations introduced in the introduction (??).

In this section we exclusively concentrate on one statistics, namely the area statistics. (More remarks, that here are basic things, also important for the folowing??) To be more concrete, let P be a path from (a, b) to (c, d). We denote the signed area of the region between P and the x-axes by area(P). For the path P_{1} in Figure 12 we would have area(P_{1}) = 7, for the path P_{2} we would have area(P_{2}) = 1.

• _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•}^{. . . . .}^{. . . . .}^{. . . . .} •  •^{P_{1}}  _{•} _{•} _{•} _{•} _{•} _{•} _{•}_{•} _{•} _{•} _{•} _{•} _{•} _{•}_{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}_{. . . . .}_{. . . . .}  ^{. . . . .}^{. . . . .}^{. . . . .} •  •^{P_{2}}^{Figure 12}

Let P be a lattice path from (a, b) to (c, d). Recalling the encoding y(P) = (y_{1}, y_{2}, …, y_{c−a}) in terms of heights of horizontal steps of P (see Section 5), we observe that

area(P) = y_{1} + y_{2} + ⋯ + y_{c−a} .  (2.70)

The study of the area statistics arises in ?? Also it immediately connects to partition theory. This connection is elaborated in the next paragraph. A general reference for partition theory is Andrews [?].

A (linear) partition of the positive integer m is a sequence λ = (λ_{1}, λ_{2}, …, λ_{r}) of positive integers λ_{1} ≥ λ_{2} ≥ ⋯ ≥ λ_{r} > 0 and λ_{1} + λ_{2} + ⋯ + λ_{r} = m. If λ is a partition of m we also say that λ partitions m and write λ ⊢ n. The components λ_{i} are called parts of λ. We write n(λ) = λ_{1} + λ_{2} + ⋯ + λ_{r} for the sum of all parts of λ and call it the norm of λ. A partition λ is frequently viewed pictorially in form of a so-called Ferrer’s diagram. In English convention, the Ferrer’s diagram of λ = (λ_{1}, λ_{2}, …, λ_{r}) is the collection of r left-justified rows, the i-th row consisting of λ_{i} cells (see Figure 13).

λ = (6, 6, 5, 2, 1)

A partition and its Ferrer’s diagram

Figure 13

An immediate observation is that, by the encoding in terms of heights of horizonatal steps, lattice paths from (0, 0) to (c, d) with area m bijectively correspond to partitions of m with at most c parts, each part being at most d. (This is also seen pictorially by reflecting the region between such a lattice path and the x-axes with respect to the line x = y. Thus this region is turned into a Ferrer’s diagram corresponding to one of the above described partitions.) To enumerate the latter is a basic question in partition theory. A closed expression is not available. But a nice answer can be given in generating function form. The next theorem (see in particular (2.72)) tells that the generating function ∑_{λ}q^{n(λ)} for these partitions λ is a q-binomial coefficient. Starting with this expression for the generating function, Andrews and Almkvist [?] derived an asymptotic expression for the number of restricted partitions. We remark that the theorem below is a q-analogue of (2.1).

Theorem 2.15. The generating function ∑_{P} q^{area(P)} for all paths from (a, b) to (c, d) is

GF (L((a, b) → (c, d)); q^{area(.)}) = q^{b(c−a)} [\begin{matrix}c + d − a − b \\ c − a \end{matrix}]_{q} .  (2.71)

Proof. By a translation in direction (−a, −b) we see that

GF (L((a, b) → (c, d)); q^{area(.)}) = q^{b(c−a)}GF (L((0, 0) → (c − a, d − b)); q^{area(.)}).

Therefore it suffices to establish

GF (L((0, 0) → (c, d)); q^{area(.)}) = [\begin{matrix}c + d \\ c \end{matrix}]_{q} .  (2.72)

This is easily verified inductively, by using the lattice path recurrence

GF (L((0, 0) → (c, d); q^{area(.)}) = q^{d} GF (L((0, 0) → (c − 1, d)); q^{area(.)})

+ GF (L((0, 0) → (c, d − 1)); q^{area(.)}),

and comparing with the recurrence (??) for q-binomial coefficients.

We point out yet another equivalent formulation of (2.72). Recall (see ??) that lattice paths from (0, 0) to (c, d) can be seen as multiset pemutations of {1^{c}, 2^{d}} by writing a 1 for each horizontal step and a 2 for each vertical step. A statistics for permutations π of multisets, introduced by MacMahon [?, ??], is the number of inversions of π, for which we write inv(π). If π = π_{1}π_{2} … π_{r} then an inversion is a pair (i, j) with i < j and π_{i} > π_{j}. ??Example!! A moment’s thought shows that if P is a path from (0, 0) to (c, d), and π_{P} is the permutation of {1^{c}, 2^{d}} corresponding to P, then area(P) = inv(π_{P}), which can be checked in Figure 12. Thus (2.72) can also be formulated as

∑_{π} q^{inv(π)} = [\begin{matrix}c + d \\ c \end{matrix}]_{q} ,  (2.74)

where the sum is over all permutations of {1^{c}, 2^{d}}. This result is due to MacMahon [?, ??]. A multidimensional generalization of both Theorem 2.15 and (2.72) will be given in ??. Lattice path counting based on other permutation statistics is discussed in Chapter ??, Section ??.

Having answered the question of area counting of unrestricted paths, we proceed by considering the area counting of paths with restrictions. The simplest restriction to consider would be a linear boundary of the type x = μy. Disappointingly, neither of the Theorem 2.1, Corollary 2.2, Theorems 2.3–2.6 generalize. (However, Theorems 2.1, 2.3, and Corollary 2.2 do generalize, if a different statistics is considered. This connects to counting paths by their number of turns. See Chapter 4, Section ??). There is only a q-analogue of the generating function identity (2.15), which gives at least an implicit answer.

Theorem 2.16. Let μ, k be nonnegative integers. Then

∑_{n≥0} GF (L((0, 0) → (μn + k, n) | x ≥ μy); q^{area(.)})

\frac{q^{\binom{n}{2}}z^{n}}{(1 + z)(1 + qz) ⋯ (1 + q^{(μ+1)n+k}z)} = 1. (2.75)

Proof. Our proof works by multiplying both sides of (2.75) by (1 + z)(1 + qz) … (1 + q^{(μ+1)N+k}z), for some nonnegative integer N, and verifying that in the resulting formal power series the coefficients of z^{l} agree for all l = 0, 1, …, N. ??In den Notizen behaupte ich, dass es reicht z^{N} statt z^{l} zu betrachten?? Having proved this to hold for all N, (2.75) would be established.

If we do this multiplication, (2.75) is turned into

∑_{n≥0} GF (L((0, 0) → (μn + k, n) | x ≥ μy); q^{area(.)})

q^{\binom{n}{2}}z^{n}(1 + q^{(μ+1)n+k+1}z) ⋯ (1 + q^{(μ+1)N+k})

= (1 + z)(1 + qz) ⋯ (1 + q^{(μ+1)N+k}z). (2.76)

Note that if n > N then the expression in the second line of (2.76) should read as

q^{\binom{n}{2}} \frac{z^{n}}{(1 + q^{(μ+1)N+k+1}z) · (1 + q^{(μ+1)n+k})}.

In order to prove that the coefficients of z^{l} on both sides of (2.76) agree, we would have to prove

∑_{n≥0}_{∑_{P ∈L((0,0)→(μn+k,n)|x≥μy)} ; U∈S_{l−n}(\{(μ+1)n+k+1,...,(μ+1)N+k\})} q^{\binom{n}{2}}q^{area(P)}q^{∥U∥} = ∑_{V ∈S_{l}(\{0,1,...,(μ+1)N+k\})} q^{∥V ∥},(2.77)

where S_{k}(M) was defined before (2.18), and ∥M∥ stands for the sum of all the elements of a set M. We prove (2.77) by exhibiting a weight-preserving bijection φ between the “left-hand side objects” (n, P, U), n a nonnegative integer, P ∈ L((0, 0) → (μn + k, n) | x ≥ μy), U ∈ S_{l−n}({(μ + 1)n + k + 1, …, (μ + 1)N + k}), and the “right-hand side objects” V ∈ S_{l}({0, 1, …, (μ + 1)N + k}). By weightpreserving we mean that if φ((n, P, U)) = V then

\binom{n}{2} + area(P) + ∥U∥ = ∥V ∥ .  (2.78)

Let (n, P, U) be a left-hand side object. We represent P by the sequence z(P) = (z_{1}, z_{2}, …, z_{n}), where z_{i} denotes the distance of the (n−i+1)-st vertical step from the vertical line x = μn + k. Note that area(P) = z_{1} + z_{2} + ⋯ + z_{n}. (Figure??) Then we define

φ((n, P, U)) = {z_{1}, z_{2} + 1, …, z_{n} + n − 1} ∪ U.

Evidently, φ((n, P, U)) is in S_{l}({0, 1, …, (μ + 1)N + k}) and satisfies (2.78).

To construct the inverse, take a V = {v_{1}, v_{2}, … v_{l}} ∈ S_{l}({0, 1, …, (μ+1)N +k}). Let n be maximal with v_{n} ≤ (μ + 1)(n − 1) + k. Then it can be verified that φ^{−1} is given by

φ^{−1}(V) = (n, P, {v_{n+1}, v_{n+2}, …, v_{l}}),

where the path P is determined by defining the height of the i-th vertical step to be (μ + 1)n + k − v_{n−i+1} + i − 1. ??Example of map and inverse!!

This establishes that the coefficients of z^{l}, l = 0, 1, …, N, in (2.76) agree, thus completing the proof of (2.75).

What we basically did in the first proof of Theorem 2.4 was extraction of the coefficients of z^{n} in (2.15) (actually in the equivalent identity (2.13)) by use of the Lagrange inversion. Thus, in attempting to compute the coefficients in (2.75), which we are intested in, it is natural to ask for q-analogues of the Lagrange inversion that would be applicable to (2.75). Of particular interest, of course, is the special case μ = 1, k = 0 which gives rise to a q-analogue of the Catalan numbers. (??Bring Andrews q-Catalan numbers [?] at some place??) Indeed, motivated by finding expressions for these q-Catalan numbers, Andrews [?], Gessel [?], Garsia [?] developed a q-Lagrange theory that basically centers around the question of finding the coefficients in the expansion

g(z) = ∑_{n≥0} c_{n}f(z)f(qz) ⋯ f(q^{n−1}z),  (2.79)

where g(z) = g_{0} + g_{1}z + g_{2}z^{2} + ⋯ and f(z) = f_{1}z + f_{2}z^{2} + ⋯ are given formal power series. In spite of the beauty of the theory, in particular in [?] and [?], the applicability of the theory, even in the simple case f(z) = z/(1 + z)(1 + \sqrt{q}z) (which would correspond to our q-Catalan numbers with q replaced by \sqrt{q}) is limited, as far as explicit expressions are concerned. For the interested reader we record Andrews’ expansion for these q-Catalan numbers,

GF (L((0, 0) → (n, n) | x ≥ y); q^{area(.)}) =??  (2.80)

The search for q-Lagrange inversion formulas that was initiated by the μ = 1, k = 0 case of (2.75) also gave rise to other q-Lagrange inversion formulas in Hofbauer [?], Krattenthaler [?], Gessel and Stanton [?], [?], Garsia and Remmel[?], which apply to expansions of different type than (2.79). We shall meet one of these in ??, also in connection with q-Catalan numbers. See also Stanton [?] for a survey article on q-Lagrange inversion.

??More on q-Catalan numbers, recurrences,...??

??Mention explicitly the name Gould numbers??

If we look for an explicit expression for the coefficients in (2.75) we have to be satisfied with a determinant. This is also true for general μ and k. In fact, both Theorem 2.12 and the dummy path technique described in Section 5 generalize to area counting. Theorem 2.17 below gives the q-analogue of Theorem 2.12. It is Theorem ?? in Handa and Mohanty [?]. So, replacing n by μn + k, a_{i} by ⌊i/μ⌋, b_{i} by 0 in Theorem 2.17 gives an explicit expression for the coefficients in (2.75).

Theorem 2.17. Let a = (a_{1}, a_{2}, …, a_{n}) and b = (b_{1}, b_{2}, …, b_{n}) be integer sequences with a_{1} ≤ a_{2} ≤ ⋯ ≤ a_{n}, b_{1} ≤ b_{2} ≤ ⋯ ≤ b_{n}, and a_{i} ≥ b_{i}, i = 1, 2, …, n. The generating function ∑_{P} q^{area(P)} of all paths P from (0, b_{1}) to (n, a_{n}) satisfying the property that for all i = 1, 2, …, n the height of the i-th horizontal step of P is between b_{i} and a_{i} is given by

GF (L((0, b_{1}) → (n, a_{n}) | a ≥ y ≥ b); q^{area(P)})

= det_{1≤i,j≤n} (q^{b_{j}(j−i+1)+j(j−i)} [\begin{matrix}a_{i} − b_{j} + 1 \\ j − i + 1 \end{matrix}]_{q}) . (2.81)

Proof. Of course, we use the correspondence of Section 5 (??give it a name??) between the paths under consideration and families (P_{1}, P_{2}, …, P_{n}) of nonintersecting lattice paths, where P_{i} runs from (n−i, b_{i} +i) to (n−i+1, a_{i} +i). Observe that under this correspondence the weight property

area(P) = area(P_{1}) + area(P_{2}) + ⋯ + area(P_{n}) − \binom{n + 1}{2}  (2.82)

holds. Thus, if we define

w_{area}((P_{1}, P_{2}, …, P_{n})) = q^{∑_{i=1}^{n} area(P_{i})},  (2.83)

by (2.82) the generating function GF (L((0, b_{1}) → (n, a_{n}) | a ≥ y ≥ b); q^{area(P)})

equals q^{−\binom{n+1}{2}} times the generating function ∑_{P} w_{area}(P), where the sum is over all families of nonintersecting lattice paths which were described above. Application of Theorem 3.2 with 𝕊 = {(1, 0), (0, 1)}, A_{i} = (n − i, b_{i} + i), E_{i} = (n − i + 1, a_{i} + i), w_{area} as in (2.83), and use of (2.71) give

GF (L((0, b_{1}) → (n, a_{n}) | a ≥ y ≥ b); q^{area(P)})

= q^{−\binom{n+1}{2}} det_{1≤i,j≤n} (GF (L((n − j, b_{j} + j) → (n − i + 1, a_{i} + i)); q^{area(.)}))

= q^{−\binom{n+1}{2}} det_{1≤i,j≤n} (q^{(b_{j}+j)(j−i+1)} [\begin{matrix}a_{i} − b_{j} + 1 \\ j − i + 1 \end{matrix}]_{q} ),

which can be transformed into (2.81).

Finally, we turn to the dummy path technique of Section 5. As already remarked, it generalizes to area counting. This is because Lemma 2.13 has a straight-forward q-analogue.

Lemma 2.18. Let C_{1}, C_{2}, …, C_{n} be pairwise distinct points in ℤ^{2}. Then the generating function ∑_{P} q^{area(P)} for all lattice paths P from (a, b) to (c, d) which avoid C_{1}, C_{2}, …, C_{n} is given by

det_{1≤i,j≤n+1} (GF (L(A_{j} → E_{i}); q^{area(.)})),  (2.84)

where A_{1} = (a, b), A_{2} = C_{1}, …, A_{n+1} = C_{n+1}, E_{1} = (c, d), E_{2} = C_{1}, …, E_{n+1} = C_{n}.

Proof. Use the arguments of the proof of Lemma 2.13 and apply Theorem 3.2 with w_{area}((P_{1}, P_{2}, …, P_{n+1})) = q^{∑_{i+1}^{n+1} area(P_{i})}.

Thus, in order to obtain an alternative determinant for GF (L((0, b_{1}) → (n, a_{n}) |

a ≥ y ≥ b); q^{area(P)}), one would use Lemma 2.18 with (a, b) = (0, b_{1}), (c, d) = (n, a_{n}), C_{1} = (0, a_{1} + 1), …, C_{n−1} = (n − 2, a_{n−1} + 1), C_{n} = (2, b_{1} − 1), …, C_{2n−2} = (n, b_{n} − 1) in combination with (2.71). Again, the reader should be aware that this alternative determinant would reduce in size if some of the bounds are negligible, i.e. if a_{i} = a_{n} or b_{i} = b_{1} for some i. We content ourselves with stating the q-analogue of Theorem 2.14.

Theorem 2.19. Let a = (a_{1}, a_{2}, …, a_{n}) be a sequence of integers satisfying a_{1} ≤ a_{2} ≤ ⋯ ≤ a_{n}. The generating function ∑_{P} q^{area(P)} of all lattice paths P from (0, 0) to (n, a_{n}) satisfying the property that for i = 1, 2, …, n the height of the i-th horizontal step is at most a_{i} equals

GF (L((0, 0) → (n, a_{n}) | a ≥ y); q^{area(.)}) = det_{1≤i,j≤n}(d_{ij}),  (2.85)

where

d_{ij} = \begin{cases}[ \begin{matrix}a_{n}+n \\ n \end{matrix}]_{q} & i = j = 1 \\ [ \begin{matrix}a_{n−i+1}+n−i+1 \\ n−i \end{matrix} ]q & i ≠ 1, j = 1 \\ q^{(a_{n−j}+1)(j+1)} [ \begin{matrix}a_{n}−a_{n−j+1}+j−1 \\ j \end{matrix} ]_{q} & i = 1, j ≠ 1 \\ q^{(a_{n−j}+1)(j−i)} [ \begin{matrix}a_{n−i+1}−a_{n−j+1}+j−i \\ j−i \end{matrix} ]_{q} & i ≠ 1, j ≠ 1. \end{cases}

??Handa-Mohanty Lehmann formula ??refer to Steck higher dimensional determinant

44 II. SIMPLE LATTICE PATHS IN TWO DIMENSIONS

CHAPTER 3 Nonintersecting lattice paths

A family P = (P_{1}, P_{2}, …, P_{n}) of lattice paths P_{i} (not necessarily simple paths) is called nonintersecting if no two paths of P have a point in common. Otherwise P is called intersecting. Figure 14.a displays a family of nonintersecting lattice paths, Figure 14.b a family of intersecting lattice paths.

• _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} • • • • • •^{A_{3}} ^{A_{2}}  ^{A_{1}}  ^{E_{3}} ^{E_{2}}  ^{E_{1}}^{P_{3}}  ^{P_{2}}  ^{P_{1}}

a. Family of nonintersecting paths

• _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} • • • • • •^{A_{3}} ^{A_{2}}  ^{A_{1}}  ^{E_{3}} ^{E_{2}} ^{E_{1}}^{P_{3}}  ^{P_{2}}  ^{P_{1}}

b. Family of intersecting paths

Figure 14

As we have already seen in Chapter ??, Section 5, nonintersecting lattice paths can be very effective in solving particular counting problems. In fact, nonintersecting lattice paths arise very naturally in so many situations that we shall devote this whole chapter to developing their theory and applications.

??cite origin (Lindström), applications in representation theory, algebraic geometry, commutative algebra ??Overview: main theorem, very simple idea < − > determinant

1. The main theorem

Let A_{1}, A_{2}, …, A_{n} and E_{1}, E_{2}, …, E_{n} be lattice points in ℤ^{2} and 𝕊 be a set of steps in ℤ^{2}. What we want to count is the number of all families P = (P_{1}, P_{2}, …, P_{n}) of nonintersecting lattice paths, where P_{i} consists of steps from 𝕊 and runs from A_{i} to E_{i}, i = 1, 2, …, n. For convenience, we have to extend our lattice path notation to families of lattice paths. We write A = (A_{1}, A_{2}, …, A_{n}) and E = (E_{1}, E_{2}, …, E_{n}). By L(A → E; 𝕊 | R) we mean the set of all families P = (P_{1}, P_{2}, …, P_{n}) of lattice paths, where P_{i} consists of steps from 𝕊 and runs from A_{i} to E_{i} (i.e. is an element of L(A_{i} → E_{i}; 𝕊)), i = 1, 2, …, n, that obey the restriction R. So what we are interested in is the cardinality of L(A → E; 𝕊 | nonintersecting).

For the following recall that given a permutation σ ∈ 𝔖_{n} and vector v = (v_{1}, v_{2}, …, v_{n}) by v_{σ} we mean (v_{σ(1)}, v_{σ(2)}, …, v_{σ(n)}). Under the constraint (which is true in many situations) that L(A_{σ} → E; 𝕊 | nonintersecting) is only nonempty for σ = id, or with other words any family P = (P_{1}, P_{2}, …, P_{n}), where P_{i} runs from A_{σi} to E_{i}, must be intersecting, the number of nonintersecting lattice paths can be given in form of a very simple determinant. We call this constraint the (σ = id)-condition.

Theorem 3.1. Let 𝕊 be a set of steps in ℤ^{2}. Let A = (A_{1}, A_{2}, …, A_{n}) and E = (E_{1}, E_{2}, …, E_{n}) be sequences of lattice points in ℤ^{2} such that the (σ = id)- condition is true. Then the number of families (P_{1}, P_{2}, …, P_{n}) of nonintersecting lattice paths, where P_{i} consists of steps from 𝕊 and runs from A_{i} to E_{i}, i = 1, 2, …, n, is given by

|L(A → E; 𝕊 | nonintersecting)| = det_{1≤i,j≤n} ( |L(A_{j} → E_{i}; 𝕊)| ).  (3.1)

Proof. Again, as done several times before, the proof is by a cancelling involution. We start by giving a combinatorial description of the right-hand side of (3.1). By expanding the determinant we obtain

det_{1≤i,j≤n} ( |L(A_{j} → E_{i}; 𝕊)| ) = ∑_{σ∈𝔖_{n}} sgn σ ∏_{i=1}^{n} |L(A_{σ(i)} → E_{i}; 𝕊)|
= _{∑_{(σ,P)} ; σ∈𝔖_{n}, P∈L(A_{σ}→E;𝕊)}sgn σ.  (3.2)

The sum in (3.2) expresses the determinant in (3.1) as a generating function for pairs (σ, P) in the set

⋃_{σ∈𝔖_{n}} L(A_{σ} → E; 𝕊).  (3.3)

Below we shall define a sign-reversing involution on the set of all pairs (σ, P) in (3.2) with the property that P is intersecting. The meaning of sign-reversing will be explained in a moment. The involution appeared first in a paper by Lindström [?], although the same idea can be found earlier in Karlin and McGregor [?] (see also [?]) in the probabilistic language. It was rediscovered by Gessel and Viennot [?, ?] who gave many important applications of interest to us. For that reason we refer to this involution as the Gessel-Viennot involution and denote it by ϕ_{GV}. Now, sign-reversing means that if φ_{GV} ((σ, P)) = (σ_{φ_{GV}} , P_{φ_{GV}}) then sgn σ = − sgn σ_{φ_{GV}}. Suppose that this had been done. Then in the sum (3.2) all contributions of pairs (σ, P) in (3.3) where P is intersecting would cancel. Only contributions of pairs (σ, P) in (3.3) where P is nonintersecting would survive. However, because of our assumption, we must have σ = id for such pairs. Hence, in the sum (3.2) only the contributions of pairs (id, P) where P is an element of L(A → E; 𝕊 | nonintersecting) remain, proving (3.1).

Next we construct the sign-reversing involution φ_{GV}. Let (σ, P) be in L(A_{σ} → E; 𝕊) where P is intersecting. In the left-hand picture of Figure 15 an example with 𝕊 = {(1, 0), (0, 1)}, n = 3, σ = 213 is displayed. Among all the meeting points between any two paths of P choose the right-most, and among all those the highest. Denote this meeting point by M. In our example the meeting points are (3, 2), (3, 3), (4, 5), the right-most and highest being (4, 5). Among all the paths of P running through M choose P_{i} and P_{j}, i < j, with i and j being minimal. Now we interchange the initial portions of P_{i} and P_{j} up to M. To be more precise, we form the new paths

P′_{i} = [subpath of P_{j} from A_{σ(j)} to M joined with subpath of P_{i} from M to E_{i}]

and

P′_{j} = [subpath of P_{i} from A_{σ(i)} to M joined with subpath of P_{j} from M to E_{j}].

Then we define

φ_{GV} ((σ, P)) = φ_{GV} ((σ, (P_{1}, …, P_{i}, …, P_{j}, …, P_{n})))
= (σ ∘ (ij), (P_{1}, …, P′_{i}, …, P′_{j}, …, P_{n})),

where (ij) denotes the transposition interchanging i and j. The right-hand picture in Figure 15 shows what is obtained by this operation in our example.

• _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} . . . . .. . . . .. . . . .. . . . .^{.....}^{.....} • • •  • • • ∘ A^{1} A_{2} A_{3}  E_{1} E_{2} E_{3} M P_{1} P_{2} P_{3} σ = 213  ⟷  _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{.....}^{.....}^{.....}. . . . .. . . . .^{.....} • • •  • • • ∘ A^{1} A_{2} A_{3}  E_{1} E_{2} E_{3} M P_{1} P′_{3} P′_{2} σ′ = 231^{Figure 15}

The image φ_{GV} ((σ, P)) is again an element of the set in (3.3) since the new permutation of the starting points of P is exactly σ∘(ij). Besides, (P_{1}, …, P′_{i}, …, P′_{j}, …, P_{n}) is intersecting since P′_{i} and P′_{j} are. From all this it is obvious that when φ_{GV} is applied to φ_{GV} ((σ, P)) we arrive back at (σ, P). Hence, φ_{GV} is an involution. Since σ and σ ∘ (ij) differ in sign, φ_{GV} is sign-reversing.

??Discuss when assumption is satisfied.

Actually, the same argument also proves a weighted version of Theorem 3.1. Let w be a function that maps each step s ∈ 𝕊 to some element w(s) in a commutative ring R. Usually, R will be some ring of polynomials. We extend w to paths P by defining w(P) = ∏_{s} w(s) where the product is over all steps s of the path P. Finally, we extend w to families (P_{1}, P_{2}, …, P_{n}) of paths by defining it to be the product of all the weights of paths in the family, w((P_{1}, P_{2}, …, P_{n})) = ∏_{i=1}^{n} w(P_{i}). Then for the generating function ∑_{P} w(P) for families P of nonintersecting lattice paths we have the following result, which is completely analogous to Theorem 3.1.

Theorem 3.2. Let 𝕊 be a set of steps in ℤ^{2}. Let A = (A_{1}, A_{2}, …, A_{n}) and E = (E_{1}, E_{2}, …, E_{n}) be sequences of lattice points in ℤ^{2} such that the (σ = id)-condition is true. Then the generating function ∑_{P} w(P) for families P = (P_{1}, P_{2}, …, P_{n}) of nonintersecting lattice paths, where P_{i} consists of steps from 𝕊 and runs from A_{i} to E_{i}, i = 1, 2, …, n, is given by

GF (L(A → E; 𝕊 | nonintersecting); w) = det_{1≤i,j≤n} (GF (L(A_{j} → E_{i}; 𝕊); w)).

(3.4)

One might also ask, what happens if the assumption in Theorems 3.1 and 3.2, that nonintersecting lattice paths can only occur if the starting and end points are connected in the “right” order, is dropped. What the argument in the proof of Theorem 3.1??T2.2 shows is the following.

Theorem 3.3. Let 𝕊 be a set of steps in ℤ^{2}. Let A = (A_{1}, A_{2}, …, A_{n}) and E = (E_{1}, E_{2}, …, E_{n}) be sequences of lattice points in ℤ^{2}. Then there holds

∑_{σ∈𝔖_{n}} sgn σGF (L(A_{σ} → E; 𝕊 | nonintersecting); w) = det_{1≤i,j≤n} (GF (L(A_{j} → E_{i}; 𝕊); w)).

(3.5)

This looks to be a very artificial statement perhaps being of no use. However, even this result does have applications in lattice path counting. We shall need it for the proof of the determinant formula for higher-dimensional path counting under a general two-sided restriction (see Chapter 5, Section ??).

??To be added: Stembridge’s pfaffian theory Cylindric partitions nonintersecting lattics paths plus reflection principle

??Therefore: New chapter for applications of nonintersecting lattice paths??

2. Applications to plane partition and tableau counting

One of the most important applications of nonintersecting lattice paths is in counting plane partitions and tableaux. A number of definitions will follow now, all of them being very related.

Let λ = (λ_{1}, λ_{2}, …, λ_{r}) and μ = (μ_{1}, μ_{2}, …, μ_{r}) be r-tuples of integers such that λ ≥ μ, i.e., λ_{i} ≥ μ_{i} for all i. A plane partition of shape λ/μ is a planar array π of integers of the form

\begin{aligned}& π_{1,μ_{1}+1} & . . . . . . . . . . . . . . & π_{1,λ_{1}} \\ π_{2,μ_{2}+1} … & π_{2,μ_{1}+1} & . . . . . . . . & π_{2,λ_{2}} \\ ... & ⋮ & ... \\ π_{r,μ_{r}+1} & . . . . . . . . . . . . . . . . . . . . . . . . & π_{r,λ_{r}} \end{aligned}  (3.6)

such that the rows and columns are weakly decreasing, i.e.

π_{i,j} ≥ π_{i,j+1} and π_{i,j} ≥ π_{i+1,j}.  (3.7)

Figure 16 shows a plane partition of shape (7, 6, 4, 4)/(3, 1, 1, 0).

9 7 6 2 11 11 8 7 4 10 8 8 9 8 5 5^{Figure 16}

In particular, a linear partition (see Chapter 2, Section ??) is a one-rowed plane partition, or a one-columned plane partition. There are other special classes of plane partitions to be considered. A plane partition is called column-strict if the entries along each column are strictly decreasing. It is called row-strict if the entries along each row are strictly decreasing. To have a convenient terminology, following Krattenthaler [?] we introduce the notion of (α, β)-plane partitions. An (α, β)-plane partition of shape λ/μ is an array π of integers of the form (3.6) such that the decrease along rows is at least α and the decrease along columns is at least β, more precisely

π_{i,j} + α ≥ π_{i,j+1} and π_{i,j} + β ≥ π_{i+1,j}.  (3.8)

So in particular, (0, 1)-plane partitions are column-strict plane partitions, and (1, 0)-plane partitions are row-strict plane partitions.

Finally, we introduce various types of reverse plane partitions. A reverse plane partition of shape λ/μ is an array π of integers of the form (3.6) such that rows and columns are weakly increasing, i.e.

π_{i,j} ≤ π_{i,j+1} and π_{i,j} ≤ π_{i+1,j}.  (3.9)

It is called column-strict if the columns are strictly increasing, it is called row-strict if the rows are strictly increasing. An (α, β)-reverse plane partition of shape λ/μ is an array π of integers of the form (3.6) such that the increase along rows is at least α and the increase along columns is at least β, more precisely

π_{i,j} ≤ π_{i,j+1} + α and π_{i,j} ≤ π_{i+1,j} + β.  (3.10)

Of particular importance are column-strict reverse plane partitions consisting only of positive entries. These are called semistandard tableaux (sometimes only tableaux for short) and play an essential role in the representation theory of GL(n, ℂ), invariant theory, parts of commutative algebra [?],... The left-hand array π_{0} in Figure 17 is a semistandard tableau of shape (5, 4, 3)/(1, 0, 0). A semistandard tableau is called standard Young tableau if its entries are exactly 1, 2, …, p where p is the number of entries of the tableau. Standard Young tableaux are a basic ingredient of the representation theory of the symmetric group 𝔖_{n}. The righthand array π_{1} in Figure 17 is a standard Young tableau of shape (5, 4, 3)/(1, 0, 0).

π_{0} = 2 3 4 6 1 4 4 6 5 5 7  π_{1} = 1 5 7 11 2 3 6 10 4 8 9^{Figure 17}

Basically, (α, β)-reverse plane partitions and (α, β)-plane partitions are the same objects. In fact, an (α, β)-reverse plane partition π of shape λ/μ is turned into an (α, β)-plane partition π̃ of shape λ̃/μ̃ by the 180^{∘}-rotation

π̃_{ij} = π_{n−i+1,n−j+1},  (3.11)

where λ̄ = (n − μ_{r}, …, n − μ_{2}, n − μ_{1}) and μ̄ = (n − λ_{r}, …, n − λ_{2}, n − λ_{1}). There are two reasons to consider both. The main reason is that historically both have been considered. So, it is for the reader’s convenience to formulate results for both types. Second, some results come in a more attractive form when formulated by the “right” type.

We shall not only consider pure counting of (various types of) plane partitions but also weighted counting. We shall consider two different weights, one being a special case of the other. Let x = (x_{1}, x_{2}, …) be an infinite sequence of variables. Then we define the weight w_{x} of an array π of the form (3.6) by

w_{x}(π) = ⋯ x_{1}^{|1′s in π|}x_{2}^{|2′s in π|} · · · .  (3.12)

Thus, the weight of the column-strict reverse plane partition π_{0} in the left-hand picture of Figure 17 is w_{x}(π_{0}) = x_{1}x_{2}x_{3}x_{4}^{3}x_{5}^{2}x_{6}^{2}x_{7}. We call the sum ∑ π_{i,j} of all the entries of a plane partition π the norm of π and denote it by n(π), as we did earlier for the sum of all parts of a (linear) partition in Chapter 2, Section 6??. For example, the norm of the column-strict reverse plane partition π_{0} in Figure 17 is n(π_{0}) = 47. The weight w_{x}(π) and the norm n(π) are related in the following way. If in w_{x}(π) we specialize x_{i} = q^{i} we obtain q^{n(π)}.

Now, after having finished the definition part, we may start with the enumeration of (various types of) plane partitions. Since the connection between (various types of) plane partitions and nonintersecting lattice paths is most evident for column-strict reverse plane partitions, we shall start with describing the standard bijection between column-strict reverse plane partitions and nonintersecting lattice paths.

Let λ = (λ_{1}, λ_{2}, …, λ_{n}) and μ = (μ_{1}, μ_{2}, …, μ_{n}) be n-tuples of integers such that

λ_{1} ≥ λ_{2} ≥ λ_{3} ≥ ⋯ ≥ λ_{n},  (3.13a)
μ_{1} ≥ μ_{2} ≥ μ_{3} ≥ ⋯ ≥ μ_{n},  (3.13b)
and λ ≥ μ.??

One could allow slightly more general restrictions for λ and μ and would thus also cover plane partitions and tableaux of shifted shape which are very important in the projective representation theory of the symmetric group and of GL(n, ℂ) (see Stembridge [?, ?]) and several enumeration problems Andrews [?, ?]. However, for sake of simplicity we shall not pursue this here.

Let a = (a_{1}, a_{2} …, a_{n}) and b = (b_{1}, b_{2}, …, b_{n}) be sequences of integers such that

a_{1} ≤ a_{2} ≤ ⋯ ≤ a_{n}  (3.14a)
b_{1} ≤ b_{2} ≤ ⋯ ≤ b_{n},  (3.14b)
and a ≥ b.??

We claim that column-strict reverse partitions of shape λ/μ where the entries in row i are at most a_{i} and at least b_{i} bijectively correspond to families (P_{1}, P_{2}, …, P_{n}) of nonintersecting lattice paths P_{i}, where P_{i} runs from (μ_{i} − i, b_{i}) to (λ_{i} − i, a_{i}).

This is seen as follows. Let π be a column-strict reverse partition of shape λ/μ where the entries in row i are at most a_{i} and at least b_{i}.

−2 ≤  1 1 4 5 ≤ 7
−1 ≤  5 5 7  ≤ 8
0 ≤  2 5 6 9 9  ≤ 9
1 ≤ 1 5 6 8  ≤ 10

a. Column-strict reverse plane partition

⟷
• _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•}_{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•}_{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} • • • •  • • • •^{P_{4}}  ^{P_{3}}  ^{P_{2}}  ^{P_{1}}^{1} ^{1} ^{4} ^{5}^{5} ^{5} ^{7}^{2} ^{5} ^{6} ^{9} ^{9}^{1} ^{5} ^{6} ^{8}

b. Family of nonintersecting lattice paths

Figure 18

The column-strict reverse plane partition π is mapped to a family of lattice paths by associating the i-th row of π with a path P_{i} from (−λ_{i} + i, b_{i}) to (−μ_{i} + i, a_{i}) where the entries in the i-th row are interpreted as heights of the horizontal steps in the path P_{i}. Thus from π we obtain the family P = (P_{1}, …, P_{n}) of lattice paths. The right-hand picture of Figure 18 displays the family of lattice paths that in this way results from the array displayed in the left-hand picture of Figure 18.

Clearly, the property that the columns of π are strictly decreasing translates into the condition that (P_{1}, P_{2}, …, P_{n}) is nonintersecting.

It is obvious that this correspondence between column-strict reverse partitions π of a given shape with given row bounds and families of nonintersecting lattice paths with given starting and end points is a bijection. Thus, in order to enumerate column-strict reverse partitions, we have to enumerate nonintersecting lattice paths which we have already done in Theorems 3.1 and 3.2. Hence we are in the position to formulate our first plane partition enumeration result.

Theorem 3.4. Let λ = (λ_{1}, λ_{2}, …, λ_{n}) and μ = (μ_{1}, μ_{2}, …, μ_{n}) be sequences of integers satisfying (3.13). Let a = (a_{1}, a_{2}, …, a_{n}) and b = (b_{1}, b_{2}, …, b_{n}) be sequences of integers satisfying

a_{1} ≤ a_{2} ≤ ⋯ ≤ a_{n}  and  b_{1} ≤ b_{2} ≤ ⋯ ≤ b_{n}.  (3.15)

Then we have the following:

(i) The number of all column-strict reverse plane partitions π of shape λ/μ where the entries in row i are at most a_{i} and at least b_{i} equals

det_{1≤i,j≤n} (\binom{a_{i} − b_{j} + λ_{i} − μ_{j} − i + j}{λ_{i} − μ_{j} − i + j}) .  (3.16)

(ii) The generating function ∑_{π} q^{n(π)} for the same set of column-strict reverse plane partitions π equals

det_{1≤i,j≤n} (q^{b_{j}(λ_{i}−μ_{j}−i+j)} [\begin{matrix}a_{i} − b_{j} + λ_{i} − μ_{j} − i + j \\ λ_{i} − μ_{j} − i + j \end{matrix}]_{q}) .  (3.17)

(iii) More generally, the generating function ∑_{π} w_{x}(π) for the same set of columnstrict reverse plane partitions π equals

det_{1≤i,j≤n} (h_{λ_{i}−μ_{j}−i+j}(x_{b_{j}}, x_{b_{j}+1}, …, x_{a_{i}})),  (3.18)

where h_{m}(z_{1}, z_{2}, …, z_{r}) = ∑_{1≤i_{1}≤i_{2}≤···≤i_{m}≤r} z_{i_{1}}z_{i_{2}} ⋯ z_{i_{m}} denotes the complete homogeneous symmetric function of order m (cf. Macdonald [?, p. ??]).

(iv) Moreover, the number of all column-strict reverse plane partitions π of shape λ/μ where the entries in column i are at most a_{i} and at least b_{i} equals

det_{1≤i,j≤n} (e_{λ′_{i}−μ′_{j}−i+j}(x_{b_{j}}, x_{b_{j}+1}, …, x_{a_{i}})),  (3.19)

where e_{m}(z_{1}, z_{2}, …, z_{r}) = ∑_{1≤i_{1}<i_{2}<···<i_{m}≤r} z_{i_{1}}z_{i_{2}} ⋯ z_{i_{m}} denotes the elementary symmetric function of order m (cf. Macdonald [?, ??]), and λ′ and μ′, respectively, denote the partitions conjugate to λ and μ (see ??).

Proof. For establishing (3.16) we use the above correspondence between columnstrict reverse plane partitions and families of nonintersecting lattice paths and apply Theorem 3.1 with 𝕊 = {(1, 0), (0, 1)}, A_{i} = (μ_{i} − i, b_{i}), and E_{i} = (λ_{i} − i, a_{i}), i = 1, 2, …, n. Observe that the (σ = id)-condition is satisfied. In fact, (3.13) and (3.15) imply that for i = 1, 2, …, n − 1 the point A_{i+1} lies in the north-west of A_{i}, the same being true for E_{i+1} and E_{i}. By our discussion (where??), this guarantees the (σ = id)-condition to hold.

For the proof of (3.17) we observe that if the column-strict reverse plane partition π corresponds to P = (P_{1}, P_{2}, …, P_{n}) under the above correspondence, then there holds the relation

n(π) = area(P_{1}) + area(P_{2}) + ⋯ + area(P_{n}),

with area(.) having been defined in Chapter ??, Section 6. Thus, if we define area(P) = ∑_{i=1}^{n} area(P_{i}), then the above correspondence carries the norm n(π) of π into the area area(P) of P = (P_{1}, P_{2}, …, P_{n}).

Now, if we introduce the step weight w_{area}, which we define as

w_{area}(s) = \begin{cases}1 & s is a vertical step \\ q^{h} & s is a horizontal step at height h, \end{cases}  (3.20)

and extend it to families of lattice paths as was described above, then we have w_{area}(P) = q^{area(P)} for any path P and w_{area}(P) = q^{area(P)} for any family P of paths.

Therefore the above correspondence between column-strict reverse plane partitions π and families P = (P_{1}, P_{2}, …, P_{n}) of nonintersecting lattice paths has the property q^{n(π)} = w_{area}(P) (if π and P correspond to each other). Thus, the generating function ∑_{π} q^{n(π)} for the column-strict reverse plane partitions under discussion equals

GF (L(A → E | nonintersecting); w_{area}),

where A = (A_{1}, A_{2}, …, A_{n}) and E = (E_{1}, E_{2}, …, E_{n}) with A_{i} and E_{i} as above. Finally, application of Theorem 3.2 with 𝕊 = {(1, 0), (0, 1)}, A_{i} = (μ_{i} − i, b_{i}), E_{i} = (λ_{i} − i, a_{i}), w = w_{area}, and use of (2.71) lead to (3.17).

The proof for (3.18) is analogous. This time as step weight we take

w_{x}(s) = \begin{cases}1 & s is a vertical step \\ x_{h} & s is a horizontal step at height h, \end{cases}  (3.21)

and extend it to paths and families of paths. Then the above correspondence between column-strict reverse plane partitions π and families P = (P_{1}, P_{2}, …, P_{n}) of nonintersecting lattice paths has the property

w_{x}(π) = w_{x}(P).

Hence, application of Theorem 3.2 with 𝕊, A_{i}, E_{i} as before, but with w = w_{x}, together with the observation that

GF (L((a, b) → (c, d)); w_{x}) = h_{c−a}(x_{b}, x_{b+1}, …, x_{d})

lead to the desired result (3.18).

The formula (3.19) can also be derived by using nonintersecting lattice paths. Here, one would read off lattice paths from the columns of semistandard tableaux, rather than from the rows. The interested reader is referred to Sagan [?, ??] or Gessel and Viennot [?], where proofs by nonintersecting lattice paths for both (3.28) and (3.27) can be found.

??The expression (3.17) was first given by Stanley [?, ?, Prop. 21.2]. (??Maybe plane partition formulation) By defining the row-flagged Schur function of shape λ/μ as the sum

s_{λ/μ}^{r}(x; a, b) := ∑_{π} w_{x}(π),  (3.22)

where the summation is over all column-strict reverse plane partitions π of shape λ/μ, and the entries in row i are at most a_{i} and at least b_{i}, it can be seen that (3.18) gives a determinantal expression for it, namely,

s_{λ/μ}^{r}(x; a, b) = det_{1≤i,j≤n} (h_{λ_{i}−μ_{j}−i+j}(x_{b_{j}}, x_{b_{j}+1}, …, x_{a_{i}}))  (3.23)

(see Wachs [?, ??]). Similarly, if one defines the column-flagged Schur function as the sum

s_{λ/μ}^{c}(x; a, b) := ∑_{π} w_{x}(π),  (3.24)

where the summation is over all column-strict reverse plane partitions π of shape λ/μ, and the entries in column i are at most a_{i} and at least b_{i}, then (3.19) gives a determinantal expression for it, namely

s_{λ/μ}^{c}(x; a, b) = det_{1≤i,j≤n} (e_{λ′_{i}−μ′_{j}−i+j}(x_{b_{j}}, x_{b_{j}+1}, …, x_{a_{i}}))  (3.25)

(see Wachs [?, ??]). In particular, if b_{i} = 1 and a_{i} → ∞, then the flagged Schur functions specialize to (skew) Schur functions

s_{λ/μ}(x) := ∑_{π} w_{x}(π),  (3.26)

where the sum is over all semistandard tableaux of shape λ/μ. Schur functions play an essential role in the theory of symmetric functions, representation theory of GL(n, ℂ), invariant theory,...?? In particular, the Schur functions s_{λ}(x) (here λ denotes the shape λ/0) are the irreducible characters for GL(n, ℂ). Identity (3.23) in the case of Schur functions becomes

s_{λ/μ}(x) = det_{1≤i,j≤n} (h_{λ_{i}−μ_{j}−i+j}(x)),  (3.27)

which is known as the Jacobi–Trudi identity (see Macdonald [?, ??]). ??Give small example for a (skew) Schur function + Jacobi-Trudi determinant?? From (3.27) we see that s_{λ/μ}(x) is indeed a symmetric function in the x_{i}’s, which is not at all obvious from the combinatorial definition (3.26). Identity (3.25) in the case of Schur functions becomes the dual Jacobi–Trudi identity,

s_{λ/μ}(x) = det_{1≤i,j≤n} (e_{λ′_{i}−μ′_{j}−i+j}(x)).  (3.28)

Once we have solved the problem of enumerating column-strict reverse plane partitions, the enumeration of other types of plane partitions comes in a natural way. For example, let π be any (α, β)-reverse plane partition of shape λ/μ. Define the new array π̄ by

π̄_{ij} = π_{ij} − jα + (i − 1)(1 − β).  (3.29)

Obviously, (3.8) translates into π̄_{ij} ≥ π̄_{i,j+1} and π̄_{ij} > π̄_{i+1,j}. In other words, by (3.29) we obtain a column-strict reverse plane partition from an (α, β)-reverse plane partition. This correspondence will be the main tool for enumerating (α, β)- reverse plane partitions.

Theorem 3.5. Let λ = (λ_{1}, λ_{2}, …, λ_{n}) and μ = (μ_{1}, μ_{2}, …, μ_{n}) be sequences of integers satisfying (3.13). Let a = (a_{1}, a_{2}, …, a_{n}) and b = (b_{1}, b_{2}, …, b_{n}) be sequences of integers satisfying

a_{i} + α(λ_{i−1} − λ_{i}) + (1 − β) ≥ a_{i−1} and b_{i} + α(μ_{i−1} − μ_{i}) + (1 − β) ≥ b_{i−1} (3.30)

for i = 2, 3, …, n. Then we have the following:

(i) The number of all (α, β)-reverse plane partitions π of shape λ/μ where the first part in row i is at least b_{i} and the last part in row i is at most a_{i} equals

det_{1≤i,j≤n} (\binom{a_{i} − b_{j} + (1 − α)(λ_{i} − μ_{j}) − β(i − j) + α}{λ_{i} − μ_{j} − i + j}) . (3.31)

(ii) The generating function ∑_{π} q^{n(π)} for the same set of (α, β)-reverse plane partitions π equals

det_{1≤i,j≤n} (q^{b_{j}(λ_{i}−μ_{j}−i+j)+(i−j)(αμ_{j}−(1−β)(λ_{i}−i))+α\binom{λ_{i}−μ_{j}}{2}}

[\begin{matrix}a_{i} − b_{j} + (1 − α)(λ_{i} − μ_{j}) − β(i − j) + α \\ λ_{i} − μ_{j} − i + j \end{matrix}]_{q} ). (3.32)

Proof. Under the correspondence (3.29) the (α, β)-reverse plane partitions under consideration bijectively correspond to column-strict reverse plane partitions of shape λ/μ in which the entries in row i are at most a_{i} − λ_{i}i + (i − 1)(1 − β) and at least b_{i} − (μ_{i} + 1)α + (i − 1)(1 − β). Therefore, in order to obtain (3.31), apply Theorem 3.4, (3.16) with a_{i} replaced by a_{i} − λ_{i}i + (i − 1)(1 − β) and b_{i} replaced by b_{i} − (μ_{i} + 1)α + (i − 1)(1 − β).

In order to obtain (3.32) we observe that under the correspondence (3.29) we have

n(π) = n(π̄) + α ∑_{i=1}^{n} (\binom{λ_{i} + 1}{2} − \binom{μ_{i} + 1}{2}) + ∑_{i=1}^{n} ((i − 1)(1 − β)(λ_{i} − μ_{i})). (3.33)

Therefore we use Theorem 3.4, (3.17) with the same replacements as before, and multiply the resulting expression by q raised to the expression in (3.33). After some manipulation we arrive at (3.32).

The results for (α, β)-reverse plane partitions in this general form can be found in Krattenthaler [?, Cor. 6.2], but of course are implicit in Gessel and Viennot [?] and Stanley [?].

By the rotation (3.11) the Theorems 3.4 and 3.5 translate into the following results for plane partitions and (α, β)-plane partitions.

Theorem 3.6. Let λ = (λ_{1}, λ_{2}, …, λ_{n}) and μ = (μ_{1}, μ_{2}, …, μ_{n}) be sequences of integers satisfying (3.13). Let a = (a_{1}, a_{2}, …, a_{n}) and b = (b_{1}, b_{2}, …, b_{n}) be sequences of integers satisfying

a_{1} ≥ a_{2} ≥ ⋯ ≥ a_{n}  and  b_{1} ≥ b_{2} ≥ ⋯ ≥ b_{n}.  (3.34)

Then we have the following:

(i) The number of all column-strict plane partitions π of shape λ/μ where the entries in row i are at most a_{i} and at least b_{i} equals

det_{1≤i,j≤n} (\binom{a_{j} − b_{i} + λ_{i} − μ_{j} − i + j}{λ_{i} − μ_{j} − i + j}) .  (3.35)

(ii) The generating function ∑_{π} q^{n(π)} for the same set of column-strict plane partitions π equals

det_{1≤i,j≤n} (q^{b_{i}(λ_{i}−μ_{j}−i+j)} [\begin{matrix}a_{j} − b_{i} + λ_{i} − μ_{j} − i + j \\ λ_{i} − μ_{j} − i + j \end{matrix}]_{q}) .  (3.36)

(iii) More generally, the generating function ∑_{π} w_{x}(π) for the same set of columnstrict plane partitions π equals

det_{1≤i,j≤n} (h_{λ_{i}−μ_{j}−i+j}(x_{b_{i}}, x_{b_{i}+1}, …, x_{a_{j}})).  (3.37)

(iv) Moreover, the number of all column-strict plane partitions π of shape λ/μ where the entries in column i are at most a_{i} and at least b_{i} equals

det_{1≤i,j≤n} (e_{λ′_{i}−μ′_{j}−i+j}(x_{b_{i}}, x_{b_{i}+1}, …, x_{a_{j}})).  (3.38)

Next, if a = (a_{1}, a_{2}, …, a_{n}) and b = (b_{1}, b_{2}, …, b_{n}) be sequences of integers satisfying

a_{i} + α(μ_{i} − μ_{i+1}) + (1 − β) ≥ a_{i+1} and b_{i} + α(λ_{i} − λ_{i+1}) + (1 − β) ≥ b_{i+1} (3.39)

for i = 1, 2, …, n − 1, then we have the following:

(v) Then the number of all (α, β)-plane partitions π of shape λ/μ where the first part in row i is at most a_{i} and the last part in row i is at least b_{i} equals

det_{1≤i,j≤n} (\binom{a_{j} − b_{i} + (1 − α)(λ_{i} − μ_{j}) − β(i − j) + α}{λ_{i} − μ_{j} − i + j}) . (3.40)

(vi) The generating function ∑_{π} q^{n(π)} for the same set of (α, β)-plane partitions π equals

det_{1≤i,j≤n} (q^{b_{i}(λ_{i}−μ_{j}−i+j)+(j−i)(αλ_{i}−(1−β)(μ_{j}−j))+α\binom{λ_{i}−μ_{j}}{2}}

[\begin{matrix}a_{j} − b_{i} + (1 − α)(λ_{i} − μ_{j}) − β(i − j) + α \\ λ_{i} − μ_{j} − i + j \end{matrix}]_{q} ). (3.41)

Proof. By (3.11), column-strict plane partitions of shape λ/μ in which the entries in row i are at least b_{i} and at most a_{i} correspond to column-strict reverse plane partitions of shape λ̃/μ̃ in which the entries in row i are at least b_{n−i+1} and at most a_{n−i+1}, with λ̃ = (λ̃_{1}, λ̃_{2}, …, λ̃_{n}) given by λ̃_{i} = n − μ_{n−i+1} and μ̃ = (μ̃_{1}, μ̃_{2}, …, μ̃_{n}) given by μ̃_{i} = n − λ_{n−i+1}. Therefore, we obtain (3.35)–(3.37) by replacing λ_{i} by n − μ_{n−i+1}, μ_{i} by n − λ_{n−i+1}, a_{i} by a_{n−i+1}, b_{i} by b_{n−i+1} in (3.16)–(3.18). Similar reasoning deduces (3.38) from (3.19).

A similar relationship holds between column-strict (α, β)-plane partitions and (α, β)-reverse plane partitions. Thus, by the same replacements in (3.31) and (3.32) we would obtain (3.40) and (3.41).

Again, the results for (α, β)-plane partitions in this general form can be found in Krattenthaler [?, Theorem 6.1], but of course are implicit in Gessel and Viennot [?] and Stanley [?]. Formula (3.40) for α = β = 0, μ = 0, a_{i} = a, b_{i} = 0 (i.e. the case of plane partitions of shape λ, the parts ranging from 0 to a) was already found by MacMahon [?, sec. 494]. Later, Kreweras [??][?] discovered a determinant formula for the number of rectangular ?reverse plane partitions with different row bounds, which is the special case α = β = 0, λ = (c, c, …, c), μ = 0 of (3.40)??. Carlitz [?, (6.12)] generalized Kreweras’ result by finding the analogous determinant for the norm generating function, which is (3.41) in the special case of rectangular shapes. As we observed earlier (see Chapter ??, Section ??), simple lattice paths in the plane can be encoded in terms of one-columned reverse plane partitions, and therefore the determinant formulas Theorems (2.12) and (2.17) are just the special cases α = β = 0, λ_{i} = 1, μ_{i} = 0 of (3.40) and (3.41), respectively.

From (3.27) it is easy to get a formula for the number of all standard Young tableaux of shape λ/μ, which is stated in the next Corollary. This number is a significant quantity in representation theory of the symmetric group and GL(n, ℂ). For example, it gives the dimension of the representation corresponding to the shape λ/μ [?]. Another way to see standard Young tableaux is through its connection with the ballot problem, n-candidaten-candidate ballot problem (see Chapter 2, Section 1 for the 2-candidate ballot problem, and Chapter 5, Section 1 for the n-candidate problem). This, in turn, corresponds to counting n-dimensional lattice paths restricted by certain hyperplanes. This way of looking at it leads to another proof of the following Corollary, by using an n-dimensional extension of the reflection principle.

Corollary 3.7. Let λ and μ be sequences of integers satisfying (3.13). The number of standard Young tableaux of shape λ/μ is

( ∑_{i=1}^{n}(λ_{i} − μ_{i}))! det_{1≤i,j≤n} (\frac{1}{(λ_{i} − μ_{j} − i + j)!}).  (3.42)

Proof. Remember that a standard Young tableau of shape λ/μ is a semistandard tableau of shape λ/μ with entries 1, 2, …, ∑_{i=1}^{n}(λ_{i} − μ_{i}). Hence, the number of standard Young tableaux of shape λ/μ equals the coefficient of x_{1}x_{2} ⋯ x_{∑_{i=1}^{n}(λ_{i}−μ_{i})} in (3.27). We write (3.27) in the form

∑_{σ∈𝔖_{n}} sgn σ ∏_{i=1}^{n} h_{λ_{i}−μ_{σ(i)}−i+σ(i)}(x).

The coefficient of x_{1}x_{2} ⋯ x_{∑_{i=1}^{n}(λ_{i}−μ_{i})} in each of the summands equals the number of ways of partitioning the set {x_{1}, x_{2}, …, x_{∑_{i=1}^{n}(λ_{i}−μ_{i})}} of variables into blocks of respective sizes λ_{i} − μ_{σ(i)} − i + σ(i), i = 1, 2, …, n. For each fixed σ ∈ 𝔖_{n} this is a multinomial coefficient. So we obtain

∑_{σ∈𝔖_{n}} sgn σ\binom{∑_{i=1}^{n}(λ_{i} − μ_{i})}{λ_{1} − μ_{σ(1)} − 1 + σ(1), …, λ_{n} − μ_{σ(n)} − n + σ(n)},

which can be written in determinantal form as in (3.42).

3. Product formulas for plane partitions

There are cases of plane partition classes, for which there even exist closed form expressions for their cardinalities, or for their norm generating functions. The most prominent cases are MacMahon’s formula (MacMahon [?, sec. 429, proof in sec. 494]) for rectangular plane partitions and Stanley’s hook-content formulas (Stanley [?]). !!Mention bijective proof in Krattenthaler [?]!! It has been discovered in Krattenthaler [?] that all of these product formulas result from applying special cases of the following determinant lemma to special cases of the determinants in Section 2.

Lemma 3.8. Let x_{1}, x_{2}, …, x_{n}, A_{2}, …, A_{n}, B_{2}, …, B_{n} be indeterminates. Then

det_{1≤i,j≤n} ((X_{i} + A_{n}) ⋯ (X_{i} + A_{j+1})(X_{i} + B_{j}) ⋯ (X_{i} + B_{2}))

= ( ∏_{1≤i<j≤n}(X_{i} − X_{j}))( ∏_{2≤i≤j≤n}(B_{i} − A_{j})). (3.43)

Proof. The proof is based on a series of operations on the left-hand side determinant without altering its value.

First, we subtract the (n − 1)-st column from the n-th, the (n − 2)-nd from the (n − 1)-st, …, the first from the second, to obtain

det_{1≤i,j≤n} ((X_{i} + A_{r}) ⋯ (X_{i} + A_{j+1})(X_{i} + B_{j}) ⋯ (X_{i} + B_{2}))

= (B_{n} − A_{n})(B_{n−1} − A_{n−1}) ⋯ (B_{2} − A_{2})

× det_{1≤i,j≤n} ((X_{i} + A_{n}) ⋯ (X_{i} + A_{j+1})(X_{i} + B_{j−1}) ⋯ (x_{i} + B_{2})).

Next, in the determinant on the right-hand side of this equation subtract the (n − 1)-st column from the n-th, …, the second from the third (but not the first from the second!). This time the expression (B_{n−1}−A_{n})(B_{n−2}−A_{n−1}) ⋯ (B_{2}−A_{3}) can be factored out on the right-hand side. This process is continued until

det_{1≤i,j≤n} ((X_{i} + A_{n}) ⋯ (X_{i} + A_{j+1})(X_{i} + B_{j}) ⋯ (X_{i} + B_{2}))

= ∏_{2≤i≤j≤n}(B_{i} − A_{j}) det_{1≤i,j≤n} ((X_{i} + A_{n}) ⋯ (X_{i} + A_{j+1})).

The determinant on the right-hand side of this equation can be reduced to the Vandermonde determinant det(X_{i}^{n−i}) by elementary column operations. This Vandermonde determinant equals ∏_{1≤i<j≤n}(X_{i} − X_{j}).

For the formulation of the closed form results we need to introduce the hook lengths and contents of partitions. The hook-length h(i, j) of the cell (i, j) of the partition λ is the number of cells in λ that are either to the right and in the same row of λ or below and in the same column of λ, including cell (i, j). Figure 19.a shows the hook-lengths of the partition λ = (5, 4, 4, 1). The contents c(i, j) of the cell (i, j) of λ is simply c(i, j) = j − i. Figure 19.b shows the contents of the partition λ = (5, 4, 4, 1).

8 7 5 4 1 6 5 3 2 5 4 2 1 1  \begin{matrix}0 & 1 & 2 & 3 & 4 \\ −1 & 0 & 1 & 2 \\ −2 & −1 & 0 & 1 \\ −3 \end{matrix}

a. hook lengths of λ = (5, 4, 4, 1) b. contents of λ = (5, 4, 41)

Figure 19

Now we are in a position to formulate the promised closed form results. What can be observed is, that in all cases the shapes are non-skew.

Theorem 3.9. Let λ satisfy (3.13a) and λ ≥ 0. Then we have the following:

(i) The generating function ∑_{π} q^{n(π)} for (α, β)-reverse plane partitions π of shape λ in which the first part in row i is at least b−(1−β)i (or equivalently, π_{11} ≥ b − 1 + β) and the last part in row i is at most a + αλ_{i} − (1 − β)i for all i is

q^{∑_{i=1}^{n} λ_{i}(αλ_{i}/2+βi+b−α/2−1)} ∏_{(i,j)∈λ} \frac{[a + α − b + 1 + c(i, j)]_{q}}{[h(i, j)]_{q}} . (3.46)

(ii) In particular, the generating function ∑_{π} q^{n(π)} for column-strict reverse plane partitions π of shape λ with entries ranging from 1 to a is

q^{∑_{i=1}^{n} iλ_{i}} ∏_{(i,j)∈λ} \frac{[a + c(i, j)]_{q}}{[h(i, j)]_{q}} ,  (3.47)

and hence the number of these column-strict reverse plane partitions is

∏_{(i,j)∈λ} \frac{(a + c(i, j))}{h(i, j)}.  (3.48)

(iii) Also, the generating function ∑_{π} q^{n(π)} for row- and column-strict reverse plane partitions π of shape λ with positive entries is

q^{∑_{i=1}^{n} λ_{i}(λ_{i}/2−1/2+i)} ∏_{(i,j)∈λ} \frac{1}{[h(i, j)]_{q}} .  (3.49)

(iv) The generating function ∑_{π} q^{n(π)} for (α, β)-reverse plane partitions π of shape λ in which the first part in row i is at least b + βi (or equivalently, π_{11} ≥ b + β) and the last part in row i is at most a − (1 − α)λ_{i} + βi for all i is

q^{∑_{i=1}^{n} λ_{i}(αλ_{i}/2+βi+b−α/2)} ∏_{(i,j)∈λ} \frac{[a + α − b − c(i, j)]_{q}}{[h(i, j)]_{q}} .  (3.50)

(v) In particular, the generating function ∑_{π} q^{n(π)} for reverse plane partitions π of shape λ with positive entries is

q^{∑_{i=1}^{n} λ_{i}} ∏_{(i,j)∈λ} \frac{1}{[h(i, j)]_{q}} .  (3.51)

(vi) The generating function ∑_{π} q^{n(π)} for (α, β)-plane partitions π of shape λ in which the first part in row i is at most a + (1 − β)i (or equivalently, π_{11} ≤ a + 1 − β) and the last part in row i is at least b − αλ_{i} + (1 − β)i for all i is

q^{∑_{i=1}^{n} λ_{i}(−αλ_{i}/2+(2−i)β+b−α/2−1)} ∏_{(i,j)∈λ} \frac{[a + α − b + 1 + c(i, j)]_{q}}{[h(i, j)]_{q}} . (3.52)

(vii) In particular, the generating function ∑_{π} q^{n(π)} for column-strict plane partitions π of shape λ with entries ranging from 1 to a is

q^{∑_{i=1}^{n} iλ_{i}} ∏_{(i,j)∈λ} \frac{[a + c(i, j)]_{q}}{[h(i, j)]_{q}} .  (3.53)

(viii) The generating function ∑_{π} q^{n(π)} for (α, β)-plane partitions π of shape λ in which the first part in row i is at most a − βi (or equivalently, π_{11} ≤ a − β) and the last part in row i is at least b + (1 − α)λ_{i} − βi for all i is

q^{∑_{i=1}^{n} λ_{i}((1−α/2)λ_{i}−βi+b−α/2)} ∏_{(i,j)∈λ} \frac{[a + α − b − c(i, j)]_{q}}{[h(i, j)]_{q}} .  (3.54)

(ix) In particular, the generating function ∑_{π} q^{n(π)} for plane partitions π of rectangular shape (c^{n}) with entries ranging from 1 to a is

??originalformulation  (3.55)

Sketch of proof. We only prove in detail that the number of column-strict reverse plane partitions of shape λ with entries ranging from 1 to a is given by

∏_{(i,j)∈λ} \frac{(a + c(i, j))}{h(i, j)}  (3.56)

(which is the q = 1 special case of (3.47)). All the other product formulas are derived similarly. (There is some additional amount of work caused by powers of q.) The idea is to bring the largest common numerator and the least common denominator of the entries in the i-th row out of the determinant. The resulting determinant then matches with a special case of Lemma 3.8. See Krattenthaler [?, sec. 6] for details.

In order to prove that (3.56) is the number in question, take μ = 0, a_{i} = a, b_{i} = 1 in (3.35) to get the expression

det (\binom{a − 1 + λ_{i} − i + j}{λ_{i} − i + j})

for this number. Now, by bringing the largest common numerator and least common denominator of the rows out of the determinant, we may rewrite this as

∏_{i=1}^{n} \frac{(a − 1 + λ_{i} − i + 1)!}{(λ_{i} − i + n)! (a − 1)!}

det ((λ_{i} − i + n) ⋯ (λ_{i} − i + j + 1)(λ_{i} − i + a + j − 1) ⋯ (λ_{i} − i + a + 1)).

The determinant can now be evaluated by using Lemma 3.8 with X_{i} = λ_{i} − i, A_{j} = j, B_{j} = a + j. Thus we obtai

∏_{i=1}^{n} \frac{(a + λ_{i} − i)!}{(λ_{i} − i + n)! (a − 1)!} · ∏_{1≤i<j≤n}(λ_{i} − i − λ_{j} + j) · ∏_{2≤i≤j≤n}(a + i − 1 − j)

= \frac{∏_{1≤i<j≤n}(λ_{i} − i − λ_{j} + j)}{∏_{i=1}^{n}(λ_{i} − i + n)!} · ∏_{i=1}^{n} \frac{(a + λ_{i} − i)!}{(a − i)!}.

We claim that the first quotient in the last line is equal to 1/ ∏_{(i,j)∈λ}h(i, j), and that the second is equal to ∏_{(i,j)∈λ}(a+c(i, j)). While for the second this is evident, the first assertion needs a proof. The most convenient way to do it is by induction on n, assuming the assertion to be already true for the shape (λ_{2}, λ_{3}, …, λ_{n}), and inserting the first row with λ_{1} cells. The reader will not have difficulties to check.

Finally, when talking about product formulas, we must not miss the celebrated Frame–Robinson–Thrall hook formula (Frame et al. [?]) for the number of standard Young tableaux of shape λ, which is given below.

Theorem 3.10. Let λ satisfy (3.13a) and λ ≥ 0. The number of standard Young tableaux of shape λ is

\frac{( ∑_{i=1}^{n} λ_{i})!}{∏_{(i,j)∈λ}h(i, j)}.  (3.59)

Proof. Set μ = 0 in (3.42) and proceed as in the previous proof. This leads t

( ∑_{i=1}^{n} λ_{i})! det (\frac{1}{(λ_{i} − i + j)!})

= ( ∑_{i=1}^{n} )! ∏_{i=1}^{n} \frac{1}{(λ_{i} − i + n)!} det ((λ_{i} − i + n) ⋯ (λ_{i} − i + j + 1)).

The determinant in the last line can be reduced to the Vandermonde determinant det((λ_{i} − i)^{n−i}) by elementary column operations. Hence, the desired number is

( ∑_{i=1}^{n} λ_{i})!\frac{∏_{1≤i<j≤n}(λ_{i} − i − λ_{j} + j)}{∏_{i=1}^{n}(λ_{i} − i + n)!}.

In the previous proof the quotient in this expression was already identified as 1/ ∏_{(i,j)∈λ}h(i, j), which settles (3.59).

66 III. NONINTERSECTING LATTICE PATHS

CHAPTER 4 Lattice paths and their turns

In this chapter we consider turns of lattice paths. Literally, a turn of a lattice path is any vertex of a path where the direction of the path changes. The enumeration of lattice paths with a given number of turns is motivated by correlated random walks, distribution of runs, coefficients of Hilbert polynomials of determinantal and pfaffian rings, and summations for Schur functions.

Our approach for the enumeration of simple plane lattice paths with respect to their number of turns is by encoding lattice paths in terms of two-rowed arrays. It will be exhibited that most(??) of the techniques introduced in Chapter 2 have their analogues for two-rowed arrays. From this point of view we believe that tworowed arrays are the “right” tool for attacking enumeration problems involving the number of turns.

1. Simple lattice paths in the plane and two-rowed arrays

For simple lattice path in the plane there are two types of turns. We call a vertex T of a path a North-East turn (NE-turn for short) if T is reached by a step towards north and left by a step towards east. We call a vertex T of a path a east-north turn (EN-turn for short) if T is reached by a step towards east and left by a step towards north. Thus, the NE-turns of the path P_{0} in Figure 20 are (1, 1), (2, 3), and (5, 4), the EN-turns of P_{0} are (2, 1), (5, 3), and (6, 4).

• _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} • •^{P_{0}}^{Figure 20}

We denote by NE(P) the number of NE-turns of P and by EN(P) the number of EN-turns of P.

Now we describe the encoding of paths in terms of two-rowed arrays. Actually, we use two encodings, one corresponding to NE-turns, one corresponding to ENturns. Let (p_{1}, q_{1}), (p_{2}, q_{2}), …, (p_{ℓ}, q_{ℓ}) be the NE-turns of a path P. Then the NE-turn representation of P is defined by the two-rowed array

\begin{matrix}p_{1} & p_{2} & … & p_{ℓ} \\ q_{1} & q_{2} & … & q_{ℓ}, \end{matrix}  (4.1)

which consists of two strictly increasing sequences. Clearly, if P runs from (a, b) to (c, d) then a ≤ p_{1} < p_{2} < ⋯ < p_{ℓ} ≤ c − 1 and b + 1 ≤ q_{1} < q_{2} < ⋯ < q_{ℓ} ≤ d are satisfied. If we wish to make this fact transparent, we write

\begin{aligned}a ≤ & p_{1} & p_{2} & … & p_{ℓ} & ≤ c − 1 \\ b + 1 ≤ & q_{1} & q_{2} & … & q_{ℓ} & ≤ d. \end{aligned}  (4.2)

For a given starting point and a given final point, by definition the empty array is the representation for the only path that has no NE-turn. For the path in our running example we obtain the NE-turn representation

1 2 5 1 3 4 ,

or with bounds included,

\begin{aligned}1 ≤ & 1 & 2 & 5 & ≤ 5 \\ 0 ≤ & 1 & 3 & 4 & ≤ 6 \end{aligned} .

Similarly, if (p_{1}, q_{1}), (p_{2}, q_{2}), …, (p_{ℓ}, q_{ℓ}) denote the EN-turns of a path P, then (4.1) is called the EN-turn representation of P. If P runs from (a, b) to (c, d) then a + 1 ≤ p_{1} < p_{2} < ⋯ < p_{ℓ} ≤ c and b ≤ q_{1} < q_{2} < ⋯ < q_{ℓ} ≤ d − 1 are satisfied. Again, as earlier, we write

\begin{aligned}a + 1 ≤ & p_{1} & p_{2} & … & p_{ℓ} & ≤ c \\ b ≤ & q_{1} & q_{2} & … & q_{ℓ} & ≤ d − 1. \end{aligned}  (4.3)

For a given starting point and a given final point, by definition the empty array is the representation for the only path that has no EN-turn. For the path in our running example we obtain the EN-turn representation

2 5 6 1 3 4 ,

or with bounds included,

\begin{aligned}2 ≤ & 2 & 5 & 6 & ≤ 6 \\ −1 ≤ & 1 & 3 & 4 & ≤ 5 \end{aligned} .

Also two-rowed arrays with its rows being of unequal length will be considered. These arrays will also have the property that the rows are strictly increasing. So by convention, whenever we speak of two-rowed arrays we mean two-rowed arrays with strictly increasing rows. For these arrays we will use a notation of the kind (4.2) or (4.3) as well. We shall frequently use the short notation (a | b) for tworowed arrays, where a denotes the sequence (a_{i}) of elements of the first row, and b denotes the sequence (b_{i}) of elements of the second row.

From (4.2) we see at once that the number of all paths from (a, b) to (c, d) with exactly ℓ NE-turns equals the number of ℓ-element subsets of {a, a + 1, …, c − 1} times the number of ℓ-element subsets of {b + 1, b + 2, …, d}. A similar argument holds for EN-turns. Thus we have proved

|L((a, b) → (c, d) | NE(.) = ℓ)| = |L((a, b) → (c, d) | EN(.) = ℓ)|

= \binom{c − a}{ℓ}\binom{d − b}{ℓ}. (4.4)

2. From NE-turns to runs

A lattice path statistic that is frequently used is the number of runs of a lattice path. A run in a path P is a maximal subpath of P consisting of steps of equal type. We write run(P) for the number of runs of P. The runs of the path P_{0} in Figure 20 are the subpaths from (1, −1) to (1, 1), from (1, 1) to (2, 1), from (2, 1) to (2, 3), from (2, 3) to (5, 3), from (5, 3) to (5, 4), from (5, 4) to (6, 4), and from (6, 4) to (6, 6). Thus we have run(P_{0}) = 7. Obviously, the number of runs of a path is exactly one more than the total number of turns (both, NE-turns and ENturns). Besides, there is also a close relation between NE-turns and runs, which allows us to translate any enumeration result for NE-turns into one for runs. The objective of this section is to explain this relation. ??What is the condition on R??no condition??see turnsurv.tex!!

To avoid case by case formulations, depending on whether the number of runs is even or odd, we prefer to consider generating functions. Suppose we know the number of all paths from A to E satisfying some property R and containing a given number of NE-turns. Then we know the generating function GF (L(A → E | R); x^{NE(.)}). For brevity, let us denote it by F (A → E | R; x). We define four refinements of F (A → E | R; x). Let F_{hv}(A → E | R; x) be the generating function ∑_{P} x^{NE(P)} where the sum is over all paths in L(A → E | R) that start with a horizontal step and end with a vertical step. The notations F_{hh}(A → E | R; x), F_{vh}(A → E | R; x), and F_{vv}(A → E | R; x) are defined analogously. The relation between enumeration by runs and enumeration by NE-turns is given by

GF (L(A → E | R); x^{run(.)}) = xF_{hh}(A → E | R; x^{2}) + x^{2}F_{hv}(A → E | R; x^{2})

+ F_{vh}(A → E | R; x^{2}) + xF_{vv}(A → E | R; x^{2}). (4.5)

??Figure

All the four refinements of the NE-turn generating function can be expressed in terms of NE-turn generating functions. This is seen by setting up a few linear equation and solving them. Evidently, the following is true:

F (A → E | R; x) = F_{hh}(A → E | R; x) + F_{hv}(A → E | R; x)

+ F_{vh}(A → E | R; x) + F_{vv}(A → E | R; x).

Besides, if E_{1} = (1, 0) and E_{2} = (0, 1) denote the standard unit vectors, we have

F_{hh}(A → E | R; x) + F_{hv}(A → E | R; x) = F (A + E_{1} → E | R; x)
F_{hv}(A → E | R; x) + F_{vv}(A → E | R; x) = F (A → E − E_{2} | R; x)
F_{hv}(A → E | R; x) = F (A + E_{1} → E − E_{2} | R; x).

3. SIMPLE PATHS IN THE PLANE RESTRICTED BY LINES PARALLEL TO x = y71

Solving for F_{hh}, F_{hv}, F_{vh}, F_{vv} we get

F_{hh}(A → E | R; x) = F (A + E_{1} → E | R; x) − F (A + E_{1} → E − E_{2} | R; x) (4.7a)
F_{hv}(A → E | R; x) = F (A + E_{1} → E − E_{2} | R; x)  (4.7b)
F_{vh}(A → E | R; x) = F (A → E | R; x) + (A + E_{1} → E − E_{2} | R; x)  (4.7c)
− F (A + E_{1} → E | R; x) − F (A → E − E_{2} | R; x)
F_{vv}(A → E | R; x) = F (A → E − E_{2} | R; x) − F (A + E_{1} → E − E_{2} | R; x). (4.7d)

We will find that (4.7a)–(4.7d) give us refined information about NE-turn enumeration which will be useful for the correlated random walk.??

3. Simple paths in the plane restricted by lines parallel to x = y

??Derive many things as corollaries of plane partitions and cylindric partitions?? As we know from Chapter 2, Section 1, counting paths restricted by x = y, or

even by two lines x = y + t and x = y + s, is effectively solved by the reflection principle. Of course, reflection by itself is useless for counting paths by turns, since the reflection of portions of paths does not take care of turns. It might introduce new turns or make turns disappear. However, there are “analogues” of reflection for two-rowed arrays, which are due to Krattenthaler and Mohanty [?]. These are the main tools of the proofs in this section. In addition, they allow us to derive q-analogues, see Chapter ??, Section ??.

Theorem 4.1. Let a ≥ b and c ≥ d. The number of all paths from (a, b) to (c, d) staying below x = y with exactly ℓ NE-turns is given by

|L((a, b) → (c, d) | x ≥ y, NE(.) = ℓ)|

= \binom{c − a}{ℓ}\binom{d − b}{ℓ} − \binom{c − b − 1}{ℓ − 1}\binom{d − a + 1}{ℓ + 1}, (4.8)

and with exactly ℓ EN-turns is given by

|L((a, b) → (c, d) | x ≥ y, EN(.) = ℓ)|

= \binom{c − a}{ℓ}\binom{d − b}{ℓ} − \binom{c − b + 1}{ℓ}\binom{d − a − 1}{ℓ}. (4.9)

Proof. We start with proving (4.8). By the NE-turn representation (4.2), the paths from (a, b) to (c, d) staying below x = y with exactly ℓ NE-turns can be represented by (??mention that these are two-rowed plane partitions??)

\begin{aligned}a ≤ & p_{1} & p_{2} & … & p_{ℓ} & ≤ c − 1 \\ b + 1 ≤ & q_{1} & q_{2} & … & q_{ℓ} & ≤ d, \end{aligned}  (4.10a)

where

p_{i} ≥ q_{i},  i = 1, 2, …, ℓ.  (4.10b)

Following the argument in the proof of Theorem 2.1, the number of these two-rowed arrays is the number of all two-rowed arrays of the type (4.10a) minus those tworowed arrays of the type (4.10a) which violate (4.10b), i.e. where p_{i} < q_{i} for some i between 1 and ℓ. We know the first number from (4.4).

Concerning the second number, we claim that two-rowed arrays of the type (4.10a) which violate (4.10b) are in one-to-one correspondence with two-rowed arrays of the type

\begin{aligned}b + 1 ≤ & & & p̄_{2} & … & p̄_{ℓ} & ≤ c − 1 \\ a ≤ & q̄_{0} & q̄_{1} & q̄_{2} & … & q̄_{ℓ} & ≤ d. \end{aligned}  (4.11)

The number of all these two-rowed arrays is \binom{c−b−1}{ℓ−1}\binom{d−a+1}{ℓ+1}, as desired. So it remains only to construct the one-to-one correspondence.

Take a two-rowed array (p | q) of the type (4.10a) such that p_{i} < q_{i} for some i. Let I be the largest integer such that p_{I} < q_{I}. Then map (p | q) to

\begin{aligned}& & q_{1} & . . . . . . & q_{I−1} & p_{I+1} & … & p_{ℓ} \\ p_{1} & p_{2} & . . . . . . & p_{I} & q_{I} & q_{I+1} & … & q_{ℓ} \end{aligned} .  (4.12)

Note that both rows are strictly increasing because of q_{I−1} < q_{I} < q_{I+1} ≤ p_{I+1} and p_{I} < q_{I}. By some case by case analysis it can be seen that (4.12) is of type (4.11). For example, if I = ℓ then we must check q_{I−1} ≤ c − 1, among others. Clearly, this follows from the inequalities q_{I−1} < q_{I} ≤ d ≤ c.

The inverse of this map is defined in the same way. Let (p̄ | q̄) be a two-rowed array of the type (4.11). Let Ī be the largest integer such that p̄Ī < q̄Ī. If there are none, take Ī = 2. Then map (p̄ | q̄) to

\begin{matrix}q̄_{0} & . . . . . . & q̄_{Ī−1} & p̄_{Ī+1} & … & p̄_{ℓ} \\ p̄_{2} & … & p̄Ī & q̄Ī & … & q̄_{ℓ} \end{matrix} .  (4.13)

It is not difficult to check that the mappings (4.12) and (4.13) are inverses of each other. This completes the proof of (4.8).

The second identity, (4.9), is established similarly. This time we represent paths from (a, b) to (c, d) staying below x = y with exactly ℓ EN-turns by the EN-turn representation (4.3), that is two-rowed arrays of the type

\begin{aligned}a + 1 ≤ & p_{1} & p_{2} & … & p_{ℓ} & ≤ c \\ b ≤ & q_{1} & q_{2} & … & q_{ℓ} & ≤ d − 1, \end{aligned}  (4.14a)

where

p_{i} ≥ q_{i+1},  i = 1, 2, …, ℓ,  (4.14b)

with the convention q_{ℓ+1} = d. All we have to do is to enumerate those two-rowed arrays of the type (4.14a) which violate (4.14b). We claim that these are in oneto-one correspondence with two-rowed arrays of the type

\begin{aligned}b ≤ & p̄_{1} & … & p̄_{ℓ} & ≤ c \\ a + 1 ≤ & q̄_{1} & … & q̄_{ℓ} & ≤ d − 1. \end{aligned}  (4.15)

In order to see this, take a two-rowed array (p | q) of the type (4.14a) such that p_{i} < q_{i+1} for some i. Let I be the largest integer such that p_{I} < q_{I+1}, 1 ≤ I ≤ ℓ. Then map this (p | q) to

\begin{matrix}q_{1} & … & q_{I} & p_{I+1} & … & p_{ℓ} \\ p_{1} & … & p_{I} & q_{I+1} & … & q_{ℓ} \end{matrix} .  (4.16)

Again, note that both rows are strictly increasing because of q_{I} < q_{I+1} < q_{i+2} ≤ p_{I+1} and p_{I} < q_{I+1}. Also, it can be seen that (4.16) is of type (4.15).

For the inverse of this mapping, suppose that (p̄ | q̄) is a two-rowed array of the type (4.15). Choose Ī to be largest integer such that p̄Ī < q̄_{Ī+1}. If there are none, take Ī = ℓ. Then map (p̄ | q̄) to

\begin{matrix}q̄_{1} & … & q̄Ī & p̄_{Ī+1} & … & p̄_{ℓ} \\ p̄_{1} & … & p̄Ī & q̄_{Ī+1} & … & q̄_{ℓ} \end{matrix} .  (4.17)

Again it is not difficult to check that the mappings (4.16) and (4.17) are inverses of each other. This completes the proof of (4.9).

??run formulation

If we combine both mappings, (4.12)/(4.13) and (4.16)/(4.17), (actually slight modifications of them), then we are able to derive a refinement of Theorem 2.3 taking into account turns. We only give the result for NE-turns since, by a rotation by 180^{∘}, it can easily transformed into a result for EN-turns.

Theorem 4.2. Let a + t ≥ b ≥ a + s and c + t ≥ d ≥ c + s. The number of all paths from (a, b) to (c, d) staying below the line y = x + t and above the line y = x + s with exactly ℓ NE-turns is given by

|L((a, b) → (c, d) | x + t ≥ y ≥ x + s, NE(.) = ℓ)|

= ∑_{k∈ℤ}(\binom{c − a − k(t − s)}{ℓ + k}\binom{d − b + k(t − s)}{ℓ − k}

− \binom{c − b − k(t − s) + s − 1}{ℓ + k}\binom{d − a + k(t − s) − s + 1}{ℓ − k}). (4.18)

??refer to cylindric partitions!!

Proof. By the NE-turn representation, the paths under consideration are in one-to-one correspondence with two-rowed arrays of the type

\begin{aligned}a ≤ & p_{1} & … & p_{ℓ} & ≤ c − 1 \\ b + 1 ≤ & q_{1} & … & q_{ℓ} & ≤ d \end{aligned}  (4.19a)

where

p_{i} + t ≥ q_{i} ≥ p_{i+1} + s.  (4.19b)

Similar to the proof of Theorem 2.3, the proof of this theorem is by a cancelling bijection. But not on paths, but certain two-rowed arrays, which we introduce now. In fact, there are two types of arrays. Let us call two-rowed arrays of the type

\begin{aligned}a + k(t − s) ≤ & p_{1−m} & … & p_{1+m} & … & p_{ℓ} & ≤ c − 1 \\ b + 1 − k(t − s) ≤ & & & q_{1+m} & … & q_{ℓ} & ≤ d \end{aligned} for m ≥ 0

and

\begin{aligned}a + k(t − s) ≤ & & & p_{1−m} & … & p_{ℓ} & ≤ c − 1 \\ b + 1 − k(t − s) ≤ & q_{1+m} & … & q_{1−m} & … & q_{ℓ} & ≤ d \end{aligned} for m < 0

type I arrays. Similarly, we call two-rowed arrays of the type

\begin{aligned}b + 1 − s + k(t − s) ≤ & p_{1−m} & … & p_{1+m} & … & p_{ℓ} & ≤ c − 1 \\ a + s − k(t − s) ≤ & & & q_{1+m} & … & q_{ℓ} & ≤ d \end{aligned} for m ≥ 0

and

\begin{aligned}b + 1 − s + k(t − s) ≤ & & & p_{1−m} & … & p_{ℓ} & ≤ c − 1 \\ a + s − k(t − s) ≤ & q_{1+m} & … & q_{1−m} & … & q_{ℓ} & ≤ d \end{aligned} for m < 0

type II arrays. The set of type I arrays is the “analogue” of the path set in (2.7a), and the set of type II arrays is the “analogue” of the path set in (2.7b). We shall set up a bijection between type I arrays not being of the type (4.19) (which means that (4.19b) must be violated if both rows have equal length) and type II arrays. Given such a bijection, we could deduce

|{type I arrays}| − |{type II arrays}| = |{arrays of type (4.19)}| . (4.20)

The arrays of type (4.19) exactly correspond to the paths we are intending to enumerate. By definition of type I and type II arays, the left-hand side in (4.20) equals the right-hand side in (4.18). Thus (4.18) would be established.

The definition of the bijection and its inverse can be given in a unified form. Let (p | q) be either a type I array which is not of the type (4.19), or a type II array, and be represented by

\begin{matrix}p_{1−m} & . . . . . . . . . . . . . & p_{ℓ} \\ q_{1+m} & … & q_{ℓ} \end{matrix}.

(This representation has to be understood symbolically. m could be also negative, whence the upper row would be shorter.) Let I be the largest integer, 1 ≤ I ≤ ℓ?? such that either

q_{I} > p_{I} + t or I = −m,  (4.21)

or

q_{I} < p_{I+1} + s or I = m.  (4.22)

If (4.21) is satisfied then map (p | q) to

\begin{matrix}(q_{1+m} − t) & . . . . . . . . . . . & (q_{I−1} − t) & p_{I+1} & … & p_{ℓ} \\ (p_{1−m} + t) & . . . . . . . . . . . . . . . . . . & (p_{I} + t) & q_{I} & . . . . . . . . . & q_{ℓ} \end{matrix}.

Note that both rows are strictly increasing because of q_{I−1} < q_{I+1} ≤ p_{I+1} and p_{I} + t < q_{I}. If (4.21) is not satisfied, but hence (4.22) is, map (p | q) to

\begin{matrix}(q_{1+m} − s) & … & (q_{I} − s) & p_{I+1} & … & p_{ℓ} \\ (p_{1−m} + s) & . . . . . . . . . . . . . . . . . . . & (p_{I} + s) & q_{I+1} & … & q_{ℓ} \end{matrix}.

Again note that both rows are strictly increasing, this time because of q_{I} −s < p_{I+1} and p_{I} + s < p_{I+2} + s ≤ q_{I+1}.

It is not difficult to verify that this mapping maps type I arrays not being of type (4.19) to type II arrays and vice versa. Besides, by applying this map to an array twice, one would receive the array back. Therefore this mapping is the desired bijection.

??Translation to runs 4. Simple paths in the plane restricted by lines parallel to x = μy

As in the first proof of Theorem 2.4, we use the Lagrange inversion to enumerate paths below x = μy with a given number of turns for which we need the following refinements of (2.13).

Proposition 4.3. Let μ be a nonnegative integer. Then

∑_{n≥0} GF (L((0, 0) → (μn + k, n) | x ≥ μy); t^{NE(.)})\frac{z^{n}}{(1 + z)^{n+1}(1 + tz)^{μn+k}} = 1, (4.23)

and

∑_{n≥0} GF (L((0, 0) → (μn + k, n) | x ≥ μy); t^{EN(.)})\frac{z^{n}}{(1 + z)^{n}(1 + tz)^{μn+k+1}} = 1. (4.24)

Proof. We use the same idea as in the proof of Theorem 2.16. Of course, here we do not encode paths by their coordinates of vertical steps, but by their coordinates of turns.

To start with (4.23), multiply (4.23) by (1 + z)^{N+1}(1 + tz)^{μN+k}, and obtain

GF (L((0, 0) → (μn + k, n) | x ≥ μy); t^{NE(.)})z^{n}(1 + z)^{N−n}(1 + tz)^{μ(N−n)}

= (1 + z)^{N+1}(1 + tz)^{μN+k}. (4.25)

We want to show that the coefficients of z^{N}t^{ℓ} on both sides agree. The coefficients of z^{N}t^{ℓ} on the left-hand and right-hand side of (4.25) are

∑_{n≥0} ∑_{i≥0} L((0, 0) → (μn + k, n) | x ≥ μy, NE(.) = ℓ − i)\binom{μ(N − n)}{i}\binom{N − n}{i} (4.26)

and

\binom{μN + k}{ℓ}\binom{N + 1}{ℓ + 1},  (4.27)

respectively. We need a combinatorial interpretation of (4.26) and (4.27). In order to explain it, it is convenient to introduce the following notation. Given two sets A and B of integers, we denote the set of all two-rowed arrays (a_{1} … a_{m} | b_{1} … b_{n}) with a_{i} ∈ A and b_{i} ∈ B by S_{m,n}(A, B). Then, in combinatorial terms, establishing that the quantities (4.26) and (4.27) agree could be shown by setting up a one-to-one correspondence between

⋃_{n≥0} ⋃_{i≥0} L((0, 0) → (μn + k, n) | x ≥ μy, NE(.) = ℓ − i)

4. SIMPLE PATHS IN THE PLANE RESTRICTED BY LINES PARALLEL TO x = μy77

× S_{i,i}({μn + k + 1, …, μN + k}, {n + 1, …, N}) (4.28) and

S_{ℓ,ℓ+1}({1, 2, …, μN + k}, {0, 1, …, N}).  (4.29)

Let (P, (r_{1} … r_{i} | s_{1} … s_{i})) be in (4.28). Given that (p_{1} … p_{ℓ−i} | q_{1} … q_{ℓ−i}) is the NE-turn representation of P, we map (P, (r | s)) to

\begin{matrix}μn + k − p_{ℓ−i} & … & μn + k − p_{1} & r_{1} & … & r_{i} \\ n − q_{ℓ−i} & … & n − q_{1} & n & s_{1} & … & s_{i} \end{matrix}.  (4.30)

This two-rowed array is an element of (4.29). At this point, let us observe that if (e | f) represents the two-rowed array in (4.30) then we have e_{j} ≤ μf_{j} + k for j = 1, 2, …, ℓ − i, but e_{ℓ−i+1} > μf_{ℓ−i+1} + k (since e_{ℓ−i+1} = r_{1} ≥ μn + k + 1 and f_{ℓ−i+1} = n, see (4.30)).

For the construction of the inverse mapping, suppose that (u_{1} … u_{ℓ} | v_{1} … v_{ℓ+1}) is in (4.29). Let I be the largest integer such that u_{I} ≤ μv_{I} + k. If there are none, then let I = 0. (For motivation of this definition, recall the observation at the end of the previous paragraph.) Then map (u | v) to (P, (u_{I+1}, …, u_{ℓ} | v_{I+2}, …, v_{ℓ+1})), where P is the path from (0, 0) to (μv_{I+1} + k, v_{I+1}) whose NEturns are (μv_{I+1} +k −u_{I+1−j}, v_{I+1} −v_{I+1−j}), j = 1, 2, …, I. If I = 0, P is the path from (0, 0) to (0, 0). It is not difficult to verify that this mapping and the mapping (4.30) are inverses of each other. Thus we have shown that the coefficients of z^{N}t^{ℓ} in (4.25) agree. This establishes (4.23).

For the proof of (4.24) we use the same method. Here, we want to show that the coefficients of z^{N}t^{ℓ} in

GF (L((0, 0) → (μn + k, n) | x ≥ μy); t^{EN(.)})z^{n}(1 + z)^{N−n}(1 + tz)^{μ(N−n)}

= (1 + z)^{N}(1 + tz)^{μN+k+1}

agree on both sides, for all ℓ. These are

∑_{n≥0} ∑_{i≥0} L((0, 0) → (μn + k, n) | x ≥ μy, EN(.) = ℓ − i)\binom{μ(N − n)}{i}\binom{N − n}{i}

and

\binom{μN + k + 1}{ℓ}\binom{N}{ℓ},

respectively. In combinatorial terms, we would have to set up a one-to-one correspondence between

⋃_{n≥0} ⋃_{i≥0} L((0, 0) → (μn + k, n) | x ≥ μy, EN(.) = ℓ − i)

× S_{i,i}({μn + k + 1, …, μN + k}, {n + 1, …, N}) (4.32) and

S_{ℓ,ℓ}({0, 1, …, μN + k}, {1, 2, …, N}).  (4.33)

Let (P, (r_{1} … r_{i} | s_{1} … s_{i})) be in (4.32) and (p_{1} … p_{ℓ−i} | q_{1} … q_{ℓ−i}) the NE-turn representation of P. It should be noted that q_{1} necessarily is 0. We map (P, (r | s)) to

\begin{matrix}μn + k − p_{ℓ−i} & … & μn + k − p_{1} & r_{1} & … & r_{i} \\ n − q_{ℓ−i} & … & n − q_{1} & s_{1} & … & s_{i} \end{matrix}.  (4.34)

For the inverse mapping, suppose that (u_{1} … u_{ℓ} | v_{1} … v_{ℓ}) is in (4.33). Let I be the largest integer such that u_{I+1} ≤ μv_{I} + k. If there are none, then let I = 0. Then we map (u | v) to (P, (u_{I+1}, …, u_{ℓ} | v_{I+1}, …, v_{ℓ})), where P is the path from (0, 0) to (μv_{I} + k, v_{I}) whose NE-turns are (μv_{I} + k − u_{I+1−j}, v_{I} − v_{I+1−j}), j = 1, 2, …, I. If I = 0, P is the path from (0, 0) to (0, 0). We leave it to the reader to verify that this mapping is the inverse of (4.34). This completes the proof of (4.24).

Applying the Lagrange inversion (??) with f(z) = (1 + z)(1 + tz)^{μ} to (4.23) and (4.24) gives the desired enumeration results for paths staying below x = μy with a given number of turns.

Theorem 4.4. Let μ be a positive integer and c ≥ μd. The number of all lattice paths from the origin to (c, d) which stay below x = μy with exactly ℓ NE-turns is given by

|L((0, 0) → (c, d) | x ≥ μy, NE(.) = ℓ)| = \binom{c}{ℓ}\binom{d}{ℓ} − μ\binom{c − 1}{ℓ − 1}\binom{d + 1}{ℓ + 1},(4.35)

and with exactly ℓ EN-turns is given by

|L((0, 0) → (c, d) | x ≥ μy, EN(.) = ℓ)|

= \frac{c − μd + 1}{c + 1}\binom{c + 1}{ℓ}\binom{d − 1}{ℓ − 1} = \binom{c + 1}{ℓ}\binom{d − 1}{ℓ − 1} − μ\binom{c}{ℓ − 1}\binom{d}{ℓ}. (4.36)

There are also nice combinatorial proofs for (4.35) and (4.36), making use of two-rowed arrays. This is what we describe next.

Combinatorial Proof of Theorem 4.4. We start with a combinatorial proof of (4.35). We represent our paths from (0, 0) to (c, d) staying below x = μy with exactly ℓ NE-turns, by their NE-turn representation. It is given by

\begin{aligned}0 ≤ & p_{1} & p_{2} & … & p_{ℓ} & ≤ c − 1 \\ 1 ≤ & q_{1} & q_{2} & … & q_{ℓ} & ≤ d, \end{aligned}  (4.37a)

where

p_{i} ≥ μq_{i},  i = 1, 2, …, ℓ.  (4.37b)

Again, the number of these two-rowed arrays is the number of all two-rowed arrays of the type (4.37a) minus those two-rowed arrays of the type (4.37a) which violate (4.37b), i.e. where p_{i} < μq_{i} for some i between 1 and ℓ. The first number is clearly \binom{c}{ℓ}\binom{d}{ℓ}.

We claim that there are as many two-rowed arrays of the type (4.37a) which violate (4.37b) as μ times the number of two-rowed arrays of the type

\begin{aligned}1 ≤ & & & r_{2} & … & r_{ℓ} & ≤ c − 1 \\ 0 ≤ & s_{0} & s_{1} & s_{2} & … & s_{ℓ} & ≤ d. \end{aligned}  (4.38)

The number of all these two-rowed arrays is \binom{c−1}{ℓ−1}\binom{d+1}{ℓ+1}, as desired. What remains to do is to find a (1 : μ) correspondence between the two-rowed arrays of type (4.37a) violating (4.37b), and those of type (4.38).

Take a two-rowed array (p | q) of the type (4.37a) such that p_{i} < μq_{i} for some i. Let I be the largest integer such that p_{I} < μq_{I}. The two-rowed array (p | q) then looks like

\begin{aligned}0 ≤ & p_{1} & . . . . . . . . . & p_{I} & … & p_{ℓ} & ≤ c − 1 \\ 1 ≤ & q_{1} & … & q_{I−1} & q_{I} & … & q_{ℓ} & ≤ d \end{aligned} .  (4.39)

Now we fix the right portion, i.e., the entries p_{I+1}, …, p_{ℓ} and q_{I}, …, q_{ℓ}. With this fixed right portion there are

\binom{μq_{I}}{I} · \binom{q_{I} − 1}{I − 1}  (4.40)

possible left portions.

On the other hand, let (r | s) be a two-rowed array of the type (4.38). Let J be maximal with r_{J} < μs_{J} (if there is no such J take J = 1), so that (r | s) looks like

\begin{aligned}1 ≤ & & & r_{2} & . . . . . . . . . & r_{J} & … & r_{ℓ} & ≤ c − 1 \\ 0 ≤ & s_{0} & s_{1} & s_{2} & … & s_{J−1} & s_{J} & … & s_{ℓ} & ≤ d \end{aligned} .  (4.41)

Again, fix the right portion, i.e., the entries r_{J+1}, …, r_{ℓ} and s_{J}, …, s_{ℓ}. Furthermore, assume that the right-hand portion in (4.41) is equal to the right portion in (4.39), i.e., assume that J = I, r_{i} = p_{i}, i = I + 1, …, ℓ, and s_{i} = q_{i}, i = I, …, ℓ. With this fixed right portion in (4.41) there are

\binom{μq_{I} − 1}{I − 1} · \binom{q_{I}}{I} = \frac{1}{μ}\binom{μq_{I}}{I} · \binom{q_{I} − 1}{I − 1}  (4.42)

possible left portions. By comparing with (4.40), we see that, for a fixed right portion, there are μ times as many two-rowed arrays of the type (4.39) with p_{I} < μq_{I}, as there are two-rowed arrays of the type (4.41) with r_{I} < μs_{I} = μq_{I}. This settles our claim and hence finishes the proof of (4.35).

The second assertion, (4.36), can be established in much the same way as the above proof of (4.35). To be precise, this sort of argument would give the second expression in (4.36).

Interestingly, there is also a nice combinatorial argument to explain the first expression in (4.36), by using the cycle lemma 2.5. The argument given here can be considered as a refinement of the one in the combinatorial proof of Theorem 2.4 (given after the proof of the cycle lemma 2.5).

We want to count paths from (0, 0) to (c, d) staying (weakly) below x = μy which have exactly ℓ EN-turns. Again, to fit with the cycle lemma we adjoin a horizontal step at the beginning and shift everything by one unit to the right. Thus we are now asking for the number of paths from (0, 0) to (c + 1, d) staying strictly below x = μy, except for the starting point (0, 0), which have exactly ℓ EN-turns. To find this number, N say, we count in two different ways the set of all pairs (P, Q), where

(1) P is any path from (0, 0) to (c + 1, d) which starts with a horizontal step and which has exactly ℓ EN-turns,

(2) Q is a path from (0, 0) to (c+1, d) which stays strictly below x = μy except for (0, 0) and which has exactly ℓ EN-turns,

(3) P and Q are related by a cyclic permutation of their steps.

Figure 21 displays an example of such a pair (P, Q), with μ = 2, c = 6, d = 2, ℓ = 2, where Q is the cyclic permutation of P which starts with the third step of P. The bold steps in the Figure indicate the possible beginnings of a path Q.

[figure: •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; P; (c + 1, d); x = μy; •; •]

[figure: •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; Q; (c + 1, d); x = μy; •; •]

a. Arbitrary path

b. Path staying strictly below x=μy Figure 21

5. NONINTERSECTING LATTICE PATHS AND THEIR NUMBER OF TURNS 81

It is not difficult to see that there are \binom{c+1}{ℓ}\binom{d−1}{ℓ−1} possible paths P, and, by the cycle lemma 2.5, for each P there are exactly c + 1 − μd cyclic permutations Q staying strictly below x = μy except for (0, 0). So our set of pairs (P, Q) has cardinality (c + 1 − μd)\binom{c+1}{ℓ}\binom{d−1}{ℓ−1}. On the other hand, there are N possible paths Q, and for each Q there are c + 1 cyclic permutations P, namely those which start with one of the c + 1 horizontal steps of Q. Therefore the cardinality of our set of pairs (P, Q) is also equal to (c + 1)N. Equating the two expressions for the cardinality gives the desired result for the number N that we are looking for.

??translate into Mohanty–Narayana results

??translate into runs

??two-boundaries, x = μy + t, x = μy + s, start in first line, end in second, formula similar to above Theorem?

??general boundary − > multiple sums.

5. Nonintersecting lattice paths and their number of turns

In this section we are concerned with a problem that arose from the study of the Hilbert polynomial of certain determinantal and pfaffian rings [?]. Let A = (A_{1}, A_{2}, …, A_{n}) and E = (E_{1}, E_{2}, …, E_{n}) be points in ℤ^{2}. How many families P = (P_{1}, P_{2}, …, P_{n}) of nonintersecting lattice paths, where P_{i} runs from A_{i} to E_{i}, i = 1, 2, …, n, are there such that the total number of NE-turns in P is some fixed number, ℓ say. In a sense, the results of this section refine the results of Chapter 3. Of course, as in Chapter 3 we will assume that the starting and end points are located such that the σ = id-condition holds. We shall even assume the stronger condition that ...?? The case that is actually relevant for the determination of Hilbert polynomials of determinantal rings is the one where the starting points are on one vertical line and the end points are on one horizontal line. The problem can however be solved for the general case by using, not surprisingly, two-rowed arrays.

We are going to prove the following three theorems about the counting of nonintersecting lattice paths with a given number of turns. The first theorem concerns counting families of nonintersecting lattice paths with given starting and end points with a given number of NE-turns.

Theorem 4.5. Let A_{i} = (a_{1}^{(i)}, a_{2}^{(i)}) and E_{i} = (e_{1}^{(i)}, e_{2}^{(i)}) be lattice points satisfying

a_{1}^{(1)} ≤ a_{1}^{(2)} ≤ ⋯ ≤ a_{1}^{(n)}, a_{2}^{(1)} > a_{2}^{(2)} > ⋯ > a_{2}^{(n)},

and

e_{1}^{(1)} < e_{1}^{(2)} < ⋯ < e_{1}^{(n)}, e_{2}^{(1)} ≥ e_{2}^{(2)} ≥ ⋯ ≥ e_{2}^{(n)}.

The number of all families P = (P_{1}, P_{2}, …, P_{n}) of nonintersecting lattice paths P_{i} : A_{i} → E_{i}, such that the paths of P altogether contain exactly ℓ NE-turns, is

∑_{ℓ_{1}+···+ℓ_{n}=ℓ} det_{1≤i,j≤n} (\binom{e_{1}^{(j)} − a_{1}^{(i)} + i − j}{ℓ_{i} + i − j}\binom{e_{2}^{(j)} − a_{2}^{(i)} − i + j}{ℓ_{i}}).  (4.43)

The second theorem concerns counting families of nonintersecting lattice paths staying below x = y, with given starting and end points, by their number of NE-turns.

Theorem 4.6. Let A_{i} = (a_{1}^{(i)}, a_{2}^{(i)}) and E_{i} = (e_{1}^{(i)}, e_{2}^{(i)}) be lattice points satisfying

a_{1}^{(1)} ≤ a_{1}^{(2)} ≤ ⋯ ≤ a_{1}^{(n)}, a_{2}^{(1)} > a_{2}^{(2)} > ⋯ > a_{2}^{(n)},
e_{1}^{(1)} < e_{1}^{(2)} < ⋯ < e_{1}^{(n)}, e_{2}^{(1)} ≥ e_{2}^{(2)} ≥ ⋯ ≥ e_{2}^{(n)},

and a_{1}^{(i)} ≥ a_{2}^{(i)}, e_{1}^{(i)} ≥ e_{2}^{(i)}, i = 1, 2, …, n. The number of all families P = (P_{1}, P_{2}, …, P_{n}) of nonintersecting lattice paths P_{i} : A_{i} → E_{i}, which stay below the line x = y, and where the paths of P altogether contain exactly ℓ NE-turns, is

∑_{ℓ_{1}+···+ℓ_{n}=ℓ} det_{1≤i,j≤n} (\binom{e_{1}^{(j)} − a_{1}^{(i)} + i − j}{ℓ_{i} + i − j}\binom{e_{2}^{(j)} − a_{2}^{(i)} − i + j}{ℓ_{i}}

− \binom{e_{1}^{(j)} − a_{2}^{(i)} − i − j + 1}{ℓ_{i} − j}\binom{e_{2}^{(j)} − a_{1}^{(i)} + i + j − 1}{ℓ_{i} + s}). (4.44)

In the third theorem (basically) the same families of nonintersecting lattice paths as before are counted, but with respect to their number of EN-turns. By a rotation by 180^{∘} this could be translated into a result about counting families of nonintersecting lattice paths staying above x = y, with given starting and end points, with respect to NE-turns.

Theorem 4.7. Let A_{i} = (a_{1}^{(i)}, a_{2}^{(i)}) and E_{i} = (e_{1}^{(i)}, e_{2}^{(i)}) be lattice points satisfying

a_{1}^{(1)} < a_{1}^{(2)} < ⋯ < a_{1}^{(n)}, a_{2}^{(1)} ≥ a_{2}^{(2)} ≥ ⋯ ≥ a_{2}^{(n)},
e_{1}^{(1)} ≤ e_{1}^{(2)} ≤ ⋯ ≤ e_{1}^{(n)}, e_{2}^{(1)} > e_{2}^{(2)} > ⋯ > e_{2}^{(n)},

and a_{1}^{(i)} ≥ a_{2}^{(i)}, e_{1}^{(i)} ≥ e_{2}^{(i)}, i = 1, 2, …, n. The number of all families P = (P_{1}, P_{2}, …, P_{n}) of nonintersecting lattice paths P_{i} : A_{i} → E_{i}, P_{i} : A_{i} → E_{i}, which stay below the line x = y, and where the paths of P altogether contain exactly ℓ EN-turns, is

∑_{ℓ_{1}+···+ℓ_{n}=ℓ} det_{1≤i,j≤n} (\binom{e_{1}^{(j)} − a_{1}^{(i)} + i − j}{ℓ_{i} + i − j}\binom{e_{2}^{(j)} − a_{2}^{(i)} − i + j}{ℓ_{i}}

− \binom{e_{1}^{(j)} − a_{2}^{(i)} − i − j + 3}{ℓ_{i} − j + 1}\binom{e_{2}^{(j)} − a_{1}^{(i)} + i + j − 3}{ℓ_{i} + i − 1}). (4.45)

These theorems are more conveniently proved after before having modified the problem. This modification is described next.

Suppose, P = (P_{1}, P_{2}, …, P_{n}) is a family of nonintersecting lattice paths P_{i} : A_{i} → E_{i}. Now we shift the i-th path P_{i} in the direction (−i + 1, i − 1) thus obtaining the new path P′_{i}, i = 1, 2, …, n. The new family P′ = (P′_{1}, P′_{2}, …, P′_{n}) might be intersecting, however it is non-crossing (see Figure 22).

• _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•} • • • • • • P_{1}P_{2}  P^{3}  ⟶  _{•} _{•} _{•} _{•} _{•} _{•} _{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•} ^{•} ^{•} ^{•} ^{•} ^{•} ^{•}^{•  •} •  •^{•  •} P′_{1} P′_{2}  P′_{3}
nonintersecting lattice paths  non-crossing lattice paths

Figure 22

The actual meaning of “non-crossing” stems from applying the shifting process to nonintersecting paths. Hence, let P_{1}, P_{2} be two paths, P_{1} : A → E, P_{2} : B → F, where A = (a_{1}, a_{2}), B = (b_{1}, b_{2}), E = (e_{1}, e_{2}), F = (f_{1}, f_{2}) with

a_{1} ≤ b_{1}, a_{2} > b_{2}, e_{1} < f_{1}, e_{2} ≥ f_{2}.

Roughly speaking, these inequalities say that A is located in the north-West of B (strictly in direction north and weakly in direction West), and E is located in the north-West of F (weakly in direction north and strictly in direction West). Let the array representations of P_{1} and P_{2} be

P_{1} : \begin{aligned}a_{1} ≤ & p_{1} & … & p_{k} & ≤ e_{1} − 1 \\ a_{2} + 1 ≤ & q_{1} & … & q_{k} & ≤ e_{2} \end{aligned}  (4.46)

and

P_{2} : \begin{aligned}b_{1} ≤ & r_{1} & … & r_{l} & ≤ f_{1} − 1 \\ b_{2} + 1 ≤ & s_{1} & … & s_{l} & ≤ f_{2} \end{aligned} ,  (4.47)

respectively.

Suppose that P_{1} and P_{2} intersect, i.e. have a point in common. Let M be a meeting point of P_{1} and P_{2}. By definition set p_{k+1} := e_{1} and q_{0} := a_{2}. (Note that the thereby augmented sequences a and b remain strictly increasing.)

• • •^{M}^{(r_{J}, s_{J})} _{(p_{I}, q_{I−1})}  ^{P_{1}}  ^{P_{2}}^{Figure 23}

Considering the east-north turn (p_{I}, q_{I−1}) in P_{1} immediately preceding M (and being allowed to be equal to M) and the north-east turn (r_{J}, s_{J}) in P_{2} immediately preceding M (and being allowed to be equal to M), we get the inequalities (cf. Figure 23)

r_{J} ≤ p_{I},  (4.48a)
q_{I−1} ≤ s_{J},  (4.48b)

where

1 ≤ I ≤ k + 1, 1 ≤ J ≤ l.  (4.48c)

Of course, k, l, p_{I}, q_{I}, r_{J}, s_{J}, etc., refer to the array representations of P_{1} and P_{2}. It now becomes apparent that the above assignments for p_{k+1} and q_{0} are needed for the inequalities (4.48a,b) to make sense for I = 1 or I = k + 1. Note that M = (p_{I}, s_{J}). Vice versa, if (4.48a,b,c) is satisfied then there must be a meeting point between P_{1} and P_{2} (because of the particular location of the starting and end points A, B, E, F).

Summarizing, the existence of I, J satisfying (4.48a,b,c) characterize the array representations of intersecting pairs of paths.

Now, if P_{2} is shifted in direction (−1, 1), we obtain the path P′_{2} with array representation

\begin{aligned}b′_{1} ≤ & c′_{1} & … & c′_{l} & ≤ f′_{1} − 1 \\ b′_{2} + 1 ≤ & d′_{1} & … & d′_{l} & ≤ f′_{2} \end{aligned} ,  (4.49)

where c′_{i} = r_{i} − 1, d′_{i} = s_{i} + 1, b′_{1} = b_{1} − 1, b′_{2} = b_{2} + 1, f′_{1} = f_{1} − 1, f′_{2} = f_{2} + 1. The conditions (4.48a,b,c) become

c′_{J} < p_{I},  (4.50a)
q_{I−1} < d′_{J},  (4.50b)

where

1 ≤ I ≤ k + 1, 1 ≤ J ≤ l.  (4.50c)

We take (4.50a,b,c) as definition of two paths P_{1} and P_{2} with array representations (4.46) and (4.49), respectively, being non-crossing. We call the point (p_{I}, d′_{J}) crossing point of P_{1} and P_{2}.

Modulo the above described shifting process, Theorems 4.5–4.7 are equivalent to the following three Theorems. Aside from translating “nonintersecting” into “noncrossing”, we also transformed (4.43)–(4.45) into generating functions form. Thus the results become more compact and also more closely resemble the similarity to the determinant formulas (??), (??) for counting without turns.

Theorem 4.8. Let A_{i} = (a_{1}^{(i)}, a_{2}^{(i)}) and E_{i} = (e_{1}^{(i)}, e_{2}^{(i)}) be lattice points satisfying

a_{1}^{(1)} + 1 ≤ a_{1}^{(2)} + 2 ≤ ⋯ ≤ a_{1}^{(n)} + n, a_{2}^{(1)} ≥ a_{2}^{(2)} ≥ ⋯ ≥ a_{2}^{(n)}, (4.51)

and

e_{1}^{(1)} ≤ e_{1}^{(2)} ≤ ⋯ ≤ e_{1}^{(n)}, e_{2}^{(1)} − 1 ≥ e_{2}^{(2)} − 2 ≥ ⋯ ≥ e_{2}^{(n)} − n. (4.52)

The generating function ∑_{P} t^{NE(P)} where the sum is over all families P = (P_{1}, P_{2}, …, P_{n}) of non-crossing lattice paths P_{i} : A_{i} → E_{i}, is

GF (L((A → E | non-crossing)); t^{NE(.)})

= det_{1≤i,j≤n} (∑_{ℓ_{i}} \binom{e_{1}^{(j)} − a_{1}^{(i)}}{ℓ_{i} + i − j}\binom{e_{2}^{(j)} − a_{2}^{(i)}}{ℓ_{i}}) . (4.53)

Theorem 4.9. Let A_{i} = (a_{1}^{(i)}, a_{2}^{(i)}) and E_{i} = (e_{1}^{(i)}, e_{2}^{(i)}) be lattice points satisfying (4.51) and (4.52) and

a_{1}^{(1)} ≥ a_{2}^{(1)} and e_{1}^{(1)} ≥ e_{2}^{(1)}.  (4.54)

The generating function ∑_{P} t^{NE(P)} where the sum is over all families P = (P_{1}, P_{2}, …, P_{n}) of non-crossing lattice paths P_{i} : A_{i} → E_{i}, such that P_{1} stays below the line x = y, is

GF (L((A → E | x ≥ y, non-crossing)); t^{NE(.)})

= det_{1≤i,j≤n} ( ∑_{ℓ_{i}} (\binom{e_{1}^{(j)} − a_{2}^{(i)}}{ℓ_{i} + i − j}\binom{e_{2}^{(j)} − a_{2}^{(i)}}{ℓ_{i}}

− \binom{e_{1}^{(j)} − a_{2}^{(i)} − 1}{ℓ_{i} − j}\binom{e_{2}^{(j)} − a_{1}^{(i)} + 1}{ℓ_{i} + i})t^{ℓ_{i}}). (4.55)

Theorem 4.10. Let A_{i} = (a_{1}^{(i)}, a_{2}^{(i)}) and E_{i} = (e_{1}^{(i)}, e_{2}^{(i)}) be lattice points satisfying

a_{1}^{(1)} ≤ a_{1}^{(2)} ≤ ⋯ ≤ a_{1}^{(n)}, a_{2}^{(1)} − 1 ≥ a_{2}^{(2)} − 2 ≥ ⋯ ≥ a_{2}^{(n)} − n, (4.56)
e_{1}^{(1)} + 1 ≤ e_{1}^{(2)} + 2 ≤ ⋯ ≤ e_{1}^{(n)} + n, e_{2}^{(1)} ≥ e_{2}^{(2)} ≥ ⋯ ≥ e_{2}^{(n)},  (4.57)

and (4.54). The generating function ∑_{P} t^{EN(P)} where the sum is over all families P = (P_{1}, P_{2}, …, P_{n}) of non-crossing lattice paths P_{i} : A_{i} → E_{i}, such that P_{1} stays below the line x = y, is

GF (L((A → E | x ≥ y, non-crossing)); t^{EN(.)})

= det_{1≤i,j≤n} ( ∑_{ℓ_{i}} (\binom{e_{1}^{(j)} − a_{1}^{(i)}}{ℓ_{i} + i − j}\binom{e_{2}^{(j)} − a_{2}^{(i)}}{ℓ_{i}}

− \binom{e_{1}^{(j)} − a_{2}^{(i)} + 1}{ℓ_{i} − j + 1}\binom{e_{2}^{(j)} − a_{1}^{(i)} − 1}{ℓ_{i} + i − 1})t^{ℓ_{i}}). (4.58)

Proof of Theorem 4.8. Since in the proof we are also considering skew tworowed arrays, we introduce some terminology for convenience. Let j > 0. We say that the two-rowed array P is of the type j if P has the form

\begin{aligned}p_{−j+1} & p_{−j+2} & … & p_{−1} & p_{0} & p_{1} & … & p_{k} \\ & & & & & q_{1} & … & q_{k} \end{aligned}

for some k ≥ 0. We say that P is of the type −j if P has the form

\begin{aligned}& & & & & p_{1} & … & p_{k} \\ q_{−j+1} & q_{−j+2} & … & q_{−1} & q_{0} & q_{1} & … & q_{k} \end{aligned}

for some k ≥ 0. Note that the placement of indices is chosen such that non-positive indices can occur only in one row of P, while the positive indices occur in both rows of P.

First we give the combinatorial interpretation of the determinant (4.53) in terms of two-rowed arrays. It is easy to see that (4.53) is the generating function

∑_{(P,σ)} sgn σ w_{x,y}(P),  (4.59)

where the sum is over all pairs (P, σ) of permutations σ in 𝔖_{n}, the symmetric group of order n, and families P = (P_{1}, P_{2}, …, P_{n}) of two-rowed arrays, P_{i} being of type σ(i) − i and the bounds for the entries of P_{i} being as follows,

\begin{aligned}a_{1}^{(σ(i))} ≤ & … p_{ℓ_{i}}^{(i)} & ≤ e_{1}^{(i)} − 1 \\ a_{2}^{(σ(i))} + 1 ≤ & … q_{ℓ_{i}}^{(i)} & ≤ e_{2}^{(i)} \end{aligned} ,  (4.60)

i = 1, 2, …, n.

The outline of the proof is as follows. Next we extend the notion of being noncrossing to two-rowed arrays. We then show that in the sum (4.59) all contributions corresponding to pairs (P, σ) where P is a crossing family (to be explained below) of two-rowed arrays cancel. This is done by constructing a weight-preserving, sign-reversing involution on those pairs. Finally we show that in a pair (P, σ) with σ ≠ id the family P must be crossing. This establishes that only pairs (P, id) where P is a non-crossing family of two-rowed arrays contribute to the sum (4.59). But these pairs exactly correspond to the families of non-crossing paths under consideration, hence Theorem 4.8 would be proved.

Let M_{1} and M_{2} be two-rowed arrays, given by

M_{1} : \begin{aligned}a_{1} ≤ & … p_{k} & ≤ e_{1} − 1 \\ a_{2} + 1 ≤ & … q_{k} & ≤ e_{2} \end{aligned}

and

M_{2} : \begin{aligned}b_{1} ≤ & … r_{l} & ≤ f_{1} − 1 \\ b_{2} + 1 ≤ & … s_{l} & ≤ f_{2} \end{aligned} ,

respectively. By definition we set p_{k+1} := e_{1}, we set q_{0} := a_{2} in case that the sequence … q_{0} is empty, and we set r_{0} := b_{1} − 1 in case that the sequence … r_{0} is empty. We say that (p_{I}, s_{J}) is a crossing point of M_{1} and M_{2} if

r_{J} < p_{I}  (4.61a)
q_{I−1} < s_{J}  (4.61b)

and

1 ≤ I ≤ k + 1, 0 ≤ J ≤ l.  (4.61c)

These inequalities should be understood to hold only if all variables are defined. In particular, if the sequence … s_{0} is empty the inequality (4.61b) does not make sense. However, if the sequence … r_{0} is empty the inequality (4.61a) makes sense because of the conventional assignment for r_{0} above.

Let (P, σ) be a pair under consideration for the sum (4.59). Besides, we assume that P contains two two-rowed arrays P_{i} and P_{i+1} with consecutive indices that have a crossing point. In the sequel two-rowed arrays with consecutive indices will be called neighbouring two-rowed arrays. A pair (P, σ) where P contains neighbouring two-rowed arrays with a crossing point will be called crossing. Otherwise it will be called non-crossing. We are going to construct a weight-preserving (with respect to the weight function w_{x,y}) and sign-reversing (with respect to sgn σ) involution on crossing pairs (P, σ). Consider all crossing points of neighbouring arrays. Among these points choose those with maximal x-coordinate, and among all those choose the crossing point with maximal y-coordinate. Denote this crossing point by M. Let i be minimal such that M is a crossing point of P_{i} and P_{i+1}. Let P_{i} = (a | b) = (… p_{ℓ_{i}} | … q_{ℓ_{i}}) and P_{i+1} = (c | d) = (… r_{ℓ_{i+1}} | … s_{ℓ_{i+1}}). Recall that P_{i} is of type σ(i) − i and P_{i+1} is of type σ(i + 1) − i − 1 and that the bounds of the entries in P_{i} and P_{i+1} are determined by (4.60). By (4.61), M being a crossing point of P_{i} and P_{i+1} means that there exist I and J such that P_{i} looks like

\begin{aligned}a_{1}^{(σ(i))} ≤ & … & p_{I−1} & p_{I} & … & p_{ℓ_{i}} & ≤ e_{1}^{(i)} − 1 \\ a_{2}^{(σ(i))} + 1 ≤ & … & q_{I−1} & q_{I} & … & q_{ℓ_{i}} & ≤ e_{2}^{(i)} \end{aligned} ,  (4.62)

P_{i+1} looks like

\begin{aligned}a_{1}^{(σ(i+1))} ≤ & . . . . . . . . . & r_{J} & r_{J+1} & … & r_{ℓ_{i+1}} & ≤ e_{1}^{(i+1)} − 1 \\ a_{2}^{(σ(i+1))} + 1 ≤ & … & s_{J−1} & s_{J} & . . . . . . . . . & s_{ℓ_{i+1}} & ≤ e_{2}^{(i+1)} \end{aligned} , (4.63)

M = (p_{I}, s_{J}),

r_{J} < p_{I}  (4.64a)
q_{I−1} < s_{J}  (4.64b)

and

1 ≤ I ≤ ℓ_{i} + 1, 0 ≤ J ≤ ℓ_{i+1}.  (4.64c)

Because of the construction of M the indices I and J are maximal with respect to (4.64a,b,c).

We map (P, σ) to the pair (P̄, σ ∘ (i, i + 1)) ((i, i + 1) denotes the transposition exchanging i and i + 1), where P̄ = (P_{1}, …, P_{i−1}, P̄_{i}, P̄_{i+1}, P_{i+2}, …, P_{n}) with P̄_{i} being given by

\begin{matrix}… & r_{J} & p_{I} & … & p_{ℓ_{i}} \\ … & s_{J−1} & q_{I} & … & q_{ℓ_{i}} \end{matrix} ,  (4.65a)

P̄_{i+1} being given by

\begin{matrix}. . . . . . . . . & p_{I−1} & r_{J+1} & … & r_{ℓ_{i+1}} \\ … & q_{I−1} & s_{J} & . . . . . . . . . & s_{ℓ_{i+1}} \end{matrix} .  (4.65b)

First of all, this operation is well-defined, i.e., all the rows in (4.65a) and (4.65b) are strictly increasing. To see this we have to check r_{J} < p_{I}, s_{J−1} < q_{I}, p_{I−1} < r_{J+1}, and q_{I−1} < s_{J}. This is obvious for the first and last inequality, because of (4.64a) and (4.64b). As for the second inequality, let us suppose s_{J−1} ≥ q_{I}. Then, by (4.64a), we have r_{J} < p_{I} < p_{I+1} and q_{I} ≤ s_{J−1} < s_{J}. This means that (p_{I+1}, s_{J}) is a crossing point of P_{i} and P_{i+1}, with an x-coordinate larger than that of S = (p_{I}, s_{J}), contradicting the “maximality” of S. Similarly, if we assume p_{I−1} ≥ r_{J+1}, we have r_{J+1} ≤ p_{I−1} < p_{I} and, by (4.64b), q_{I−1} < s_{J} < s_{J+1}. This means that (p_{I}, s_{J+1}) is a crossing point of P_{i} and P_{i+1}, with a y-coordinate larger than that of S = (p_{I}, s_{J}), again contradicting the “maximality” of S.

We claim that (P̄, σ(i, i + 1)) is again a pair under consideration for the generating function (4.59). That is, we claim that P̄_{i} is of type (σ ∘ (i, i + 1))(i) − i = σ(i + 1) − i, that P̄_{i+1} is of type (σ ∘ (i, i + 1))(i + 1) − i − 1 = σ(i) − i − 1, and that the bounds for the entries of P̄_{i} are given by

\begin{aligned}a_{1}^{(σ(i+1))} ≤ & … & r_{J} & p_{I} & … & p_{ℓ_{i}} & ≤ e_{1}^{(i)} − 1 \\ a_{2}^{(σ(i+1))} + 1 ≤ & … & s_{J−1} & q_{I} & … & q_{ℓ_{i}} & ≤ e_{2}^{(i)} \end{aligned} ,  (4.66)

and that those for P̄_{i+1} are given by

\begin{aligned}a_{1}^{(σ(i))} ≤ & . . . . . . . . . & p_{I−1} & r_{J+1} & … & r_{ℓ_{i+1}} & ≤ e_{1}^{(i+1)} − 1 \\ a_{2}^{(σ(i))} + 1 ≤ & … & q_{I−1} & s_{J} & . . . . . . . . . & s_{ℓ_{i+1}} & ≤ e_{2}^{(i+1)} \end{aligned} . (4.67)

The claims concerning the types of P̄_{i} and P̄_{i+1} are trivial. The claim concerning the bounds requires some case-by-case analysis, which we leave to the reader. One may also consult Krattenthaler [?]. Obviously the map (4.65) reverses the signs of the associated permutations. Besides, it can be checked that it is an involution.

Finally we have to show that, given a pair (P, σ), P = (P_{1}, P_{2}, …, P_{n}), σ ≠ id, there exist neighbouring two-rowed arrays P_{i} and P_{i+1} having a crossing point. If σ ≠ id then there must exist an i with σ(i + 1) < i + 1. Without loss of generality we may assume that i is minimal with this property. Then we have σ(i) ≥ i. Besides, from σ(i) ≥ i ≥ σ(i + 1) it follows that σ(i) > σ(i + 1). Let P_{i} = (a | b) = (… p_{ℓ_{i}} | … q_{ℓ_{i}}) and P_{i+1} = (c | d) = (… r_{ℓ_{i+1}} | … s_{ℓ_{i+1}}). Because of σ(i) − i ≥ 0 the sequence … q_{0} is empty, hence the assignment q_{0} := a_{2}^{(σ(i))} applies. Because of σ(i + 1) − i − 1 < 0 the sequence … r_{0} is empty, hence the assignment r_{0} := a_{1}^{(σ(i+1))} − 1 applies. Because of σ(i) > σ(i + 1) and (4.51) the inequalities r_{0} = a_{1}^{(σ(i+1))} − 1 ≤ a_{1}^{(σ(i))} < p_{1} and q_{0} = a_{2}^{(σ(i))} ≤ a_{2}^{(σ(i+1))} < s_{0} hold. By (4.61) this means that (p_{1}, s_{0}) is a crossing point of P_{i} and P_{i+1}.

Proof of Theorem 4.9. Again we work with families of two-rowed arrays. This time we consider triples (P, σ, η), where σ is a permutation in 𝔖_{n}, η ∈ {−1, 1}^{r}, and P = (P_{1}, P_{2}, …, P_{n}) is a family of two-rowed arrays, with P_{i} being of type η_{i}σ(i) − i and the bounds of P_{i} being given by

\begin{aligned}a_{1}^{(σ(i))} ≤ & … & ≤ e_{1}^{(i)} − 1 \\ a_{2}^{(σ(i))} + 1 ≤ & … & ≤ e_{2}^{(i)} \end{aligned} , for η = 1,  (4.68)

respectively

\begin{aligned}a_{2}^{(σ(i))} + 1 ≤ & … & ≤ e_{1}^{(i)} − 1 \\ a_{1}^{(σ(i))} ≤ & … & ≤ e_{2}^{(i)} \end{aligned} , for η = −1.  (4.69)

Define sgn η := ∏_{i=1}^{n} η_{i}. It is easy to see that (4.55) is the generating function

∑_{(P,σ,η)} sgn η sgn σ w_{x,x}(P)  (4.70)

where the sum is over all triples which were described above.

We adopt the notion of “crossing of neighbouring arrays” from the previous proof. In addition, we have to consider crossings of P_{1} with the line x = y. Let P_{1} = (a | b), or with bounds included

\begin{aligned}L_{1} ≤ & . . . . . . . . . & p_{I} & p_{I+1} & … & p_{ℓ_{1}} & ≤ e_{1}^{(1)} − 1 \\ L_{2} ≤ & … & q_{I−1} & q_{I} & . . . . . . . . . & q_{ℓ_{1}} & ≤ e_{2}^{(1)} \end{aligned} .  (4.71)

Recall that L_{1} and L_{2} are a_{1}^{(σ(1))} + 1 and a_{2}^{(σ(1))} or the other way round, depending on whether η_{1} = 1 or η = −1. Let us, by convention, set p_{0} := L_{1} − 1. We say that (q_{I}, q_{I}) is a crossing point of P_{1} and x = y if I ≥ 0 and p_{I} < q_{I}. Again it is understood that this inequality only holds if both p_{I} and q_{I} are defined. In particular, this means that in case I = 0 the inequality only makes sense if the sequence … p_{0} is empty (in which case the assignment p_{0} := L_{1} − 1 applies). We say that a family P = (P_{1}, P_{2}, …, P_{n}) is a crossing family if either there is a crossing of neighbouring arrays or there is a crossing of P_{1} and x = y.

Similarly to the proof of Theorem 4.8 we shall show that in the sum (4.70) all contributions corresponding to triples (P, σ, η) where P is a crossing family of two-rowed arrays cancel by constructing a weight-preserving (with respect to w_{x,x}), sign-reversing (with respect to sgn η sgn σ) involution on those triples. Finally we show that in a triple (P, σ, η) with (σ, η) ≠ (id, (1, 1, …, 1)) the family P must be crossing. This establishes that only triples (P, id, (1, 1, …, 1)) where P is a noncrossing family of two-rowed arrays contribute to the sum (4.70). But these triples exactly correspond to the families of non-crossing paths under consideration, hence Theorem 4.9 would be proved.

Let (P, σ, η) be a triple where P = (P_{1}, P_{2}, …, P_{n}) is a crossing family of tworowed arrays. Consider all crossing points between neighbouring arrays and all crossing points of P_{1} and x = y. Among these points choose those with maximal x-coordinate, and among all those choose the crossing point with maximal y-coordinate. Denote this crossing point by M. If M is a crossing point between neighbouring arrays let i be minimal such that M is a crossing point of P_{i} and P_{i+1}. Map (P, σ, η) to (P̄, σ ∘ (i, i + 1), η^{(i,i+1)}), where η^{(i,i+1)} = (η_{1}, …, η_{i+1}, η_{i}, …, η_{n}) and P̄ = (…, P̄_{i}, P̄_{i+1}, …) with P̄_{i} and P̄_{i+1} being constructed by (4.66) and (4.67), respectively, as in the proof of Theorem 4.8. Note that sgn η^{(i,i+1)} sgn (σ ∘

(i, i + 1)) = − sgn η sgn σ so that the map indeed changes the sign of the corresponding terms in the generating function (4.70).

If M is no crossing point between neighbouring arrays, then it has to be a crossing point of P_{1} and x = y. In this case we map (P, σ, η) to the triple (P̄, σ, η^{(1)}) where η^{(1)} = (−η_{1}, η_{2}, …, η_{n}) and P̄ = (P̄_{1}, P_{2}, …, P_{n}) with P̄_{1} being constructed basically according to the “two-rowed arrays reflection” (4.12). To be precise, let P_{1} be given by (4.71) and let I be the index such that M = (q_{I}, q_{I}). Then P̄_{1} is defined by

\begin{aligned}L_{2} ≤ & . . . . . . . & q_{I−1} & p_{I+1} & … & p_{ℓ_{1}} & ≤ e_{1}^{(1)} − 1 \\ L_{1} ≤ & … & p_{I} & q_{I} & . . . . . . . . . & q_{ℓ_{1}} & ≤ e_{2}^{(1)} \end{aligned} .  (4.72)

To check that this is well-defined we have to verify q_{I−1} < p_{I+1}. In fact, if we assume q_{I−1} ≥ p_{I+1}, we have p_{I+1} < q_{I+1} (because otherwise p_{I+1} ≥ q_{I+1} > q_{I} > q_{I−1}, which is a contradiction to our assumption). But this means that (q_{I+1}, q_{I+1}) is a crossing point of P_{1} and x = y with larger coordinates than S = (q_{I}, q_{I}), which contradicts the “maximality” of S. Also for being well-defined, we have to check L_{1} ≤ q_{0} in case that I = 0 and … p_{0} is empty. But in this case the assignment p_{0} := L_{1} − 1 applies. And, since I = 0 we have p_{0} < q_{0}. But this is exactly equivalent to L_{1} ≤ q_{0}.

Next we note that the type of P̄_{1} is −η_{1}σ(1)+1−2 = −η_{1}σ(1)−1 = η_{1}^{(1)}σ(1)−1. Trivially, sgn η^{(1)} = − sgn η so also this mapping changes the sign of the corresponding terms in the generating function (4.70). Therefore (P̄, σ, η^{(1)}) is indeed again a triple under consideration for the generating function (4.70).

That (q_{I}, q_{I}) is also a crossing point of P̄_{1} is obvious. It is also maximal in the sense explained above since the entries to the right of p_{I+1} and q_{I} were not changed. Therefore, applying the map again to the triple (P̄, σ, η^{(1)}) gives (P, σ, η).

Finally we have to show that if (σ, η) ≠ (id, (1, 1, …, 1)) in the triple (P, σ, η) the family P must be a crossing family. Let η ≠ (1, 1, …, 1). Let i be minimal with η_{i} = −1. If i = 1 then either there is an I ≥ 1 with p_{I} < q_{I}, i.e. a crossing of P_{1} and x = y. Or for all I ≥ 1 there holds p_{I} ≥ q_{I}. By (4.51) and (4.54) we have q_{0} ≥ L_{2} + σ(1) = a_{1}^{(σ(1))} + σ(1) ≥ a_{1}^{(1)} + 1 > a_{2}^{(1)} ≥ a_{2}^{(σ(1))} = L_{1} − 1 = p_{0}. Hence (q_{0}, q_{0}) is a crossing of P_{1} and x = y. (Note that q_{−1} and q_{0} exist since for η_{1} = −1 the type of P_{1} is −σ(1) − 1 ≤ −2.) Now suppose that i ≥ 2. In addition assume that there is no crossing point of neighbouring arrays. We claim that again there must be a crossing point of P_{1} and x = y. Because of η_{i} = −1, P_{i} is of the type −σ(i) − i. Let P_{i−1} = (a | b) and P_{i} be given by

\begin{aligned}a_{2}^{(σ(i))} + 1 ≤ & & & r_{1} & … & r_{ℓ_{i}} & ≤ e_{1}^{(i)} − 1 \\ a_{1}^{(σ(i))} ≤ & … & s_{0} & s_{1} & … & s_{ℓ_{i}} & ≤ e_{2}^{(i)} \end{aligned} .

By assumption P_{i} and P_{i−1} are non-crossing. Because … r_{0} is empty the assignment r_{0} := a_{2}^{(σ(i))} applies. By (4.51) there hold the inequalities s_{0} ≥ a_{1}^{(σ(i))} +σ(i)+ i − 1 ≥ a_{1}^{(1)} + 1 + i − 1 > a_{2}^{(1)} ≥ a_{2}^{(σ(i))} = r_{0}. Now let j be maximal with p_{j} ≤ r_{0}. Then there holds r_{0} < p_{j+1}. If q_{j} < s_{0} then (p_{j+1}, s_{0}) would be a crossing point of P_{i−1} and P_{i} which contradicts our assumptions. Hence we have p_{j} ≤ r_{0} < s_{0} ≤ q_{j}. The same argument is repeated with P_{i−2} and P_{i−1}, etc., until we arrive at P_{1}. Obviously, this gives us a crossing point of P_{1} and x = y, as was claimed.

If σ ≠ id and η = (1, 1, …, 1), the same arguments as in the proof of Theorem 4.8 apply to establish that there must be a crossing of neighbouring arrays.

Sketch of Proof of Theorem 4.10. Here we use the EN-turn representation of paths. For the combinatorial interpretation of the determinant (4.58) we consider triples (P, σ, η), where σ is a permutation in 𝔖_{n}, η ∈ {−1, 1}^{r}, and P = (P_{1}, P_{2}, …, P_{n}) is a family of two-rowed arrays, with P_{i} being of type η_{i}σ(i) − i + 1 − η_{i} and the bounds of P_{i} being given by

\begin{aligned}a_{1}^{(σ(i))} + 1 ≤ & … & ≤ e_{1}^{(i)} \\ a_{2}^{(σ(i))} ≤ & … & ≤ e_{2}^{(i)} − 1 \end{aligned} , for η = 1,  (4.73a)

respectively

\begin{aligned}a_{2}^{(σ(i))} ≤ & … & ≤ e_{1}^{(i)} \\ a_{1}^{(σ(i))} + 1 ≤ & … & ≤ e_{2}^{(i)} − 1 \end{aligned} , for η = −1.  (4.73b)

It is easy to see that (4.58) is the generating function

∑_{(P,σ,η)} sgn η sgn σ w_{x,x}(P)  (4.74)

where the sum is over all triples which were described above.

Since now we are considering EN-turns, we have to redefine what we mean by a crossing of two-rowed arrays and a crossing of a two-rowed array and x = y. Let M_{1} and M_{2} be two-rowed arrays, given by

M_{1} : \begin{aligned}a_{1} + 1 ≤ & … p_{k} & ≤ e_{1} \\ a_{2} ≤ & … q_{k} & ≤ e_{2} − 1 \end{aligned}

and

M_{2} : \begin{aligned}b_{1} + 1 ≤ & … r_{l} & ≤ f_{1} \\ b_{2} ≤ & … s_{l} & ≤ f_{2} − 1 \end{aligned} ,

respectively. By definition we set s_{l+1} := f_{2}, we set r_{0} := b_{1} in case that the sequence … r_{0} is empty, and we set q_{0} := a_{2} − 1 in case that the sequence … q_{0} is empty. We say that (p_{I}, s_{J}) is a crossing point of M_{1} and M_{2} if

r_{J−1} < p_{I}  (4.75a)
q_{I} < s_{J}  (4.75b)

and

0 ≤ I ≤ k, 1 ≤ J ≤ l + 1.  (4.75c)

To define crossings of a two-rowed array M with x = y, let M be given by

\begin{aligned}L_{1} ≤ & … & p_{ℓ_{1}} & ≤ U_{1} \\ L_{2} ≤ & … & q_{ℓ_{1}} & ≤ U_{2} \end{aligned} .

By convention we set q_{ℓ_{1}+1} := U_{2} + 1. The point (q_{I+1}, q_{I+1}) is called a crossing point of M and x = y if 1 ≤ I ≤ ℓ_{1} + 1 and p_{I} < q_{I+1}. Note that if M_{1}, M_{2}, M correspond to lattice paths (by the EN-turn encoding) P_{1}, P_{2}, P, respectively, the first definition exactly is equivalent to P_{1} and P_{2} having a crossing, while the second is equivalent to P and x = y having a crossing.

The arguments are now completely analogous to those in the proof of Theorem 4.9. So we shall not give all the details. Again we construct a weightpreserving (with respect to w_{x,x}), sign-reversing (with respect to sgn η sgn σ) involution on triples (P, σ, η) where P is a crossing (in the new sense) family. Also here we have to distinguish between two cases. Either there is a crossing of neighbouring paths, P_{i} and P_{i+1} say. Or else their is a crossing of P_{1} and x = y. In the first case assume that P_{i} is given by

\begin{matrix}… & p_{I−1} & p_{I} & . . . . . . . . \\ . . . . . . . . . & q_{I} & q_{I+1} & … \end{matrix} ,

and P_{i+1} be given by

\begin{matrix}… & r_{J−1} & r_{J} & … \\ … & s_{J−1} & s_{J} & … \end{matrix} .

As in the proof of Theorem 4.9 it is assumed that M = (p_{I}, s_{J}) is a “maximal” crossing point, and in particular a crossing point of P_{i} and P_{i+1}. Then we map (P, σ, η) to (P̄, σ ∘ (i, i + 1), η^{(i,i+1)}) where P = (P_{1}, …, P̄_{i}, P̄_{i+1}, …, P_{n}) with P̄_{i} being given by

\begin{matrix}… & r_{J−1} & p_{I} & . . . . . . . . \\ . . . . . . . . . & s_{J−1} & q_{I+1} & … \end{matrix} ,  (4.76a)

and P̄_{i+1} being given by

\begin{matrix}… & p_{I−1} & r_{J} & … \\ … & q_{I} & s_{J} & … \end{matrix} .  (4.76b)

In the second case let P_{1} be given by

\begin{matrix}… & p_{I} & p_{I+1} & … \\ … & q_{I} & q_{I+1} & … \end{matrix} .

Here we assume that M = (q_{I+1}, q_{I+1}) is a “maximal” crossing point of P_{1} and x = y. Then we map (P, σ, η) to (P̄, σ, η^{(1)}) where P = (P̄_{1}, P_{2}, …, P_{n}) with P̄_{1} being given by

\begin{matrix}… & q_{I} & p_{I+1} & … \\ … & p_{I} & q_{I+1} & … \end{matrix} .  (4.77)

It can be shown that this mapping is a weight-preserving, sign-reversing involution and that “non-crossing triples” can only occur for (σ, η) = (id, (1, 1, …, 1)). These exactly correspond to the families of non-crossing paths under consideration.

Remark 4.11. A special case of Theorem 4.5 is of relevance in the computation of Hilbert polynomials of determinantal rings. This was shown by several authors, Conca and Herzog [?], Kulkarni [?], and Modak [?]. In fact, Kulkarni [?, Main Theorem 5] derived this special case (r = p, ℓ = E, A_{i} = (0, a_{p−i+1}), E_{i} = (m(2) − b_{p−i+1}, m(1))) from Abhyankar’s formula (Abhyankar [?, (20.14.4), p. 484]), while Conca and Herzog [?] used it to give an alternative proof of Abhyankar’s formula, see also Krattenthaler [?]. On the other hand, Modak [?] gave an independent (manipulative) proof of this special case. Slight variations of Theorem 4.5 solve the computation of Hilbert series for rings generated by minors of a symmetric matrix as considered by Conca [?], see Krattenthaler [?]. Theorem 4.6 can be applied for the computation of the Hilbert series for certain ladder determinantal rings (one sided, with a diagonal upper bound) and perhaps also for pfaffian ringsOn the other hand, Modak [?] gave an independent (manipulative) proof of this special case. Slight variations of Theorem 1 solve the computation of Hilbert series for rings generated by minors of a symmetric matrix as considered by Conca [?], see Krattenthaler [?]. For arbitrary one-sided ladders see Krattenthaler [?] and Krattenthaler and Prohaska [?]. Similar ideas should also give answers for the Hilbert series of more general ladder determinantal rings and symmetric ladders (cf. Conca [?]). In particular, the extension of the dummy path idea in Krattenthaler and Mohanty [?] should be useful in this context.

96 IV. LATTICE PATHS AND THEIR TURNS

CHAPTER 5 Multidimensional lattice paths

This chapter contains enumeration results for lattice paths in spaces of higher dimension.

??ℤ^{d}, remind of meaning of simple lattice paths

Denote coordinates by x_{1}, x_{2}, …, x_{d}.

Of course, as a basis to start with, we need the number of all simple paths in ℤ^{d} from (a_{1}, a_{2}, …, a_{d}) to (e_{1}, e_{2}, …, e_{d}). Since these lattice paths can be seen as permutations of e_{1} − a_{1} steps in x_{1}-direction, e_{2} − a_{2} steps in x_{2}-direction, …, e_{d} − a_{d} steps in x_{d}-direction, the answer is a multinomial coefficient,

|L((a_{1}, …, a_{d}) → (e_{1}, …, e_{d}))| = \binom{∑_{i=1}^{d}(e_{i} − a_{i})}{e_{1} − a_{1}, e_{2} − a_{2}, …, e_{d} − a_{d}}. (5.1)

1. Simple lattice paths subject to boundaries x_{i} ≥ x_{i+1}

The results of this and the following section generalize those of Chapter 2, Section 1.

First we want to find the number of all paths from (a_{1}, a_{2}, …, a_{d}) to (e_{1}, e_{2}, …, e_{d}) which always stay in the region x_{1} ≥ x_{2} ≥ ⋯ ≥ x_{d}. This problem is related to the d-candidate ballot problem (for the 2-candidate ballot problem see Chapter 2, Section 1). Let E_{1}, E_{2}, …, E_{d} be candidates in an election, E_{1} receiving e_{1} votes, E_{2} receiving e_{2} votes, …, E_{d} receiving e_{d} votes. How many ways of counting the votes are there, such that at any stage during the counting E_{1} has at least as many votes as E_{2}, E_{2} has at least as many votes as E_{3}, etc.? It is evident that by encoding each vote for candidate E_{i} by a step in x_{i}-direction this ballot problem is transferred into counting paths from the origin to (e_{1}, e_{2}, …, e_{d}) which are staying in the region x_{1} ≥ x_{2} ≥ ⋯ ≥ x_{d}. It has been known for a long time [?] that the answer is a determinant. But only not too long back, it was Zeilberger [?] who discovered how to generalize André’s reflection principle (which was described in Chapter 2, Section 1) to the d-dimensional problem.

Theorem 5.1. Let A = (a_{1}, a_{2}, …, a_{d}) and E = (e_{1}, e_{2}, …, e_{d}) be points in ℤ^{d} with a_{1} ≥ a_{2} ≥ ⋯ ≥ a_{d} and e_{1} ≥ e_{2} ≥ ⋯ ≥ e_{d}. The number of all paths from A to E staying in the region x_{1} ≥ x_{2} ≥ ⋯ ≥ x_{d} equals

|L(A → E | x_{1} ≥ x_{2} ≥ ⋯ ≥ x_{d})| = ( ∑_{i=1}^{d}(e_{i} − a_{i}))! det_{1≤i,j≤d} (\frac{1}{(e_{i} − a_{j} − i + j)!}) . (5.2)

Proof. Following the idea of the proof of Theorem 2.1 one would argue that the desired number is the number of all paths from A to E minus those which cross one of the hyperplanes x_{i} = x_{i+1}. If a path crosses x_{i} = x_{i+1} then it has to touch the hyperplane H_{i} given by

H_{i} : x_{i} − x_{i+1} = −1, i = 1, 2, …, d − 1.  (5.3)

For a fixed i, the number of paths touching H_{i} can be found using the reflection principle with respect to H_{i}. What the reflection with respect to H_{i} does is to map

(…, y_{i}, y_{i+1}, …) to (…, y_{i+1} − 1, y_{i} + 1, …).  (5.4)

But there are also paths touching some H_{i} and H_{j}, etc. So one would have to argue by inclusion-exclusion, which leads to the task of finding the number of paths touching H_{i_{1}}, H_{i_{2}}, …, H_{i_{k}}. Experienced with the proof of Theorem 2.3, one would apply iterated reflection, see Watanabe and Mohanty [?]. Ultimately, we would consider lattice paths starting from points that result from reflecting A with respect to H_{i}’s, any number of times, to E. These reflections of A can be made explicit. The underlying mechanism will be described in the setting of “Weyl chambers” in Section 2.

In fact, all reflections of A = (a_{1}, a_{2}, …, a_{d}) with respect to H_{i}’s are the points

(a_{σ(1)} + 1 − σ(1), a_{σ(2)} + 2 − σ(2), …, a_{σ(d)} + d − σ(d)),

where σ is some permutation in 𝔖_{d}. Let us write σ(A) for (a_{σ(1)}, a_{σ(2)}, …, a_{σ(d)}), and V_{σ} for (1 − σ(1), 2 − σ(2), …, d − σ(d)) so that

A_{σ} + V_{σ} = (a_{σ(1)} + 1 − σ(1), a_{σ(2)} + 2 − σ(2), …, a_{σ(d)} + d − σ(d)). (5.5)

So, what we consider is the set of paths

⋃_{σ∈𝔖_{d}} L(σ(A) + V_{σ} → E).

1. SIMPLE LATTICE PATHS SUBJECT TO BOUNDARIES x_{i} ≥ x_{i+1} 99

We claim that

|L(A → E | x_{1} ≥ x_{2} ≥ ⋯ ≥ x_{d})| = ∑_{σ∈𝔖_{d}} sgn σ |L(σ(A) + V_{σ} → E)| . (5.6)

It is easily checked that by (5.1) this identity immediately implies (5.2). We may rewrite (5.6) in the form

|L(A → E | x_{1} ≥ x_{2} ≥ ⋯ ≥ x_{d})| = ∑_{(σ,P)} sgn σ,  (5.7)

where the sum is over all pairs (σ, P) with σ ∈ 𝔖_{d} and P ∈ L(σ(A) + V_{σ} → E). The proof of (5.7) is done by a sign-reversing involution on the set of all such pairs (σ, P) where P touches at least one of the hyperplanes H_{i} defined in (5.3). Sign-reversing has to be understood with respect to sgn σ. Provided the existence of such an involution, the only contributions to the sum in (5.7) would be by pairs (σ, P) where P does not touch any of the hyperplanes H_{i}. This can be only the case for σ = id. For, if σ ≠ id then there exists an i with σ(i) > σ(i + 1). This implies a_{σ(i)} ≤ a_{σ(i+1)}, and hence a_{σ(i)} + i − σ(i) < a_{σ(i+1)} + i + 1 − σ(i + 1). Since e_{i} ≥ e_{i+1}, the path P from σ(A) + V_{σ} to E, would have to cross x_{i} = x_{i+1} and therefore would touch H_{i}. This would prove (5.7) and hence the theorem.

Now we construct the promised involution. Let (σ, P) be a pair with σ ∈ 𝔖_{d} and P ∈ L(A_{σ}+V_{σ} → E) with P touching at least one of the hyperplanes H_{i}. Consider all meeting points of P with hyperplanes H_{i}. Choose the last meeting point and denote it by M. Let I be minimal such that H_{I} is a hyperplane that contains M. Then we form the new path P′ by reflecting the portion of P from the starting point σ(A) + V_{σ} up to M with respect to H_{I} and leaving the portion from M to P invariant. Using (5.5) and (5.4), it is easily checked that the starting point of P′ is A_{σ∘(I,I+1)} +V_{σ∘(I,I+1)} (with (I, I +1) denotes the transposition interchanging I and I+1.) Hence, (σ∘(I, I+1), P′) is a pair under consideration for the sum in (5.7) and P′ touches one of the hyperplanes H_{i} (namely H_{I}). This mapping is an involution since nothing was changed after M. Besides, we have sgn σ ∘ (I, I + 1) = − sgn σ. Therefore it is also sign-reversing. This completes the proof of the theorem.

The result (5.2) should remind us of Theorem 3.7, (3.42). There, the number of standard Young tableaux of a given shape is computed. Indeed, given λ = (λ_{1}, λ_{2}, …, λ_{n}) and μ = (μ_{1}, μ_{2}, …, μ_{n}) a standard Young tableaux T of shape λ/μ can be converted into a lattice path in ℤ^{n} from λ to μ staying in the region x_{1} ≥ x_{2} ≥ ⋯ ≥ x_{n}. This is seen by starting the path at μ and for i = 1, 2, …, ∑_{k=1}^{n}(λ_{k} −μ_{k}) joining a step in x_{j}-direction if i is in the j-th row of T. Thus both enumeration problems are equivalent. But they were derived in different ways, namely by using nonintersecting lattice paths for Theorem 3.7 and multidimensional reflection principle for Theorem 5.1, respectively. That both methods, nonintersecting lattice paths in the plane and multidimensional reflection principle, are in fact not too far apart, can be seen from the fact that n nonintersecting lattice paths in the plane can be “merged together” into a single lattice path in ℤ^{n} staying in the region x_{1} ≥ x_{2} ≥ ⋯ ≥ x_{n}, and vice versa. The “merging” of a family (P_{1}, P_{2}, …, P_{n}) of lattice paths is done (see for example Fisher [?]) by scanning the strips S_{p} := {p − 1 ≤ x + y ≤ p}, p = …, −1, 0, 1, …, and joining a step in direction (ε_{1}, ε_{2}, …, ε_{n}), where ε_{i} is 0 or 1 depending on whether the path P_{i} takes a step in horizontal or vertical direction in this strip. On the other hand, a simple lattice path P in ℤ^{d} from (a_{1}, a_{2}, …, a_{d}) to (e_{1}, e_{2}, …, e_{d}) staying in the region x_{1} ≥ x_{2} ≥ ⋯ ≥ x_{d} corresponds to the family P = (P_{1}, P_{2}, …, P_{d}) of nonintersecting lattice paths in the plane, P_{i} : (−a_{i} +i, a_{i} −i) → (−e_{i} +i+∑_{j=1}^{d}(e_{i} −a_{i}), e_{i} −i), where in each strip S_{p} there is exactly one vertical step among the paths of P. The p-th step of P is in direction x_{j} if and only if the vertical step in the region S_{p} belongs to P_{j}. So, in a sense, enumeration of nonintersecting lattice paths in the plane and the enumeration of paths in ℤ^{d} staying in the region x_{1} ≥ x_{2} ≥ ⋯ ≥ x_{d} are equivalent.

??Only for m = 2 or 3, then mention that it is also possible with m groups?? The same idea as that in the proof of Theorem 5.1 would also enable us to count lattice paths in a region which is determined by several disjoint inequality chains, which after a suitable permutation of coordinates could be described as

x_{1} ≥ x_{2} ≥ ⋯ ≥ x_{i_{1}}, x_{i_{1}+1} ≥ x_{i_{1}+2} ≥ ⋯ ≥ x_{i_{2}}, …, x_{i_{r−1}+1} ≥ x_{i_{r−1}+2} ≥ ⋯ ≥ x_{i_{r}} (5.8)

(1 ≤ i_{1} < i_{2} < ⋯ < i_{r} ≤ d). The answer would be of the form

∑_{σ_{1}∈𝔖_{i_{1}}, ...,σ∈𝔖_{ir}} multinomial(σ_{1}, …, σ_{r}, A, E),

where the multinomial coefficients would count the number of paths from some (iterated) reflection of A to E. The reflections are with respect to hyperplanes x_{i} − x_{i+1} = −1, i ∈ {1, …, i_{1} − 1, i_{1} + 1, …, i_{r−1} − 1, i_{r−1} + 1, …, i_{r}} and encoded by the permutations σ_{1}, σ_{2}, …, σ_{r}. See the next section for a precise formulation and generalizations.

However, if the inequality chains are not disjoint, then the situation becomes truly difficult. In particular, the reflection principle does not seem to help anymore. One could consider reflections, as in the proof of Theorem 5.1, but the nice property that for permutations other than the identity permutation some hyperplane has to be touched would fail. That there is some difference in complexity can also seen from the fact that only the most simple problem has been solved and that no really nice and enlightening solution is known. This problem was originally formulated in terms of a three-candidate ballot problem. Let E_{1}, E_{2}, E_{3} be candidates in an election, E_{1} receiving e_{1} votes, E_{2} receiving e_{2} votes, and E_{3} receiving e_{3} votes, e_{1} ≥ max{e_{2}, e_{3}}. How many ways of counting the votes are there such that at any stage during the counting candidate E_{1} has at least as many votes as E_{2} and as least as many votes as E_{3}? In lattice paths formulation this would mean to count all simple lattice paths in ℤ^{3} from the origin to (e_{1}, e_{2}, e_{3}) staying in the region {(x_{1}, x_{2}, x_{3}) : x_{1} ≥ x_{2} and x_{1} ≥ x_{3}}. We state the result below. Solutions were given by Kreweras [?] and Niederhausen [?, ?], see also Gessel [?, ??]. It is possible to get a determinantal solution on the basis of Theorem 5.7, but it is not as nice as the one given below.

Theorem 5.2. Let e_{1} ≥ max{e_{2}, e_{3}}. The number of all lattice paths in ℤ^{3} from (0, 0, 0) to (e_{1}, e_{2}, e_{3}) subject to x_{1} ≥ x_{2} and x_{1} ≥ x_{3} is given by

|L((0, 0, 0) → (e_{1}, e_{2}, e_{3}) | x_{1} ≥ max{x_{2}, x_{3}})|

= \binom{e_{1} + e_{2} + e_{3}}{e_{1}, e_{2}, e_{3}} − \frac{e_{2} + e_{3}}{1 + e_{1}} \binom{e_{1} + e_{2} + e_{3}}{e_{1}, e_{2}, e_{3}}

+ ∑_{i,j≥1}(−1)^{i+j} \frac{(e_{1} + e_{2} + e_{3})! (2i + 2j − 2)! (i + j − 2)!}{i! (e_{3} − i)! j! (e_{2} − j)! (2i − 1)! (2j − 1)! (i + j + e_{1})!}. (5.9)

In particular, if e_{1} = e_{2} this number simplifies to

|L((0, 0, 0) → (e_{1}, e_{2}, e_{3}) | x_{1} ≥ max{x_{2}, x_{3}})|

= 2^{2e_{3}+1} (\frac{e_{1}!}{(e_{1} − e_{3})!})^{2} \frac{(2e_{1} − 2e_{3} + 1)!}{(2e_{1} + 2)!}. (5.10)

2. Hyperplane boundaries and the reflection principle

In the previous section we saw how to apply the reflection principle to count simple lattice paths in the region x_{1} ≥ x_{2} ≥ ⋯ ≥ x_{d}. We also indicated some generalizations that would be obtainable by means of the reflection principle. It is natural to ask: How far can we go with reflection principle? What is the most general situation where it applies? This question was raised and answered by Gessel and Zeilberger [?], ??see Magyar [?]??.

Clearly, the typical situation where reflection principle may be applied is the problem of counting lattice paths in ℤ^{d} from some starting point A = (a_{1}, a_{2}, …, a_{d}) to some end point E = (e_{1}, e_{2}, …, e_{d}) with steps in a set 𝕊 ⊆ ℤ^{d}, which stay in a region bounded by a set H of hyperplanes. Let us review Theorem 5.1 and its proof to see what the underlying principle is.

The formula which we proved in Theorem 5.1 is (5.2), or equivalently, equation (5.6), which is

|L(A → E | x_{1} ≥ x_{2} ≥ ⋯ ≥ x_{d})| = ∑_{σ∈𝔖_{d}} sgn σ |L(σ(A) + V_{σ} → E)| ,

where σ(A) = (a_{σ(1)}, a_{σ(2)}, …, a_{σ(d)}) and V_{σ} = (1 − σ(1), 2 − σ(2), …, d − σ(d)). It is easy to see, by a translation in direction (−1, −2, …, −d), that this is equivalent to

|L(A → E | x_{1} > x_{2} > ⋯ > x_{d})| = ∑_{σ∈𝔖_{d}} sgn σ |L(σ(A) → E)| . (5.11)

In words, given A = (a_{1}, a_{2}, …, a_{d}) and E = (e_{1}, e_{2}, …, e_{d}) with a_{1} > a_{2} > ⋯ > a_{d} and e_{1} > e_{2} > ⋯ > e_{d}, the number of lattice paths from A to E which do not touch any of the hyperplanesH_{i} given by

H_{i} : x_{i} − x_{i+1} = 0, i = 1, 2, …, d − 1.  (5.12)

(note the slight difference with (5.3)), is given by a signed sum of path numbers, each number counting the paths from some “permutation” σ(A) of the starting point A to the end point E. Recall that the appearance of these “permuted starting points” is caused by the reflection principle which makes it necessary to consider all possible points that arise from A by iterated reflection with respect to the hyperplanes H_{i} in (5.12). Phrased differently, the group of linear orthogonal transformations of ℝ^{d} (ℝ denoting the set of real numbers) which is generated by the reflections with respect to the hyperplanes H_{i} in (5.12) is the group 𝔖_{d} of permutations. It acts on the points of ℝ^{d} by permuting coordinates.

Hence, the natural environment for a “general reflection principle” is within the setting of reflection groups. A reflection group is simply a group which is generated by all reflections with respect to the hyperplanes H in a given set H of hyperplanes (in some ℝ^{d}). We review the facts about reflection groups that are relevant for us below. For an excellent exposition of the subject see Humphreys [?]. The situation of Theorem 5.1 will be our running example.

As above, let H be a (finite??) set of hyperplanes in some ℝ^{d}. Let W denote the group that is generated by the corresponding reflections. The group W consists of linear orthogonal transformations of ℝ^{d}. Some of these transformations happen to be reflections with respect to a hyperplane (not necessarily belonging to H), and let R denote the collection of all these hyperplanes. Of course, R contains H. In the example when H is the set of hyperplanes in (5.12), we already saw that the group W is the permutation group 𝔖_{d}, acting on ℝ^{d} by permutating coordinates.

All the reflections in this group are the interchanges of two coordinates x_{i} and x_{j}, 1 ≤ i < j ≤ d, corresponding to the transpositions (i, j) in 𝔖_{d}. Hence, the corresponding set R of hyperplanes in this case is

R = {x_{i} − x_{j} = 0 : 1 ≤ i < j ≤ d}.  (5.13)

??Figure The hyperplanes in R cut the space into different regions. The connected components of the complement of ⋃_{H∈R} H in ℝ^{d} are called chambers. Each chamber is enclosed by a set R_{0} of bordering hyperplanes. Clearly, R_{0} is a subset of R. In our running example a typical chamber is the region

{(x_{1}, x_{2}, …, x_{d}) : x_{1} > x_{2} > ⋯ > x_{d}},  (5.14)

which is bounded by the hyperplanes in (5.12). Actually, in this special case any chamber has the form

{(x_{1}, x_{2}, …, x_{d}) : x_{σ(1)} > x_{σ(2)} > ⋯ > x_{σ(d)}},  (5.15)

where σ is some permutation in 𝔖_{d}.

It can be shown that the reflections with respect to the hyperplanes in R_{0} generate the complete reflection group W. Another important fact is that, given one chamber C, all chambers are w(C), where w runs through the elements of the reflection group W, all w(C) being distinct.

For the analysis of reflection groups it is convenient to associate to each hyperplane H in R a pair {r_{H}, −r_{H}} of vectors perpendicular to H. The collection of all these vectors is called a root system Φ associated to the reflection group W. The elements of Φ are called roots. In our running example, the set of hyperplanes is given by (5.13), hence an associated root system is Φ = {ε_{i} − ε_{j} : 1 ≤ i ≠ j ≤ d}, where ε_{1}, ε_{2}, …, ε_{d} are the standard basis vectors. The properties of a root system Φ are

(1) If α belongs to Φ then also −α.

(2) The reflection of α ∈ Φ with respect to a hyperplane perpendicular to some β ∈ Φ is again an element of Φ.

If a particular chamber C is fixed, the roots which are associated with the bordering hyperplanes and which point inside the chamber C are called simple roots. In our running example, the simple roots associated to the chamber (5.14) are {ε_{1} − ε_{2}, ε_{2} − ε_{3}, …, ε_{d−1} − ε_{d}}.

Now we are in a position to formulate and prove Gessel and Zeilberger’s result [?, Theorem 1]. The motivation for the technical conditions in the statement of the theorem will become apparent from the proof. ??signum von w!! ??see the Remark below??

Theorem 5.3. Let C be a chamber of some reflection group W, determined by the hyperplanes in the set R_{0}. Let 𝕊 be a set of steps which is invariant under W, i.e., w(𝕊) = 𝕊, and with the property that for all hyperplanes H ∈ R_{0} and all steps s ∈ 𝕊 the Euclidean inner product (s, r_{H}) is either 0 or ±k_{H}, where k_{H} is a fixed constant, depending only on the hyperplane H. As above, r_{H} is a root associated to H, i.e., a vector perpendicular to H. Furthermore, let A, E be lattice points inside the chamber C such that also w(A), w(E) are lattice points for all w ∈ W, and such that for all hyperplanes H ∈ R_{0} the Euclidean inner product (A, r_{H}) is an integral multiple of k_{H}.

Then the number of all lattice paths from A to E, with exactly m steps from 𝕊, and which stay strictly inside the chamber C equals

|L_{m}??(A → E; 𝕊 | inside C)| = ∑_{w∈W} sgn w |L_{m}??(w(A) → E; 𝕊)| . (5.16)

Remark 5.4. Technical conditions??

It is obvious that (5.16) specializes to formula (5.11) in the case of our running example, where W is the group of permutations of coordinates, C is given by (5.14), the set of hyperplanes is (5.12), the set of steps is {(1, 0), (0, 1)}, all constants k_{H} being 1.

In fact, we already considered a second special case of Theorem 5.3, namely in Theorem ??. In this case, … ???

What other examples are covered by the general setup in Theorem 5.3? The answer is that all reflection groups that are “relevant” in our context are completely classified. The meaning of “relevant” is as follows. In order our formula (5.16) to make sense, the sum on the right-hand side of (5.16) should be finite. So, only reflection groups that are “discrete” and act “locally finite” will interest us. And it is exactly these reflection groups that are precisely known (see Humphreys [?, sec. 4.10], Bourbaki [?, ch. V, VI]).

The classification of reflection groups is most conveniently described by starting with the classification of root systems. Recall that the roots are vectors orthogonal to the reflecting hyperplanes of the reflection group. Given a root system Φ it may happen that it decomposes as a disjoint union Φ = Φ_{1} ∪ Φ_{2}, where Φ_{1} and Φ_{2} are root systems orthogonal to each other, i.e., the Euclidean inner product (α_{1}, α_{2}) vanishes for all α_{1} ∈ Φ_{1} and α_{2} ∈ Φ_{2}. In this case the root system Φ is called reducible, otherwise irreducible. Obviously, every root system is the disjoint union of irreducible root systems.

There exist the following irreducible root systems, the five infinite families I_{2}(m) (m = 1, 2, …), A_{n}, B_{n}, C_{n}, D_{n} (n = 1, 2, …), and the seven exceptional root systems G_{2}, F_{4}, E_{6}, E_{7}, E_{8}, and H_{3}, H_{4}. (The indices mark the dimension of the vector space that is spanned by the roots.) To each root system there is associated a finite reflection group and an affine reflection group, which is infinite, except that for m ≠ 2, 4, 6, 8, 12?? there is only a finite reflection group associated to I_{2}(m), the same being true for H_{3}, H_{4}. The finite reflection group is generated by all reflections with respect to the hyperplanes which run through a given point (we assume that this is the origin) and are perpendicular to a root in the root system. The affine reflection group is generated by a larger set of hyperplanes, which includes the aforementioned hyperplanes plus certain translates of them. The root systems B_{n} and C_{n} have the same associated reflection groups. Therefore we give a joint description. The reflection groups corresponding to G_{2} are the same as those for I_{2}(6), therefore we need not consider G_{2}.

Below is the list of the infinite families of reflection groups. For the exceptional cases we refer the reader to Bourbaki [?, cartes blanches??]. It goes without saying that the hyperplane realizations are unique only up to translations and orthogonal transformations of the underlying space. We give realizations in some ℝ^{d}, so x always denotes the vector (x_{1}, x_{2}, …, x_{d}).

Type I_{2}(m): The root system is Φ = {cos(2πk/m)ε_{1} + sin(2π/m)ε_{2} : k = 0, 1, …, m−1}. The set of reflecting hyperplanes is therefore R = {cos(2πk/m)x_{1}+ sin(2πk/m)x_{2} = 0 : k = 0, 1, …, m − 1}.

So, the associated finite reflection group is the symmetry group of a regular m-gon. A typical chamber is {x : x_{1} ≥ 0, cos(2π/m)x_{1} + sin(2π/m)x_{2} ≥ 0}, the corresponding simple roots are ε_{1}, cos(2π/m)ε_{1} + sin(2π/m)ε_{2}.

Type A_{n}: The root system is Φ = {ε_{i} − ε_{j} : 1 ≤ i ≠ j ≤ n + 1}. The set of reflecting hyperplanes is therefore R = {x_{i} − x_{j} = 0 : 1 ≤ i < j ≤ n + 1}. Obviously, the reflection with respect x_{i} − x_{j} = 0 acts by interchanging the i-th and j-th coordinate.

So, the associated finite reflection group is the group of permutations of the coordinates x_{1}, x_{2}, …, x_{n+1}, which is isomorphic to the permutation group 𝔖_{n+1}. A typical chamber is C = {x : x_{1} > x_{2} > ⋯ > x_{n+1}}, the corresponding system of simple roots is {ε_{1} − ε_{2}, ε_{2} − ε_{3}, …, ε_{n} − ε_{n+1}}.

The associated affine reflection group is generated by the reflections with respect to the hyperplanes R = {x_{i} − x_{j} = k : 1 ≤ i < j ≤ n + 1, k ∈ ℤ}. The elements of this group are called affine permutations. They act by permuting the coordinates x_{1}, x_{2}, …, x_{n+1} and adding a vector (k_{1}, k_{2}, …, k_{n+1}, 0, 0, …) with k_{1} + k_{2} + ⋯ + k_{n+1} = 0. A typical chamber is C = {x : x_{1} > x_{2} > ⋯ > x_{n+1} > x_{1} − 1}, with the same corresponding system of simple roots.

Type B_{n}, C_{n}: The root system B_{n} is Φ = {±ε_{i} ± ε_{j} : 1 ≤ i ≠ j ≤ n} ∪ {±ε_{i} : 1 ≤ i ≤ n}. The root system C_{n} is Φ = {±ε_{i} ± ε_{j} : 1 ≤ i ≠ j ≤ n} ∪ {±2ε_{i} : 1 ≤ i ≤ n}. The set of reflecting hyperplanes in either case is therefore R = {±x_{i} ± x_{j} = 0 : 1 ≤ i < j ≤ n} ∪ {x_{i} = 0}. Obviously, the reflection with respect x_{i}−x_{j} = 0 acts by interchanging the i-th and j-th coordinate, the reflection with respect to x_{i} + x_{j} = 0 acts by interchanging the i-th and j-th coordinate and changing the sign of both, while the reflection with respect to x_{i} = 0 acts by changing sign of the i-th coordinate.

The associated finite reflection group is the group of signed permutations of the coordinates x_{1}, x_{2}, …, x_{n}, which acts by permuting and changing signs of (some of) the coordinates x_{1}, x_{2}, …, x_{n}. This group is isomorphic to the semidirect product 𝔖_{n}⋉??(ℤ/2ℤ)^{n}. A typical chamber is C = {x : x_{1} > x_{2} > ⋯ > x_{n} > 0}, the corresponding system of simple roots is {ε_{1} − ε_{2}, ε_{2} − ε_{3}, …, ε_{n−1} − ε_{n}, ε_{n}}.

The associated affine reflection group is generated by the reflections with respect to the hyperplanes R = {±x_{i} ± x_{j} = k : 1 ≤ i < j ≤ n, k ∈ ℤ} ∪ {x_{i} = k : 1 ≤ i ≤ n, k ∈ ℤ}. The elements of this group act by permuting and changing signs of (some of) the coordinates x_{1}, x_{2}, …, x_{n} and adding a vector (k_{1}, k_{2}, …, k_{n}, 0, 0, …) with k_{1} + k_{2} + ⋯ + k_{n} ≡ 0 (mod 2). A typical chamber is C = {x : 1 > x_{1} > x_{2} > ⋯ > x_{n} > 0}??, with the same corresponding system of simple roots.

Type D_{n}: The root system D_{n} is Φ = {±ε_{i} ± ε_{j} : 1 ≤ i ≠ j ≤ n}. The set of reflecting hyperplanes is therefore R = {±x_{i} −x_{j} = 0 : 1 ≤ i < j ≤ n}∪{x_{i} = 0}. Obviously, D_{n} is a subset of B_{n} or C_{n}. The action of the reflection with respect to a hyperplane ±x_{i} ± x_{j} = 0 was already explained there.

The associated finite reflection group is the group of signed permutations of the coordinates x_{1}, x_{2}, …, x_{n} with an even number of sign changes. It acts by permuting the coordinates x_{1}, x_{2}, …, x_{n} and changing an even number of signs thereof. This group is isomorphic to the semidirect product 𝔖_{n}⋉??(ℤ/2ℤ)^{n−1}. A typical chamber is C = {x : x_{1} > x_{2} > ⋯ > x_{n−1} > |x_{n}|}, the corresponding system of simple roots is {ε_{1} − ε_{2}, ε_{2} − ε_{3}, …, ε_{n−1} − ε_{n}, ε_{n−1} + ε_{n}}.

The associated affine reflection group is generated by the reflections with respect to the hyperplanes R = {±x_{i} ± x_{j} = k : 1 ≤ i < j ≤ n, k ∈ ℤ}. The elements of this group act by permuting the coordinates x_{1}, x_{2}, …, x_{n}, changing an even number of signs thereof??, and adding a vector (k_{1}, k_{2}, …, k_{n}, 0, 0, …) with k_{1} + k_{2} + ⋯ + k_{n} ≡ 0 (mod 2). A typical chamber is C = {x : 1 > x_{1} > x_{2} > ⋯ > x_{n−1} > |x_{n}|}??, with the same corresponding system of simple roots.

??Figures!!, also exceptional types??

??list equalities: I_{2}(4) = C_{2}, etc.

It is now apparent that Theorem 5.1 follows from Theorem 5.3 specialized to A_{d−1}. Also, Theorem ?? follows from Theorem 5.3 in the special case of the affine reflection group corresponding to A_{1}. To make things more explicit, we state the specialization of Theorem 5.3 to affine A_{d−1}, for arbitrary d. It is equivalent to a result which was first discovered by Filaseta [?, Theorem ??].

Theorem 5.5. Let A = (a_{1}, a_{2}, …, a_{d}) and E = (e_{1}, e_{2}, …, e_{d}) be points in ℤ^{d} with a_{1} > a_{2} > ⋯ > a_{d} and e_{1} > e_{2} > ⋯ > e_{d}. The number of all (simple) paths from A to E staying in the region x_{1} > x_{2} > ⋯ > x_{d} > x_{1} − K equals

|L(A → E | x_{1} > x_{2} > ⋯ > x_{d} > x_{1} − K)|

= ( ∑_{i=1}^{d}(e_{i} − a_{i}))! ∑_{k_{1}+...k_{d}=0} det_{1≤i,j≤d} (\frac{1}{(e_{i} − a_{j} + u_{i}K)!}) . (5.17)

Proof. We use Theorem 5.3 with 𝕊 = {(1, 0), (0, 1)} and m = ∑_{i=1}^{d}(e_{i} − a_{i}) in the special case of affine A_{d−1}. The region which we consider is {x : x_{1} > x_{2} > ⋯ > x_{d} > x_{1} − K}. Therefore, in the prototypical description of the A_{d−1} case we have to transform the underlying space by a scalar multiplication by the constant K. Thus, the reflection group W which we have to consider is the group of permutations of coordinates and additions of vectors (k_{1}K, k_{2}K, …, k_{d}K) with k_{1} + k_{2} + ⋯ + k_{d} = 0. Then, by (5.16) we have

|L(A → E | x_{1} > x_{2} > ⋯ > x_{d} > x_{1} − K)|

= ∑_{σ∈𝔖_{d}} ∑_{k_{1}+...k_{d}=0} |L(A_{σ} + (k_{1}K, …, k_{d}K) → E)| .

It is easily checked that this identity immediately implies (5.17).

Next we illustrate the decomposition of reflection groups. We should remark that if a root system Φ, with associated reflection group W, decomposes into the disjoint union Φ = Φ_{1} ∪ … Φ_{r} of root systems, then W is the product W = W_{1}×⋯×W_{r}, if W_{i} is the reflection group associated to Φ_{i}, i = 1, 2, …, r. Actually we already encountered such a case, namely when considering regions of the form (5.8). It is straight-forward to check that this is a chamber for the root system A_{i_{1}−1} ∪ A_{i_{2}−1} ∪ ⋯ ∪ A_{i_{r}−1}, with associated reflection group 𝔖_{i_{1}} ∪ 𝔖_{i_{2}} ∪ ⋯ ∪ 𝔖_{i_{r}}, which matches with what we did there.

Let us conclude with another example. {K > x_{1} > 0, x_{2} > x_{3} > 0}.

Finally we prove Theorem 5.3.

Proof of Theorem 5.3. We proceed along the lines of the proof of Theorem 5.1.

We may rewrite (5.6) in the form

|L_{m}??(A → E; 𝕊 | inside C)| = ∑_{w,P} sgn w,  (5.19)

where the sum is over all pairs (w, P) with w ∈ W and P ∈ L_{m}(w(A) → E). The proof of (5.19) is by a sign-reversing involution on the set of all such pairs (w, P) where P touches at least one of the hyperplanes in H_{0}. Sign-reversing has to be understood with respect to sgn w. Provided the existence of such an involution, the only contributions to the sum in (5.19) would be by pairs (w, P) where P does not touch any of the hyperplanes in H_{0}. We claim that this can be only the case for w = id. In fact, as we already mentioned, it is one of the properties of a reflection group W that, given one chamber C, all chambers are w(C), w ∈ W, and all w(C)’s are distinct. Therefore, if A is in C and w ≠ id then w(A) must be in a different chamber and so cannot be in C. Thus evidently, any path from w(A) to E, which is inside C, must touch at least one of the bordering hyperplanes. This would prove (5.19) and hence the theorem.

Now we construct the promised involution. Fix some order of the hyperplanes in H_{0}. Let (w, P) be a pair with w ∈ W and P ∈ L(w(A) → E) with P touching at least one of the hyperplanes in H_{0}. Consider all meeting points of P with hyperplanes in H_{0}. Choose the last meeting point and denote it by M. M must be a lattice point because of the assumptions that involve the constants k_{H}. Let H be the first hyperplane (in the chosen order) that meets P. Then we form the new path P′ by reflecting the portion of P from the starting point w(A) up to M with respect to H and leaving the portion from M to P invariant. By assumption, reflection of a step from 𝕊 is again in a step 𝕊. So also P′ consists of steps from 𝕊 only. Evidently, the starting point of P′ is w_{H}w(A), where w_{H} denotes the reflection with respect to H. Hence, (w_{H}w, P′) is a pair under consideration for the sum in (5.19) and P′ touches one of the hyperplanes in H_{0} (namely H). This mapping is an involution since nothing was changed after M. Besides, we have sgn w_{H}w = − sgn w. Therefore it is also sign-reversing. This completes the proof of the theorem.

3. Simple lattice paths bounded by x_{0} = ∑_{i=1}^{d} μ_{i}x_{i}

In this section we consider simple lattice paths in ℤ^{d+1} restricted by a hyperplane of the form x_{0} = ∑_{i=1}^{d} μ_{i}x_{i} where μ_{i}, i = 0, 1, …, d, are nonnegative integers. The results of this section generalize those of Chapter 2, Section 2. It should be noted that the reflection principle does not apply because in general the set of steps (??more precisely) is not invariant under reflection with respect to such a hyperplane (??except of course when μ_{i} = 1 for all i, in which case the reflection principle applies). It will be seen that the proof methods for Theorem 2.4, path decomposition combined with generating function methods and cycle lemma, respectively, carry over to this higher-dimensional problem.

Theorem 5.6. Let μ_{0}, μ_{1}, …, μ_{d} be nonnegative integers and c_{0}, c_{1}, …, c_{d} integers such that c_{0} ≥ ∑_{i=1}^{d} μ_{i}c_{i}. The number of all lattice paths from the origin to

3. SIMPLE LATTICE PATHS BOUNDED BY x_{0} = ∑_{i=1}^{d} μ_{i}x_{i} 109

(c_{0}, c_{1}, …, c_{d}) not crossing the hyperplane x_{0} = ∑_{i=1}^{d} μ_{i}x_{i} is given by

|L(0 → (c_{0}, c_{1}, …, c_{d}) | x_{0} ≥ ∑_{i=1}^{d} μ_{i}x_{i})| = \frac{c_{0} − ∑_{i=1}^{d} μ_{i}c_{i} + 1}{1 + ∑_{i=0}^{d} c_{i}} \binom{1 + ∑_{i=0}^{d} c_{i}}{c_{0} + 1, c_{1}, c_{2}, …, c_{d}}. (5.20)

Proof. Let us denote the hyperplane x_{0} = ∑_{i=1}^{d} μ_{i}x_{i} by H. For k ≥ 0 let L_{k} denote the set of all paths from the origin to some point on the hyperplane x_{0} = k + ∑_{i=1}^{d} μ_{i}x_{i} never crossing H. Let s_{i} denote the unit step in x_{i}-direction. Very analogous to the decomposition (??) in the proof of Theorem 2.4, the set L_{k} can be decomposed in the form

L_{k} = L_{0}(s_{0}L_{0})^{k}.  (5.21)

Namely, a path P in L_{k} is decomposed into

P_{0}s_{0}P_{1}s_{0} … P_{k−1}s_{0}P_{k},

where P_{j} starts at the point immediately following the last intersection of P with x_{0} = j − 1 + ∑_{i=1}^{d} μ_{i}x_{i} and runs until the last intersection of P with x_{0} = j + ∑_{i=1}^{d} μ_{i}x_{i}. The set L_{0} itself can be decomposed with respect to the last step of paths,

L_{0} = {ε} ∪ L_{μ_{1}}s_{1} ∪ L_{μ_{2}}s_{2} ∪ ⋯ ∪ L_{μ_{d}}s_{d}.  (5.22)

The decompositions (5.21) and (5.22) by ?? translate into equations for the generating functions ??explain multi-notation??

F_{k}(z) = ∑_{n≥0} |L(0 → (k + ∑_{i=1}^{d} μ_{i}n_{i}, n_{1}, n_{2}, …, n_{d}) | x_{0} ≥ ∑_{i=1}^{d} μ_{i}x_{i})| z^{n}.

The decomposition (5.21) translates into

F_{k}(z) = F_{0}(z)^{k+1},  (5.23)

and (5.22) translates into

F_{0}(z) = 1 + z_{1}F_{μ_{1}}(z) + ⋯ + z_{d}F_{μ_{d}}(z).  (5.24)

Inserting (5.23) into (5.24) gives

F_{0}(z) = 1 + z_{1}F_{0}(z)^{μ_{1}+1} + ⋯ + z_{d}F_{0}(z)^{μ_{d}+1}.  (5.25)

To be able to apply Lagrange inversion, we write F_{0}(z) = 1 + G_{0}(z). Thus, by replacing z_{i} by tz_{i} and a little rearrangement, (5.25) becomes

\frac{G_{0}(tz)}{∑_{i=1}^{d} z_{i}(1 + G_{0}(tz))^{μ_{i}+1}} = t.

From this we infer that, as a formal power series in t, G_{0}(tz) is the compositional inverse of t/ ∑_{i=1}^{d} z_{i}(1 + t)^{μ_{i}}. We are interested in the coefficients of the series F_{k}(z) which by (5.23) equals (1 + G_{0}(z))^{k+1}. By using Lagrange inversion ?? with f(t) = t/ ∑_{i=1}^{d} z_{i}(1 + t)^{μ_{i}+1} and g(t) = (1 + G_{0}(tz))^{k+1} we get

⟨t^{n}⟩(1 + G_{0}(tz))^{k+1} = \frac{1}{n}⟨t^{−1}⟩(k + 1)(1 + t)^{k}t^{−n}(∑_{i=1}^{d} z_{i}(1 + t)^{μ_{i}+1})^{n}

= \frac{k + 1}{n}⟨t^{n−1}⟩ ∑_{n_{1}+...n_{d}=n} \binom{n}{n_{1}, n_{2}, …, n_{d}}z^{n}(1 + t)^{k+∑_{i=1}^{d}(μ_{i}+1)n_{i}}

= \frac{k + 1}{n} ∑_{n_{1}+···+n_{d}=n} \binom{n}{n_{1}, n_{2}, …, n_{d}}\binom{k + ∑_{i=1}^{d}(μ_{i} + 1)n_{i}}{n − 1}z^{n}

= ∑_{n_{1}+···+n_{d}=n} \frac{k + 1}{k + 1 + ∑_{i=1}^{d}(μ_{i} + 1)n_{i}} \binom{k + 1 + ∑_{i=1}^{d}(μ_{i} + 1)n_{i}}{k + 1 + ∑_{i=1}^{d} μ_{i}n_{i}, n_{1}, n_{2}, …, n_{d}}z^{n}.

Hence we have

F_{k}(z) = ∑_{n≥0} \frac{k + 1}{k + 1 + ∑_{i=1}^{d}(μ_{i} + 1)n_{i}} \binom{k + 1 + ∑_{i=1}^{d}(μ_{i} + 1)n_{i}}{k + 1 + ∑_{i=1}^{d} μ_{i}n_{i}, n_{1}, n_{2}, …, n_{d}}z^{n}.

To obtain the desired number of paths we have to take the coefficient of z_{1}^{c_{1}}z_{2}^{c_{2}} ⋯ z_{d}^{c_{d}} in F_{c_{0}−∑_{i=1}^{d} μ_{i}c_{i}}. This gives exactly the expression in (5.20).

4. Simple paths with a general boundary

In this section we generalize the enumeration problem of Chapter 2, Section 5 to arbitrary dimensions. Let n_{1}, n_{2}, …, n_{d} be nonnegative integers and a and b be increasing integer functions defined on the box ∏_{i=1}^{d}{0, 1, …, n_{i}} such that a ≥ b. a is increasing means that a(i) ≤ a(j) whenever i ≤ j in the usual product order. We ask for the number of all paths in ℤ^{d+1} from (0, b(0)) to (n, a(n)) that always stay in the region “that is bounded by a and b” by which we mean the region

{(i, y) : b(i) ≤ y ≤ a(i)}.  (5.26)

The generalization of Theorem 2.12 reads as follows.

Theorem 5.7. Let n_{1}, n_{2}, …, n_{d} be nonnegative integers and p = ∏_{i=1}^{d} n_{i}. Assume that the points in the box ??[0, n] are 0 = u_{0}, u_{1}, u_{2}, …, u_{p} = n, ordered lexicographically. Then the number of all lattice paths in ℤ^{d+1} from (0, b(0)) to (n, a(n)) that always stay in the region (5.26) equals

|L((0, b(0)) → (n, a(n)) | a ≥ y ≥ b)|

= (−1)^{∑_{i=1}^{d} n_{i}+∏_{i=1}^{d} n_{i}} det ; 0≤i,j≤∑_{i=1}^{d} n_{i}−1 (\binom{a(u_{i}) − b(u_{j+1}) + 1}{u_{j+1} − u_{i}}) . (5.27)

Proof. As in the proof of Theorem 2.12 we transform the enumeration problem into one on nonintersecting lattice paths.

Define two sequences of points, A = ((u_{i−1}, a(u_{i−1}) + ∥u_{i−1}∥))_{i=1,2,...,∥n∥} and E = ((u_{i}, b(u_{i}) + ∥u_{i}∥ − 1))_{i=1,2,...,∥n∥}. We map each lattice path in L((0, b(0)) →

(n, a(n)) | a ≥ y ≥ b) to a family of nonintersecting lattice paths, consisting of unit steps in positive x_{i}-direction, i = 1, 2, …, d, and of unit steps in negative x_{d+1}-direction, which connect some permutation of the points of A (the starting points) to those of E (the end points). This correspondence is defined as follows. See Figure 24. Let P be a path in L((0, b(0)) → (n, a(n)) | a ≥ y ≥ b). Let s be any step of P that is not a step in x_{d+1}-direction, say s = (u_{i}, y) → (u_{j}, y). Then shift s by ∥u_{i}∥ units in x_{d+1}-direction and add a(u_{i}) − y steps in (negative) x_{d+1}-direction prior to s and y − b(u_{j}) steps in (negative) x_{d+1}-direction after s. Thus a path from (u_{i}, a(u_{i}) + ∥u_{i}∥) to (u_{j}, b(u_{j}) + ∥u_{j}∥ − 1) is formed, consisting only of steps in x_{d+1}-direction with the exception of the shifted step s.

Figure!!^{Figure 24}

All other points are connected “vertically”, to be precise, (u_{i}, a(u_{i}) + ∥u_{i}∥) is connected with (u_{i}, b(u_{i}) + ∥u_{i}∥ − 1) whenever this does not intersect P. It is not difficult to see that thus we obtain a family of nonintersecting lattice paths which connects the points of A (in some order) with those of E.

Now we claim two things.

(1) All families of nonintersecting lattice paths that connect some permutation of A with E can be generated by a lattice paths P in L((0, b(0)) →

(n, a(n)) | a ≥ y ≥ b) in that way. Therefore, L((0, b(0)) → (n, a(n)) |

a ≥ y ≥ b) is in bijection with these families of nonintersecting lattice paths.

(2) The permutations of the starting points corresponding to these families of nonintersecting lattice paths all have equal sign. In fact the sign is (−1)^{∑_{i=1}^{d} n_{i}+∏_{i=1}^{d} n_{i}}.

Provided these two assertions hold true, Theorem 3.3 can be applied to infer that the number of lattice paths in question is

(−1)^{∑_{i=1}^{d} n_{i}+∏_{i=1}^{d} n_{i}} det_{0≤i,j≤p−1} ( |L(A_{j} → E_{i})| ),

where A_{j} is the j-th point in the sequence A and E_{i} is the i-th point in the sequence E. Evidently, by (5.1) this expression agrees with the right-hand side of (5.27).

So it remains to prove the two assertions. We start with proving item 2). Let P be a path in L((0, b(0)) → (n, a(n)) | a ≥ y ≥ b) and let 0 = u_{0}, u_{i_{1}}, u_{i_{2}}, …, u_{i_{∥n∥−1}}, u_{p} = n be its projection on the hyperplane x_{d+1} = 0. Then the permutation corresponding to the family of lattice paths associated with P is

(\begin{matrix}1 & i_{1} + 1 & i_{2} + 1 & … & i_{∥n∥−1} + 1 & j_{1} + 1 & j_{2} + 1 & … & j_{p−∥n∥} + 1 \\ i_{1} & i_{2} & i_{3} & … & p & j_{1} & j_{2} & … & j_{p−∥n∥} \end{matrix}) ,

where u_{j_{1}}, u_{j_{2}}, … are the points in the box [0, n] not in the projection of P, ordered such that j_{1} < j_{2} < · · ·. In order to determine the sign of this permutation, we transform it to the identity permutation and record how many transpositions we need. First we rearrange the permutation such that the top row becomes 1, 2, …, p. (This does not change the permutation.) Now notice that j_{1} < j_{2} < ⋯ and that each value j_{k} in the bottom row is offset one place to the right. Hence for each j_{k} we need exactly one transposition to put it into its right place, beginning with j_{1}, continuing with j_{2}, etc. After that the permutation will already have been converted into the identity permutation. The number of transpositions needed was the number of j_{k}’s, which is p − ∥n∥ = ∏_{i=1}^{d} n_{i} − ∑_{i=1}^{d} n_{i}. Therefore the sign of the permutation is as claimed.

For the proof of item 1) take any family P of nonintersecting lattice paths connecting some permutation of A with E. We have to prove the following:

(a) Each path of P has at most one step not in x_{d+1}-direction.

(b) For every k = 0, 1, …, p − 1 there is exactly one non-straight path that starts at a point (i, a(i) + ∥i∥) with ∥i∥ = k.

In order to prove (a), suppose in contrary that there is some path Q in P containing at least 2 steps not in x_{d+1}-direction. Suppose that Q starts at (l, a(l)+ ∥l∥) with ∥i∥ = k. Consider all paths ending at some point (j, b(j) + ∥j∥) where ∥j∥ ≤ k. All these paths have to start at some point (i, a(i) + ∥i∥) with ∥i∥ ≤ k. The number of the latter points is exactly one more than the number of the former, therefore the additional path that starts from one of the latter points is exactly Q. And, by assumption, Q leads to an end point (j, b(j) + ∥j∥) with ∥j∥ ≥ k + 2. Therefore, all the points (i, a(i) + ∥i∥) with ∥i∥ = k + 1 are connected straight with the points (i, b(i) + ∥i∥). But one of these paths must intersect Q, which is absurd.

A similar argument leads to a contradiction if we assume that for some k there are at least two non-straight paths starting at a point (i, a(i) + ∥i∥) with ∥i∥ = k. In fact, in this case there would not be enough starting points left to connect to points (j, b(j) + ∥j∥) with ∥j∥ ≤ k.