ENUMERATION OF ad-NILPOTENT

𝔟-IDEALS FOR SIMPLE LIE ALGEBRAS

Christian Krattenthaler^{†} Luigi Orsina Paolo Papi

Dedicated to the memory of Rodica Simion

Abstract. We provide explicit formulas for the number of ad-nilpotent ideals of a Borel subalgebra of a complex simple Lie algebra having fixed class of nilpotence.

§1 Introduction

In this paper we provide formulas for the number of ad-nilpotent ideals of a Borel subalgebra of a complex simple Lie algebra 𝔤 having fixed class of nilpotence. Our results continue and complete the analysis of [OP] and [AKOP], where Lie algebras of type A have been treated.

The basic framework is the following. Let 𝔤 be a complex simple Lie algebra of rank n. Let 𝔥 ⊂ 𝔤 be a fixed Cartan subalgebra, Δ the corresponding root system of 𝔤. Fix a positive system Δ^{+} in Δ. For each α ∈ Δ^{+} let 𝔤_{α} be the root space of 𝔤 relative to α, 𝔫 = ⨁_{α∈Δ^{+}} 𝔤_{α}, and 𝔟 be the Borel subalgebra 𝔟 = 𝔥 ⊕ 𝔫.

Let I^{n} denote the set of ad-nilpotent ideals (i.e., consisting of ad-nilpotent elements) of 𝔟. Then 𝔦 ∈ I^{n} if and only if 𝔦 decomposes as 𝔦 = ⨁_{α∈Φ} 𝔤_{α}, with Φ ⊆ Δ^{+}

being a dual order ideal in the poset (Δ^{+}, ≤) of positive roots (i.e., a subset of the positive roots with the property that whenever α ≥ β with β ∈ Δ^{+}, then also α ∈ Δ^{+}; the partial order ≤ being the restriction to Δ^{+} of the usual partial order on the root lattice). Clearly, an ideal 𝔦 ∈ I^{n} is nilpotent. We denote its class of nilpotence by n(𝔦). (By definition, the class of nilpotence of an ideal 𝔦 is the smallest number m such that m-fold bracketing of 𝔦 with itself gives the zero ideal. Thus, Abelian ideals are exactly those with class of nilpotence at most 1.)

ad-Nilpotent ideals were studied by Kostant in [K1], [K2]. He introduced this notion in [K1], where he used it to obtain a useful equivalence criterion for certain decomposably-generated simple G-submodules of Λ(𝔤) in terms of I^{n} (here, G is a compact semi-simple Lie group, and 𝔤 is the complexification of its Lie algebra). Moreover, he gave an explicit description of the eigenspace relative to the maximal

Typeset by AMS-TEX

1991 Mathematics Subject Classification. Primary: 17B20; Secondary: 05A15 05A19 05E15 17B30.

Key words and phrases. ad-nilpotent ideal, Lie algebra, order ideal, Dyck path, Chebyshev polynomial.

†Partially supported by the Austrian Science Foundation FWF, grant P13190-MAT.

eigenvalue of the Laplace-Beltrami operator of G by means of the set of Abelian ideals. This subclass has been studied much more recently in [K2] in connection with discrete series representations. The latter paper was partly motivated by a striking enumerative result due to D. Peterson: the Abelian ideals are 2^{n} in number, independently of the type of 𝔤. (In contrast, as we shall see below, the cardinality of I^{n} depends on the type). Peterson’s result has a twofold interpretation: on the one hand, looking at its proof one gets an encoding of the Abelian ideals by means of elements of the affine Weyl group of 𝔤; on the other hand it is in its own right an elegant enumerative result for ideals whose class of nilpotence is at most 1.

These results lead to ask similar questions for ad-nilpotent ideals in general: find an appropriate encoding and enumerate I^{n} by class of nilpotence. The first problem was settled in [CP], where an encoding in the spirit of Kostant–Peterson was found. The second question, for 𝔤 of type A, was addressed and given a solution in [OP] and [AKOP]. Interestingly, it turns out that in type A ideals in I^{n} with class of nilpotence K are equinumerous with Dyck paths of length 2n + 2 and of height K + 1 (see [AKOP, Theorem 4.4]). In fact, in [AKOP, Sec. 5] an explicit bijection between the former set of ideals and the latter set of Dyck paths is constructed. Several explicit formulas for the number of these ideals can be given (see [OP] and [AKOP, Theorems 4.2, 4.5, 4.6]).

In the present paper we resolve the enumerative question in all other cases. In types B, C, and D we give explicit formulas, expressed in terms of Chebyshev polynomials of the second kind, for the generating function for ideals in I^{n} with class of nilpotence K (see the theorems labelled B, C, and D in Section 2). In types E, F, and G we have computed the number of ideals in I^{n} with a given class of nilpotence in a rather straightforward way, using a computer (see Section 8).

In type C we are again able to establish a relationship between the ideals in I^{n} and certain paths: in Theorem C.3 we prove that in type C ideals in I^{n} with class of nilpotence K are equinumerous with paths with step vectors (1, 1) and (1, −1) of length 2n and height K + 1, which start at the origin and never pass below the x-axis. However, in contrast to type A, we are not able to provide an explicit bijection between this set of ideals and this set of paths. In types B and D we do not even know any relationship between the ideals in I^{n} with class of nilpotence K and any set of paths. It seems indeed that paths are not the right objects to consider in the case of type B and type D. We believe that a fruitful direction for further investigation is to try to relate ideals in I^{n} in any of the types A, B, C, or D to non-crossing partitions of the corresponding types. Indeed, in any of these types, the sequence of numbers of ideals in I^{n} is identical with the sequence of numbers of non-crossing partitions (cf. [Si] and [R, Cor. 10]). We speculate that there is a naturally defined statistics on non-crossing partitions, such that, in any of the types A, B, C, or D, the ideals in I^{n} with class of nilpotence K are equinumerous with the non-crossing partitions (in the set corresponding to I^{n}) on which this statistics has the value K. (In fact, a rougher version of that problem was already stated in Remark 2 of [R]. There, the problem is posed to relate non-crossing partitions to non-nesting partitions. However, non-nesting partitions are by definition antichains of roots, which in turn are in bijection with our ad-nilpotent ideals. See also the final paragraph of the introduction.)

Our paper is organized as follows. In the next section we first summarize the enumerative results from [OP] and [AKOP] in type A, in order to make a comparison with our results in types B, C, and D easy. We state the results in types B, C and D subsequently, also in Section 2. In Section 3 we review the encoding of adnilpotent 𝔟-ideals in type A in terms of subdiagrams of the staircase shape, and the geometric algorithm to determine class of nilpotence in type A given in [AKOP]. This algorithm is of particular importance for our analysis in types B, C, and D. For in Section 4 we show that ad-nilpotent 𝔟-ideals in types B, C, and D can be encoded as certain subdiagrams of a shifted staircase (as done originally by Shi [Sh]), and, for the determination of class of nilpotence, we may in fact apply the type A algorithm to diagrams that are obtained by conjugation operations from the subdiagrams of the shifted staircase. We use these observations to prove the enumerative results of Section 2 in the subsequent sections. Section 5 is devoted to the proofs in type C, Section 6 to the proofs in type B, and Section 7 to the proofs in type D. While the results follow quite easily in type C, types B and D need a more refined (though still elementary) analysis. Finally, our enumerative results in the exceptional types are reported in Section 8.

It is interesting to note that ad-nilpotent ideals in a Borel subalgebra have appeared in various (equivalent) disguises in the literature. In [Sh] Shi studied ⊕-sign types, a relevant notion in the theory of cells for Weyl groups. As he shows these are in bijection with dual order ideals in the set of positive roots, which in turn are in bijection with our ad-nilpotent ideals, as we have argued right at the beginning. In another direction, Postnikov introduced, for any of the classical and exceptional root systems, non-nesting partitions (see [R, Remark 2]). As we already mentioned, these are by definition antichains of positive roots (a definition that translates to a nice combinatorial interpretation in the classical types), which in turn are in bijection with dual order ideals in the set of positive roots, and, hence, again with our ad-nilpotent ideals. Postnikov also showed that non-nesting partitions are in bijection with the regions of the Catalan arrangement contained in the fundamental chamber. These hyperplane arrangements have for instance been investigated by Athanasiadis [A1, A2]. Interestingly, Reiner [R, Remark 2] states a uniform formula for the total number of non-nesting partitions, and, thus, for the total number of ad-nilpotent 𝔟-ideals in a Borel subalgebra for 𝔤 of any type, namely

∏_{i=1}^{n} \frac{h + e_{i} + 1}{e_{i} + 1},  (1.1)

where h is the Coxeter number and e_{1}, e_{2}, …, e_{n} are the exponents of Δ (cf. [Hu; Sec. 3.19]). This formula can easily be checked case by case, by comparing it with the known results on these numbers. For the classical types one may consult [CP, Theorem 3.1] or [Sh, Sec. 3], and for the exceptional types [Sh, Sec. 3], except that the number for E_{6} in [Sh] has to be corrected; see the last paragraph and Table 1 in Section 8. Alternatively, for the classical types, one may consult [A1, Theorem 5.5] (cf. also [A2, Theorem 3.1]). However, it would be desirable to find a uniform proof for this formula.

§2 The enumeration of ad-nilpotent 𝔟-ideals in the classical types

We begin by recalling the enumerative results from [OP] and [AKOP] in type A.

Theorem A.1. [OP], [AKOP, Theorem 4.2] Let 𝔤 be of type A_{n}. Let α_{n}(K) denote the number of ideals in I^{n} with class of nilpotence K. Then

α_{n}(K) = ∑_{0=i_{0}<i_{1}<···<i_{K}<i_{K+1}=n+1} ∏_{j=0}^{K−1} \binom{i_{j+2} − i_{j} − 1}{i_{j+1} − i_{j}}.  (A.1)

The combinatorial meaning of the previous formula has been analysed in detail in [AKOP]. In particular, it connects to the enumeration of Dyck paths. Recall that a Dyck path is a lattice path with diagonal step vectors (1, 1) and (1, −1), which starts at the origin and returns to the x-axis, and which does not pass below the x-axis. We define the height of a Dyck path to be the maximum ordinate of its peaks.

Theorem A.2. [AKOP, Theorem 4.4] For 𝔤 of type A_{n}, the number of ideals in I^{n} with class of nilpotence K is exactly the same as the number of Dyck paths from (0, 0) to (2n + 2, 0) with height K + 1.

Generating function results for Dyck paths translate into the following result for ideals with a given class of nilpotence (cf. the fact quoted in the paragraph before Theorem C.3). Let U_{n}(x) denote the nth Chebyshev polynomial of the second kind, U_{n}(cos t) = sin((n + 1)t)/ sin t, or, explicitly,

U_{n}(x) = ∑_{j≥0}(−1)^{j}\binom{n − j}{j}(2x)^{n−2j}.

In the statement of the following result, and also later in the paper, we write Ũ_{k} as short-hand for U_{k} (1/2\sqrt{x}).

Theorem A.3. [AKOP, Theorem 4.6] Let 𝔤 be of type A_{n}. Let α_{n}^{≤}(h) denote the number of ideals in I^{n} with class of nilpotence at most h. Then

1 + ∑_{n=0}^{∞} α_{n}^{≤}(h)x^{n+1} = \frac{Ũ_{h+1}}{\sqrt{x} Ũ_{h+2}} .  (A.3)

A nice implication of this theorem is that not only is the number of Abelian ideals given by a nice compact formula, namely 2^{n}, but there are also similarly nice formulae for ideals with class of nilpotence at most 2 and 3.

Corollary A.4. [AKOP, Corollary 4.7] For 𝔤 of type A_{n}, the number of ideals in I^{n} with class of nilpotence at most 2 is the Fibonacci number F_{2n}. The number of ideals in I^{n} with class of nilpotence at most 3 is (3^{n} + 1)/2.

Denote by α_{n}(h, K) the number of such ideals with dimension h and class of nilpotence K and set

C_{n}(q, t) = ∑_{h,K≥0} α_{n}(h, K)t^{h}q^{K}.  (A.4)

This notation reflects the fact that C_{n}(1, 1) is the (n + 1)st Catalan number \frac{1}{n+2} \binom{2n+2}{n+1}, so that the polynomials C_{n}(q, t) are (q, t)-analogues of the Catalan numbers^{1}. In [AKOP] we proved the following formula for C_{n}(q, t).

1Although these (q, t)-analogues of the Catalan numbers are different from the (q, t)-Catalan

Theorem A.5. [AKOP, Theorem 6.1] We have

C_{n}(q, t) = ∑_{K=0}^{n} ( ∑_{0=i_{0}<i_{1}<···<i_{K}<i_{K+1}=n+1} ∏_{j=0}^{K−1} t^{i_{j+1}(i_{j+3}−i_{j+2})} [ \begin{matrix}i_{j+2} − i_{j} − 1 \\ i_{j+1} − i_{j} \end{matrix} ]_{t}) q^{K},

(A.5)

with i_{K+2} = n + 2. Here, [ \begin{matrix}m \\ n \end{matrix} ]_{t} is the t-binomial coefficient, defined by

[ \begin{matrix}m \\ n \end{matrix} ]_{t} = \{ \begin{matrix}\frac{(1−t^{m})(1−t^{m−1})···(1−t^{m−n+1})}{(1−t^{n})(1−t^{n−1})···(1−t)} & if m ≥ n > 0, \\ 1 & if n = 0, \\ 0 & in any other case. \end{matrix}

In this paper we provide analogues of Theorems A.1 and A.3 and of Corollary A.4 for 𝔤 of type B, C, and D. We are able to give an analogue of Theorem A.2 only in type C. Finally, again in type C, we also provide an analogue of Theorem A.5, thus obtaining a (q, t)-analogue of the binomial coefficient \binom{2n}{n}. In principle, we would also be able to write down analogues of Theorem A.5 in types B and D. However, the corresponding formulas are rather unwieldy, so that we prefer to omit these for the sake of brevity.

Now we state these results. We begin with the type C analogue of Theorem A.1.

Theorem C.1. Let 𝔤 be of type C_{n}. Let γ_{n}(K) denote the number of ideals in I^{n} with class of nilpotence K. Then

γ_{n}(K) = \{ \begin{matrix}∑_{0<i_{1}<···<i_{k}<i_{k+1}=n} ∏_{j=1}^{k−1} \binom{i_{j+2}−i_{j}−1}{i_{j+1}−i_{j}} · ∑_{ℓ=0}^{i_{2}−i_{1}−1} \binom{i_{1}+i_{2}−1}{ℓ} \\ if K = 2k, \\ ∑_{−i_{2}<i_{1}≤0<i_{2}<···<i_{k}<i_{k+1}=n} 2^{i_{1}+i_{2}−1} ∏_{j=1}^{k−1} \binom{i_{j+2}−i_{j}−1}{i_{j+1}−i_{j}} \\ if K = 2k − 1, \end{matrix}

(C.1)

where i_{K+2} = n + 1.

Next we state the type C analogue of Theorem A.3.

Theorem C.2. Let 𝔤 be of type C_{n}. Let γ_{n}^{≤}(h) be the number of ideals in I^{n} with class of nilpotence at most h. Then the generating function ∑_{n≥0} γ_{n}^{≤}(h)x^{n} is given by

\begin{matrix}∑ & {⌈h/2⌉ \atop i=0} & Ũ_{h+1−2i} \\ \sqrt{x} & Ũ_{h+2} \end{matrix} = \frac{Ũ_{h+1} + Ũ_{h−1} + Ũ_{h−3} + ⋯}{\sqrt{x} Ũ_{h+2}} .  (C.2)

Theorem C.2 allows to relate the ideals of given class of nilpotence again to path combinatorics. Appealing to the fact (see for example [Kr, Theorem A2 + Fact A3]) that the generating function ∑_{P} x^{ℓ(P)/2} for paths P with step vectors (1, 1) and (1, −1), which start at the origin, never exceed height k (the height being again defined as the maximum ordinate of the peaks of the path), and end at height s (ℓ(P) denoting the length of P) is given by

numbers of Garsia and Haiman [GHn], a combinatorial interpretation of the latter (q, t)-Catalan numbers has recently been found by Haglund [H] (the proof [GHd] jointly with Garsia) which is amazingly similar to the combinatorial interpretation of (A.4) of [AKOP] (recalled in Section 2)

in terms of two statistics on Ferrers diagrams contained in a staircase. Whereas the first statistics is (basically) identical in both cases (in [AKOP] it is the area of the diagram, respectively in [H] and [GHd] it is the difference of the area of the staircase and the area of the diagram), the difference in the second statistics can be best described by saying that the statistics of [AKOP] (the number of touching points on the diagonal line that bounds the Ferrers diagram; see Figure 1)

is a descent-like statistics, whereas Haglund’s statistics is the corresponding major-like statistics.

\frac{Ũ_{k−s}}{\sqrt{x} Ũ_{k+1}} ,

we obtain the following type C analogue of Theorem A.2.

Theorem C.3. For 𝔤 of type C_{n}, the number of ideals in I^{n} with class of nilpotence K is exactly the same as the number of paths with step vectors (1, 1) and (1, −1), which start at the origin, have length 2n and height K + 1, and never pass below the x-axis.

As we already said in the introduction, it would be desirable to find an explicit bijection between the ideals and the paths in the above theorem.

This lattice path interpretation of the number of ideals with given class of nilpotence, together with the iterated reflection principle formula for paths that stay between two parallel lines (cf. [Mo, Ch. 1, Th. 2]), implies, upon little simplification, an explicit formula for this number. This formula must be preferred over the one in (C.1), as it is much simpler and computationally superior. (An analogous formula in type A has been given in [AKOP, Eq. (4.6)].)

Corollary C.4. For 𝔤 of type C_{n}, the number of ideals in I^{n} with class of nilpotence at most h is given by

∑_{s=0}^{⌊h/2⌋} ∑_{k∈ℤ} \frac{1 + 2s + 2k(h + 2)}{2n + 1}\binom{2n + 1}{n − s − k(h + 2)}.  (C.4)

Using Theorem C.2 again, it is easy to deduce the following type C analogue of Corollary A.4. There, F_{n} denotes again the nth Fibonacci number.

Corollary C.5. Let 𝔤 be of type C_{n}. Then for n ≥ 1 the number of ideals in I^{n} with class of nilpotence at most 2 is F_{2n}, and the number of ideals with class of nilpotence at most 3 is 2 · 3^{n−1}.

Finally, we state the type C analogue of Theorem A.5.

Theorem C.6. Let 𝔤 be of type C_{n}. Let γ_{n}(h, K) be the number of ideals in I^{n} with dimension h and class of nilpotence K. Then

∑_{h,K≥0} γ_{n}(h, K)t^{h}q^{K} = ∑_{k=0}^{n} ∑_{−i_{2}<i_{1}<i_{2}<···<i_{k}<i_{k+1}=n} q^{2k−χ(i_{1}≤0)}

· ( ∏_{j=1}^{k−1} t^{(i_{j+1}+n)(i_{j+3}−i_{j+2})} [ \begin{matrix}i_{j+2} − i_{j} − 1 \\ i_{j+1} − i_{j} \end{matrix} ]_{t}

· ∑_{ℓ=0}^{i_{2}−i_{1}−1} [ \begin{matrix}i_{1} + i_{2} − 1 \\ ℓ \end{matrix} ]_{t} t^{(i_{1}+n)(i_{3}−i_{2})−\binom{n−i_{2}+1}{2}+\binom{ℓ+1}{2}}), (C.6)

where i_{k+2} = n + 1, [ \begin{matrix}m \\ n \end{matrix} ]_{t} is the t-binomial coefficient defined in the statement of Theorem A.5, and χ is the usual truth function, χ(A) = 1 if A is true and χ(A) = 0 otherwise.

Clearly, since in type C the total number of ideals in I^{n} is the central binomial coefficient \binom{2n}{n} (see Corollary 4.2), the expression in (C.6) is a (q, t)-analogue of \binom{2n}{n}.

Remark. The innermost sum in (C.6) does in fact simplify if i_{1} ≤ 0. For, by means of the q-binomial theorem (cf. [GR, Ex. 1.2(vi)]), we have

∑_{ℓ=0}^{i_{2}−i_{1}−1} [ \begin{matrix}i_{1} + i_{2} − 1 \\ ℓ \end{matrix} ]_{t} t^{\binom{ℓ+1}{2}} = ∑_{ℓ=0}^{∞} [ \begin{matrix}i_{1} + i_{2} − 1 \\ ℓ \end{matrix} ]_{t} t^{\binom{ℓ+1}{2}}
= (1 + t)(1 + t^{2}) ⋯ (1 + t^{i_{1}+i_{2}−1}).  (C.7)

In view of this, it is now obvious that the special case t = 1 of Theorem C.6 implies Theorem C.1.

Now we state our results for type B.

Theorem B.1. Let 𝔤 be of type B_{n}. Let β_{n}(K) denote the number of ideals in I^{n} with class of nilpotence K. Then the generating function ∑_{n≥0} β_{n}(K)x^{n} is given by

\{ \begin{matrix}\frac{Ũ_{2k} + Ũ_{k}Ũ_{k+1}Ũ_{2k−1}}{\sqrt{x} Ũ_{2k}Ũ_{2k+1}Ũ_{2k+2}} & if K = 2k, \\ \frac{Ũ_{2k} + Ũ_{k+1}^{2}Ũ_{2k−2} + Ũ_{k−1}^{2}Ũ_{2k−2} + Ũ_{2}}{Ũ_{2k−1}Ũ_{2k}Ũ_{2k+1}} & if K = 2k − 1. \end{matrix}  (B.1)

Analogues of formula (A.1) for type B will be established in the proof of the above theorem in Section 6. An easy consequence of Theorem B.1 is the following type B analogue of Theorem A.3. It can be proved by a straightforward induction on h.

Corollary B.2. Let 𝔤 be of type B_{n}. Let β_{n}^{≤}(h) denote the number of ideals in I^{n} with class of nilpotence at most h. Then the generating function ∑_{n≥0} β_{n}^{≤}(h)x^{n} is given by

\frac{∑_{i=1}^{h/2} (2 i + 1) Ũ_{2i−1} + (h + 1)Ũ_{h+1} + ∑_{i=1}^{h/2} 2i Ũ_{2h+3−2i}}{Ũ_{h+1} Ũ_{h+2}}  if h is even,

\frac{∑_{i=1}^{(h−1)/2} (2 i + 1) Ũ_{2i−1} + (h + 1)Ũ_{h} + ∑_{i=1}^{(h+1)/2} 2i Ũ_{2h+3−2i}}{Ũ_{h+1} Ũ_{h+2}} if h is odd.

(B.2)

The above theorem implies a type B analogue of Corollary A.4.

Corollary B.3. Let 𝔤 be of type B_{n}. Then for n ≥ 1 the number of ideals in I^{n} with class of nilpotence at most 2 is F_{2n} + F_{2n−2} − 2^{n−1}, and the number of ideals with class of nilpotence at most 3 is \frac{1}{2}(5 · 3^{n−1} + 1) − F_{2n−2}.

Next we state our results for type D.

Theorem D.1. Let 𝔤 be of type D_{n}. Let δ_{n}(K) denote the number of ideals in I^{n} with class of nilpotence K. Then the generating function ∑_{n≥0} δ_{n}(K)x^{n} is given by x/(1 − x) if K = 0, and otherwise by

\{ \frac{\sqrt{x} (2 − Ũ_{2k} + 2Ũ_{2k+2} + 3Ũ_{k}Ũ_{k+1}Ũ_{2k−1})}{Ũ_{2k}Ũ_{2k+1}Ũ_{2k+2}}  if K = 2k, ; \frac{x (2Ũ_{2k+2} + Ũ_{2k} − 3Ũ_{2k−2} + Ũ_{k}Ũ_{k+1}Ũ_{2k−1} + 4Ũ_{k}^{2}Ũ_{2k−2} + Ũ_{k−1}Ũ_{k}Ũ_{2k−3} + 2)}{Ũ_{2k−1}Ũ_{2k}Ũ_{2k+1}} ; if K = 2k − 1.

(D.1)

Analogues of formula (A.1) for type D will be established in the proof of the above theorem in Section 7. An easy consequence of Theorem D.1 is the following type D analogue of Theorem A.3. Once again, it can be proved by a straightforward induction on h.

Corollary D.2. Let 𝔤 be of type D_{n}. Let δ_{n}^{≤}(h) denote the number of ideals in I^{n} with class of nilpotence at most h. Then the generating function ∑_{n≥0} δ_{n}^{≤}(h)x^{n} is given by x/(1 − x) if h = 0, by x(1 + 2x)/(1 − 2x) if h = 1, and otherwise by

\{ \begin{matrix}x(6Ũ_{1} + ∑_{i=1}^{(h−2)/2} (6i + 8) Ũ_{2i+1} + (3h + 4)Ũ_{h+1} \\ + ∑_{i=0}^{(h−2)/2}(6i + 5) Ũ_{2h+1−2i} + Ũ_{2h+3})/Ũ_{h+1}Ũ_{h+2} & if h is even, \\ x(6Ũ_{1} + ∑_{i=1}^{(h−3)/2} (6i + 8) Ũ_{2i+1} + (3h + 4)Ũ_{h} \\ + ∑_{i=0}^{(h−1)/2}(6i + 5) Ũ_{2h+1−2i} + Ũ_{2h+3})/Ũ_{h+1}Ũ_{h+2} & if h is odd. \end{matrix} (D.2)

The above theorem implies a type D analogue of Corollary A.4.

Corollary D.3. Let 𝔤 be of type D_{n}. Then for n ≥ 2 the number of ideals in I^{n} with class of nilpotence at most 2 is 5F_{2n−3} − 2^{n−2}, and the number of ideals with class of nilpotence at most 3 is \frac{13}{2} · 3^{n−2} − \frac{3}{2} + 4F_{2n} − 7F_{2n−1}.

§3 ad-nilpotent ideals in type A

Throughout this section, 𝔤 will be of type A_{n}, i.e., 𝔤 is the Lie algebra sl(n+1, ℂ) of (n + 1) × (n + 1) traceless matrices. In the following paragraphs we collect the findings from [AKOP] on how to encode ideals in I^{n} and how to efficiently compute their class of nilpotence.

Recall from the introduction that an ideal 𝔦 is in I^{n}, i.e., is an ad-nilpotent ideal of our fixed Borel subalgebra 𝔟, if and only if 𝔦 can be written as 𝔦 = ⨁_{α∈Φ} 𝔤_{α}, where

Φ ⊆ Δ^{+} is a dual order ideal in the poset (Δ^{+}, ≤) of positive roots. This allows us to represent ad-nilpotent ideals conveniently in a geometric fashion, which will be crucial in all subsequent considerations. Clearly, any positive root in A_{n} can be written as a sum of simple roots. Explicitly, with α_{1}, α_{2}, …, α_{n} denoting the simple roots, let us write τ_{i,j} = α_{i} + ⋯ + α_{n−j+1}, 1 ≤ i ≤ n, 1 ≤ j ≤ n − i + 1. If we place the roots τ_{i,j}, j = 1, 2, …, n − i + 1, in the ith row of a diagram, then this defines an arrangement of the positive roots in a staircase fashion. For example, for A_{3} we obtain the arrangement

α_{1} + α_{2} + α_{3} α_{1} + α_{2} α_{1}
α_{2} + α_{3}  α_{2}
α_{3}

Obviously, the above defines an identification of the positive roots with the cells of the staircase diagram (n, n − 1, …, 1) (for all partition notation we refer the reader to [Ma, Ch. I, Sec. 1]), in which the root τ_{i,j} is identified with the cell (i, j). Given an ad-nilpotent ideal 𝔦, written as 𝔦 = ⨁_{α∈Φ} 𝔤_{α}, for some collection Φ of positive

roots, we can use the above identification to represent 𝔦 as the set of cells that corresponds to the roots in Φ. Since, as we noted above, 𝔦 is a dual order ideal, the set of cells obtained forms a (Ferrers diagram of a) partition. For example, the ideal 𝔤_{α_{1}+α_{2}+α_{3}} ⊕ 𝔤_{α_{1}+α_{2}} ⊕ 𝔤_{α_{1}} ⊕ 𝔤_{α_{2}+α_{3}} corresponds to the partition (3, 1, 0). This defines a bijection between ad-nilpotent ideals in sl(n + 1, ℂ) and subdiagrams of (n, n − 1, …, 1).

Now, let m_{i,j} be the maximal number m such that the root space 𝔤_{τ_{i,j}} occurs in

𝔦^{m} := \underbrace{[⋯ [𝔦, 𝔦], …]}_{m occurrences of 𝔦} .

Alternatively, m_{i,j} is the maximal number m such that there is a decomposition of τ_{i,j} of the form τ_{i,j} = β_{1} + ⋯ + β_{m} with all β_{i}’s in Φ, the collection of positive roots corresponding to 𝔦. Written succinctly,

m_{i,j} = max{m : τ_{i,j} = β_{1} + ⋯ + β_{m}, for some β_{l} ∈ Φ, l = 1, 2, …, m}. (3.1)

By definition, the class of nilpotence of 𝔦 is given by max_{i,j} m_{i,j}. We claim, first, that the numbers m_{i,j} satisfy m_{i,j} ≥ m_{i+1,j} and m_{i,j} ≥ m_{i,j+1} for all i and j, and, second, that the m_{i,j} (hence, in particular, the class of nilpotence) can be obtained by the following algorithm (see [AKOP, Prop. 3.1]): let λ be the subdiagram of (n, n − 1, …, 1) that corresponds to 𝔦 according to the identification explained above. Define a filling (t_{i,j})_{1≤i≤n, 1≤j≤n−i+1} of the cells of (n, n−1, …, 1) by recursively setting

t_{i,j} = \{ \begin{matrix}0 & if (i, j) ∉ λ, \\ 1 & if (i, j) is a corner cell of λ, \\ max_{j<k≤n−i+1} \{t_{i,k} + t_{n−k+2,j}\} & otherwise. \end{matrix}  (3.2)

It is easy to see that the above rule uniquely defines a filling of the shape (n, n − 1, …, 1), whose nonzero entries are precisely those corresponding to the cells of λ. E.g., when n = 4, the fillings corresponding to (2, 1, 0, 0), (3, 3, 2, 1), (4, 3, 2, 1) are respectively

1 1 0 0 1 0 0 0 0 0  3 2 1 0 3 2 1 2 1 1  4 3 2 1 3 2 1 2 1 1

The claim is that m_{i,j} = t_{i,j} for all i and j.

Let us go through a proof of these two facts that allows to be “recycled” when we shall discuss the computation of class of nilpotence in types B, C, and D. For proving the first claim, suppose that there is a representation of τ_{i+1,j} as

τ_{i+1,j} = β_{1} + ⋯ + β_{m},  (3.3)

with all β_{l}’s in Φ. Then one of the β_{l}’s must be equal to some τ_{i+1,k}, because otherwise the sum on the right-hand side of (3.3), when expanded as a sum of simple roots, would either contain no α_{i+1} or in addition to α_{i+1} also α_{i}. Without loss of generality let β_{1} = τ_{i+1,k}. Then

τ_{i,j} = α_{i} + τ_{i+1,j} = α_{i} + β_{1} + ⋯ + β_{m} = τ_{i,k} + β_{2} + ⋯ + β_{m}.

Hence we have m_{i,j} ≥ m_{i+1,j}. A similar argument proves m_{i,j} ≥ m_{i,j+1}.

For proving the second claim, we do a reverse induction on i + j. The claim is obvious if i + j = n + 1. For the induction step, assume that the claim is right for all (i′, j′) with i′ + j′ > i + j. Now consider the root τ_{i,j}. If j < k ≤ n − i + 1, then we can write it as

τ_{i,j} = τ_{i,k} + τ_{n−k+2,j}.  (3.4)

By the induction hypothesis and (3.1) we know that

t_{i,k} = max{m : τ_{i,k} = β_{1} + ⋯ + β_{m}, for some β_{l} ∈ Φ, l = 1, 2, …, m}

and

t_{n−k+2,j} = max{m : τ_{n−k+2,j} = β_{1} + ⋯ + β_{m}, for some β_{l} ∈ Φ, l = 1, 2, …, m}

(3.5)

Combined with (3.4), this implies that

t_{i,k} + t_{n−k+2,j} ≤ m_{i,j}  (3.6)

as long as t_{i,k} and t_{n−k+2,j} are nonzero. But (3.6) is also true if one or both of t_{i,k} and t_{n−k+2,j} should be zero. If both are zero, then there is nothing to prove. If, for instance, t_{i,k} is nonzero, then we have t_{i,k} = m_{i,k} ≤ m_{i,j}, the equality being true by induction hypothesis, the inequality being true because of our first claim.

But m_{i,j} cannot be larger than t_{i,k} + t_{n−k+2,j}. For, as we already noted (just replace i by i − 1 in the previous argument), in any decomposition

τ_{i,j} = β_{1} + ⋯ + β_{m},  (3.7)

with the β_{l}’s being positive roots, one of the β_{l}’s must be equal to some τ_{i,k}. Again, without loss of generality let β_{1} = τ_{i,k}. Then we have β_{2} + ⋯ + β_{m} = τ_{i,j} − τ_{i,k} = τ_{n−k+2,j}. By induction hypothesis we have (3.5), and hence m − 1 ≤ t_{n−k+2,j}. It follows that m ≤ 1 + t_{n−k+2,j} ≤ t_{i,k} + t_{n−k+2,j} ≤ t_{i,j}, as required.

Since m_{i,j} ≥ m_{i+1,j} and m_{i,j} ≥ m_{i,j+1}, the same property must be true for the t_{i,j}’s. In particular, this implies that the maximum of all entries is entry t_{1,1}, so that in fact the class of nilpotence of an ideal 𝔦 is equal to t_{1,1}.

If one is only interested to quickly compute the class of nilpotence of some ideal 𝔦 (i.e., just the entry t_{1,1} of the filling constructed by (3.2)), then there is a short-cut through the algorithm (3.2). For a convenient statement of the result, we write, in abuse of notation, n(λ_{1}, λ_{2}, …, λ_{n}) for n(𝔦), given that the partition corresponding to 𝔦 is (λ_{1}, λ_{2}, …, λ_{n}).

Proposition 2.1. [AKOP, Prop. 3.2] Let 𝔦 ∈ I^{n} and let λ = (λ_{1}, …, λ_{n}) be the corresponding partition. We have n(0, 0, …, 0) = 0, and otherwise

n(λ_{1}, λ_{2}, …, λ_{n}) = n(λ_{n+2−λ_{1}}, …, λ_{n}) + 1.  (3.8)

It should be noted that on the left-hand side of (3.8) there appears the class of nilpotence of an ideal in I^{n}, whereas on the right-hand side there appears the class of nilpotence of an ideal in I^{λ_{1}−1} (with corresponding partition (λ_{n+2−λ_{1}}, …, λ_{n})). The computation, however, can be carried out completely formally, without reference to ideals, as we now demonstrate by an example.

Example. Let 𝔦 ∈ I^{13} be the ideal which corresponds to the partition (10, 10, 9, 6, 5, 4, 4, 3, 1, 1, 1, 1, 0). (This is the partition in Figure 1. At this point, all dotted lines should be ignored.) Then, by applying Proposition 2.1 iteratively, we obtain for the class of nilpotence of 𝔦:

n(𝔦) = n(10, 10, 9, 6, 5, 4, 4, 3, 1, 1, 1, 1, 0)
= n(5, 4, 4, 3, 1, 1, 1, 1, 0) + 1
= n(1, 1, 1, 0) + 2
= 3.

As is obvious from the example, iterated application of Proposition 2.1 provides a very efficient algorithm for the determination of the class of nilpotence of a given ideal 𝔦.

Since it will be essential subsequently, we wish to point out that this algorithm has a very nice geometric rendering. Let, as before, λ = (λ_{1}, λ_{2}, …, λ_{n}) be the partition corresponding to 𝔦. Consider the Ferrers diagram of λ. As it is contained in the staircase diagram (n, n − 1, …, 1), it must not cross the antidiagonal line x + y = n + 1. We draw a broken ray as follows (see Figure 1, where n = 13 and λ = (10, 10, 9, 6, 5, 4, 4, 3, 1, 1, 1, 1, 0)): we start on the vertical edge on the right of cell (1, λ_{1}), and move downward until we touch the antidiagonal x + y = n + 1. At the touching point we turn direction from vertical-down to horizontal-left, and move on until we touch a vertical part of the Ferrers diagram. At the touching point we turn direction from horizontal-left to vertical-down. Now the procedure is iterated, until we reach the line x = 0. The class of nilpotence of the ideal 𝔦 is equal to the number of touching points on x + y = n + 1. In Figure 1, the resulting broken ray is the dotted line outside the Ferrers diagram of (10, 10, 9, 6, 5, 4, 4, 3, 1, 1, 1, 1, 0). There are three touching points on x + y = n + 1 = 14, in accordance with n(10, 10, 9, 6, 5, 4, 4, 3, 1, 1, 1, 1, 0) = 3, as we computed just above.

§4 ad-nilpotent ideals in types B, C, D

Let 𝔤 be a complex simple Lie algebra of type B_{n}, C_{n}, or D_{n}. In this section we describe a diagrammatic encoding of the positive roots (cf. [Sh, Sec. 2–3]) similar to the one introduced in the previous section for type A. Throughout this section, for any of the three types, the vectors α_{1}, α_{2}, …, α_{n} denote a basis of simple roots corresponding to the chosen system Δ^{+} of positive roots.

We arrange the positive roots in a shifted staircase diagram of shape (2n−1, 2n− 3, …, 1) for B_{n} and C_{n}, and of shape (2n − 2, 2n − 4, …, 2) for D_{n} as follows.

[figure: .....; . . . . .; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; x + y = n + 1; •; •; •]_{Figure 1}

If 𝔤 is of type C_{n}, then we associate the cell (i, j), 1 ≤ i ≤ j ≤ 2n − i, with the positive root

\{ \begin{matrix}α_{i} + ⋯ + α_{j−1} + 2(α_{j} + ⋯ + α_{n−1}) + α_{n} & if j ≤ n − 1, \\ α_{i} + ⋯ + α_{2n−j} & if n ≤ j ≤ 2n − i. \end{matrix} (4.C)

For example, if n = 3, this defines the following arrangement of positive roots,

2α_{1} + 2α_{2} + α_{3}  α_{1} + 2α_{2} + α_{3}  α_{1} + α_{2} + α_{3}  α_{1} + α_{2}  α_{1}
2α_{2} + α_{3}  α_{2} + α_{3}  α_{2}
α_{3}

Likewise, if 𝔤 is of type B_{n}, then we associate the cell (i, j), 1 ≤ i ≤ j ≤ 2n − i, with the positive root

\{ \begin{matrix}α_{i} + ⋯ + α_{j} + 2(α_{j+1} + ⋯ + α_{n}) & if j ≤ n − 1, \\ α_{i} + ⋯ + α_{2n−j} & if n ≤ j ≤ 2n − i. \end{matrix}  (4.B)

For example, if n = 3, this defines the following arrangement of positive roots,

α_{1} + 2α_{2} + 2α_{3}  α_{1} + α_{2} + 2α_{3}  α_{1} + α_{2} + α_{3}  α_{1} + α_{2}  α_{1}
α_{2} + 2α_{3}  α_{2} + α_{3}  α_{2}
α_{3}

If 𝔤 is of type D_{n}, then we associate the cell (i, j), 1 ≤ i ≤ j ≤ 2n − 1 − i, with the positive root

\{ \begin{matrix}α_{i} + ⋯ + α_{j} + 2(α_{j+1} + ⋯ + α_{n−2}) + α_{n−1} + α_{n} & if j ≤ n − 2, \\ α_{i} + ⋯ + α_{n−2} + α_{n} & if j = n − 1, \\ α_{i} + ⋯ + α_{2n−j−1} & if n ≤ j ≤ 2n − 1 − i. \end{matrix}

(4.D)

For example, if n = 4, this defines the following arrangement of positive roots,

α_{1} + 2α_{2} + α_{3} + α_{4} α_{1} + α_{2} + α_{3} + α_{4} α_{1} + α_{2} + α_{4} α_{1} + α_{2} + α_{3} α_{1} + α_{2} α_{1}
α_{2} + α_{3} + α_{4} α_{2} + α_{4} α_{2} + α_{3} α_{2}
α_{4}  α_{3}

Now, for 𝔤 of any of the types B_{n}, C_{n}, or D_{n}, we again associate an ideal 𝔦 ∈ I^{n}, represented as before as 𝔦 = ⨁_{α∈Φ} 𝔤_{α}, to a collection of cells, via the above

identification of positive roots with cells. The resulting collections are characterized in the proposition below. For convenience, if A ⊆ (2n − 2, 2n − 4, …, 2) is a collection of cells, we denote by A^{•} the collection of cells obtained from A by switching columns n−1 and n. By a subdiagram of some shifted staircase we mean as usual a collection of cells contained in the shifted staircase which forms a shifted Ferrers diagram (cf. [Ma, Ch. I, Sec. 1, Ex. 9]).

Proposition 4.1. If 𝔤 is of type B_{n} or C_{n}, then, under the above identification, the ideals in I^{n} correspond bijectively to the subdiagrams of (2n − 1, 2n − 3, …, 1). If 𝔤 is of type D_{n}, the above identification defines a bijection between the ideals in I^{n} and the collection of cells A ⊆ (2n − 2, 2n − 4, …, 2) such that either A or A^{•} is a subdiagram of (2n − 2, 2n − 4, …, 2).

Corollary 4.2. The number of ideals in I^{n} is given by

|I^{n}| = \{ \begin{matrix}\binom{2n}{n} & if 𝔤 is of type B_{n} or C_{n}, \\ \binom{2n}{n} − \binom{2n−2}{n−1} & if 𝔤 is of type D_{n}. \end{matrix}

Remark. The previous proposition and corollary are also proved in [CP, Theorem 3.1] and, in equivalent form, in [Sh, Sec. 3]. In particular, Corollary 4.2 follows almost immediately from Proposition 4.1. The most direct argument (which is different from the ones in [CP] and [Sh]) is as follows: for 𝔤 of type B_{n} or C_{n}, one maps the shifted subdiagrams of (2n − 1, 2n − 3, …, 1) to lattice paths which start at the origin, never pass below the x-axis, and consist of 2n steps, the steps being up-steps (1, 1) and down-steps (1, −1), as before. The correspondence between subdiagrams and paths is best explained with an example at hand. Let n = 9 and consider the shifted partition (16, 13, 11, 8, 7, 5, 3), see Figure 4. Rotate the figure by 45^{∘} in the positive direction and then flip it across a vertical line. Then the zig-zag line which forms the (right) border of the shifted partition becomes such a lattice path (on attaching a few up-steps at the beginning of the path). See Figure 2 for the path corresponding to the partition of Figure 4. (To make a comparison easy, the steps which correspond to thick segments in Figure 4 are also made thick in Figure 2.)

[figure: •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •; •]_{Figure 2}

Now one can resort to the well-known result (see e.g. [F, Theorem 1, (4.6) in Ch. III, Sec.4]) that the number of these lattice paths is \binom{2n}{n}. The result for type D_{n} is then an easy consequence. (A very different proof can be found in [CP], where an enumeration result for trapezoidal plane partitions due to Proctor is used.) It can be checked that in all cases the formulas are in accordance with formula (1.1).

We explain now in a rough form how to calculate the class of nilpotence by means of the algorithm used in Section 3 for type A. For convenience, let us write S_{2n−1} for the shifted staircase (2n−1, 2n−3, …, 1), S_{2n−2} for the shifted staircase (2n − 2, 2n − 4, …, 2), and T_{N} for the ordinary staircase diagram (N, N − 1, …, 1). The general procedure will be as follows: we embed the shifted staircase S_{N}, where N = 2n − 1 or N = 2n − 2, respectively, into the ordinary staircase T_{N}, in such a way that to any collection of cells A ⊆ S_{N} corresponds uniquely a collection of cells Ã ⊆ T_{N}. We shall eventually prove in Proposition 4.3 that the class of nilpotence of an ideal 𝔦 ∈ I^{n} can be obtained by applying the algorithm of Section 3 to Ã, respectively to Ã^{•} in the case of type D_{n} if A is not a diagram, where A is the collection of cells correponding to 𝔦 in the way that was explained earlier.

To explain this in detail, let first 𝔤 be of type C_{n}. We fill the ordinary staircase T_{2n−1} with the positive roots of C_{n} by associating the cells of the “upper half” of T_{2n−1} with positive roots as in (4.C), and by associating the cells in the “lower half” of T_{2n−1} in a symmetric fashion in such a way that cell (i, j) gets associated with the same root as cell (j, i). (I.e., the line formed by the cells (i, i), i = 1, 2, …, n, constitutes a symmetry axis of this arrangement of the positive roots.) Thus, if n = 3, this defines the arrangement

2α_{1} + 2α_{2} + α_{3}  α_{1} + 2α_{2} + α_{3}  α_{1} + α_{2} + α_{3}  α_{1} + α_{2}  α_{1}
α_{1} + 2α_{2} + α_{3}  2α_{2} + α_{3}  α_{2} + α_{3}  α_{2}
α_{1} + α_{2} + α_{3}  α_{2} + α_{3}  α_{3}
α_{1} + α_{2}  α_{2}
α_{1}

Now, given an ideal 𝔦 ∈ I^{n}, written as 𝔦 = ⨁_{α∈Φ} 𝔤_{α}, with associated collection of cells

A ⊆ S_{2n−1} (which is in fact of the form of a shifted Ferrers diagram), we let Ã be the collection of all cells in T_{2n−1} that contain a root α ∈ Φ. Phrased differently, Ã is the union of A with its mirror image about the symmetry axis. Hence, Ã forms a self-conjugate (cf. [Ma, Ch. I, Sec. 1]) partition. For example, according to this description, the ideal 𝔤_{2α_{1}+2α_{2}+α_{3}} +𝔤_{α_{1}+2α_{2}+α_{3}} +𝔤_{α_{1}+α_{2}+α_{3}} +𝔤_{2α_{2}+α_{3}} corresponds to the self-conjugate partition (3, 2, 1).

If 𝔤 is of type B_{n}, we fill the ordinary staircase T_{2n−1} with elements of the root lattice of B_{n} by associating the cells of the “upper half” of T_{2n−1} with positive roots as in (4.B), by associating the cell (i, i − 1) with the element of the root lattice 2α_{i} + 2α_{i+1} + ⋯ + 2α_{n} (this element is not a root!), i = 2, 3, …, n, and by associating the cells in the “lower half” of T_{2n−1} in a symmetric fashion in such a way that cell (i, j) gets associated with the same root as cell (j + 1, i − 1). (I.e., here the line formed by the cells (i, i − 1), i = 2, 3, …, n, constitutes a symmetry axis of this arrangement of elements of the root lattice.) Thus, if n = 3, this defines the arrangement

α_{1} + 2α_{2} + 2α_{3}  α_{1} + α_{2} + 2α_{3}  α_{1} + α_{2} + α_{3}  α_{1} + α_{2}  α_{1}
2α_{2} + 2α_{3}  α_{2} + 2α_{3}  α_{2} + α_{3}  α_{2}
α_{2} + 2α_{3}  2α_{3}  α_{3}
α_{2} + α_{3}  α_{3}
α_{2}

Here, given an ideal 𝔦 ∈ I^{n}, written as 𝔦 = ⨁_{α∈Φ} 𝔤_{α}, with associated collection of cells

A ⊆ S_{2n−1} (which is in fact of the form of a shifted Ferrers diagram), we let Ã be the collection of all cells in T_{2n−1} that contain a root α ∈ Φ, together with all cells (i, i− 1) that are in the same row as some cell of A. Phrased differently, Ã is the union of A with its mirror image about the line formed by the cells (i, i − 1), i = 2, 3, …, n, including the cells on that line which are necessary to “fill the holes”. Hence, if we write Ã as a partition λ = (λ_{1}, λ_{2}, …, λ_{2n−1}), then (λ_{2}, …, λ_{2n−1}) is a selfconjugate partition. Moreover, we have λ_{i} ≠ i−1 for i ≥ 2. For example, according to this description, the ideal 𝔤_{α_{1}+2α_{2}+2α_{3}} + 𝔤_{α_{1}+α_{2}+2α_{3}} + 𝔤_{α_{2}+2α_{3}} corresponds to the partition (2, 2, 1).

Finally, if 𝔤 is of type D_{n}, we fill the ordinary staircase T_{2n−2} with elements of the root lattice of D_{n} in a similar fashion as in type B. To be precise, we associate the cells of the “upper half” of T_{2n−2} with positive roots as in (4.D), we associate cell (i, i−1) with the element of the root lattice 2α_{i}+2α_{i+1}+⋯+2α_{n−2}+α_{n−1}+α_{n} (this element is not a root!), i = 2, 3, …, n − 1, and we associate the cells in the “lower half” of T_{2n−1} in a symmetric fashion in such a way that cell (i, j) gets associated with the same root as cell (j + 1, i − 1). (I.e., the line formed by the cells (i, i − 1), i = 2, 3, …, n − 1, constitutes again a symmetry axis of this arrangement of elements of the root lattice.) In particular, we do not associate the cell (n, n − 1) with any element of the root lattice. Thus, if n = 4, this defines the arrangement (there, a cross indicates the cell that is not associated to anything)

α_{1} + 2α_{2} + α_{3} + α_{4} α_{1} + α_{2} + α_{3} + α_{4} α_{1} + α_{2} + α_{4} α_{1} + α_{2} + α_{3} α_{1} + α_{2} α_{1}
2α_{2} + α_{3} + α_{4}  α_{2} + α_{3} + α_{4}  α_{2} + α_{4}  α_{2} + α_{3}  α_{2}
α_{2} + α_{3} + α_{4}  α_{3} + α_{4}  α_{4}  α_{3}
α_{2} + α_{4}  α_{4}  ×
α_{2} + α_{3}  α_{3}

α_{2}

Given an ideal 𝔦 ∈ I^{n}, with associated collection of cells A ⊆ S_{2n−2}, we form Ã in the same way as in type B_{n}, i.e., Ã is the union of A with its mirror image about the line formed by the cells (i, i − 1), i = 2, 3, …, n − 1, including the cells on that line which are in the same row as some cell in A. Should A not be a shifted diagram (and, hence, Ã not be an ordinary diagram), then we shall interchange the (n−1)st and the nth column and the nth and (n+1)st row, and call the resulting (ordinary) diagram Ã^{•}. Whichever of Ã or Ã^{•} is a diagram, it is contained in T_{2n−2}, and when written as a partition (λ_{1}, λ_{2}, …, λ_{2n−2}), it has the property that (λ_{2}, …, λ_{2n−2}) is a self-conjugate partition and that λ_{i} ≠ i − 1 for i ≥ 2. For example, according to this description, the set Ã^{•} corresponding to the ideal

𝔤_{α_{1}+2α_{2}+α_{3}+α_{4}} + 𝔤_{α_{1}+α_{2}+α_{3}+α_{4}} + 𝔤_{α_{1}+α_{2}+α_{4}} + 𝔤_{α_{1}+α_{2}+α_{3}} + 𝔤_{α_{2}+α_{3}+α_{4}} + 𝔤_{α_{2}+α_{3}} is the partition (4, 3, 1, 1).

After “having set the scene,” we come to the crucial result of this section. As promised earlier, it allows to compute the class of nilpotence of ideals in any of the types B_{n}, C_{n}, or D_{n}, by applying the algorithm of Section 3 to a suitable partition.

Proposition 4.3. Let 𝔤 be of type B_{n}, C_{n}, or D_{n}. Let 𝔦 be in I^{n}, and let λ be the corresponding ordinary partition according to the case-by-case description given above, i.e., either it stands for Ã or for Ã^{•}, and, in addition, λ = (λ_{1}, λ_{2}, …) is self-conjugate in the case of type C_{n}, whereas in types B_{n} and D_{n} it is (λ_{2}, …) which is self-conjugate. Then n(𝔦) = n(λ), where n(λ) stands for the result of the algorithm of Proposition 2.1.

Remark. We should clarify what is meant by “the result of the algorithm of Proposition 2.1.” For type B_{n} and type C_{n} the partition λ is contained in T_{2n−1}. Accordingly we write λ = (λ_{1}, …, λ_{2n−1}) (i.e., with 2n − 1 components) and apply (3.8) with n replaced by 2n − 1, and then iterate. For type D_{n}, however, the partition λ is contained in T_{2n−2}. Thus in this case we write λ = (λ_{1}, …, λ_{2n−2}) (i.e., with 2n − 2 components) and apply (3.8) with n replaced by 2n − 2, and then iterate. In the geometric rendering of the algorithm (see Figure 1), the touching points of the broken ray are on the line x + y = 2n for type B_{n} and C_{n}, and on x + y = 2n − 1 for type D_{n}.

Proof. Construct the filling (t_{i,j}) corresponding to λ according to the algorithm (3.2). Let τ_{i,j} denote the element of the root lattice associated with cell (i, j).

Now we copy the arguments of Section 3 that show that t_{i,j} is equal to the right-hand side of (3.1). Since (3.4) is also valid for our τ_{i,j}’s (regardless whether we consider B_{n}, C_{n}, or D_{n}!), everything runs through smoothly, with just two modifications. First, the collection of roots Φ (which define the ideal 𝔦) must be replaced by the elements of the root lattice contained in the cells of A. (The reader must recall that in types B_{n} and D_{n} this includes some nonroots.) Second, if i > j, the argument that in any decomposition (3.3) of τ_{i+1,j}, with the β_{l}’s being positive roots, one of the β_{l}’s must be equal to some τ_{i+1,k}, must now be modified by maintaining that one of the β_{l}’s must be equal to some τ_{h,j} (because, otherwise, the sum on the right-hand side of (3.3) would either contain no α_{j+1} or in addition to α_{j+1} also α_{j}). Clearly, if i = j, then a decomposition (3.3) of τ_{i+1,j} must contain some τ_{i+1,k} as well as some τ_{h,j}. An analogous modification has to be made for the argument following the decomposition (3.7) of τ_{i,j}.

Let us denote the set of the elements of the root lattice contained in the cells of A by Φ′. Since we had to replace Φ by Φ′, the conclusion of the above arguments is that t_{i,j} is equal to

max{m : τ_{i,j} = β_{1} + ⋯ + β_{m}, for some β_{l} ∈ Φ′, l = 1, 2, …, m}.

This expression differs from the right-hand side of (3.1) by a replacement of Φ by Φ′. This indeed makes a difference if the type that we consider is B_{n} or D_{n}. However, our goal is actually to prove that it equals exactly the right-hand side of (3.1) as long as i ≠ j +1, for any of the types B_{n}, C_{n}, or D_{n}. To see that this is true in type B_{n}, let in a decomposition (3.7) of τ_{i,j}, where all the β_{l}’s are in Φ′, one of the β_{l}’s be equal to a nonroot, to τ_{r,r−1} = 2α_{r} + 2α_{r+1} + ⋯ + 2α_{n} say. Then some other β_{l} must be necessarily equal to some τ_{s,2n−r+1} = α_{s} +α_{s+1} +⋯+α_{r−1}. Then we can replace these two elements by τ_{r,r} and τ_{s,2n−r}, both of which are now roots in Φ.

For, both of which are clearly roots, and in addition τ_{s,2n−r} > τ_{s,2n−r+1}, whence τ_{s,2n−r} must belong to Φ since τ_{s,2n−r+1} does (here we use that Φ is a dual order ideal), and τ_{r,r} must be in Φ because otherwise we would have λ_{r} = r − 1 (that is, the cell (r, r − 1) would be the rightmost cell in row r). An analogous argument holds in type D_{n}.

Now that we have shown that t_{i,j} is equal to the right-hand side of (3.1), and since also here t_{1,1} is the maximum of all entries t_{i,j}, it follows that t_{1,1} is the class of nilpotence of the ideal 𝔦. Since Proposition 2.1 shows that t_{1,1} can be obtained by repeated application of (3.8), the proof is complete. □

In view of this proposition, we have reduced the problem of counting ideals in I^{n} with a given class of nilpotence to the problem of counting certain partitions contained in a staircase with respect to the outcome of the algorithm in Proposition 2.1. This partition enumeration problem is already in a convenient form for being tackled directly in the cases of types B_{n} and C_{n}. We shall do that in Sections 6 and 5, respectively. However, in the case of type D_{n} we have the additional complication that the partition λ is either given by Ã or by Ã^{•}. It means that every partition in which the (n − 1)st and nth column have the same length is counted once, whereas every partition whose (n − 1)st and nth column differ in length is actually counted twice. Therefore we have to distinguish between these two cases. So, for a Lie algebra 𝔤 of type D_{n}, let, as in Theorem D.1, δ_{n}(K) denote the number of ideals in I^{n} with class of nilpotence K, let δ_{n}^{(1)}(K) denote the number of partitions λ = (λ_{1}, λ_{2}, …, λ_{2n−2}) in the staircase T_{2n−2} with the property that (λ_{2}, …, λ_{2n−2}) is self-conjugate, that λ_{i} ≠ i − 1 for i ≥ 2, and that n(λ) = K, and let δ_{n}^{(2)}(K) denote the number of partitions λ = (λ_{1}, λ_{2}, …, λ_{2n−2}) in the staircase T_{2n−2} that have the property that (λ_{2}, …, λ_{2n−2}) is self-conjugate, that λ_{i} ≠ i − 1 for i ≥ 2, that the (n − 1)st and the nth column have the same length, and that n(λ) = K. Then the above arguments show that

δ_{n}(K) = 2δ_{n}^{(1)}(K) − δ_{n}^{(2)}(K).  (4.1)

In Section 7 we shall show how to compute δ_{n}^{(1)}(K). Very conveniently, the computation of δ_{n}^{(2)}(K) can be reduced to the computation of the number of ideals with class of nilpotence K in type B_{n−1}.

Proposition 4.4. For 𝔤 a Lie algebra of type B_{n}, let, as in Theorem B.1, β_{n}(K) denote the number of ideals in I^{n} with class of nilpotence K. Then δ_{n}^{(2)}(K) = β_{n−1}(K).

Proof. By Proposition 4.3 we know that β_{n−1}(K) is equal to the number of partitions μ = (μ_{1}, μ_{2}, …, μ_{2n−3}) contained in the staircase T_{2n−3} with the property that (μ_{2}, …, μ_{2n−3}) is self-conjugate, that μ_{i} ≠ i − 1 for i ≥ 2, and that it satisfies n(μ) = K, where n(μ) is the result of the algorithm of Proposition 2.1.

Thus it suffices to set up a one-to-one correspondence between the partitions counted by δ_{n}^{(2)}(K) and those counted by β_{n−1}(K). Such a correspondence is easily set up. Given a partition counted by δ_{n}^{(2)}(K) we delete the (n − 1)st column and the nth row. Clearly, we obtain a partition μ = (μ_{1}, μ_{2}, …, μ_{2n−3}) contained in T_{2n−3} with the property that (μ_{2}, …, μ_{2n−3}) is self-conjugate and that μ_{i} ≠ i − 1 for i ≥ 2. It is equally obvious that the mapping can easily be reversed. What

[figure: _{.....}; _{.....}; . . . . .; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; x + y = 2n − 1; •; •; •; .....; .....; . . . . .; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; x + y = 2n − 2; •; •; •]

a. A partition counted by δ_{n}^{(2)}(K) b. A partition counted by β_{n−1}(K)

Figure 3

we need in addition is that the result of the algorithm of Proposition 2.1 yields the same value, namely K, for the original partition as well as for the “reduced” partition. This is most obvious from the geometric rendering of the algorithm, see Figure 3, which displays an example with n = 7 and K = 3. In Part (a) of the figure, the (n − 1)st column and the nth row are marked, while in Part (b) the places are marked where that column and that row were cut out. When we delete the (n − 1)st column and the nth row of the original partition then the effect on the broken ray is plainly that a horizontal piece of unit length is cut out (the one that passed exactly under this column) and that a vertical piece of unit length is cut out (the one that passed exactly to the right of that row). Otherwise the ray is completely identical. In particular, the number of touching points with the bounding line must be the same, and this number is exactly equal to the outcome of the algorithm of Proposition 2.1. □

§5 Proofs in type C

Let 𝔤 be of type C_{n}. We have to prove Theorems C.2 and C.6, upon which the other theorems and corollaries labelled C follow, as we have described in Section 2.

By Proposition 4.3 we know that the number of ideals in I^{n} with class of nilpotence K is equal to the number of self-conjugate partitions λ = (λ_{1}, λ_{2}, …, λ_{2n−1}) contained in the staircase (2n−1, 2n−2, …, 1) with n(λ) = K, where n(λ) denotes the result of the algorithm of Proposition 2.1.

The strategy which we use to count the latter partitions is based on the following observation: instead of considering the self-conjugate partitions (λ_{1}, λ_{2}, …, λ_{2n−1}), we consider just the “upper halfs,” the shifted partitions with row lengths (λ_{1}, λ_{2} − 1, λ_{3} − 2, …). (In fact, this is the collection of cells A that contains the roots that define the corresponding ideal; see Section 4.) Draw the shifted partition (λ_{1}, λ_{2} − 1, λ_{2} − 2, …), see Figures 4 and 5. Start as in the A_{2n−1} algorithm. I.e., begin on the vertical edge on the right of cell (1, λ_{1}), and move downward until we touch the antidiagonal x + y = 2n. At the touching point we turn direction from vertical-down to horizontal-left, and move on until we touch a vertical part of the Ferrers diagram. At the touching point we turn direction from horizontal-left to vertical-down. We iterate until we meet the diagonal x = y. Let k be the number of touching points on the antidiagonal x + y = 2n. Then we claim that if we meet the diagonal x = y while travelling horizontal-left, then the class of nilpotence (of the corresponding ideal) is 2k, whereas if we meet the diagonal while travelling vertical-downward, then the class of nilpotence is 2k + 1. For example, if n = 9, then for the shifted partition (16, 13, 11, 8, 7, 5, 3) in Figure 4 (corresponding to the self-conjugate partition (16, 14, 13, 11, 11, 10, 9, 6, 6, 5, 4, 3, 3, 2, 1, 1)) the class of nilpotence is 2·3 = 6, whereas for the partition (16, 13, 11, 8, 5, 3, 1) in Figure 5 (corresponding to the self-conjugate partition (16, 14, 13, 11, 9, 8, 7, 6, 5, 4, 4, 3, 3, 2, 1, 1)) the class of nilpotence is 2·2+1 = 5. (At this point, the thick segments are without relevance and should therefore be ignored.)

[figure: ^{.....}; ^{.....}; ^{.....}; . . . . .; ^{.....}; ^{.....}; . . . . .; . . . . .; ^{.....}; ^{.....}; . . . . .; . . . . .; x + y = 2n; x = y; •; •; •]_{Figure 4}

These claims become immediately obvious from the geometrical picture. Consider the self-conjugate partition together with the broken ray that is obtained according to the geometric rendering of the algorithm of Proposition 2.1. Figure 6.a shows the self-conjugate partition (16, 14, 13, 11, 11, 10, 9, 6, 6, 5, 4, 3, 3, 2, 1, 1) (corresponding to the shifted partition in Figure 4) together with the corresponding broken ray. In the figure the ray hits the line x = y while travelling horizontal-left, so that we are in the first of the two possible cases. If we simply reflect the portion of the ray which is below the line x = y about this line (see Figure 6.b), then it is obvious that the touching points below the line x = y get reflected just in between the touching points above x = y. Thus, in the first case the total number of touching points of the ray generated by the original self-conjugate partition is exactly twice the number of the touching points above x = y, while in the second case it is twice this number plus 1. This proves our claims.

We are now in the position to prove Theorem C.6 (and thereby Theorem C.1, which follows by setting t = 1 in Theorem C.6). Subsequently, we shall prove Theorem C.2.

[figure: ^{.....}; ^{.....}; ^{.....}; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; x + y = 2n; x = y; •; •]_{Figure 5}

[figure: . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; x + y = 2n; x = y; ∘; ∘; ∘; •; •; •; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; x + y = 2n; x = y; •; •; •; ∘; ∘; ∘]

a. A self-conjugate partition

b. One half of the self-conjugate partition

Figure 6

Proof of Theorem C.6. For a fixed K, we aim at computing the generating function ∑_{h≥0} γ_{n}(h, K)t^{h} for ideals 𝔦 ∈ I^{n} with class of nilpotence K. Once this is done, Equation (C.6) will follow immediately. Clearly the dimension of an ideal 𝔦 is equal to the size of the corresponding shifted partition, i.e., to the sum of its parts. Let us denote the size of a partition μ by |μ|. Thus, what we are asking for is to compute the generating function ∑_{μ} t^{|μ|} for shifted partitions μ contained in the shifted staircase S_{2n−1} such that the broken ray construction yields ⌊K/2⌋ touching points on x + y = 2n − 1, and x = y is hit while travelling horizontal-left if K is even, respectively vertically-down if K is odd.

Suppose we fix a particular broken ray. How do we obtain all the shifted partitions whose corresponding ray is exactly this fixed one? The answer is immediate once we observe that every time a broken ray hits a vertical part of the shifted partition while travelling horizontal-left, then next the broken ray will continue by a unit step vertically down. These vertical unit steps are shown as thick segments in Figures 4 and 5. In turn, every shifted partition which contains these vertical segments will generate exactly the given broken ray. Let the x-coordinates of these vertical segments be i_{1}, i_{2}, …, i_{k}, in ascending order. For example, in Figure 4 we have i_{1} = 10, i_{2} = 13, and i_{3} = 16, while in Figure 5 we have i_{1} = 8, i_{2} = 13, and i_{3} = 16. It is easy to see that i_{1}, i_{2}, …, i_{k} determine a unique broken ray. If we now use the well-known fact that the generating function ∑_{ν} t^{|ν|}, summed over all (ordinary) partitions ν which are contained in an a × b rectangle, is equal to the t-binomial coefficient [ \begin{matrix}a+b \\ b \end{matrix} ]_{t} (cf., e.g., [Sta, Prop. 1.3.19]), then if K = 2k we obtain

∑_{h≥0} γ_{n}(h, 2k)t^{h} = ∑_{n<i_{1}<···<i_{k}<i_{k+1}=2n} ∏_{j=1}^{k−1} t^{i_{j+1}(i_{j+3}−i_{j+2})} [ \begin{matrix}i_{j+2} − i_{j} − 1 \\ i_{j+1} − i_{j} \end{matrix} ]_{t}

· ∑_{ℓ=0}^{i_{2}−i_{1}−1} [ \begin{matrix}i_{1} + i_{2} − 2n − 1 \\ ℓ \end{matrix} ]_{t} t^{i_{1}(i_{3}−i_{2})−\binom{2n−i_{2}+1}{2}+\binom{ℓ+1}{2}} (5.1)

(where i_{k+2} = 2n + 1) for the generating function for ideals with class of nilpotence 2k, and

∑_{h≥0} γ_{n}(h, 2k − 1)t^{h} = ∑_{2n−i_{2}<i_{1}≤n<i_{2}<···<i_{k}<i_{k+1}=2n} t^{i_{1}(i_{3}−i_{2})−\binom{2n−i_{2}+1}{2}}

· (1 + t)(1 + t^{2}) ⋯ (1 + t^{i_{1}+i_{2}−2n−1}) ∏_{j=1}^{k−1} t^{i_{j+1}(i_{j+3}−i_{j+2})} [ \begin{matrix}i_{j+2} − i_{j} − 1 \\ i_{j+1} − i_{j} \end{matrix} ]_{t} (5.2)

(where again i_{k+2} = 2n + 1) for the generating function for ideals with class of nilpotence 2k − 1. Because of (C.7), both formulas can be combined. After having also done the substitution i_{j} → i_{j} + n, the result is (C.6). □

Proof of Theorem C.2. We start by first concentrating on the case that h is even, h = 2k say. We want to compute the number of ideals with class of nilpotence at most 2k. In principle, we could sum up the expressions (5.1) and (5.2) with t = 1, but it is more convenient to use an algorithm which determines whether the class of nilpotence is at most 2k (as opposed to equal to 2k). This algorithm works as follows. Again we are given a shifted partition λ. This time we start at the diagonal x = y (!), at the lowest intersection point with the Ferrers diagram of λ. We move right until we touch the antidiagonal x + y = 2n. At the touching point we turn direction from horizontal-right to vertical-up, and move on until we touch a horizontal part of the Ferrers diagram. At the touching point we turn direction from vertical-up to horizontal-right. Etc. Then the class of nilpotence is at most twice the number of touching points on x + y = 2n. See Figure 7, where this procedure is applied to the partition of Figure 4.

Let j_{1}, j_{1}+j_{2}, …, j_{1}+j_{2}+⋯+j_{k} denote the y-coordinates of the touching points of the broken ray on x + y = 2n, in ascending order. For example, in Figure 7 we

[figure: ^{.....}; . . . . .; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; x + y = 2n; x = y; •; •; •]_{Figure 7}

have j_{1} = 1, j_{2} = 2, and j_{3} = 4. In addition, let us write j_{k+1} for n−j_{1}−j_{2}−⋯−j_{k}. Then, if we follow the line of argument of the proof of Theorem C.6 (in a slightly modified form), we obtain

∑_{j_{1}+j_{2}+···+j_{k+1}=n} ; j_{1},j_{2},...,j_{k}≥1, j_{k+1}≥0\binom{j_{1} + j_{2} − 1}{j_{1}}\binom{j_{2} + j_{3} − 1}{j_{2}} ⋯ \binom{j_{k−1} + j_{k} − 1}{j_{k−1}}\binom{j_{k} + 2j_{k+1}}{j_{k}}

for the number of ideals with class of nilpotence 2k or 2k − 1, and therefore, as a moment’s thought shows, we obtain

∑_{j_{1}+j_{2}+···+j_{k+1}=n} ; j_{1},j_{2},...,j_{k},j_{k+1}≥0\binom{j_{1} + j_{2} − 1}{j_{1}}\binom{j_{2} + j_{3} − 1}{j_{2}} ⋯ \binom{j_{k−1} + j_{k} − 1}{j_{k−1}}\binom{j_{k} + 2j_{k+1}}{j_{k}}

for the number of ideals with class of nilpotence at most 2k. Now we can easily compute the generating function ∑_{n≥0} γ_{n}^{≤}(2k)x^{n} for the ideals with class of nilpotence at most 2k. We substitute the previous expression in the generating function, and obtain

∑_{j_{1},j_{2},...,j_{k},j_{k+1}≥0}x^{j_{1}+···+j_{k}+j_{k+1}} \binom{j_{1} + j_{2} − 1}{j_{1}}\binom{j_{2} + j_{3} − 1}{j_{2}}

⋯ \binom{j_{k−1} + j_{k} − 1}{j_{k−1}}\binom{j_{k} + 2j_{k+1}}{j_{k}}.

Now we perform the summation over j_{1} and obtain (1/(1 − x))^{j_{2}} in the sum. Next the summation over j_{2} is performed, etc., thus slowly building up continued fractions of the form of the left-hand side in (5.3) below. If we use the fact that

\frac{1}{1 − \frac{x}{1 − \frac{x}{\frac{...}{1 − x}}}} = \frac{Ũ_{h}}{\sqrt{x} Ũ_{h+1}} ,  (5.3)

where x occurs h times in the continued fraction (again, Ũ_{k} is short-hand for U_{k}(1/2\sqrt{x})), then we end up with

∑_{j_{k+1}≥0} \frac{1}{\sqrt{x}} (\frac{Ũ_{k}}{Ũ_{k+1}} )^{2j_{k+1}+1} .

The sum is easily evaluated as it is just a geometric series. This gives

\begin{matrix}Ũ_{k} & Ũ_{k+1} \\ \sqrt{x} & (Ũ_{k+1}^{2} − Ũ_{k}^{2}) \end{matrix}.

Now, it is easy to check that

U_{k}(x)U_{k+1}(x) = U_{2k+1}(x) + U_{2k−1}(x) + ⋯ + U_{1}(x)

and

U_{k+1}^{2}(x) − U_{k}^{2}(x) = U_{2k+2}(x).

Hence, our generating function equals

\frac{Ũ_{2k+1} + Ũ_{2k−1} + ⋯ + Ũ_{1}}{\sqrt{x} Ũ_{2k+2}} .

This proves Theorem C.2 in the case that h is even.

Next we compute the generating function for ideals with class of nilpotence equal to 2k − 1, i.e., ∑_{n≥0} γ_{n}(2k − 1)x^{n}, where, as before, γ_{n}(h) is the number of adnilpotent ideals in I^{n} with class of nilpotence equal to h. The case k = 1 has to be done separately, but this is trivial. (This is already included in the Kostant– Peterson result.) So let us assume k ≥ 2. If we substitute formula (5.2) with t = 1 in the generating function, then we obtain

∑_{n≥0} ∑_{2n−i_{2}<i_{1}≤n<i_{2}<···<i_{k}<2n} x^{n} ( \begin{matrix}2n − i_{k−1} − 1 \\ i_{k} − i_{k−1} \end{matrix} ) ( \begin{matrix}i_{k} − i_{k−2} − 1 \\ i_{k−1} − i_{k−2} \end{matrix} )

⋯ ( \begin{matrix}i_{3} − i_{1} − 1 \\ i_{2} − i_{1} \end{matrix} ) 2^{i_{1}+i_{2}−2n−1}

for our generating function. As before, the sums over n, i_{k}, i_{k−1}, …, i_{3} are easily computed. On using again that the resulting continued fractions can be expressed in terms of Chebyshev polynomials by means of (5.3), and after having replaced i_{1} by i_{1} + n and i_{2} by i_{2} + n, we obtain

∑_{−i_{2}<i_{1}≤0<i_{2}} x^{i_{2}+k−1} (\frac{Ũ_{1}}{x^{k/2−1} Ũ_{k−1}} )^{2} (\frac{Ũ_{k−1}}{\sqrt{x} Ũ_{k}} )^{i_{2}−i_{1}+1} 2^{i_{1}+i_{2}−1}.

Next we sum over i_{1}, and subsequently over i_{2}. In both cases, it is just geometric series that have to be summed, a terminating series when summing over i_{1} and two nonterminating series when summing over i_{2}. The result, after cancellation of factors, is

\frac{Ũ_{k}}{(Ũ_{k} − 2\sqrt{x} Ũ_{k−1}) (Ũ_{k}^{2} − Ũ_{k−1}^{2})}.

It is now routine to verify that this is equal to

\frac{Ũ_{2k} + Ũ_{2k−2} + ⋯ + Ũ_{0}}{\sqrt{x} Ũ_{2k+1}} − \frac{Ũ_{2k−1} + Ũ_{2k−3} + ⋯ + Ũ_{1}}{\sqrt{x} Ũ_{2k}} .

We already know that the second term in this difference is the generating function for ideals with class of nilpotence at most 2k − 2. Hence the first term must be the generating function for ideals with class of nilpotence at most 2k − 1.

At this point Theorem C.2 is completely proved. □

§6 Proofs in type B

We have to prove Theorem B.1, upon which Corollaries B.2 and B.3 follow, as we have described in Section 2.

Proof of Theorem B.1. By Proposition 4.3 we know that β_{n}(K) is equal to the number of partitions (λ_{1}, λ_{2}, …, λ_{2n−1}) contained in the staircase (2n − 1, 2n − 2, …, 1) with the property that (λ_{2}, …, λ_{2n−1}) is self-conjugate, that λ_{i} ≠ i − 1 for i ≥ 2, and that n(λ) = K, where n(λ) stands for the result of the algorithm of Proposition 2.1. In slight abuse of terminology, we shall refer to n(λ) as the “class of nilpotence of λ.”

First of all, the cases K = 0 and K = 1 can be treated directly, the case of K = 1 being contained in the Kostant–Peterson result.

For the proof of the theorem for K ≥ 2, we follow a similar idea as in the proof of Theorems C.6 and C.2 in Section 5. Instead of considering the above (ordinary) partitions (λ_{1}, λ_{2}, …, λ_{2n−1}), we consider again just the “upper halfs,” the shifted partitions with row lengths (λ_{1}, λ_{2} − 1, λ_{3} − 2, …). (In fact, this is the collection of cells A that contains the roots that define the corresponding ideal; see Section 4.) Again we need an algorithm which, given the shifted partition, allows us to find the class of nilpotence n(λ) without having to go back to the ordinary partition λ. The particular construction that we are going to use requires two broken rays (as opposed to just one as in Section 5). These two broken rays are constructed as follows: ray 1 starts on the vertical edge on the right of cell (1, λ_{1}) (that is, on the vertical right border of the first row), whereas ray 2 starts on the vertical edge on the right of cell (2, λ_{2}) (that is, on the vertical right border of the second row). Both rays are determined in an algorithmic manner: starting on that edge, we move downward until we touch the antidiagonal x + y = 2n. At the touching point we turn direction from vertical-down to horizontal-left, and move on until we touch a vertical part of the Ferrers diagram. At the touching point we turn direction from horizontal-left to vertical-down. Now the procedure is iterated, until we reach the diagonal x = y − 1. See Figures 7–14 for typical examples, with ray 1 (starting at the right border of the first row) marked as a thin line, and ray 2 (starting at the right border of the second row) marked as a dotted line.

In order to do the computations, we need to divide the shifted partitions (λ_{1}, λ_{2}− 1, …) into 7 subclasses. To which subclass a particular partition belongs depends on whether the corresponding rays reach x = y − 1 while travelling vertical-down or horizontally-left, and whether one ray is above the other or not, as is detailed below. At the same time we shall be able to read off the class of nilpotence from the broken rays.

Below we list the 7 subclasses. It is easy to see that they cover all possibilities. For each subclass we provide a precise characterization, a typical example, and we describe how to read off the class of nilpotence.

Case 1. A partition belongs to subclass 1, if ray 2 reaches x = y −1 while travelling horizontal-left, and if either ray 1 reaches x = y − 1 while travelling horizontal-left weakly below ray 2, after having touched x + y = 2n one more time than ray 2 (see Figure 8) or ray 1 reaches x = y − 1 while travelling vertical-down strictly to the right of ray 2 (see Figure 9; that some edges are thick is irrelevant at the moment).

[figure: ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; x + y = 2n; x = y − 1; •; •; •; •; •; ∘]_{Figure 8}

[figure: ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; x + y = 2n; x = y − 1; •; •; •; •; ∘]_{Figure 9}

Let k be the number of touching points of ray 2 on the antidiagonal x + y = 2n. Then the class of nilpotence of an ideal in this subclass is 2k + 1.

[figure: ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; x + y = 2n; x = y − 1; •; •; •; •; ∘]_{Figure 10}

Case 2. A partition belongs to subclass 2, if ray 2 reaches x = y −1 while travelling horizontal-left, and if ray 1 reaches x = y − 1 while travelling vertical-down weakly to the left of ray 2 (see Figure 10).

Let k be the number of touching points of ray 2 on the antidiagonal x + y = 2n. Then the class of nilpotence of an ideal in this subclass is 2k. Case 3. A partition belongs to subclass 3, if ray 2 reaches x = y −1 while travelling vertical-down, and if ray 1 reaches x = y − 1 while travelling horizontal-left strictly above ray 2 (see Figure 11).

[figure: . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; x + y = 2n; x = y − 1; •; •; •; ∘]_{Figure 11}

Let k be the number of touching points of ray 1 on the antidiagonal x + y = 2n. Then the class of nilpotence of an ideal in this subclass is 2k. Case 4. A partition belongs to subclass 4, if ray 2 reaches x = y −1 while travelling vertical-down, and if ray 1 reaches x = y − 1 while travelling vertical-down weakly to the left of ray 2, after having touched x + y = 2n one more time than ray 2 (see Figure 12).

Let k be the number of touching points of ray 1 on the antidiagonal x + y = 2n. Then the class of nilpotence of an ideal in this subclass is 2k.

[figure: . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; x + y = 2n; x = y − 1; •; •; •; ∘]_{Figure 12}

Case 5. A partition belongs to subclass 5, if ray 2 reaches x = y −1 while travelling vertical-down, and if ray 1 reaches x = y − 1 while travelling horizontal-left weakly below ray 2 (see Figure 13).

[figure: . . . . .; . . . . .; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; x + y = 2n; x = y − 1; •; •; •; ∘]_{Figure 13}

Let k be the number of touching points of ray 1 on the antidiagonal x + y = 2n. Then the class of nilpotence of an ideal in this subclass is 2k − 1. Case 6. A partition belongs to subclass 6, if ray 2 reaches x = y −1 while travelling vertical-down, and if ray 1 reaches x = y − 1 while travelling vertical-down weakly to the right of ray 2, both rays touching x + y = 2n an equal number of times (see Figure 14).

Let k−1 be the number of touching points of ray 1 on the antidiagonal x+y = 2n. Then the class of nilpotence of an ideal in this subclass is 2k − 1. Case 7. A partition belongs to subclass 7, if ray 2 reaches x = y −1 while travelling horizontal-left, and if ray 1 reaches x = y − 1 while travelling horizontal-left weakly above ray 2, both rays touching x+y = 2n an equal number of times (see Figure 15).

Let k be the number of touching points of ray 2 on the antidiagonal x + y = 2n. Then the class of nilpotence of an ideal in this subclass is 2k.

[figure: . . . . .; . . . . .; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; x + y = 2n; x = y − 1; •; •; ∘]_{Figure 14}

[figure: ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; . . . . .; ^{.....}; ^{.....}; ^{.....}; ^{.....}; ^{.....}; . . . . .; . . . . .; . . . . .; . . . . .; x + y = 2n; x = y − 1; •; •; •; •; ∘]_{Figure 15}

Let β_{n}^{(i)}(K) denote the number of partitions in the ith subclass with class of nilpotence K, i = 1, 2, …, 7. We will now, for each subclass, compute the corresponding generating function ∑_{n≥0} β_{n}^{(i)}(K)x^{n}. This is made possible by formulas

for β_{n}^{(i)}(K) for each subclass in the spirit of (A.1).

The enumeration in Case 1. For given n, the number β_{n}^{(1)}(2k + 1) of partitions in this subclass is given by the multiple summation

∑_{−i_{2}≤i_{1}≤i_{2}≤···≤i_{2k+1}≤n−1} ( ∏_{j=4}^{2k+2} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}}) ∑_{ℓ=0}^{i_{3}−i_{2}−1} \binom{i_{3} + i_{1} − 1}{ℓ},

where, as always in the sequel, i_{2k+2} = n − 1. This expression is obtained from the geometrical presentation as in Figures 7 or 8, by denoting the deviation of the first vertical edges on each downward travel along a ray (in Figures 7 and 8 these are the thick vertical edges; the thick horizontal edge marks the downmost and leftmost edge of the shifted Ferrers diagram; it must touch the diagonal x = y − 1) from the “reference point” (n, 0) (in Figures 7 and 8 it is marked by a circle) by i_{2k+1}, i_{2k}, …, i_{1}, from right to left. Thus, in Figure 8 we have i_{5} = 8, i_{4} = 6, i_{3} = 4, i_{2} = 1, i_{1} = 0, and in Figure 9 we have i_{5} = 8, i_{4} = 6, i_{3} = 4, i_{2} = 1, i_{1} = −1. Then arguments very similar to those that we used in the proofs of Theorems C.6 and C.2 in the previous section show that, for fixed i_{2k+1}, i_{2k}, …, i_{1}, the number of partitions is equal to the summand in the above sum.

We now compute the generating function ∑_{n≥0} β_{n}^{(1)}(2k + 1)x^{n}. By definition, this is

∑_{n≥0} x^{n} ∑_{−i_{2}≤i_{1}≤i_{2}≤···≤i_{2k+1}≤n−1} ( ∏_{j=4}^{2k+2} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}})

· ∑_{ℓ=0}^{i_{3}−i_{2}−1} \binom{i_{3} + i_{1} − 1}{ℓ}.

Now we would like to perform the sum over n, then the sum over i_{2k+1}, then over i_{2k}, etc., one after the other. In order to do this conveniently, we have to split the range of the sum into the possibilities

−i_{2} ≤ i_{1} ≤ i_{2}, i_{2} + ℓ + 1 ≤ i_{3} < ⋯ < i_{2k+1} < n − 1

and

0 ≤ i_{1} = i_{2}, i_{2} + ℓ + 1 ≤ i_{3} = i_{4} < i_{5} = i_{6} < ⋯ < i_{2j−1} = i_{2j} < i_{2j+1} < ⋯ < i_{2k+1} < n − 1,

for some j between 2 and k + 1. These two do indeed cover all possibilities which contribute to the sum. For, if we have i_{s} = i_{s−1} for some s ≥ 4, then, in order that the summand does not vanish, we must also have i_{s−2} = i_{s−3}, etc. The case of s being even is covered by the second possibility above. If s should be odd, then we would also have i_{3} = i_{2}. Since then the sum over ℓ is empty, this does not contribute to the sum.

Now, in each of the two cases the sums over n, i_{2k+1}, …, i_{4} are easily carried out by the binomial theorem. Thereby, continued fractions as in the left-hand side of (5.3) are slowly built up.

To finish the calculation, we use the identity

∑_{ℓ=0}^{i_{3}−i_{2}−1} \binom{i_{3} + i_{1} − 1}{ℓ} = ∑_{ℓ=0}^{i_{3}−i_{2}−1} \binom{i_{1} + i_{2} + ℓ}{ℓ} · \{ \begin{matrix}2^{i_{3}−i_{2}−ℓ−2} & if ℓ < i_{3} − i_{2} − 1, \\ 1 & if ℓ = i_{3} − i_{2} − 1. \end{matrix}

(6.1)

For, now the sums over i_{3}, ℓ, i_{1}, and finally i_{2}, in that order, can easily be carried out, as this amounts again to just summing binomial or even just geometric series. To get rid of the continued fractions, we use Equation (5.3). The result is

\begin{matrix}\sqrt{x} & Ũ_{k+1} & (Ũ_{k}^{2} + Ũ_{k+1}^{2}) \\ Ũ_{k}^{2} & Ũ_{2k+2}^{2} & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix} .

The computation for the second possible range is similar. The result is

\begin{matrix}\sqrt{x} & Ũ_{k+1}^{2} \\ Ũ_{k−j+1} & Ũ_{k−j+2} & Ũ_{k} & Ũ_{2k+2} & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix} .

In order to obtain the overall generating function for the ideals in this subclass, we have to take the first expression and add to it the sum of the second expressions over j from 2 to k + 1. Using the identity

\frac{1}{U_{0}(x)U_{1}(x)} + \frac{1}{U_{1}(x)U_{2}(x)} + ⋯ + \frac{1}{U_{s}(x)U_{s+1}(x)} = \frac{U_{s}(x)}{U_{s+1}(x)},

we obtain

\begin{matrix}\sqrt{x} & Ũ_{k+1} & (Ũ_{k}^{2} + Ũ_{k+1}^{2} + Ũ_{k−1} Ũ_{k+1} Ũ_{2k+2}) \\ Ũ_{k}^{2} & Ũ_{2k+2}^{2} & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix}.

The enumeration in Case 2. By arguments analogous to the ones in Case 1, for given n the number β_{n}^{(2)}(2k) of partitions in this subclass is given by the multiple summation

∑_{0≤i_{2}≤···≤i_{2k+1}≤n−1} ; −i_{3}+1≤i_{1}≤−i_{2}−1( ∏_{j=4}^{2k+2} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}})2^{i_{3}+i_{1}−1}.

We now compute the generating function ∑_{n≥0} β_{n}^{(2)}(2k)x^{n}. We proceed as in Case 1. Here, the analysis is in fact easier. First, there is no sum over ℓ, so that we do not have to use the transformation formula (6.1). Second, both i_{2} = i_{1} and i_{3} = i_{2} produce vanishing summands, so that we can restrict the summation to

−i_{3} + 1 ≤ i_{1} ≤ −i_{2} − 1, 0 ≤ i_{2} < ⋯ < i_{2k+1} < n − 1.

As a result, we obtain for the generating function the expression

\begin{matrix}\sqrt{x} & Ũ_{k−1} & Ũ_{k+1} \\ Ũ_{k}^{2} & Ũ_{2k+2} & (Ũ_{k+1} − Ũ_{k−1}) & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix}.

The enumeration in Case 3. By arguments analogous to the ones in Case 1, for given n the number β_{n}^{(3)}(2k) of partitions in this subclass is given by the multiple summation

∑_{−i_{2}≤i_{1}<0≤i_{2}≤···≤i_{2k}≤n−1}( ∏_{j=4}^{2k+1} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}}) ∑_{ℓ=0}^{i_{3}−i_{2}−1} \binom{i_{3} + i_{1} − 1}{ℓ}.

We now compute the generating function ∑_{n≥0} β_{n}^{(3)}(2k)x^{n}. We proceed as in Case 1. Since both i_{2} = i_{1} and i_{3} = i_{2} produce vanishing summands, we can restrict the summation to

−i_{2} ≤ i_{1} < 0 ≤ i_{2} < ⋯ < i_{2k} < n − 1.

As a result, we obtain for the generating function the expression

\begin{matrix}\sqrt{x} & Ũ_{k−1} & Ũ_{k+1} \\ Ũ_{k}^{2} & Ũ_{2k} & (Ũ_{k+1} − Ũ_{k−1}) & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix} .

The enumeration in Case 4. By arguments analogous to the ones in Case 1, for given n the number β_{n}^{(4)}(2k) of partitions in this subclass is given by the multiple summation

∑_{−i_{3}+1≤i_{1}≤i_{2}<0≤i_{3}≤···≤i_{2k+1}≤n−1}( ∏_{j=4}^{2k+2} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}})2^{i_{3}+i_{1}−1}.

We now compute the generating function ∑_{n≥0} β_{n}^{(4)}(2k)x^{n}. We proceed as in Case 1. We split the range of summation into the possibilities

−i_{3} + 1 ≤ i_{1} ≤ i_{2} < 0 ≤ i_{3} < ⋯ < i_{2k+1} < n − 1

and

−i_{3} + 1 ≤ i_{1} = i_{2} < 0 ≤ i_{3} = i_{4} < ⋯ < i_{2j−1} = i_{2j} < i_{2j+1} < ⋯ < i_{2k+1} < n − 1,

for some j between 2 and k + 1. The contribution to the generating function of the first range is

\begin{matrix}\sqrt{x} & Ũ_{k−1}^{2} \\ Ũ_{k}^{2} & Ũ_{2k} & (Ũ_{k+1} − Ũ_{k−1}) & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix} ,

while the contribution of the second range is

\begin{matrix}\sqrt{x} & Ũ_{k−1}^{2} \\ Ũ_{k−j+1} & Ũ_{k−j+2} & Ũ_{k} & Ũ_{2k} & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix} .

Summing the second expression over j from 2 to k + 1, and adding the result to the first expression gives

\begin{matrix}\sqrt{x} & Ũ_{k−1}^{2} \\ Ũ_{k}^{2} & Ũ_{2k} & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix} (\frac{1}{Ũ_{k+1} − Ũ_{k−1}} + Ũ_{k−1}) .

The enumeration in Case 5. By arguments analogous to the ones in Case 1, for given n the number β_{n}^{(5)}(2k−1) of partitions in this subclass is given by the multiple summation

∑_{0≤i_{2}≤···≤i_{2k}≤n−1} ; −i_{3}+1≤i_{1}≤−i_{2}−1( ∏_{j=4}^{2k+1} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}})2^{i_{3}+i_{1}−1}.

We now compute the generating function ∑_{n≥0} β_{n}^{(5)}(2k − 1)x^{n}. We proceed as in Case 1. Since both i_{2} = i_{1} and i_{3} = i_{2} produce vanishing summands, we can restrict the summation to

−i_{3} + 1 ≤ i_{1} ≤ −i_{2} − 1, 0 ≤ i_{2} < ⋯ < i_{2k} < n − 1

As a result, we obtain for the generating function the expression

\begin{matrix}\sqrt{x} & Ũ_{k−1} \\ Ũ_{2k}^{2} & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix}.

The enumeration in Case 6. By arguments analogous to the ones in Case 1, for given n the number β_{n}^{(6)}(2k−1) of partitions in this subclass is given by the multiple summation

∑_{−i_{3}+1≤i_{1}≤i_{2}<0≤i_{3}≤···≤i_{2k}≤n−1}( ∏_{j=4}^{2k+1} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}})2^{i_{3}+i_{1}−1}.

We now compute the generating function ∑_{n≥0} β_{n}^{(6)}(2k − 1)x^{n}. We proceed as in Case 1. We split the range of summation into the possibilities

−i_{3} + 1 ≤ i_{1} ≤ i_{2} < 0 ≤ i_{3} < ⋯ < i_{2k} < n − 1

and

− i_{3} + 1 ≤ i_{1} = i_{2} < 0 ≤ i_{3} = i_{4} < ⋯ < i_{2j−1} = i_{2j} < i_{2j+1} < ⋯ < i_{2k} < n − 1,

for some j between 2 and k. The contribution to the generating function of the first range is

\begin{matrix}\sqrt{x} & Ũ_{k−1} \\ Ũ_{2k}^{2} & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix},

while the contribution of the second range is

\begin{matrix}\sqrt{x} & Ũ_{k−1}^{2} \\ Ũ_{k−j} & Ũ_{k−j+1} & Ũ_{k} & Ũ_{2k} & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix} .

Summing the second expression over j from 2 to k, and adding the result to the first expression gives

\begin{matrix}\sqrt{x} & Ũ_{k−1} \\ Ũ_{2k}^{2} & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix} (1 + \frac{Ũ_{k−2} Ũ_{2k}}{Ũ_{k}} ) .

The enumeration in Case 7. By arguments analogous to the ones in Case 1, for given n the number β_{n}^{(7)}(2k) of partitions in this subclass is given by the multiple summation

∑_{0≤i_{1}≤i_{2}≤···≤i_{2k}≤n−1} ( ∏_{j=4}^{2k+1} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}}) ∑_{ℓ=0}^{i_{3}−i_{2}−1} \binom{i_{3} + i_{1} − 1}{ℓ}.

We now compute the generating function ∑_{n≥0} β_{n}^{(7)}(2k)x^{n}. We proceed as in Case 1. We split the range of summation into the possibilities

0 ≤ i_{1} ≤ i_{2} < ⋯ < i_{2k} < n − 1

and

0 ≤ i_{1} = i_{2} < i_{3} = i_{4} < ⋯ < i_{2j−1} = i_{2j} < i_{2j+1} < ⋯ < i_{2k} < n − 1,

for some j between 2 and k. The contribution to the generating function of the first range is

\begin{matrix}\sqrt{x} & Ũ_{k+1}^{3} \\ Ũ_{k−1} & Ũ_{k}^{2} & Ũ_{2k+2} & (Ũ_{k+1} − Ũ_{k−1}) & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix} ,

while the contribution of the second range is

\begin{matrix}\sqrt{x} & Ũ_{k+1}^{2} \\ Ũ_{k−j} & Ũ_{k−j+1} & Ũ_{k} & Ũ_{2k+2} & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix} .

Summing the second expression over j from 2 to k, and adding the result to the first expression gives

\begin{matrix}\sqrt{x} & Ũ_{k+1}^{2} \\ Ũ_{k−1} & Ũ_{k}^{2} & Ũ_{2k+2} & (Ũ_{k} − 2 \sqrt{x} Ũ_{k−1}) \end{matrix} (\frac{Ũ_{k+1}}{Ũ_{k+1} − Ũ_{k−1}} + Ũ_{k−2} Ũ_{k}) .

Finally, in order to complete the proof of Theorem B.1, in case that K is even, the expressions of Cases 2, 3, 4 and 7 have to be summed, while in case that K is odd, the expressions of Cases 1 (with k replaced by k − 1), 5 and 6 have to be summed. Some simplification yields the claimed expressions. □

§7 Proofs in type D

We have to prove Theorem D.1, upon which Corollaries D.2 and D.3 follow, as we have described in Section 2.

Proof of Theorem D.1. According to (4.1) and Proposition 4.4 we have to compute

2 ∑_{n≥0} δ_{n}^{(1)}(K)x^{n} − ∑_{n≥0} β_{n−1}(K)x^{n}.  (7.1)

By Theorem B.1, we already know an expression for the second sum. It remains to compute ∑_{n≥0} δ_{n}^{(1)}(K)x^{n}, where δ_{n}^{(1)}(K) is the number of partitions λ = (λ_{1}, λ_{2}, …, λ_{2n−2}) in the staircase T_{2n−2} with the property that (λ_{2}, …, λ_{2n−2}) is self-conjugate, that λ_{i} ≠ i − 1 for i ≥ 2, and that n(λ) = K.

We proceed by imitating the arguments used in Case B in the previous section. First, instead of considering the above (ordinary) partitions (λ_{1}, λ_{2}, …, λ_{2n−2}), we consider again just the “upper halfs,” the shifted partitions with row lengths (λ_{1}, λ_{2} − 1, λ_{3} − 2, …). We apply again the construction of the two broken rays as in Case B, with the slight modification that it is now the line x+y = 2n−1 (instead of x + y = 2n) where the rays get reflected. We divide the shifted partitions that we consider here again into 7 subclasses. The characterization of the 7 subclasses and the description of how to determine the corresponding class of nilpotence are identical to those in Case B, except that, of course, again the line x + y = 2n has to be replaced by the line x + y = 2n − 1. We therefore omit to repeat them here and instead refer the reader to Section 6. For writing down formulas for the number of partitions in each subclass we also follow the derivations in Case B. In particular, we choose again (n, 0) as the reference point with respect to which deviations of the first vertical edges on each downward travel along a ray are measured.

We now discuss each of the 7 subclasses. Since everything is completely parallel to the computations in Case B, given in the previous section, we can be brief here. For each subclass, we provide the formula in the spirit of (A.1) for the number of shifted partitions in that subclass, and the corresponding generating function.

The enumeration in Case 1: For given n, the number of shifted partitions in this subclass is given by the multiple summation

∑_{−i_{2}−1≤i_{1}≤i_{2}≤···≤i_{2k+1}≤n−2} ( ∏_{j=4}^{2k+2} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}}) ∑_{ℓ=0}^{i_{3}−i_{2}−1} \binom{i_{3} + i_{1}}{ℓ},

where i_{2k+2} = n − 1. For the range

−i_{2} − 1 ≤ i_{1} ≤ i_{2}, i_{2} + ℓ + 1 ≤ i_{3} < ⋯ < i_{2k+1} < n − 2

we obtain

\frac{2 x Ũ_{k+1}^{2}}{Ũ_{k}Ũ_{2k+2}^{2} (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})},

for the corresponding generating function, while for the range

0 ≤ i_{1} = i_{2}, i_{2} + ℓ + 1 ≤ i_{3} = i_{4} < i_{5} = i_{6} < ⋯ < i_{2j−1} = i_{2j} < i_{2j+1} < ⋯ < i_{2k+1} < n − 2,

for some j between 2 and k + 1, we obtain

\frac{x Ũ_{k+1}}{Ũ_{k−j+1} Ũ_{k−j+2} Ũ_{2k+2} (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})}.

The overall generating function for this subclass is

\frac{x Ũ_{k+1} (2Ũ_{k+1} + Ũ_{k−1}Ũ_{2k+2})}{Ũ_{k} Ũ_{2k+2}^{2} (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})}.

The enumeration in Case 2: For given n, the number of shifted partitions in this subclass is given by the multiple summation

∑_{0≤i_{2}≤···≤i_{2k+1}≤n−2} ; −i_{3}≤i_{1}≤−i_{2}−2( ∏_{j=4}^{2k+2} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}})2^{i_{3}+i_{1}},

ENUMERATION OF ad-NILPOTENT 𝔟-IDEALS FOR SIMPLE LIE ALGEBRAS 35

where i_{2k+2} = n − 1. We can restrict the sum to the range

−i_{3} ≤ i_{1} ≤ −i_{2} − 2, 0 ≤ i_{2} < ⋯ < i_{2k+1} < n − 2.

We obtain

\frac{x Ũ_{k−1}}{Ũ_{k} Ũ_{2k+2} (Ũ_{k+1} − Ũ_{k−1}) (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})}

for the corresponding generating function. The enumeration in Case 3: For given n, the number of shifted partitions in this subclass is given by the multiple summation

∑_{−i_{2}−1≤i_{1}<0≤i_{2}≤···≤i_{2k}≤n−2} ( ∏_{j=4}^{2k+1} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}}) ∑_{ℓ=0}^{i_{3}−i_{2}−1} \binom{i_{3} + i_{1}}{ℓ},

where i_{2k+1} = n − 1. We can restrict the sum to the range

−i_{2} − 1 ≤ i_{1} < 0 ≤ i_{2} < ⋯ < i_{2k} < n − 2.

We obtain

\frac{x Ũ_{k+1}}{Ũ_{k} Ũ_{2k} (Ũ_{k+1} − Ũ_{k−1}) (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})}

for the corresponding generating function. The enumeration in Case 4: For given n, the number of shifted partitions in this subclass is given by the multiple summation

∑_{−i_{3}≤i_{1}≤i_{2}<0≤i_{3}≤···≤i_{2k+1}≤n−2} ( ∏_{j=4}^{2k+2} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}})2^{i_{3}+i_{1}},

where i_{2k+2} = n − 1. For the range

−i_{3} ≤ i_{1} ≤ i_{2} < 0 ≤ i_{3} < ⋯ < i_{2k+1} < n − 2

we obtain

\frac{x Ũ_{k−1}}{Ũ_{k} Ũ_{2k} (Ũ_{k+1} − Ũ_{k−1}) (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})}

for the corresponding generating function, while for the range

−i_{3} ≤ i_{1} = i_{2} < 0 ≤ i_{3} = i_{4} < ⋯ < i_{2j−1} = i_{2j} < i_{2j+1} < ⋯ < i_{2k+1} < n − 2, for some j between 2 and k + 1, we obtain

\frac{x Ũ_{k−1}}{Ũ_{k−j+1} Ũ_{k−j+2} Ũ_{2k} (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})}.

The overall generating function is

\frac{x Ũ_{k−1}}{Ũ_{k} Ũ_{2k} (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})} (\frac{1}{Ũ_{k+1} − Ũ_{k−1}} + Ũ_{k−1}) .

The enumeration in Case 5: For given n, the number of shifted partitions in this subclass is given by the multiple summation

∑_{0≤i_{2}≤···≤i_{2k}≤n−2} ; −i_{3}≤i_{1}≤−i_{2}−2( ∏_{j=4}^{2k+1} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}})2^{i_{3}+i_{1}},

where i_{2k+1} = n − 1. We can restrict the sum to the range

−i_{3} ≤ i_{1} ≤ −i_{2} − 2, 0 ≤ i_{2} < ⋯ < i_{2k} < n − 2.

We obtain

\frac{x Ũ_{k−1}^{2}}{Ũ_{2k}^{2} Ũ_{k} (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})}

for the corresponding generating function.

The enumeration in Case 6: For given n, the number of shifted partitions in this subclass is given by the multiple summation

∑_{−i_{3}≤i_{1}≤i_{2}<0≤i_{3}≤···≤i_{2k}≤n−2} ( ∏_{j=4}^{2k+1} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}})2^{i_{3}+i_{1}},

where i_{2k+1} = n − 1. For the range

−i_{3} ≤ i_{1} ≤ i_{2} < 0 ≤ i_{3} < ⋯ < i_{2k} < n − 2

we obtain

\frac{x Ũ_{k}}{Ũ_{2k}^{2} (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})}

for the corresponding generating function, while for the range

− i_{3} + 1 ≤ i_{1} = i_{2} < 0 ≤ i_{3} = i_{4} < ⋯ < i_{2j−1} = i_{2j} < i_{2j+1} < ⋯ < i_{2k} < n − 1,

for some j between 2 and k, we obtain

\frac{x Ũ_{k−1}}{Ũ_{k−j+1} Ũ_{k−j} Ũ_{2k} (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})}.

The overall generating function is

\frac{x Ũ_{k}}{Ũ_{2k}^{2} (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})} (1 + \frac{Ũ_{k−2}Ũ_{2k}}{Ũ_{k}} ) .

The enumeration in Case 7: For given n, the number of shifted partitions in this subclass is given by the multiple summation

∑_{0≤i_{1}≤i_{2}≤···≤i_{2k+1}≤n−2} ( ∏_{j=4}^{2k+2} \binom{i_{j} − i_{j−1} + i_{j−2} − i_{j−3} − 1}{i_{j−2} − i_{j−3}}) ∑_{ℓ=0}^{i_{3}−i_{2}−1} \binom{i_{3} + i_{1}}{ℓ},

where i_{2k+2} = n − 1. For the range

0 ≤ i_{1} ≤ i_{2} < ⋯ < i_{2k} < n − 2

we obtain

\frac{x Ũ_{k+1}^{2}}{Ũ_{k} Ũ_{k−1} Ũ_{2k+2} (Ũ_{k+1} − Ũ_{k−1}) (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})}

for the corresponding generating function, while for the range

0 ≤ i_{1} = i_{2} < i_{3} = i_{4} < ⋯ < i_{2j−1} < i_{2j} < i_{2j+1} < ⋯ < i_{2k} < n − 2,

for some j between 2 and k, we obtain

\frac{x Ũ_{k+1}}{Ũ_{k−j+1} Ũ_{k−j} Ũ_{2k+2} (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})}.

The overall generating function is

\frac{x Ũ_{k+1}}{Ũ_{k−1} Ũ_{k} Ũ_{2k+2} (Ũ_{k} − 2\sqrt{x}Ũ_{k−1})} (\frac{Ũ_{k+1}}{Ũ_{k+1} − Ũ_{k−1}} + Ũ_{k−2}Ũ_{k}) .

Finally, in order to complete the proof of Theorem D.1, we have to combine all our results to obtain the generating function (7.1). In order to obtain the generating function ∑_{n≥0} δ_{n}^{(1)}(K)x^{n}, in case that K is even, the expressions of Cases 2, 3, 4 and 7 have to be summed, while in case that K is odd, the expressions of Cases 1 (with k replaced by k − 1), 5 and 6 have to be summed. The result is then substituted in (7.1), together with the result of the previous section for ∑_{n≥0} β_{n−1}(K)x^{n}. Some simplification eventually yields the claimed expressions. □

n(𝔦)  E_{6}  E_{7}  E_{8}  F_{4}  G_{2}
0  1  1  1  1  1
1  63  127  255  15  3
2  210  662  2200  28  2
3  217  894  3804  21  1
4  150  766  3872  14  0
5  92  576  3372  12  1
6  51  403  2752  5
7  28  279  2182  4
8  12  175  1656  2
9  6  115  1277  2
10  2  68  955  0
11  1  44  737  1
12  23  536
13  14  412
14  7  300
15  4  227
16  1  157
17  1  123
18  81
19  61
20  40
21  30
22  18
23  14
24  7
25  5
26  3
27  2
28  0
29  1

total

number  833  4160  25080  105  8

Table 1

§8 The exceptional types

It is not difficult to write down a computer program which determines explicitly the descending central series of a given ideal. The final results are given in Table 1.

Just a few remarks on how to get the data in the table. Clearly we can work at the level of the root system. If 𝔦 ∈ I^{n} is encoded by Φ ⊆ Δ^{+}, then n(𝔦) equals the maximal integer k such that θ ∈ Φ^{k} (here θ is the highest root of Δ and Φ^{k} is inductively defined as Φ^{1} = Φ, Φ^{k} = (Φ^{k−1} + Φ) ∩ Δ). The input for the program (i.e., the Φ’s) can be reduced to the determination of the antichains of the root poset. (Indeed, for any finite poset, there is a canonical bijection mapping the antichain {a_{1}, …, a_{k}} to the dual order ideal which is the union of the principal dual order ideals V_{a_{1}}, …, V_{a_{k}}.) In turn, the calculation of the antichains has been done using the Maple program Coxeter [Ste]. Note that the total number of ideals in type E_{6} is 833, one more than the number given in [Sh, Theorem 3.6]. Our counting agrees with formula (1.1).

References

[AKOP] G. E. Andrews, C. Krattenthaler, L. Orsina and P. Papi, ad-nilpotent 𝔟-ideals in sl(n) having a fixed class of nilpotence: combinatorics and enumeration, Trans. Amer. Math. Soc. (to appear), math.RA/0004107.

[A1] C. A. Athanasiadis, Characteristic polynomials of subspace arrangements and finite fields, Adv. in Math. 122 (1996), 193–233.

[A2] C. A. Athanasiadis, Deformations of Coxeter hyperplane arrangements and their characteristic polynomials, Arrangements — Tokyo 1998 (M. Falk and H. Terao, eds.), Kinokuniya Company Ltd., Advanced Studies in Pure Mathematics, vol. 27, 2000, pp. 1– 26.

[CP] P. Cellini and P. Papi, ad-nilpotent ideals of a Borel subalgebra, J. Algebra 225 (2000), 130–141.

[F] W. Feller, An introduction to probability theory and its applications, Vol. 1, 2nd ed., Wiley, New York, 1957.

[GHd] A. M. Garsia and J. Haglund, A proof of the q, t-Catalan positivity conjecture, Discrete Math. (to appear).

[GHn] A. M. Garsia and M. Haiman, A remarkable q, t-Catalan sequence and q-Lagrange inversion, J. Alg. Combin. 5 (1996), 191–244.

[GR] G. Gasper and M. Rahman, Basic hypergeometric series, Encyclopedia of Math. And Its Applications 35, Cambridge University Press, Cambridge, 1990.

[H] J. Haglund, Conjectured statistics for the q, t-Catalan numbers, Adv. in Math. (to appear).

[Hu] J. E. Humphreys, Reflection groups and Coxeter groups, Cambridge University Press, Cambridge, 1990.

[K1] B. Kostant, Eigenvalues of a Laplacian and commutative Lie subalgebras, Topology 3, suppl. 2 (1965), 147–159.

[K2] B. Kostant, The Set of Abelian ideals of a Borel Subalgebra, Cartan Decompositions, and Discrete Series Representations, Internat. Math. Res. Notices 5 (1998), 225–252.

[Kr] C. Krattenthaler, Permutations with restricted patterns and Dyck paths, Adv. Appl. Math. 27 (2001), 510–530.

[Ma] I. G. Macdonald, Symmetric Functions and Hall Polynomials, second edition, Oxford University Press, New York/London, 1995.

[Mo] S. G. Mohanty, Lattice path counting and applications, Academic Press, 1979.

[OP] L. Orsina and P. Papi, Enumeration of ad-nilpotent ideals of a Borel subalgebra in type A by class of nilpotence, Comptes Rendus Acad. Sciences Paris Ser. I Math. 330 (2000), 651–655.

[R] V. Reiner, Non-crossing partitions for classical reflection groups, Discrete Math. 177 (1997), 195–222.

[Sh] J. Shi, The number of ⊕-sign types, Quart. J. Math. Oxford 48 (1997), 93–105.

[Si] R. Simion, Noncrossing partitions, Discrete Math. 217 (2000), 367–409.

[Sta] R. P. Stanley, Enumerative Combinatorics, Vol. 1, Wadsworth & Brooks/Cole, Pacific Grove, California, 1986, reprinted by Cambridge University Press, Cambridge, 1998.

[Ste] J. R. Stembridge, Coxeter, Maple package, available at:

http://www.mat.lsa.umich.edu/~jrs/maple.html#coxeter.

Institut für Mathematik der Universität Wien, Strudlhofgasse 4, A-1090 Wien, Austria. e-mail: KRATT@Ap.Univie.Ac.At

Dipartimento di Matematica, Istituto G. Castelnuovo Università di Roma “La Sapienza” Piazzale Aldo Moro 5 00185 Rome — ITALY e-mail: orsina@mat.uniroma1.it, papi@mat.uniroma1.it