%% if you are submitting an initial manuscript then you should have submission as an option here %% if you are submitting a revised manuscript then you should have revision as an option here %% otherwise options taken by the article class will be accepted \documentclass[finalversion]{FPSAC2020} \articlenumber{76} %% but DO NOT pass any options (or change anything else anywhere) which alters page size / layout / font size etc %% note that the class file already loads {amsmath, amsthm, amssymb} \theoremstyle{plain} \newtheorem{thm}{Theorem}[section] \newtheorem{lem}[thm]{Lemma} \theoremstyle{definition} \newtheorem{dfn}[thm]{Definition} \usepackage{mathtools} \usepackage{ytableau} \usepackage{tikz} \usepackage{paralist} %for compact lists \usepackage{leftidx} %Ermöglicht linke Hochzahlen durch \leftidx{^a}{x} %%%%%%%%%% MathOperators %%%%%%%%%% \DeclareMathOperator{\asym}{\mathbf{ASym}} \DeclareMathOperator{\sym}{\mathbf{Sym}} \DeclareMathOperator{\sgn}{sgn} \DeclareMathOperator{\ct}{CT} \DeclareMathOperator{\E}{E} \DeclareMathOperator{\fd}{\Delta} %\DeclareMathOperator{\bd}{\delta} \DeclareMathOperator{\id}{id} \DeclareMathOperator{\Qfd}{\leftidx{^\textit{Q}}{\Delta}{}} %\DeclareMathOperator{\Qbd}{\leftidx{^\textit{Q}}{\delta}{}} \DeclareMathOperator{\subsets}{\mathbf{Subsets}} \DeclareMathOperator{\fpsacdelta}{\leftidx{^\textit{Q}}{\delta}{}} %%%%%%%%%% MONOTONE TRIANGLES %%%%%%%%%% % Three counters \newcounter{xmt} \newcounter{ymt} \newcounter{nhmt} % Definition of \mt % To draw a MT, take the rows from bottom to top as sets in the argument, e.g. \begin{tikzpicture}\mt{{1,3,4},{1,4},{2}}\end{tikzpicture} \newcommand\mt[1]{ \setcounter{ymt}{-1} \foreach \p in {#1} { \addtocounter{ymt}{1} \setcounter{xmt}{\value{ymt}-1} \foreach \q in \p { \addtocounter{xmt}{2} \node at (\value{xmt}*.5,\value{ymt}*.5) {\q}; } } } %% define your title in the usual way \title{On a fourfold refined enumeration of alternating sign trapezoids} %% define your authors in the usual way %% use \addressmark{1}, \addressmark{2} etc for the institutions, and use \thanks{} for contact details \author[Hans H{\"o}ngesberg]{Hans H{\"o}ngesberg\thanks{\href{mailto:hans.hoengesberg@univie.ac.at}{hans.hoengesberg@univie.ac.at}. Supported by the Austrian Science Foundation FWF (SFB grant F50).}\addressmark{1}} %% then use \addressmark to match authors to institutions here \address{\addressmark{1}Fakult{\"a}t f{\"u}r Mathematik, Universit{\"a}t Wien, Vienna, Austria} %% put the date of submission here \received{\today} %% leave this blank until submitting a revised version %\revised{} %% put your English abstract here, or comment this out if you don't have one yet %% please don't use custom commands in your abstract / resume, as these will be displayed online %% likewise for citations -- please don't use \cite, and instead write out your citation as something like (author year) \abstract{Alternating sign trapezoids have recently been introduced as a generalization of alternating sign matrices. Fischer established a threefold refined enumeration of alternating sign trapezoids and provided three statistics on column strict shifted plane partitions with the same joint distribuition. We extend this result by another statistic that generalizes the number of $-1$'s in alternating sign matrices.} %% put your French abstract here, or comment this out if you don't have one %\resume{Abstract in French} %% put your keywords here, or comment this out if you don't have them yet \keywords{Alternating sign trapezoids, plane partitions, constant term formula} %% you can include your bibliography however you want, but using an external .bib file is STRONGLY RECOMMENDED and will make the editor's life much easier %% regardless of how you do it, please use numerical citations, ie. [xx, yy] in the text %% this sample uses biblatex, which (among other things) takes care of URLs in a more flexible way than bibtex %% but you can use bibtex if you want \addbibresource{ASM.bib} %% note the \printbibliography command at the end of the file which goes with these biblatex commands \begin{document} \maketitle %% note that you DO NOT have to put your abstract here -- it is generated by \maketitle and the \abstract and \resume commands above \section{Introduction} Since their introduction in the early 1980s, alternating sign matrices have aroused great interest among combinatorialists. Mills, Robbins, and Rumsey \cite{MRR83} conjectured them to be equinumerous with descending plane partitions, which had been enumerated by Andrews \cite{And79} a few years earlier; it was finally independently proved over a decade later by Zeilberger \cite{Zei96a} and Kuperberg \cite{Kup96}. Since then, the spellbinding research of alternating sign matrices has revealed new equinumerous classes of combinatorial objects but finding bijections between them remains one of the most challenging problems. Equally distributed statistics on these objects might finally lead to those eagerly awaited bijections. Embracing this idea, we provide a fourfold refined enumeration of alternating sign trapezoids, a recently defined generalization of alternating sign matrices. Moreover, we establish four statistics on certain column strict shifted plane partitions with the same joint distribution. Thus, we generalize the recent refined enumerations of alternating sign trapezoids and of column strict shifted plane partitions by Fischer \cite{Fis}. \section{Preliminaries} We start by introducing \emph{alternating sign trapezoids} and \emph{column strict shifted plane partitions} together with four statistics on each of these classes of objects. \begin{dfn} For given integers~$n \geq 1$ and $l \geq 2$, an \emph{$(n,l)$-alternating sign trapezoid} is an array of $-1$s, $0$s, and $+1$s in a trapezoidal shape with $n$ rows of the following form \begin{equation*} \begin{array}[t]{ccccccccc} a_{1,1}&a_{1,2}&\cdots&\cdots&\cdots&\cdots&\cdots&\cdots&a_{1,2n+l-2}\\ &a_{2,2}&\cdots&\cdots&\cdots&\cdots&\cdots&a_{2,2n+l-3}&\\ &&\ddots&&&&\reflectbox{$\ddots$}&&\\ &&&a_{n,n}&\cdots&a_{n,n+l-1}&&& \end{array} \end{equation*} such that \begin{compactitem} \item the nonzero entries alternate in sign in each row and each column, \item the topmost nonzero entry in each column is $1$ (if existent), \item the entries in each row sum up to $1$, and \item the entries in the central $l-2$ columns sum up to $0$. \end{compactitem} An \emph{$(n,1)$-alternating sign trapezoid} is defined as above with the exception that the entry in the bottom row can be $0$ or $1$. \end{dfn} The entries in each column of an alternating sign trapezoid sum up to $0$ or $1$. A column whose entries sum up to $1$ is called a \emph{$1$-column}. If, in addition, the bottom entry of a $1$-column is $0$, we call that column a \emph{$10$-column}. Note that the number of $1$-columns in any $(n,l)$-alternating sign trapezoid is exactly $n$ if $l \neq 1$; otherwise, it is $n$ or $n-1$. % \begin{figure}[ht] \centering \begin{equation*} \begin{array}[t]{cccccccccc} 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 & 0 & 0\\ & 1 & 0 & 0 & 0 & -1 & 0 & 1 & 0 & \\ & & 0 & 0 & 0 & 0 & 1 & 0 & & \\ & & & 1 & 0 & 0 & 0 & & & \\ \end{array} \end{equation*} \caption{$(4,4)$-alternating sign trapezoid with weight $Q R^2 T^2$} \label{fig:AST} \end{figure} We introduce four different statistics on alternating sign trapezoids by associating the following weight to $(n,l)$-alternating sign trapezoids if $l \ge 2$: % \begin{multline*} Q^{\text{\# $-1$s}}\, R^{\text{\# $1$-columns within the $n$ leftmost columns}}\\ \times S^{\text{\# $10$-columns within the $n$ leftmost columns}}\, T^{\text{\# $10$-columns within the $n$ rightmost columns}}. \end{multline*} % An example of a $(4,4)$-alternating sign trapezoid is given in \cref{fig:AST}. For $(n,1)$-alternating sign trapezoids, however, we have to adapt the weight in the following way: % \begin{multline*} Q^{\text{\# $-1$s}}\, R^{\text{\# $1$-columns within the $n$ leftmost columns}}\\ \times S^{\text{\# $10$-columns within the $n-1$ leftmost columns}}\, T^{\text{\# $10$-columns within the $n-1$ rightmost columns}}\\ \times (S+T-Q)^{\left[\text{the central column is a $10$-column}\right]}, \end{multline*} % where we make use of the \emph{Iverson bracket}: For any logical proposition~$P$, $[P]=1$ if $P$ is satisfied and $[P]=0$ otherwise. Ayyer, Behrend, and Fischer \cite{ABF} showed that $n \times n$-alternating sign matrices are equinumerous with $(n-1,3)$-alternating sign trapezoids. As a corollary of \cite[Theorem 1.2]{ABF}, the statistic~$Q$ generalizes the number of $-1$s in alternating sign matrices. \begin{dfn} A \emph{shifted Young diagram} is a finite collection of cells arranged in rows of strictly decreasing lengths such that each row is indented by one cell compared to the row above. The \emph{shape} of a shifted Young diagram is the sequence $\lambda = \left( \lambda_1,\dots,\lambda_n \right)$ of its row lengths. Note that $\lambda$ is a \emph{strict partition}, that is, a sequence of strictly decreasing positive integers. \begin{figure}[ht] \centering \hfill\begin{minipage}{.4\textwidth} \ydiagram{5,1+3,2+2} \end{minipage}% \begin{minipage}{.4\textwidth} \begin{ytableau} 9 & 8 & 8 & 7 & 3 \\ \none & 7 & 7 & 5 \\ \none & \none & 6 & 1 \end{ytableau} \end{minipage} \caption{A shifted Young diagram of shape $(5,3,2)$ and a column strict shifted plane partition of class $4$ of the same shape with weight~$Q^2 R^3 S T$ for $d=3$} \label{fig:SYD+CSSPP} \end{figure} A \emph{column strict shifted plane partition} is a filling of a shifted Young diagram with positive integers such that the entries weakly decrease along each row and strictly decrease down each column. It is of \emph{class k} if the first entry of each row $i$ is exactly $k+\lambda_i$, that is, exactly $k$ plus its corresponding row length. \end{dfn} Note that we cannot always associate a class to a given column strict shifted plane partition. Column strict shifted plane partitions of class $2$ correspond to \emph{descending plane partitions}. We introduce four different statistics on column strict shifted plane partitions of class $k$ of which two depend on a fixed parameter~$d \in \{1,\dots,k\}$: % \begin{compactitem} \item $Q$ counts the number of parts equal to $\{2,3,\dots,j-i+k\} \setminus \{j-i+d\}$, \item $R$ counts the number of rows, \item $S$ counts the number of parts equal to $j-i+d$, and \item $T$ counts the number of $1$s, \end{compactitem} % where $i$ is the row and $j$ is the column of the respective part. An example of a shifted Young diagram and a column strict shifted plane partition is presented in \cref{fig:SYD+CSSPP}. Note that the parts counted by the statistic~$Q$ generalize the parts in descending plane partitions that are referred to as \emph{special parts} by Mills, Robbins, and Rumsey \cite{MRR83} and enumerated by Behrend, Di Francesco, and Zinn-Justin \cite{BFZ12}. Fischer \cite{Fis} established refined enumerations of alternating sign trapezoids and column strict shifted plane partition taking account of the statistics~$S$, $T$, and $P$. We extend her proof by adding the fourth statistic in order to prove the following main theorem of this paper: \begin{thm} \label{thm:maintheorem} Let $n,l \ge 1$ and $1 \le d \le l-1$. Then the joint distribution of the corresponding statistics~$Q$, $R$, $S$, and $T$ on $(n,l)$-alternating sign trapezoids and on column strict shifted plane partitions of class $l-1$ with at most $n$ entries in the first row coincide. \end{thm} % Note that we can generalize \cref{thm:maintheorem} by providing a combinatorial interpretation for the case $d=0$. \section{Weighted Enumeration of Alternating Sign Trapezoids} \label{sec:EnumAST} First, we provide the generating function of $(n,l)$-alternating sign trapezoids. For this purpose, we heavily exploit the correspondence between alternating sign trapezoids and truncated \emph{monotone triangles}, both as defined below. For the sake of simplicity, we assume that $l \ge 2$ throughout the extended abstract. However, note that \cref{thm:maintheorem} includes the case $l = 1$. \begin{dfn} For a given integer~$n \geq 1$, a \emph{monotone triangle of order $n$} is an array of integers in a triangular shape with $n$ rows of the following form % \begin{center} \begin{tikzpicture}[xscale=2] \mt{{$a_{n,1}$,$a_{n,2}$,$a_{n,3}$,\dots,$a_{n,n}$}, {$a_{n-1,1}$,$a_{n-1,2}$,\dots,$a_{n-1,n-1}$}, {\dots,\dots,\dots}, {$a_{2,1}$,$a_{2,2}$}, {$a_{1,1}$}} \end{tikzpicture} \end{center} % such that the entries strictly increase along rows and weakly increase both along $\nearrow$-diagonals and $\searrow$-diagonals. \end{dfn} \begin{dfn} For given integers $p,q \geq 0$ and $n \geq 1$ such that $p+q \le n$ as well as a weakly decreasing sequence $\mathbf{s}=(s_1,s_2,\dots,s_p)$ and a weakly increasing sequence $\mathbf{t}=(t_{n-q+1},t_{n-q+2},\allowbreak \dots,\allowbreak t_n)$ of nonnegative integers, we define an \emph{$(\mathbf{s}$,$\mathbf{t})$-tree} as an array of integers which arises from a monotone triangle of order~$n$ by truncating the diagonals as follows: for each $1 \leq i \leq p$, we delete the $s_i$ bottom entries of the $i^{\text{th}}$ $\nearrow$-diagonal; for each $n-q+1 \leq i \leq n$, we delete the $t_i$ bottom entries of the $i^{\text{th}}$ $\searrow$-diagonal. All diagonals are counted from left to right. We say that an $(\mathbf{s}$,$\mathbf{t})$-tree has \emph{bottom row $\mathbf{k}=(k_1, \dots, k_{n})$} if the following holds true: for all $i$ such that $1 \le i \le p$ or $n-q+1 \le i \le n$, the integer $k_i$ is the bottom entry of the $i^{\text{th}}$ $\nearrow$-diagonal or the $i^{\text{th}}$ $\searrow$-diagonal, respectively; and, for all $p < i < n-q+1$, the integer $k_i$ is equal to the entry $a_{n,i}$ in the bottom row of the original monotone triangle. \end{dfn} \begin{figure}[ht] \centering \begin{tikzpicture}[xscale=2] \mt{{,-1,2,},{,2,3},{-3,3},{1}} \end{tikzpicture} \caption{$\left((2),(1)\right)$-tree with bottom row $(-3,-1,2,3)$} \label{fig:Tree} \end{figure} We can transform an alternating sign trapezoid into a tree; see \cite{Fis19} for the detailed construction. To illustrate its main features, we number the $n$ leftmost columns of an $(n,l)$-alternating sign trapezoid from $-n$ to $-1$ and the $n$ rightmost columns from $1$ to $n$. The \emph{$1$-column vector} $\mathbf{c}=\left(c_1,\dots,c_n\right)$ records the positions of the $1$-columns of the alternating sign trapezoid; hence, $-n \le c_1 < \dots < c_m < 0 < c_{m+1} < \dots < c_n\le n$ for some $0 \le m \le n$. The construction above yields an $(\mathbf{s}$,$\mathbf{t})$-tree with bottom row $(c_1,\dots,c_m,c_{m+1}+l-3,\dots,c_{n}+l-3)$ such that $\mathbf{s} = ( -c_1-1,\dots,-c_m-1 )$ and $\mathbf{t} = ( c_{m+1}-1,\dots,c_n-1 )$. Regarding the statistics of alternating sign trapezoids, we make the following observations: a $-1$ in the alternating sign trapezoid corresponds to an entry $a_{i,j}$ in the tree which has two neighbouring entries $a_{i+1,j}$ and $a_{i+1,j+1}$ in the row below such that $a_{i+1,j} < a_{i,j} < a_{i+1,j+1}$. The positions of the $1$-columns are reflected in the bottom row of the tree, and $10$-columns cause the corresponding diagonals in the tree to have twice the same bottom entries. In \cref{fig:Tree}, we present the tree corresponding to the $(4,4)$-alternating sign trapezoid with $1$-column vector $(-3,-1,1,2)$ in \cref{fig:AST}. To enumerate monotone triangles and trees, we use operator formulae and constant term expressions. To this end, we need to introduce several operators and notations. First, we define the \emph{symmetriser} $\sym$ and the \emph{antisymmetriser} $\asym$ of a function $f(x_1,\dots,x_n)$. Let $\mathfrak{S}_n$ be the symmetric group of degree $n$. Then % \begin{align*} \sym_{x_1,\dots,x_n} f(x_1,\dots,x_n) &\coloneqq \sum_{\sigma\in\mathfrak{S}_n} f(x_{\sigma(1)},\dots,x_{\sigma(n)})\,\text{and}\\ \asym_{x_1,\dots,x_n} f(x_1,\dots,x_n) &\coloneqq \sum_{\sigma\in\mathfrak{S}_n} \sgn(\sigma) f(x_{\sigma(1)},\dots,x_{\sigma(n)}). \end{align*} % We use $\sym_{\mathbf{x}}$ and $\asym_{\mathbf{x}}$ as an abbreviation if $\mathbf{x}=(x_1,\dots,x_n)$ is clear from the context. Furthermore, $\ct_{\mathbf{x}} f(\mathbf{x})=\ct_{x_1,\dots,x_n} f(x_1,\dots,x_n)$ denotes the constant term of the function $f$ with respect to the variables $x_1,\dots,x_n$. Finally, we define the \emph{shift operator} $\E_x f(x) \coloneqq f(x+1)$, the \emph{forward difference operator} $\fd_x \coloneqq \E_x - \id$, and the \emph{backward difference operator} $\delta_x \coloneqq \id - \E_x^{-1}$, where $\id$ denotes the standard identity operator. We use the notation $\E_a f(a) \coloneqq \left. \E_x f(x)\right|_{x=a}$ for a given a variable~$x$ and an integer~$a$. This abbreviatory notation is correspondingly used for other operator expressions. %Note that all these operators commute independently of the variables they refer to. Fischer and Riegler \cite{FR13} provided a weighted enumeration of monotone triangles: \begin{thm} The generating function of monotone triangles of order $n$ with bottom row $\mathbf{k}=(k_1, \dots,\allowbreak k_{n})$ with respect to the statistic~$Q$ is given by $M_n(\mathbf{k})$ with $M_n \left(\mathbf{x}\right)$ defined as % \begin{equation} \label{eq:Q-M} \ct_{\mathbf{y}} \left( \asym_{\mathbf{y}} \left( \prod_{i=1}^n \left(1+y_i\right)^{x_i} \prod_{1 \le i < j \le n} \left( Q - (1-Q)y_i + y_j + y_i y_j \right) \right) \prod_{1 \le i < j \le n} \left( y_j - y_i \right)^{-1} \right). \end{equation} % \end{thm} The crucial observation is that if we repeatedly apply $-\fd_{x_i}$ and $\delta_{x_i}$ to $\left. M_n(\mathbf{x}) \right|_{Q=1}$, we enumerate monotone triangles with truncated diagonals. By generalising the difference operators, we obtain the following enumeration formula for trees with respect to the statistic $Q$ as a corollary of \cite[Theorem 5]{Fis18}. \begin{thm} The generating function of $(\mathbf{s}$,$\mathbf{t})$-trees with $\mathbf{s}=(s_1,s_2,\dots,s_p)$, $\mathbf{t}=(t_{n-q+1},\allowbreak t_{n-q+2},\allowbreak \dots,\allowbreak t_n)$ and bottom row $\mathbf{k}=(k_1, \dots, k_{n})$ with respect to the statistic~$Q$ is given by % \begin{equation*} \prod_{i=1}^{p} \left( -\Qfd_{k_i} \right)^{s_i} \prod_{i=n-q+1}^{n} \leftidx{^\textit{Q}}{\delta}{}_{k_i}^{t_i} M_n \left(\mathbf{k}\right), \end{equation*} % where $\Qfd_x \coloneqq (Q - (1-Q)\fd_x)^{-1}\fd_x$ and $\leftidx{^\textit{Q}}{\delta}{}_x \coloneqq (Q - (Q-1)\delta_x)^{-1}\delta_x$. \end{thm} We use the correspondence between alternating sign trapezoids and trees to obtain enumeration formulae. First, we consider alternating sign trapezoids with prescribed $1$-column vectors. The following theorem can be proved by similar means as \cite[Theorem 4.4]{Hon}: \begin{lem} The generating function of $(n,l)$-alternating sign trapezoids with $1$-column vector $\mathbf{c}$ with respect to the statistics~$Q$, $S$, and $T$ is given by % \begin{equation} \label{eq:QST-AST1columns} \prod_{i=1}^{m} \left( \id - \frac{S}{Q} \delta_{c_i} \right) \left( \id + \Qfd_{c_i} \right) \left( -\Qfd_{c_i} \right)^{-c_i-1} \hspace*{-1ex} \prod_{i=m+1}^{n} \left( \id + \frac{T}{Q} \fd_{c_i} \right) \left( \id - \leftidx{^\textit{Q}}{\delta}{}_{c_i} \right) \leftidx{^\textit{Q}}{\delta}{}_{c_i}^{c_i-1} M_n \left( \mathbf{\tilde c} \right), \end{equation} % where $\mathbf{\tilde c}=\left(c_1,\dots,c_m,c_{m+1}+l-3,\dots,c_{n}+l-3\right)$. \end{lem} Instead of evaluating the previous polynomial at $\mathbf{\tilde c}$, we can shift the argument by suitable operators and take the constant term. In particular, \eqref{eq:QST-AST1columns} is equal to % \begin{multline} \label{eq:QST-AST1columns-shifted} \ct_{\mathbf{x}} \left( \prod_{i=1}^{m} \E_{x_i}^{c_i} \left( \id - \frac{S}{Q} \delta_{c_i} \right) \left( \id + \Qfd_{c_i} \right) \left( -\Qfd_{c_i} \right)^{-c_i-1} \right.\\ % \left. \times \prod_{i=m+1}^{n} \E_{x_i}^{c_i+l-3} \left( \id + \frac{T}{Q} \fd_{c_i} \right) \left( \id - \leftidx{^\textit{Q}}{\delta}{}_{c_i} \right) \leftidx{^\textit{Q}}{\delta}{}_{c_i}^{c_i-1} M_n \left( \mathbf{x} \right) \right)\\ % \shoveleft = \ct_{\mathbf{x}} \left( \prod_{i=1}^{m} \E_{x_i}^{-1} \frac{Q-(S-Q)\fd_{x_i}}{Q-(1-Q)\fd_{x_i}} \left( \frac{-\delta_{x_i}}{Q-(1-Q)\fd_{x_i}} \right)^{-c_i-1}\right.\\ % \left. \times \prod_{i=m+1}^{n} \E_{x_i}^{l-2} \frac{Q+(T-Q)\fd_{x_i}}{Q+(1-Q)\fd_{x_i}} \left( \frac{-\fd_{x_i}}{Q+(1-Q)\delta_{x_i}} \right)^{-c_i-1} M_n \left( \mathbf{x} \right) \right). \end{multline} % We analyse how the operators in \eqref{eq:QST-AST1columns-shifted} interact with the argument of the antisymmetriser in \eqref{eq:Q-M}: The effect of the shift operator $\E_{x_i}$ is the multiplication by $1 + y_i$. Therefore, the application of the forward difference operator $\fd_{x_i}$ or of the backward difference operator $\delta_{x_i}$ is equivalent to the multiplication by $y_i$ or by $y_i(1+y_i)^{-1}$, respectively. This observation implies that \eqref{eq:QST-AST1columns-shifted} equals % \begin{multline} \label{eq:QST-AST1columns-CT} \ct_{\mathbf{y}} \left( \asym_{\mathbf{y}} \left( \prod_{i=1}^m \left(-y_i\right)^{-c_i-1} \left(1+y_i\right)^{c_i} \left(Q-(1-Q)y_i\right)^{c_i+1} \frac{Q-(S-Q)y_i}{Q-(1-Q)y_i} \right. \right.\\ % \times \prod_{i=m+1}^n y_i^{c_i-1} \left(1+y_i\right)^{c_i+l-3} \left(Q+y_i\right)^{-c_i+1} \frac{Q+T y_i}{Q+y_i}\\ % \left. \left. \times \prod_{1 \le i < j \le n} \left( Q - (1-Q)y_i + y_j + y_i y_j \right) \right) \prod_{1 \le i < j \le n} \left( y_j - y_i \right)^{-1} \right). \end{multline} % Thus far, we have considered $(n,l)$-alternating sign trapezoids with prescribed $1$-column vector $\mathbf{c}$. To sum over all $c_i$ such that $-n \le c_1 < \dots < c_m < 0 < c_{m+1} < \dots < c_n \le n$, we ignore the upper and lower bound in the summation since the polynomial in \eqref{eq:QST-AST1columns-CT} has no constant term if $c_1 < n$ or $c_n > n$. Hence, by using some geometric series evaluation, we obtain that the argument of the antisymmetriser in \eqref{eq:QST-AST1columns-CT} is equal to % \begin{multline} \label{eq:QST-ASTsumOverC} \prod_{i=1}^m \frac{1}{1+y_i} \left( \frac{-y_i}{\left( 1+y_i \right) \left( Q-(1-Q)y_i \right)} \right)^{m-i} \\ % \times \left( 1 - \prod_{j=1}^i \left( \frac{-y_j}{\left( 1+y_j \right) \left( Q-(1-Q)y_j \right)} \right) \right)^{-1} \frac{Q-(S-Q)y_i}{Q-(1-Q)y_i}\\ % \times \prod_{i=m+1}^n \left( 1+y_i \right)^{l-2} \left( \frac{y_i \left( 1+y_i \right)}{Q+y_i} \right)^{i-m-1} \left( 1 - \prod_{j=i}^n \left( \frac{y_j \left( 1+y_j \right)}{Q+y_j} \right) \right)^{-1} \frac{Q+T y_i}{Q+y_i}\\ % \times \prod_{1 \le i < j \le n} \left( Q - (1-Q)y_i + y_j + y_i y_j \right) \left( y_j - y_i \right)^{-1}. \end{multline} % Before summing over all $m$ such that $0 \le m \le n$, we apply the symmetriser to the expression~\eqref{eq:QST-ASTsumOverC}. To this end, we use the following trick by Fischer \cite{Fis}: We set $\mathfrak{S}_n^m \coloneqq \{ \sigma \in \mathfrak{S}_n \mid \sigma (i) < \sigma (j)\, \forall\, 1 \le i < j \le m\, \vee\, m+1 \le i < j \le n \}$ and define % \begin{equation*} \subsets_{x_1,\dots,x_m}^{x_{m+1},\dots,x_n} f(x_1,\dots,x_n) \coloneqq \sum_{\sigma \in \mathfrak{S}_n^m} f\left( x_{\sigma(1)},\dots,x_{\sigma(n)} \right). \end{equation*} % It follows that % \begin{equation*} \sym_{x_1,\dots,x_n} f(x_1,\dots,x_n) = \subsets_{x_1,\dots,x_m}^{x_{m+1},\dots,x_n} \sym_{x_1,\dots,x_m} \sym_{x_{m+1},\dots,x_n} f(x_1,\dots,x_n). \end{equation*} % That is, we first apply $\sym_{y_1,\dots,y_m}$ and $\sym_{y_{m+1},\dots,y_n}$ to \eqref{eq:QST-ASTsumOverC} by means of the following antisymmetriser lemma \cite{Hon}: \begin{lem} Let $n \ge 1$. Then % \begin{multline*} \asym_{\mathbf{x}} \left( \prod_{i=1}^{n} \frac{\left( \frac{x_i(1+x_i)}{Q+x_i} \right)^{i-1}}{1-\prod_{j=i}^{n} \frac{x_j(1+x_j)}{Q+x_j} } \prod_{1\leq i,thick] (-1.75,0)--(4.75,0) node[right]{$x$}; \draw[->,thick] (0,-.75)--(0,8.75) node[above]{$y$}; \fill (4,0) circle (5pt); \fill (2,0) circle (5pt); \fill (1,0) circle (5pt); \fill (0,8) circle (5pt); \fill (0,6) circle (5pt); \fill (0,5) circle (5pt); %\coordinate[label=225:$S_2$] (S2) at (0,3); %\fill (S2) circle (2pt); \draw[ultra thick] (-1,8) -- (0,8) -- (0,7) -- (2,7) -- (2,6) -- (3,6) -- (3,2) -- (4,2) -- (4,0); \node at (-0.5,8.5) {$9$}; \node at (0.5,7.5) {$8$}; \node at (1.5,7.5) {$8$}; \node at (2.5,6.5) {$7$}; \node at (3.5,2.5) {$3$}; \draw[ultra thick] (-1,6) -- (1,6) -- (1,4) -- (2,4) -- (2,0); \node at (-0.5,6.5) {$7$}; \node at (0.5,6.5) {$7$}; \node at (1.5,4.5) {$5$}; \draw[ultra thick] (-1,5) -- (0,5) -- (0,0) -- (1,0); \node at (-0.5,5.5) {$6$}; \node at (0.5,0.5) {$1$}; \end{tikzpicture} \caption{Family of nonintersecting lattice paths corresponding to the column strict shifted plane partition in \cref{fig:SYD+CSSPP}.} \label{fig:LatticePaths} \end{figure} This construction yields a bijective correspondence between column strict shifted plane partitions of class $l-1$ with at most $n$ entries in the first row and the family of nonintersecting lattice paths using only horizontal $\leftarrow$ and vertical $\uparrow$ unit steps with start points $S \subseteq \{ S_i \coloneqq (i,0) \mid 0 \le i \le n-1\}$ and end points $E \subseteq \{ E_i \coloneqq (0,i+l-1) \mid 0 \le i \le n-1\}$ such that $S_i \in S$ if and only if $E_i \in E$. By this interpretation of column strict shifted plane partitions as a family of nonintersecting lattice paths and by the Lindstr{\"o}m-Gessel-Viennot lemma, it can be shown that column strict shifted plane partitions of class $l-1$ with at most $n$ entries in the first row are enumerated by % \begin{equation} \label{eq:AndrewsDet} \det_{0 \le i,j \le n-1} \left( \binom{i+j+l-1}{i} + \delta_{i,j} \right). \end{equation} % This was first proved by Andrews \cite{And79}. In fact, this determinant~\eqref{eq:AndrewsDet} can be obtained from \eqref{eq:QRST-ASTbinom} by setting $Q=R=S=T=1$. Andrews' result can be generalized: By setting $Q=1$ in \eqref{eq:QRST-ASTbinom}, we obtain % \begin{equation*} \det_{0 \le i,j \le n-1} \left(R \sum_{k = 0}^{i} T^{i-k} \left( \binom{k+j+l-3}{k} + \binom{k+j+l-3}{k-1} S \right) + \delta_{i,j} \right). \end{equation*} % This is the generating function of column strict shifted plane partitions of class $l-1$ with at most $n$ entries in the first row with respect to the statistics~$R$, $S$, and $T$, which was proved by Fischer \cite{Fis}. In particular, we see that % \begin{equation*} \sum_{k = 0}^{i} T^{i-k} \binom{k+j+l-3}{k} \end{equation*} % is the generating function of lattice paths from $(i,0)$ to $(0,j+l-1)$ where the line $y=x+d$ is reached by a vertical step, and $T$ counts the number of horizontal steps at height $0$. As a straightforward consequence, % \begin{equation*} \sum_{k = 0}^{i} T^{i-k} \sum_{m=0}^{j} \binom{j}{m} \binom{k+l-3}{k-m} Q^{k-m} \end{equation*} % is the generating function of lattice paths from $(i,0)$ to $(0,j+l-1)$ where the line $y=x+d$ is reached by a vertical step, $T$ takes the number of horizontal steps at height $0$ into account, and, in addition, $Q$ counts the number of horizontal steps which are under the line $y=x+l-1$ and have at least height $1$, that is, which are not already counted by $T$. Similarly, % \begin{equation*} \sum_{k = 0}^{i} T^{i-k} \sum_{m=0}^{j} \binom{j}{m} \binom{k+l-3}{k-m-1} S Q^{k-m} \end{equation*} % is the generating function of lattice paths from $(i,0)$ to $(0,j+l-1)$ where the line $y=x+d$ is reached by a horizontal step which $S$ keeps track of, $T$ counts the number of horizontal steps at height $0$, and $Q$ counts the number of horizontal steps which are beneath the line $y=x+l-1$ that are not already taken into account by the statistics~$T$ and $S$. As a result, \eqref{eq:QRST-ASTbinom} is also the generating function of column strict shifted plane partitions of class $l-1$ with at most $n$ entries in the first row with respect to the statistics~$Q$, $R$, $S$, and $T$. This completes the proof of \cref{thm:maintheorem}. %\acknowledgements{} %% if you use biblatex then this generates the bibliography %% if you use some other method then remove this and do it your own way \printbibliography \end{document}