SIGNED WORDS AND PERMUTATIONS, II; THE EULER-MAHONIAN POLYNOMIALS Dominique Foata

SIGNED WORDS AND PERMUTATIONS, II;

THE EULER-MAHONIAN POLYNOMIALS Dominique Foata

Institut Lothaire, 1, rue Murner F-67000 Strasbourg, France [email protected]

Guo-Niu Han

I.R.M.A. UMR 7501, Universit´e Louis Pasteur et CNRS 7, rue Ren´e-Descartes, F-67084 Strasbourg, France

[email protected]

Submitted: May 6, 2005; Accepted: Oct. 28, 2005; Published: Nov. 08, 2005 Mathematics Subject Classifications: 05A15, 05A30, 05E15

Dans la th´eorie de Morse, quand on veut ´etudier un espace, on introduit une fonction num´erique; puis on aplatit cet espace sur l’axe de la valeur de cette fonction. Dans cette op´eration d’aplatissement, on cr´ee des singularit´es de la fonction et celles-ci sont en quelque sorte les vestiges de la topologie qu’on a tu´ee.

Ren´e Thom,Logos et Th´eorie des catastrophes,.

Dedicated to Richard Stanley,

on the occasion of his sixtieth birthday.

Abstract

As for the symmetric group of ordinary permutations there is also a statistical study of the group of signed permutations, that consists of calculating multi- variable generating functions for this group by statistics involving record values and the length function. Two approaches are here systematically explored, us- ing the flag-major index on the one hand, and the flag-inversion number on the other hand. The MacMahon Verfahren appears as a powerful tool throughout.

1. Introduction

The elements of the hyperoctahedral group B_n (n ≥ 0), usually called signed permutations, may be viewed as wordsw= x1x2. . . xn, where the letters xi are positive or negative integers and where|x₁| |x₂| . . . |x_n|is a permutation of 1 2 . . . n(see [Bo68]

p. 252–253). For typographical reasons we shall use the notation i:= −i in the sequel.

Using the χ-notation that maps each statement A onto the value χ(A) = 1 or 0

depending on whether A is true or not, we recall that the usual inversion number, invw, of the signed permutationw =x₁x₂. . . x_n is defined by

invw:= X

1≤j≤n

X

i<j

χ(x_i > x_j).

It also makes sense to introduce

invw:= X

1≤j≤n

X

i<j

χ(x_i > x_j),

and verify that thelength function (see [Bo68, p. 7], [Hu90, p. 12]), that will be denoted by “finv” (flag-inversion number) in the whole paper, can be defined, using the notation negw:= P

1≤j≤n

χ(x_j <0), by

finvw:= invw+ invw+ negw.

Another equivalent definition will be given in (7.1). The flag-major index “fmaj” and the flag descent number “fdes” were introduced by Adin and Roichman [AR01] and read:

fmajw := 2 majw+ negw;

fdesw := 2 desw+χ(x1 <0);

where majw := P

jj χ(x_j > x_j+1) denotes the usual major index of w and desw the number of descents desw:=P

jχ(xj > xj+1).

Another class of statistics needed here is the class of lower records. A letter x_i (1 ≤ i ≤ n) is said to be a lower record of the signed permutation w = x1x2. . . xn, if |x_i| < |x_j| for all j such that i+ 1 ≤ j ≤ n. When reading the lower records of w from left to right we get a signed subword, called the lower record subword, denoted by Lowerw. Denote the number of positive (resp. negative) letters in Lowerw by lowerpw (resp. lowernw).

In our previous paper [FoHa05] we gave the construction of a transformation Ψ on (arbitrary) signed words, that is, words, whose letters are positive or negative with repetitions allowed. When applied to the group B_n, the transformation Ψ has the following properties:

(a) fmajw= finv Ψ(w) for every signed permutation w;

(b) Ψ is a bijection of B_n onto itself, so that “fmaj” and “finv” are equidistributed over the hyperoctahedral group B_n;

(c) Lowerw = Lower Ψ(w), so that lowerpw = lowerp Ψ(w) and lowernw = lowern Ψ(w).

Actually, the transformation Ψ has stronger properties than those stated above, but these restrictive properties will suffice for the following derivation. Having properties (a)–(c) in mind, we see that the two three-variable statistics (fmaj,lowerp,lowern) and (finv,lowerp,lowern) are equidistributed overB_n. Hence, the two generating polynomials

fmajB_n(q, X, Y) := X

w∈Bn

q^fmajX^lowerpwY^lowernw

finvB_n(q, X, Y) := X

w∈Bn

q^finvX^lowerpwY^lowernw

are identical. To derive the analytical expression for the common polynomial we have two approaches, using the “fmaj” interpretation, on the one hand, and the “finv”

geometry, on the other. In each case we will go beyond the three-variable case, as we consider the generating polynomial for the group B_n by the five-variable statistic (fdes,fmaj,lowerp,lowern,neg)

(1.1) ^fmajB_n(t, q, X, Y, Z) := X

w∈Bn

t^fdeswq^fmajwX^lowerpwY^lowernwZ^negw

and the generating polynomial for the group B_n by the four-variable statistic (finv,lowerp,lowern,neg)

(1.2) ^finvB_n(q, X, Y, Z) := X

w∈Bn

q^finvwX^lowerpwY^lowernwZ^negw.

Using the usual notations for the q-ascending factorial (a;q)_n:=

1, ifn= 0;

(1−a)(1−aq) . . . (1−aqⁿ⁻¹), ifn≥1;

(1.3)

in its finite form and

(a;q)_∞:= lim_n(a;q)_n = Q

n≥0

(1−aqⁿ);

(1.4)

in its infiniteform, we consider the products

(1.5) H_∞(u) :=

uqZ+q

1−q² −ZY

;q²

∞

uq(Z+q)

1−q² +X

;q²

∞

,

in its infinite version, and (1.6) H_2s(u) := 1−q²

1−q²+uq^2s+1(Z +q)

uq Z +q−ZY(1−q²) 1−q²+uq^2s+1(Z+q);q²

s

u q(Z+q) +X(1−q²) 1−q²+uq^2s+1(Z+q);q²

s+1

,

as well as

(1.7) H_2s+1(u) :=

uq Z+q−ZY(1−q²)

1−q²+uq^2s+2(Zq+ 1);q²

s+1

u q(Z+q) +X(1−q²) 1−q²+uq^2s+2(Zq+ 1);q²

s+1

,

in its graded version under the form P

s≥0

t^sH_s(u).

The purpose of this paper is to prove the following two theorems and derive several applications regarding statistical distributions over B_n.

Theorem 1.1 (the “fmaj” approach). Let ^fmajBn(t, q, X, Y, Z) be the generating polynomial for the groupB_nby the five-variable statistic(fdes,fmaj,lowerp,lowern,neg) as defined in (1.1). Then

(1.8) X

n≥0

(1 +t)^fmajB_n(t, q, X, Y, Z) uⁿ

(t²;q²)_n+1 =X

s≥0

t^sH_s(u), where H_s(u) is the finite product introduced in(1.6) and (1.7).

Theorem 1.2 (the “finv” approach). Let ^finvB_n(q, X, Y, Z) be the generating polyno- mial for the group B_n by the four-variable statistic

(finv,lowerp,lowern,neg), as defined in (1.2). Then

(1.9) ^finvB_n(q, X, Y, Z) = (X+q+· · ·+qⁿ⁻¹+qⁿZ +· · ·+q²ⁿ⁻²Z +q²ⁿ⁻¹Y Z)

· · · ×(X+q+q²+q³Z +q⁴Z +q⁵Y Z)(X +q+q²Z+q³Y Z)(X+qY Z).

The proofs of those two theorems are very different in nature. For proving Theo- rem 1.1 we re-adapt theMacMahon Verfahren to make it work forsignedpermutations.

Ren´e Thom’s quotation that appears as an epigraph to this paper illustrates the essence of theMacMahon Verfahren. The topology of the signed permutations measured by the various statistics, “fdes”, “fmaj”, . . . must be reconstructed when the group of the signed permutations is mapped onto a set of plain words for which the calculation of the associated statistic is easy. There is then a combinatorial bijection between signed permutations and plain words that describes the “flattening” (“aplatissement”) process.

This is the content of Theorem 4.1.

Another approach might have been to make use of the P-partition technique introduced by Stanley [St72] and successfully employed by Reiner [Re93a, Re93b, Re93c, Re95a, Re95b] in his statistical study of the hyperoctahedral group.

Theorem 1.2 is based upon another definition of the length function for Bn (see formula (7.1)). Notice that in the two theorems we have included a variable Z, which takes the number “neg” of negativeletters into account. This allows us to re-obtain the classical results on the symmetric group by letting Z = 0.

In the next section we show that the infinite productHs(u) first appears as the gen- erating function for a class ofplainwords by a four-variable statistic (see Theorem 2.2).

This theorem will be an essential tool in section 4 in the MacMahon Verfahren for signed permutations to handle the five-variable polynomial ^fmajB_n(t, q, X, Y, Z). Sec- tion 3 contains an axiomatic definition of the Record-Signed-Euler-Mahonian Polyno- mials B_n(t, q, X, Y, Z). They are defined, not only by (1.8) (with B_n replacing ^fmajB_n), but also by a recurrence relation. The proof of Theorem 1.1 using the MacMahon Ver- fahren is found in Section 4. In Section 5 we show how to prove that the polynomials

fmajBn(t, q, X, Y, Z) satisfy the same recurrence as the polynomials Bn(t, q, X, Y, Z), using an insertion technique. The specializations of Theorem 1.1 are numerous and described in section 6. We end the paper with the proof of Theorem 1.2 and its special- izations.

2. Lower Records on Words

As mentioned in the introduction, Theorem 2.2 below, dealing withordinary words, appears to be a preparation lemma for Theorem 1.1, that takes the geometry of signed permutations into account. Consider an ordinary word c = c1c2. . . cn, whose letters belong to the alphabet {0,1, . . . , s}, that is, a word from the free monoid{0,1, . . . , s}^∗. A letterci (1≤i≤n) is said to be aneven lower record (resp.odd lower record) ofc, if c_iis even (resp. odd) and ifc_j ≥c_i (resp.c_j > c_i) for allj such that 1≤j ≤i−1. Notice the discrepancy between even and odd letters. Also, to define those even and odd lower records for words the reading is made from left to right, while for signed permutations, the lower records are read from right to left (see Sections 1 and 4). We could have considered a totally ordered alphabet with two kinds of letters, but playing with the parity of the nonnegative integers is more convenient for our applications. For instance, the even (resp. odd) lower records of the wordc=5 4 41 5 2 10403 are reproduced in boldface (resp. in italic).

For each wordclet evenlowerc(resp. oddlowerc) be the number of even (resp. odd) lower records ofc. Also let totc(“tot” stands for “total”) be thesumc1+c2+· · ·+cn of the letters ofcand oddcbe thenumberof its odd letters. Also denote itslengthby|c|and let|c|_kbe the number of letters incequal tok. Our purpose is to calculate the generating function for {0,1, . . . , s}^∗ by the four-variable statistic (tot,evenlower,oddlower,odd).

Say that c = c₁c₂. . . c_n is of minimal index k (0 ≤ k ≤ s/2), if minc :=

min{c₁, . . . , c_n} is equal to 2k or 2k + 1. Let c_j be the leftmost letter of c equal to 2k or 2k+ 1. Then, c admits a unique factorization

(2.1) c=c^′c_jc^′′,

having the following properties:

c^′ ∈ {2k+ 2,2k+ 3, . . . , s}^∗, c_j = 2k or 2k+ 1, c^′′ ∈ {2k,2k+ 1, . . . , s}^∗.

With the forementioned example we have the factorization c^′ = 5 4 4, c_j = 1, c^′′ = 5 2 1 0 4 0 3. In this example notice that c_j = 16= minc= 0.

Lemma 2.1. The numbers of even and odd lower records of a word ccan be calculated by induction as follows: evenlowerc = oddlowerc := 0 if c is empty; otherwise, let c=c^′cjc^′′ be its minimal index factorization (defined in (2.1)). Then

evenlowerc= evenlowerc^′+χ(c_j = 2k) +|c^′′|_2k; (2.2)

oddlowerc= oddlowerc^′+χ(cj = 2k+ 1).

(2.3)

Proof. Keep the same notations as in (2.1). If c_j = 2k, then c_j is an even lower record, as well as all the letters equal to 2k to the right ofcj. On the other hand, there is no even lower record equal to 2k to the left of c_j, so that (2.2) holds. If c_j = 2k+ 1, then cj is an odd lower record and there is no odd lower record equal to 2k+ 1 to the right of c_j. Moreover, there is no odd lower record to the left of c_j equal to c_j. Again (2.3) holds.

It is straightforward to verify that the fraction Hs(u) displayed in (1.6) and (1.7) can also be expressed as

H2s(u) = Y

0≤k≤s

1−u([q^2k+1(1−Y)Z +q^2k+2+· · ·+q^2s−2+q^2s−1Z+q^2s]) 1−u(q^2kX+ [q^2k+1Z +q^2k+2+· · ·+q^2s−2+q^2s−1Z +q^2s]) (2.4)

H_2s+1(u) = Y

0≤k≤s

1−u(q^2k+1(1−Y)Z + [q^2k+2+· · ·+q^2s+q^2s+1Z]) 1−u(q^2kX+q^2k+1Z+ [q^2k+2+· · ·+q^2s+q^2s+1Z]), (2.5)

where the expression between brackets vanishes whenever k = s, and that the Hs(u)’s satisfy the recurrence formula

(2.6) H₀(u) = 1

1−uX; H₁(u) = 1−uqZ(1−Y)

1−u(X+qZ) ; and for s≥1 H2s(u) = 1−u(q(1−Y)Z+q²+q³Z+· · ·+q^2s−1Z+q^2s)

1−u(X +qZ+q²+q³Z +· · ·+q^2s−1Z +q^2s) H2s−2(uq²);

H_2s+1(u) =1−u(q(1−Y)Z +q²+q³Z+· · ·+q^2s+q^2s+1Z)

1−u(X+qZ+q²+q³Z +· · ·+q^2s+q^2s+1Z) H_2s−1(uq²).

Theorem 2.2. The generating function for the free monoid {0,1, . . . , s}^∗ by the four- variable statistic (tot,evenlower,oddlower,odd) is equal to H_s(u), that is to say,

(2.7) X

c∈{0,1,...,s}^∗

u^|c|q^totcX^evenlowercY^oddlowercZ^oddc =H_s(u).

Proof. Let H_s^∗(u) denote the left-hand side of (2.7). Then, H₀^∗(u) = X

c∈{0}^∗

u^|c|q⁰X^|c|Y⁰Z⁰ = 1 1−uX.

When s = 1 the minimal index factorization of each nonempty word c reads c= c_jc^′′, so that

H₁^∗(u) = 1 +u(X+qY Z) X

c^′′∈{0,1}^∗

u^|c′′|q^|c′′|1X^|c′′|0Y⁰Z^|c′′|1

= 1 +u(X+qY Z) 1

1−u(X+qZ) = 1−uqZ(1−Y) 1−u(X+qZ). Consequently, H_s^∗(u) =H_s(u) for s= 0,1. For s≥2 we write

H_s^∗(u) = X

0≤k≤s/2

H_s,k^∗ (u) with

H_s,k^∗ (u) := X

c∈{0,1,...,s}^∗ minici=2k or 2k+1

u^|c|q^totcX^evenlowercY^oddlowercZ^oddc.

From Lemma 2.1 it follows that H_s,0^∗ (u) = X

c^′∈{2,...,s}^∗

u^|c′|q^totc′X^{evenlowerc′}Y^oddlowerc′Z^oddc′ ×u(X+qY Z)

× X

c^′′∈{0,...,s}^∗

u^|c′′|q^totc′′X^|c′′|0Z^oddc′′

= X

c^′∈{0,...,s−2}^∗

(uq²)^|c′|q^totc′X^{evenlowerc′}Y^oddlowerc′Z^oddc′×u(X+qY Z)

×X

c^′′∈{0,...,s}^∗

(uX)^|c′′|0(uqZ)^|c′′|1(uq²)^|c′′|2(uq³Z)^|c′′|3(uq⁴)^|c′′|4· · ·

=H_s−2^∗ (uq²)u(X+qY Z) 1

1−u(X +qZ +q²+q³Z+q⁴+· · ·),

the polynomial in the denominator ending with · · ·+q^s−1Z+q^s or · · ·+q^s−1+q^sZ depending on whether s is even or odd.

On the other hand, X

1≤k≤s/2

H_s,k^∗ (u) = X

c∈{2,3,...,s}^∗

u^|c|q^totcX^evenlowercY^oddlowercZ^oddc

= X

c∈{0,1,...,s−2}^∗

(uq²)^|c|q^totcX^evenlowercY^oddlowercZ^oddc

=H_s−2^∗ (uq²).

Hence,

H_s^∗(u) =

1 + u(X+qY Z)

1−u(X+qZ+q²+q³Z +q⁴+· · ·)

H_s−2^∗ (uq²)

= 1−u(qZ(1−Y) +q²+q³Z+q⁴+· · ·)

1−u(X+qZ+q²+q³Z +q⁴ +· · ·) H_s−2^∗ (uq²).

As the fractions H_s^∗(u) satisfy the same induction relation as theHs(u)’s, we conclude that H_s^∗(u) =H_s(u) for all s.

When s tends to infinity, then H_s(u) tends to H_∞(u), whose expression is shown in (1.5). In particular, we have the identity:

(2.8) X

c∈{0,1,2,...}^∗

u^|c|q^totcX^evenlowercY^oddlowercZ^oddc =H_∞(u).

3. The Record-Signed-Euler-Mahonian Polynomials Our next step is to form the series P

s≥0t^sH_s(u) and show that the series can be expanded as a series in the variableu in the form

(3.1) X

n≥0

C_n(t, q, X, Y, Z) uⁿ (t²;q²)n+1

=X

s≥0

t^sH_s(u),

where Bn(t, q, X, Y, Z) := Cn(t, q, X, Y, Z)/(1 + t) is a polynomial with nonnegative integral coefficients such that B_n(1,1,1,1,1) = 2ⁿn! .

Definition. A sequence

B_n(t, q, X, Y, Z) = P

k≥0

t^kB_n,k(q, X, Y, Z)

(n ≥ 0) of polynomials in five variablest,q,X,Y andZ is said to berecord-signed-Euler-Mahonian, if one of the following equivalent three conditions holds:

(1) The (t², q²)-factorial generating function for the polynomials (3.2) C_n(t, q, X, Y, Z) := (1 +t)B_n(t, q, X, Y, Z) is given by identity (3.1).

(2) For n≥2 the recurrence relation holds:

(3.3) (1−q²)B_n(t, q, X, Y, Z)

=

X(1−q²) + (Zq+q²)(1−t²q²ⁿ⁻²) +t²q²ⁿ⁻¹(1−q²)ZY

B_n−1(t, q, X, Y, Z)

− 1

2(1−t)q(1 +q)(1 +tq)(1 +Z)Bn−1(tq, q, X, Y, Z) + 1

2(1−t)q(1−q)(1−tq)(1−Z)Bn−1(−tq, q, X, Y, Z), while B₀(t, q, X, Y, Z) = 1, B₁(t, q, X, Y, Z) =X+tqY Z.

(3) The recurrence relation holds for the coefficients Bn,k(q, X, Y, Z):

(3.4) B_0,0(q, X, Y, Z) = 1, B_0,k(q, X, Y, Z) = 0 for all k 6= 0;

B_1,0(q, X, Y, Z) =X, B_1,1(q, X, Y, Z) =qY Z, B_1,k(q, X, Y, Z) = 0 for all k 6= 0,1;

B_n,2k(q, X, Y, Z) = (X +qZ+q²+q³Z+· · ·+q^2k)B_n−1,2k(q, X, Y, Z) +q^2kB_n−1,2k−1(q, X, Y, Z)

+ (q^2k+q^2k+1Z+· · ·+q²ⁿ⁻¹Y Z)B_n−1,2k−2(q, X, Y, Z), Bn,2k+1(q, X, Y, Z) = (X+qZ+q²+· · ·+q^2k+q^2k+1Z)Bn−1,2k+1(q, X, Y, Z)

+q^2k+1ZBn−1,2k(q, X, Y, Z)

+ (q^2k+1Z +q^2k+2+· · ·+q²ⁿ⁻²+q²ⁿ⁻¹Y Z)B_n−1,2k−1(q, X, Y, Z), for n≥2 and 0≤2k+ 1≤2n−1.

Theorem 3.1. The conditions(1),(2)and(3)in the previous definition are equivalent.

Proof. The equivalence (2) ⇔ (3) requires a lengthy but elementary algebraic argument and will be omitted. The other equivalence (1)⇔(2) involves a more elaborate q-series technique, which is now developed. LetGs(u) :=Hs(u²); then

G0(u) = 1

1−u²X; G1(u) = 1−u²qZ(1−Y) 1−u²(X +qZ) ;

and by (2.6)

G_2s(u) = 1−u²(qZ(1−Y) +q²+q³Z+· · ·+q^2s−1Z+q^2s)

1−u²(X+qZ+q²+q³Z +· · ·+q^2s−1Z +q^2s) G_2s−2(uq), G2s+1(u) = 1−u²(qZ(1−Y)+q²+q³Z+· · ·+q^2s+q^2s+1Z)

1−u²(X+qZ+q²+q³Z+· · ·+q^2s+q^2s+1Z)G2s−1(uq), for s ≥1. Working with the series P

s≥0

t^sGs(u) we obtain X

s≥0

t^2sG_2s(u)

1−u² X + Zq+q²

1−q² − q^2s

1−q²(Zq+q²)

+X

s≥0

t^2s+1G2s+1(u)

1−u² X + Zq+q²

1−q² − q^2s+2

1−q²(Zq+ 1)

= 1 +t(1−u²qZ(1−Y))

+X

s≥1

t^2sG_2s−2(qu)

1−u² −qZY + Zq+q²

1−q² − q^2s

1−q²(Zq+q²)

+X

s≥1

t^2s+1G2s−1(qu)

1−u² −qZY + Zq+q²

1−q² − q^2s+2

1−q²(Zq+ 1) , which may be rewritten as

X

s≥0

t^sG_s(u)

1−u² X+ Zq +q² 1−q²

= 1 +t(1−u²qZ(1−Y))

+X

s≥0

t^s+2G_s(qu)

1−u² −qZY + Zq+q² 1−q²

−X

s≥0

(tq)^2s G_2s(u)−t²q²G_2s(qu)

u²Zq+q² 1−q²

−X

s≥0

(tq)^2s+1 G_2s+1(u)−t²q²G_2s+1(qu)

u²qZq + 1 1−q² . Now let P

n≥0b_n(t)u²ⁿ :=P

s≥0t^sG_s(u). This gives:

X

n≥0

bn(t)u²ⁿ

1−u² X + Zq+q² 1−q²

= 1 +t(1−u²qZ(1−Y))

+X

n≥0

b_n(t)t²q²ⁿu²ⁿ

1−u² −qZY + Zq+q² 1−q²

−X

n≥0

b_n(tq) +b_n(−tq)

2 (1−t²q²ⁿ⁺²)u²ⁿ⁺²Zq+q² 1−q²

−X

n≥0

bn(tq)−bn(−tq)

2 (1−t²q²ⁿ⁺²)u²ⁿ⁺²qZq+ 1 1−q².

We then have b₀(t) = 1

1−t, b₁(t) = X+tqY Z

(1−t)(1−t²q²) and for n≥2 b_n(t)(1−t²q²ⁿ) =

X+ Zq+q²

1−q² +t²q²ⁿ⁻¹ZY −t²q²ⁿ⁻²Zq +q² 1−q²

b_n−1(t)

− bn−1(tq)

2(1−q²)(1−t²q²ⁿ)q(1 +q)(1 +Z) + bn−1(−tq)

2(1−q²) (1−t²q²ⁿ)q(1−q)(1−Z).

Because of the presence of the factors of the form (1−t²q²ⁿ) we are led to introduce the coefficients C_n(t, q, X, Y, Z) := b_n(t)(t²;q²)_n+1 (n ≥ 0). By multiplying the latter equation by (t²;q²)n we get for n≥2

(3.5) (1−q²)C_n(t, q, X, Y, Z)

=

X(1−q²) + (Zq+q²)(1−t²q²ⁿ⁻²) +t²q²ⁿ⁻¹(1−q²)ZY

C_n−1(t, q, X, Y, Z)

− 1

2(1−t²)q(1 +q)(1 +Z)Cn−1(tq, q, X, Y, Z) + 1

2(1−t²)q(1−q)(1−Z)C_n−1(−tq, q, X, Y, Z),

while C₀(t, q, X, Y, Z) = 1 +t, C₁(t, q, X, Y, Z) = (1 +t)(X +tqY Z).

Finally, with C_n(t, q, X, Y, Z) := (1 +t)B_n(t, q, X, Y, Z) (n≥ 0) we get the recur- rence formula (3.3), knowing that the factorial generating function for the polynomials C_n(t, q, X, Y, Z) = (1 +t)B_n(t, q, X, Y, Z) is given by (3.1). As all the steps are perfectly reversible, the equivalence holds.

4. The MacMahon Verfahren

Now having three equivalent definitions for the record-signed-Euler-Mahonian polynomial B_n(t, q, X, Y, Z), our next task is to prove the identity

(4.1) ^fmajBn(t, q, X, Y, Z) = Bn(t, q, X, Y, Z).

Let Nⁿ (resp. NIW(n)) be the set of all the words (resp. all the nonincreasing words) of lengthn, whose letters are nonnegative integers. As we have seen in section 2 (Theorem 2.2), we know how to calculate the generating function for words by a certain four-variable statistic. The next step is to map each pair (b, w) ∈ NIW(n)×Bn onto c ∈ Nⁿ in such a way that the geometry on w can be derived from the latter statistic on c.

For the construction we proceed as follows. Write the signed permutation w as the linear word w = x1x2. . . xn, where xk is the image of the integer k (1 ≤ k ≤ n). For each k = 1,2, . . . , n let z_k be the number of descents in the right factor x_kx_k+1. . . x_n and ǫk be equal to 0 or 1 depending on whether xk is positive or negative. Next, form the words z =z₁z₂. . . z_n and ǫ=ǫ₁ǫ₂. . . ǫ_n.

Now, take a nonincreasing word b = b1b2. . . bn and define ak := bk + zk, c^′_k := 2a_k+ǫ_k (1≤k ≤n), then a:=a₁a₂. . . a_n and c^′ :=c^′₁c^′₂. . . c^′_n. Finally, form the two-matrix

c^′₁ c^′₂ . . . c^′_n

|x₁| |x₂| . . . |x_n|

. Its bottom row is a permutation of 1 2 . . . n; rearrange the columns in such a way that the bottom row is precisely 1 2 . . . n. Then the word c= c₁c₂. . . c_n which corresponds to the pair (b, w) is defined to be the top row in the resulting matrix.

Example. Start with the pair (b, w) below and calculate all the necessary ingredi- ents:

b = 1 1 1 0 0 0 0 w = 6 5 4 1 7 3 2 z = 2 1 1 1 1 0 0 ǫ = 0 1 1 0 0 1 1 a = 3 2 2 1 1 0 0 c^′ = 6 5 5 2 2 1 1 c = 2 1 1 5 5 6 2

Theorem 4.1. For each nonnegative integer r the above mapping is a bijection of the set of all the pairs (b, w) = (b1b2. . . bn, x1x2. . . xn) ∈ NIW(n)× Bn such that 2b₁ + fdesw = r onto the set of the words c = c₁c₂. . . c_n ∈ Nⁿ such that maxc = r.

Moreover,

(4.2) 2b₁+ fdesw = maxc; 2 totb+ fmajw= totc;

(4.3) lowerpw= evenlowerc; lowernw= oddlowerc; negw = oddc.

Before giving the proof of Theorem 4.1 we derive its analytic consequences. First, it is q-routine to prove the three identities, where b₁ is the first letter of b,

1

(u;q)_N = X

n≥0

N +n−1 n

q

uⁿ; (4.4)

N +n n

q

= X

b∈NIW(N), b¹≤n

q^totb; (4.5)

1 (u;q)N+1

= X

n≥0

uⁿ X

b∈NIW(N), b¹≤n

q^totb. (4.6)

We then consider

1 +t (t²;q²)_n+1

fmajB_n(t, q, X, Y, Z), where

fmajB_n(t, q, X, Y, Z) := X

w∈Bn

t^fdeswq^fmajwX^lowerpwY^lowernwZ^negw,

which we rewrite as X

r^′≥0

(t^2r′+t^2r′+1)

n+r^′ r^′

q²

fmajB_n(t, q, X, Y, Z) [by (4.4)]

=X

r≥0

t^r

n+⌊r/2⌋

⌊r/2⌋

q²

fmajB_n(t, q, X, Y, Z) [and by (4.5)]

=X

r≥0

t^r X

b∈NIW(n),2b¹≤r

q^{2 totb} X

w∈Bn

t^fdeswq^fmajwX^lowerpwY^lowernwZ^negw

=X

s≥0

t^s X

b∈NIW(n), w∈Bn

2b¹+fdesw≤s

q^{2 totb+fmajw}X^lowerpwY^lowernwZ^negw.

By Theorem 4.1 the set {(b, w) :b∈NIW(n), w ∈Bn, 2b1+ fdesw ≤s} is in bijection with the set {0,1, . . . , s}ⁿ and (4.2) and (4.3) hold. Hence,

1 +t (t²;q²)_n+1

fmajBn(t, q, X, Y, Z) =X

s≥0

t^s X

c∈{0,...,s}ⁿ

q^totcX^evenlowercY^oddlowercZ^oddc. Now use Theorem 2.2:

X

s≥0

t^sH_s(u) =X

s≥0

t^s X

c∈{0,1,...,s}^∗

u^|c|q^totcX^evenlowercY^oddlowercZ^oddc

=X

n≥0

uⁿX

s≥0

t^s X

c∈{0,1,...,s}ⁿ

q^totcX^evenlowercY^oddlowercZ^oddc

=X

n≥0

(1 +t)^fmajB_n(t, q, X, Y, Z) uⁿ (t²;q²)_n+1. Thus, identity (4.1) is proved, as well as Theorem 1.1.

Let us now complete the proof of Theorem 4.1. First, maxc = c^′₁ = 2a₁ +ǫ₁ = 2b₁+2z₁+ǫ₁ = 2b₁+fdesw; also totc= totc^′ = 2 tota+totǫ = 2 totb+2 totz+totǫ = 2 totb+ fmajw. Hence (4.2) holds.

Next, prove that (b, w)7→cis bijective. As both sequencesbandzare nonincreasing, a = b+z is also nonincreasing. If x_k > x_k+1, then z_k = z_k+1+ 1 and a_k ≥ a_k+1+ 1, then c^′_k= 2a_k+ǫ_k ≥2a_k+1+ 2 +ǫ_k ≥2a_k+1+ 2>2a_k+1+ǫ_k+1 =c^′_k+1. Thus,

(4.7) x_k > x_k+1 ⇒c^′_k > c^′_k+1.

To construct the reverse bijection we proceed as follows. Start with a sequence c = c₁c₂. . . c_n; form the word δ = δ₁δ₂. . . δ_n, where δ_i := χ(c_i even) −χ(c_i odd) (1≤i≤n) and the two-row matrix

c1 c2 . . . cn

1δ₁ 2δ₂ . . . nδ_n

.

Rearrange the columns in such a way that ^c_iⁱ

occurs to the left of ^c_j^j

, if eitherci > cj, or c_i =c_j, i < j. The bottom of the new matrix is a signedpermutation w. After those two transformations the new matrix reads

c^′ w

=

c^′₁ c^′₂ . . . c^′_n x₁ x₂ . . . x_n

.

The sequences ǫ andz are defined as above, as well as the sequencea := (c^′−ǫ)/2.

As the sequencec^′ is nonincreasing, the inequalityc^′_k−ǫ_k ≥c^′_k+1−ǫ_k+1 holds ifǫ_k= 0 or if ǫ_k =ǫ_k+1 = 1. It also holds when ǫ_k = 1 and ǫ_k+1 = 0, because in such a case c^′_k is odd and c^′_k+1 even and then c^′_k ≥1 +c^′_k+1. Hence, a is also nonincreasing.

Finally, define b := a−z. Because of (4.7) we have z_k = z_k+1 + 1 ⇒ c^′_k > c^′_k+1. If c^′_k and c^′_k+1 are of the same parity, then c^′_k > c^′_k+1 ⇒ a_k > a_k+1. If c^′_k > c^′_k+1 holds and the two terms are of different parity, then c^′_k is even and c^′_k+1 odd. Hence, a_k =c^′_k/2>(c^′_k+1−ǫ_k+1)/2 = a_k+1. Thus z_k =z_k+1+ 1 ⇒a_k > a_k+1. As z_n = 0, we conclude by a decreasing induction that b_k = a_k−z_k ≥ a_k+1 −z_k+1 =b_k+1, so that b is a nonincreasing sequence of nonnegative integers.

There remains to prove (4.3). First, the letterc_j of c is odd, if and only if j occurs with the minus sign in w, so that negw = oddc. Next, suppose that x_j is a positive lower record of w = x1x2. . . xn. For j < i we have xj < |xi|. Hence, the following implications hold: |x_i| < x_j ⇒i < j ⇒c^′_i ≥ c^′_j ⇒c_|xj| ≤c_|xi| and c_xi is an even lower record of c. If xj is a negative lower record of w, then j < i ⇒ −xj < |xi|, so that

|x_i|<−x_j ⇒i < j ⇒c^′_i > c^′_j ⇒c_|xj| < c_|xi| and c_|xi| is an odd lower record of c.

5. The Insertion Method

Another method for proving identity (4.1) is to make use of the insertion method.

Each signed permutation w^′ = x^′₁. . . x^′_n−1 of order (n− 1) gives rise to 2n signed permutations of ordernwhennor−nis inserted to the left or to the rightw^′, or between two letters ofw^′. Assuming thatw^′ has a flag descent number equal to fdesw^′ =k, our duty is then to watch how the statistics “fmaj”, “lowerp”, “lowern”, “neg” are modified after the insertion of n or −n into the possible n slots. Such a method has already been used by Adin et al. [ABR01], Chow and Gessel [ChGe04], Haglund et al. [HLR04], for “fmaj” only. They all have observed that for each j = 0,1, . . . ,2n−1 there is one and only one signed permutation of order n derived by the insertion of nor −n whose flag-major index is increased by j.

In our case we observe that the number of positive (resp. negative) lower records remains alike, except whenn(resp.−n) is inserted to the right ofw^′, where it increases by 1. The number of negative letters increases only when−nis inserted. For controlling

“fdes” we make a distinction between the signed permutations having an even flag descent number and those having an odd one. For the former ones the first letter is positive. When n (resp. −n) is inserted to the left of w^′, the flag descent number increases by 2 (resp. by 1). For the latter ones the first letter is negative. Whenn(resp.

−n) is inserted to the left of w^′, the flag descent number increases by 1 (resp. remains invariant).

For n≥2, 0≤k≤2n−1 let

fmajB_n,k := X

w∈Bn,fdesw=k

q^fmajwX^lowerpwY^lowernwZ^negw,

and ^fmajB_0,0 := 1, ^fmajB_0,k := 0 when k 6= 0; ^fmajB_1,0 =X, ^fmajB_1,1 =qY Z. Making use of the observations above we easily see that the polynomials ^fmajB_n,k satisfy the same recurrence relation, displayed in (3.4), as theB_n,k(q, Y, Y, Z).

Remark. How to compare the MacMahon Verfahren and the insertion method?

Recurrence relations may be difficult to be truly verified. The first procedure has the advantage of giving both a new identity on ordinary words (Theorem 2.1) and a closed expression for the factorial generating function for the polynomials ^fmajB_n(t, q, X, Y, Z) (formula (1.8)).

6. Specializations

When a variable has been deleted in the following specializations of the polynomials B_n(t, q, X, Y, Z), this means that the variable has been given the value 1. For instance, B_n(q, X, Y, Z) :=B_n(1, q, X, Y, Z).

When s tends to infinity, then H_s(u) tends to H_∞(u) given in (1.1). Hence, when identity (3.1) is multiplied by (1−t) and t is replaced by 1, identity (3.1) specializes into

(6.1) X

n≥0

B_n(q, X, Y, Z) uⁿ

(q²;q²)_n=H_∞(u) =

uqZ +q

1−q² −ZY

;q²

∞

uq(Z +q)

1−q² +X

;q²

∞

.

Now expand H∞(u) by means of the q-binomial theorem [GaRa90, chap. 1]. We get:

H_∞(u) = X

n≥0

qZ+q

1−q² −Y Z q(Z+q)

1−q² +X

;q²

!

n

uq(Z +q)

1−q² +Xn.

(q²;q²)_n.

By identification

B_n(q, X, Y, Z) =

qZ +q

1−q² −Y Z q(Z +q)

1−q² +X

;q²

!

n

q(Z+q)

1−q² +Xn

,

which can also be written as

(6.2) B_n(q, X, Y, Z) = (X+qZ+q²+q³Z +· · ·+q²ⁿ⁻² +Y Zq²ⁿ⁻¹)

· · · ×(X+qZ+q²+q³Z+q⁴+q⁵Y Z)(X +qZ+q²+q³Y Z)(X+qY Z), or, by induction,

(6.3) Bn(q, X, Y, Z) = (X+qZ+q²+q³Z+· · ·+q²ⁿ⁻²+Y Zq²ⁿ⁻¹)Bn−1(q, X, Y, Z) for n≥2 and B1(q, X, Y, Z) =X+qY Z.

In particular, the polynomial ^fmajBn(q, X, Y) defined in the introduction is equal to

(6.4) B_n(q, X, Y) = (X+q+q²+q³+· · ·+q²ⁿ⁻²+Y q²ⁿ⁻¹)

· · · ×(X +q+q²+q³+q⁴+q⁵Y)(X+q+q²+q³Y)(X+qY).

Finally, when X =Y =Z := 1, identity (6.4) reads

(6.5) X

n≥0

(1 +t)Bn(t, q) uⁿ

(t²;q²)_n+1 =X

s≥0

t^s 1

1−u(1 +q+q²+· · ·+q^s),

an identity derived by several authors (Adin et al. [ABR01], Chow & Gessel [ChGe04], Haglund et al. [HLR04]) with ad hoc methods. Also notice that for X = Y = Z := 1 formula (6.2) yields Bn(q) = (q²;q²)n/(1−q)ⁿ.

For Z = 0 we get

fmajB_n(t, q, X, Y,0) = X

w∈S_n

t^fdeswq^fmajwX^lowerpw,

since the monomials corresponding to signed permutations having negative letters vanish. The summation is then over the ordinary permutations. Also notice that

fmajB_n(t, q, X, Y,0) = ^fmajB_n(t, q, X,0,0). Moreover, for each ordinary permutation w we have: fdesw = 2 desw and fmajw = 2 majw. When Y =Z = 0 we also have

H₂₊₁(u) =H_2s(u) =

uq²

1−q²+uq^2(s+1);q²

s+1

u X(1−q²) +q² 1−q²+uq^2(s+1);q²

s+1

and

X

n≥0

(1 +t)^fmajB_n(t, q, X,0,0) uⁿ

(t²;q²)_n+1 =X

s≥0

(t^2s+t^2s+1)H_2s(u).

WithA_n(t, q, X) :=^fmajB_n(t^1/2, q^1/2, X,0,0) = P

w∈S_n

t^deswq^majwX^lowerpw we obtain the identity

(6.7) X

n≥0

A_n(t, q, X) uⁿ

(t;q)_n+1 =X

s≥0

t^s

uq

1−q+uq^s+1;q

s+1

uX(1−q) +q 1−q+uq^s+1;q

s+1

,

apparently a new identity for the generating function for the symmetric groups S_n by the three-variable statistic (des,maj,lowerp), as well as the following one obtained by multiplying the identity by (1−t) and letting t go to infinity:

(6.8) X

n≥0

A_n(q, X) uⁿ (q;q)_n =

uq 1−q;q

∞

uX + uq 1−q;q

∞

.

7. Lower Records and Flag-inversion Number

The purpose of this section is prove Theorem 1.2. We proceed as follows. For each (ordinary) permutation σ =σ₁σ₂. . . σ_n of order n and for each i = 1,2, . . . , n let b_i(σ) denote the number of letters σj to the left of σi such that σj < σi. On the other hand, for each signed permutation w=x₁x₂. . . x_n of order n let absw denote the (ordinary) permutation|x₁| |x₂|. . .|x_n|. It is straightforward to see that another expression for the flag-inversion number (or the length function), finvw, of w is the following

(7.1) finvw= inv absw+ X

1≤i≤n

(2bi(absw) + 1)χ(xi <0).

The generating polynomial^finvBn(q, X, Y) can be derived as follows. Let Lowerσ denote the set of the (necessarily positive) records of the ordinary permutation σ. By (7.1) we have

finvB_n(q, X, Y, Z) =X

w∈Bn

q^finvwX^lowerpwY^lowernwZ^negw

=X

σ∈S_n

X

absw=σ

q^finvwX^lowerpwY^lowernwZ^negw

=X

σ∈S_n

q^invσ Y

σi∈Lowerσ

(X+Y Zq^2bi(σ)+1)Y

σi6∈Lowerσ

(1 +Zq^2bi(σ)+1) :=X

σ∈S_n

f(σ),

showing that^finvBn(q, X, Y, Z) is simply a generating polynomial for the groupS_nitself.

But S_n can be generated from S_n−1 by inserting the letter n into the possible n slots of each permutation of order n−1. Let σ^′=σ^′₁σ₂^′ . . . σ_n−1^′ be such a permutation.

Then

f(σ^′₁σ₂^′ . . . σ_n−2^′ σ_n−1^′ n) =f(σ^′)(X+Y Zq²ⁿ⁻¹);

f(σ^′₁σ₂^′ . . . σ_n−2^′ nσ_n−1^′ ) =f(σ^′)q(1 +Zq²ⁿ⁻³);

· · · ·

f(σ^′₁nσ^′₂. . . σ_n−2^′ σ_n−1^′ ) =f(σ^′)qⁿ⁻²(1 +Zq³);

f(nσ^′₁σ^′₂. . . σ_n−2^′ σ_n−1^′ ) =f(σ^′)qⁿ⁻¹(1 +Zq).

Hence,^finvB_n(q, X, Y, Z) = ^finvB_n−1(q, X, Y, Z)(X+q+· · ·+qⁿ⁻¹+qⁿZ+· · ·+q²ⁿ⁻²Z+ q²ⁿ⁻¹Y Z) (n≥2). As ^finvB₁(q, X, Y, Z) = X+qY Z, we get the expression displayed in (1.9).

Comparing (6.2) with (1.9) we see that the two polynomials ^finvB_n(q, Z) and Bn(q, Z) are different as soon as n ≥ 2, while ^finvBn(q, X, Y) = Bn(q, X, Y) for all n.

This means that any bijection Ψ of the group B_n onto itself having the property that fmajw = finv Ψ(w)

does not leave the number of negative letters “neg” invariant. It was, in particular, the case for the bijection constructed in our previous paper [FoHa05].

Remark1. Instead of considering lower records from right to left we can introduce lower recordsfrom left to right. Let lowerp^′w and lowern^′wdenote the numbers of such records, positive and negative, respectively and introduce

finvBn(q, X, Y, X^′, Y^′):=X

w∈Bn

q^finvwX^lowerpwY^lowernwX^{′lowerp′w}Y^{′lowern′w}.

Using the method developed in the proof of the previous theorem we can calculate this polynomial in the form

finvBn(q, X, Y, X^′, Y^′)

= (X+q+· · ·+qⁿ⁻²+qⁿ⁻¹X^′+qⁿY^′+qⁿ⁺¹+· · ·+q²ⁿ⁻²+q²ⁿ⁻¹Y)

· · · ×(X+q+q²X^′+q³Y^′+q⁴+q⁵Y)(X+qX^′+q²Y^′+q³Y)(XX^′+qY Y^′).

Remark 2. The same method may be used for ordinary permutations. For each permutationσ let upperσ denote the number of itsupper recordsfrom right to leftand define:

invA_n(q, X, V) := X

σ∈S_n

q^invσX^lowerpσV^upperσ. Then

invAn(q, X, V) =XV(X+qV)(X+q+q²V)· · ·(X+q+q²+· · ·+qⁿ⁻¹V), an identity which can be put into the form

(7.2) X

n≥0

invAn(q, X, V) uⁿ

(q;q)_n = 1− XV

X +V −1 + XV X+V −1

u

1−q −ux;q

∞

uX+ uq 1−q;q

∞

,

which specializes into

(7.3) X

n≥0

invAn(q, X) uⁿ (q;q)_n =

uq 1−q;q

∞

uX+ uq 1−q;q

∞

;

(7.4) X

n≥0

invAn(q, V) uⁿ (q;q)_n =

u

1−q −uV;q

∞

u 1−q;q

∞

.

Comparing (7.3) with (6.8) we then see thatAn(q, X) =^invAn(q, X). In other words, the generating polynomial forS_n by (maj,lowerp) is the same as the generating polynomial by (inv,lowerp). A combinatorial proof of this result is due to Bj¨orner and Wachs [BjW88], who have made use of the transformation constructed in [Fo68]. Finally, the expression of ^invAn(q, X, V) for q = 1 was derived by David and Barton ([DaBa62], chap. 10).