stacks Catalan and Schröder permutations sortable by two restricted Information Processing Letters

Contents lists available atScienceDirect

Information Processing Letters

www.elsevier.com/locate/ipl

Catalan and Schröder permutations sortable by two restricted stacks

Jean-Luc Baril

^a

, Giulio Cerbai

^b

^,

¹

, Carine Khalil

^a

, Vincent Vajnovszki

^a

^,∗

aLIB,UniversitédeBourgogneFranche-Comté,B.P.47870,21078DijonCedex,France bDipartimentodiMatematicaeInformatica“U.Dini”,UniversityofFirenze,Firenze,Italy

a rt i c l e i nf o a b s t ra c t

Articlehistory:

Received14September2020

Receivedinrevisedform15February2021 Accepted2May2021

Availableonline19May2021 CommunicatedbyAmitChakrabarti

Keywords:

Stacksorting/twostacksinseries Patternavoidingpermutations/machines CatalanandtheSchrödernumbers Combinatorialproblems

Pattern avoiding machines were introduced recently by Claesson, Cerbai and Ferrari as a particular case of the two-stacks in series sorting device. They consist of two restrictedstacks inseries,ruledby aright-greedy procedure andthe stacksavoidsome specified patterns.Someoftheobtainedresultshave beenfurthergeneralizedto Cayley permutations by Cerbai, specialized to particular patterns by Defant and Zheng, or consideredinthecontextoffunctionsoverthesymmetricgroupbyBerlow.Inthiswork we study pattern avoiding machines where the first stack avoidsa pairof patterns of length3and investigatethosepairs forwhichsortablepermutationsarecountedbythe (binomialtransformofthe)CatalannumbersandtheSchrödernumbers.

©^{2021PublishedbyElsevierB.V.}

1. Introduction

Pattern avoiding machines were recently introduced in [7] inattempttogainabetterunderstandingofsortable permutationsusingstacksinseries.Theyconsistoftwore- strictedstacksinseries,equippedwitharight-greedypro- cedure,wherethefirststackavoidsafixedpattern,reading the elements from top to bottom; and the second stack avoids thepattern21 (whichis anecessarycondition for themachinetosortpermutations).Theauthorsof [7] pro- vide a characterization of theavoided patterns forwhich sortable permutationsdonotformaclass,andthey show that those patterns are enumerated by the Catalannum- bers.Forspecificpatterns,suchas123 andthedecreasing patternofanylength,ageometricaldescriptionofsortable

*

Correspondingauthor.

E-mailaddresses:[email protected](J.-L. Baril),

giulio.cerbai@unifi.it(G. Cerbai),[email protected](C. Khalil), [email protected](V. Vajnovszki).

1 G.C. is member of the INdAM Research group GNCS; he is par- tiallysupportedbyINdAM-GNCS2020project“Combinatoriadelleper- mutazioni,delleparolee deigrafi:algoritmie applicazioni”.

permutationsis also obtained. The pattern132 has been solvedlaterin [8].Someoftheseresultshavebeenfurther generalizedto Cayley permutations in [9]. More recently, Berlow [5] exploresasinglestackversionofpatternavoid- ingmachines,wherethestackavoidsasetofpatternsand the sorting process is regarded as a function. Analogous machines,butbasedonthenotionofconsecutivepatterns, havebeenintroducedanddiscussedin [10].

In this work we study a variant of pattern-avoiding machines where the first stack avoids

( σ , τ )

, a pair of patterns oflength three.Following [7], we call it

( σ , τ )

- machine. More specifically, we restrict ourselves to those pairs of patterns for which sortable permutations are counted by either the Catalan numbers or two of their close relatives: the binomial transform of Catalan num- bers and the Schröder numbers. For the pair

(

132

,

231

)

we show that sortable permutations are those avoiding 1324 and2314,a setwhoseenumerationis givenbythe large Schröder numbers. Under certain conditions on the avoided patterns, the output of the first stack is bijec- tively related to its input (see [5,9]): it follows that for threepairsofpatterns,namely

(

123

,

213

)

,

(

132

,

312

)

and

(

231

,

321

)

,sortablepermutationsarecountedbytheCata- https://doi.org/10.1016/j.ipl.2021.106138

0020-0190/©^{2021PublishedbyElsevierB.V.}

lannumbers.Thisresultwasconjecturedin [3] andsettled independently in [4,5]. Forthe pair

(

123

,

132

)

, we prove thatsortablepermutationsarethoseavoidingthepatterns 2314, 3214,4213 and the generalizedpattern

[

2413.

¯

We prove that sortable permutations are enumerated by the Catalan numbers by showing that the distribution of the firstelement isgivenbythe well-knownCatalantriangle.

Finally, we show that for the pair

(

123

,

312

)

the corre- sponding countingsequence isthe binomial transformof Catalannumbers.

This paper is the extended version of the conference presentation[4] andsome ofthepresented resultsprevi- ouslyappearin [3].

2. Notationsandsomepreliminaryresults

We start by recalling some classical definitions about pattern avoidance on permutations (see [12] for a more detailedintroduction). Denote by Sn the set ofpermuta- tions oflengthn andlet S

= ∪

n≥⁰Sn.Giventwopermu- tations

σ

oflengthkand

π ₌ π

1

· · · π

n,wesaythat

π

con- tainsthepattern

σ

if

π

containsasubsequence

π

i1

· · · π

ik, withi1

<

i2

< · · · <

i_k,whichisorderisomorphicto

σ

.In thiscase,wesaythat

π

i1

· · · π

ikisanoccurrenceofthepat- tern

σ

in

π

.Otherwise,wesaythat

π

avoids

σ

.

We say that

π

contains an occurrence ofthe (gener- alized) pattern

[ σ

if

π

contains an occurrenceof

σ

that involvesthefirstelement

π

1 of

π

.Forinstance,anoccur- renceof

[

^{12 in}

π

correspondstoapairofelements

π

1

π

i, with i

>

1 and

π

i

> π

1. A barredpattern

σ ˜

is a pattern wheresomeentriesarebarred.Let

σ

betheclassicalpat- tern obtainedby removing all the bars from

σ ˜

.Let

τ

be the pattern whichis order isomorphic to the non-barred entries of

σ ˜

(i.e. obtained from

σ ˜

by removing all the barred entries and suitably rescaling the remaining ele- ments).Apermutation

π

avoids

σ ˜

ifeachoccurrenceof

τ

in

π

canbeextendedtoan occurrenceof

σ

.Forinstance, apermutation

π

avoidsthepattern

[

²⁴13 if

¯

foranysubse- quence

π

1

π

i

π

j,with1

<

i

<

j and

π

1

< π

j

< π

i,thereis an indext,i

<

t

<

j,suchthat

π

1

π

i

π

t

π

j isanoccurrence of2413.

Given a set of (generalized) patterns T, denote by Av_n

(

T

)

the setofpermutationsin Sn avoiding eachpat- terninT.Similarly,letAv(T

) = ∪

n≥0Av_n

(

T

)

.IfT

= { σ }

^is asingleton,wewriteAv_n

( σ )

andAv(

σ )

.Inhiscelebrated book [13], Knuth gave the following characterization of stacksortablepermutations,whichisoftenconsideredthe starting point of stack sorting and permutation patterns disciplines.

Proposition1([13]).Apermutation

π

issortableusingaclas- sicalstack(thatis,a21-avoidingstack)ifandonlyif

π

avoids thepattern231.

Let T be a set of patterns. A T-stack is a stack that isnot allowed tocontain anoccurrenceofanypatternin T,readingits elements fromtop tobottom. Givena per- mutation

π

,denotebyout^T

( π )

thepermutationobtained after passing

π

through the T-avoiding stack by apply- ing a greedy procedure, i.e. by always pushing the next element of the input, unless it creates an occurrence of

a forbidden pattern inside the stack. Denote by Sortn

(

T

)

theset oflength npermutationsthat are sortable by the T-machine,that is, by passing

π

through the T-avoiding stack and then through the 21-avoiding stack. Permuta- tions inSort_n

(

T

)

arecalled T -sortable, andSort

(

T

)

is the setof T-sortable permutationsofanylength.As aconse- quenceofProposition1,Sort

(

T

)

consistspreciselyofthose permutations

π

for which out^T

( π )

avoids 231. To ease notations,if T is either a singleton T

= { σ }

^{or a pair of} patterns T

= { σ , τ }

^{,wewill omitthecurlybracketsfrom} theabovenotations.Forinstance,wewillwriteSort

( σ , τ )

insteadofSort

( { σ , τ _} )

.

The authors of [7] showed that if

π

is a 12-sortable permutation of length n, then out¹²

( π ) =

ⁿ

(

n

−

¹

) · · ·

^1.

Moreover,by Proposition1 andapplyingthecomplement operation on the processed permutation, we have that Sort

(

12

) =

Av

(

213

)

. In order to refer to thisresult later, we state itbelowina slightlymoregeneral form.A par- tial permutation of n is an injection

π _{: {}

1

,

2

, . . . ,

k

} → {

¹

,

2

, . . . ,

n

}

^{,forsome 0}

≤

^k

≤

^{n,andtheintegerk issaid} tobe the length of

π

. Welet a 12-stackact on apartial permutation

π

ofn in the naturalway by identifying

π

withthelistofitsimages.

Proposition2.If

π

isapartialpermutationofn whichis12- sortable,thenout¹²

( π )

isthedecreasingrearrangementofthe symbolsof

π

.Moreover,

π

is12-sortableifandonlyifitavoids 213.

An entry

π

i ofa permutation

π

isa left-to-rightmin- imum if

π

i

< π

j, for each j

<

i. The left-to-rightminima decomposition (briefly ltr-mindecomposition) of

π

is

π ₌

m₁B₁m₂B₂

· · ·

^mtB_t,wherem₁

>

m₂

> · · · >

m_t aretheltr- minimaof

π

andtheblock Bicontains theelements of

π

betweenmi andmi+¹,for i

=

¹

, . . . ,

t

−

^{1. The last block} B_t contains the elements that followm_t in

π

.Note that mt

=

^{1. The notion of} left-to-rightmaximum of a permu- tation

π

isdefinedsimilarly. Theltr-maxdecompositionof

π

is

π =

^M1B1M2B2

· · ·

^MtBt,where M1

<

M2

< · · · <

Mt are theltr-maximaof

π

.In thiscase M_t

=

^{n, wheren is} thelengthof

π

.

Finally,thesequence

(

cn

)

n≥⁰,pervasiveinthispaper,is thesequenceofCatalannumbersc_n

=

_n+¹_12n

n

(A000108 in [15]).

3. Pair(132,231)

Thissectionisdevotedtotheanalysisofthe

(

123

,

231

)

- machine.

Theorem1.Considerthe

(

132

, σ )

-machine,where

σ = σ

1

· · · σ

_k−1

σ

_k

_∈

S_k,withk

≥

^3and

σ

k−1

> σ

_k.Givenapermutation

π

oflengthn,letm1B1

· · ·

^mtBt

= π

beitsltr-mindecomposi- tion.Then:

1. Everytimealtr-minimummiispushedintothe

(

132

, σ )

- stack,the

(

132

, σ )

-stackcontainstheelementsm_i−1

, . . . ,

m2

,

m1,readingfromtoptobottom.Moreover,wehave out^132,σ

( π ) = ˜

^B1

· · · ˜

^Btm_t

· · ·

^m1

,

whereB

˜

iisarearrangementofBi.

2. If

π

is

(

132

, σ )

-sortable,thenB

˜

_iisdecreasingforeachi.

Moreover,foreachi

≤

^t

−

^1,wehaveBi

>

Bi+¹(i.e.x

>

y foreachx

∈

^Bi

,

y

∈

^Bi+¹).

Proof. 1. Let usconsider theevolution of the

(

132

, σ )

- stackon input

π

.Notethat, sincek

≥

^{3,the element} m1 remains atthebottom ofthe

(

132

, σ )

-stack until theendofthe process.Now,if B1 is notempty then foreach x

∈

^B1,the elements m₂xm₁ form an occur- rence of132.Therefore theblock B1 is extractedbe- forem2 entersthe

(

132

, σ )

-stack.After m2 ispushed, the

(

132

, σ )

-stackcontainsm2m1,readingfromtopto bottom. Sincem₂

<

m₁,but

σ

k−¹

> σ

k by hypothesis, m2 cannot play the role of either

σ

k−1 in an occur- rence of

σ

orof 3 in anoccurrenceof 132.Thusm2 remains atthebottomofthe

(

132

, σ )

-stack untilthe endofthesortingprocedure.Thethesisfollowsbyit- eratingthesameargumentoneachblockB_i,fori

≥

^2.

2. Suppose that

π

is

(

132

, σ )

-sortable. Assume, for a contradiction, that B

˜

i is not decreasing, for some i.

Then therearetwoconsecutiveelements x

<

y inB

˜

_i. Therefore,bywhatprovedabove,out^132,σ

( π )

contains an occurrence xymt of 231, which is impossibledue toProposition1.Finally,supposethatx

<

y,forx

∈

^Bi and y

∈

^Bi+1. Then xym_t is an occurrenceof 231 in out^132,σ

( π )

,acontradiction.

Theorem 1 and Proposition 2 guarantee that if

π ₌

m₁B₁

· · ·

^mtB_t isthe ltr-mindecompositionofa

(

132

,

231

)

- sortable permutation

π

, then (with the notation above) B

˜

i

=

^out12

(

Bi

)

,foreachi.However,thisistrueevenwhen thesortabilityrequirementisrelaxed.

Lemma1.Let

π =

^m1B₁

· · ·

^mtB_tbetheltr-mindecomposition ofapermutation

π

.Writeout^132,231

( π ) = ˜

^B1

· · · ˜

^Btmt

· · ·

^m1 asinTheorem1.ThenB

˜

i

=

^out12

(

Bi

)

,foreachi.

Proof. Considertheinstantimmediatelyafterm_iispushed intothe

(

132

,

231

)

-stackandthenon-empty block B_i has tobeprocessed,forsomei.ByTheorem1,atthispointthe

(

132

,

231

)

-stack contains mi

,

mi−¹

, . . . ,

m1, reading from top tobottom.Wewanttoshowthat thebehaviorofthe

(

132

,

231

)

-stackon B_i isequivalent tothebehavior ofan empty12-stackoninput Bi.Weprovethatthe

(

132

,

231

)

- stack performs the pop operation of some x

∈

^Bi if and only ifthe 12-stackdoes thesame. Ifeitherthe next el- ement ofthe input ism_i+1 orx isthe last elementof

π

to be processed, then both the

(

132

,

231

)

-stack and the 12-stack perform a pop operation, as desired. Otherwise, supposethenextelementoftheinputis y,forsome y in thesameblock B_i,andthe

(

132

,

231

)

-stackpops theele- mentx

∈

^Bi.Thismeansthatthe

(

132

,

231

)

-stackcontains two elements z

,

w,withz above w,such that yzw is an occurrence of either 132 or231. Note that, since z

>

w, z isnot a ltr-minimum. Therefore yz isan occurrenceof 12 andthe12-stackperformsapopoperation,asdesired.

Conversely,supposethatthe12-stackpopstheelementx, with y

∈

^Bi the next element of the input. This implies that the 12-stack contains an element z such that z

>

y.

Thereforeyzmiisanoccurrenceof231 andthe

(

132

,

231

)

- stackperformsapopoperation,asdesired.

Corollary1.Let

π =

^m1B₁

· · ·

^mtB_tbetheltr-mindecomposi- tionofapermutation

π

.Thenthefollowingareequivalent.

1. B_iavoids213andB_i

>

B_i+1,foreachi.

2.

π

is

(

132

,

231

)

-sortable.

3.

π _∈

Av(1324

,

2314

)

.

Proof. Combining the firstpoint in Theorem 1andLem- ma1wehave:

out^132,231

( π ) =

^out12

(

B₁

) · · ·

^out12

(

B_t

)

m_t

· · ·

^m1

.

We will use this decomposition of out^132,231

( π )

throughouttherestoftheproof.

[1

⇒

2] Suppose, for a contradiction, that out^132,231

( π )

contains anoccurrencebcaof231.Notethat,since c

>

a, while mt

< · · · <

m1, c is not a ltr-minimum of

π

(and thusneitherisb).Now,ifbandcareinthesameblockBj, thenout¹²

(

B_j

)

isnotdecreasing.Thus,byProposition2,B_j contains213,whichisacontradiction.Otherwise,ifb

∈

^Bj andc

∈

^Bk,with j

<

k,then wehavea contradictionwith thehypothesis B_i

>

B_i+1 foreachi.

[2

⇒

3] Suppose, for a contradiction, that

π ∈ /

Av

(

1324

,

2314

)

.First, suppose that

π

contains an occurrenceacbd of1324.Observethatb

,

c

,

darenotltr-minimaof

π

.Now, ifbanddareinthesameblock B_j of

π

,forsome j,then Bj containsanoccurrencecbdof213.Thereforeout¹²

(

Bj

)

containsanoccurrenceof231 duetoProposition2,which contradictsthehypothesis.Otherwise,ifb

∈

^Bjandd

∈

^Bk, forsome j

<

k,thenout^132,231

( π )

contains anoccurrence bdmk of231,againa contradiction. Thepattern2314 can beaddressedanalogously,soweleaveittothereader.

[3

⇒

^{1] Let}

π ∈

Av

(

1324

,

2314

)

.If B_i contains an occur- rencebac of213,then

π

containsan occurrencemibac of 1324,whichisimpossible.Otherwise,if

π

containstwoel- ementsx

∈

^Bj, y

∈

^Bk,withx

<

y and j

<

k,thenm_jxm_ky isanoccurrenceof2314,contradictingthehypothesis.

TheenumerationofAv(1324

,

2314

)

(or asymmetryof thesepatterns) can be found for instance in [2,16]. Note thatin [1],theauthorsprovideaconstructivebijectionbe- tweenthesepermutationsandSchröderpaths.

Corollary2.Permutations oflengthn in Sort

(

132

,

231

)

are enumeratedbythelargeSchrödernumbers(sequenceA006318 in [15]).

4. The(

σ

,

σ

ˆ)-machine

Forapermutation

σ

oflengthtwoormore,denoteby

ˆ

σ

thepermutation obtained from

σ

by interchanging its firsttwoentries.Letusregard a

( σ , τ )

-stackasanopera- tor outσ,τ

:

S

→

S.By convenientlymodifying the proof of Corollary 4.5 in [9] (stated in the context of Cayley permutations), we have that outσ,τ is a length preserv- ing bijection on S if andonly if

τ = ˆ σ

. More generally, Berlow [5] showed that foraset T ofpatterns, out^T isa

lengthpreservingbijectiononSifandonlyifT isclosed under the

ˆ

^{operator. In order for the paper to be self-} contained,weshallgivethefollowingresult,whichiseas- ier to prove (although weaker): outσ,σˆ is a bijection for any pattern

σ

. An immediateconsequence will be Theo- rem2below.

LetN^∗bethesetoffinitelengthintegersequences.The actionofthe

( σ , τ )

-stackoninput

π

canbenaturallyrep- resentedasasequenceoftriples

(

r

;

^s

;

^t

) ∈ (

N^∗

)

³,wherer isthecurrentcontentoftheoutput,s isthecurrentcon- tent of the

( σ , τ )

-stack (read fromtop to bottom) andt is the current content of the input. The triple

(

r

;

^s

;

^t

)

is saidtobea stateofpassingof

π

throughthe

( σ , τ )

-stack.

Clearly,risaprefixofoutσ,τ

( π )

,tisasuffixof

π

,theini- tialstateis

(λ; λ; π )

andthefinaloneis

(

outσ,τ

( π ); λ; λ)

, where

λ

istheemptysequence.Moreoveranon-finalstate

(

p1p2

· · ·

^pa

;

^s1s2

· · ·

^sb

;

^t1t2

· · ·

^tc

)

isfollowedby eitherthe state

(

p1p2

· · ·

pas1

;

s2

· · ·

sb

;

t1t2

· · ·

tc

),

ifapopoperationisperformednext,or

(

p₁p₂

· · ·

^pa

;

^t1s₁s₂

· · ·

^sb

;

^t2

· · ·

^tc

),

ifapushoperationisperformednext.

For p

=

^p1

· · ·

^pn

∈

Nⁿ, we denote by p^r the reverse of p, that is p^r

=

^pn

· · ·

^p1. We wish to show that the behavior of the

( σ , σ ˆ )

-stack on

π

is strictly related to its behavior on

outσ,σˆ

( π )

r

.More precisely, ifo1

· · ·

^o2n is the sequence ofpush/pop operationsperformed when

π

is passed through a

( σ , σ ˆ )

-stack, then o_2n

· · ·

^o1 is the sequence of push/pop operations performed when

outσ,σˆ

( π )

r

ispassedthroughthe

( σ , σ ˆ )

-stack,whereo_i isa push(resp.pop) operation ifoi isapop (resp.push) operation. This can be equivalently expressed by saying that thestate

(

p

;

^s

;

^t

)

isfollowedby

(

u

;

^v

;

^w

)

ifandonly ifthestate

(

w^r

;

^v

;

^ur

)

isfollowedby

(

t^r

;

^s

;

^pr

)

.

Lemma2.Considertheactionofthe

( σ , σ ˆ )

-stack.Letp

,

s

,

t

∈

N^∗andx

∈

N.

1. Ifthestate

(

p

,

xs

,

t

)

isfollowedbythestate

(

px

,

s

,

t

)

(and thusapopoperationisperformed)thenthestate

(

t^r

,

s

,

xp^r

)

isfollowedbythestate

(

t^r

,

xs

,

p^r

)

(andthusapushopera- tionisperformed).

2. If the state

(

p

,

s

,

xt

)

is followed by the state

(

p

,

xs

,

t

)

(andthusapushoperationis performed),thenthestate

(

t^r

,

xs

,

p^r

)

isfollowedbythestate

(

t^rx

,

s

,

p^r

)

(andthusa popoperationisperformed).

Proof. 1. Since xs isthe content ofthe

( σ , σ ˆ )

-stack in the state

(

p

,

xs

,

t

)

, we have that xs avoids

σ

and

σ ˆ

. Thusapushoperationisperformedifsisthecontent of the

( σ , σ ˆ )

-stack and x isthe next element of the input.

2. If pisempty,thestatementholds.Otherwise,let p

=

p₁

· · ·

^pa and s

=

^s1

· · ·

^sb.Observe that p_a is the last elementthathasbeenextractedfromthe

( σ , σ ˆ )

-stack

before x enters. Therefore, when pa is extracted, pa playstheroleofeither

σ

2inanoccurrenceof

σ

orof

ˆ

σ

2 in an occurrenceof

σ ˆ

. More precisely,one of the followingfourcaseshold.Weshowthedetailsforthe first caseonly, the othersbeing similar. Let z be the lengthof

σ

.

•

^spasi3

· · ·

^siz is an occurrenceof

σ

,forsome

≥

¹ and

<

i₃

< · · · <

i_z. Then p_ass_i3

· · ·

^siz is an oc- currenceof

σ ˆ

andthereforeapopoperationisper- formed when pa is the next element of the input andxsisthecontentofthe

( σ , σ ˆ )

-stack,asdesired.

•

^sp_as_i3

· · ·

^siz is an occurrenceof

σ ˆ

,forsome

≥

¹ and

<

i3

< · · · <

iz.

•

^xpasi3

· · ·

^siz is an occurrence of

σ

, for some i3

<

· · · <

i_z.

•

^xpas_i3

· · ·

^siz is an occurrence of

σ ˆ

, for some i3

<

· · · <

iz.

Astraightforward consequence ofthe previous lemma is that the map

outσ,σˆr

:

S

→

S is its own inverse, and thus a bijection. More specifically, for anypermuta- tion

π

,wehave

outσ,σˆ₋1

( π ) =

outσ,σˆ

( π

^r

)

r

.Since

π

is

( σ , σ ˆ )

-sortableifandonlyifoutσ,σˆ

( π )

avoids231 (and thereversemapisbijective),wehavethatSort

( σ , σ ˆ )

isin bijectionwithAv(231

)

.Thenexttheoremfollows.

Theorem2.Foranypattern

σ

,outσ,σˆ

n isabijection onSn. Moreover,wehave

|

^Sortn

(

123

,

213

) | = |

^Sortn

(

132

,

312

) |

= |

^Sortn

(

231

,

321

) | =

^cn

,

thenthCatalannumber.

5. Pair(123,132)

We characterize Sort

(

123

,

132

)

in terms of pattern avoidance.Thenweshow that

(

123

,

132

)

-sortablepermu- tationsareenumeratedbytheCatalannumbersbyexhibit- ingalinkwiththeverywellstudiedCatalantriangle.

Theorem3.Apermutation

π

is

(

123

,

132

)

-sortableifandonly if

π

avoids2314,3214,4213and

[

²⁴13.

¯

Proof. Supposethat

π

is

(

123

,

132

)

-sortable.Foracontra- diction, suppose that

π

contains

τ ∈ {

²³¹⁴

,

3214

,

4213

}

^. Pick an occurrence

π

i

π

j

π

k

π

of

τ

, with i

<

j

<

k

<

, where

ischosenminimal,andk, j,andi arechosenmax- imal,inthisorder.

If

τ =

^{2314,due to our choice ofi}

,

j

,

k

,

,we have

π

i

<

π

u

< π

j, for k

<

u

<

. Now, when

π

k is pushed in the

(

123

,

132

)

-stack, at least one of

π

i and

π

j has already beenextracted:otherwisethe

(

123

,

132

)

-stackwouldcon- tainanoccurrenceof132,whichisforbidden.Foreachu, k

+

¹

≤

^u

≤

,wehave

π

k

< π

u andwhen

π

u ispushedin the

(

123

,

132

)

-stack,

π

kisstillinthe

(

123

,

132

)

-stack.In- deed,

π

u

π

x

π

y cannotbean occurrenceof123 norof132 with

π

x above

π

y,bothinthetailofthe

(

123

,

132

)

-stack

beginningby

π

k.If

π

i(resp.

π

j)isextractedbefore

π

k en- tersinthe

(

123

,

132

)

-stack,then

π

i

π

k+¹

π

k (resp.

π

j

π

π

k) createsapattern231 inout^123,132

( π )

,acontradiction.

If

τ ₌

3214, dueto our choice of i

,

j

,

k

,

, we have

π

i

>

π

u

> π

j, for k

<

u

<

; and

π

u

< π

u+1, for k

≤

^u

<

. As above, when

π

_k is pushed in the

(

123

,

132

)

-stack, at least one of

π

i and

π

j has already been extracted: oth- erwise the

(

123

,

132

)

-stack would contain an occurrence of 123. Since

π

k+¹

> π

k, the next step pushes

π

k+¹ in the

(

123

,

132

)

-stack. (i) Assumethat

π

i and

π

j bothhad left the

(

123

,

132

)

-stack. Then

π

k is just below

π

k+1 in the

(

123

,

132

)

-stack,and

π

k

< π

j

< π

k+¹.Thisimpliesthat

π

j

π

k+¹

π

k isanoccurrenceof231 in out^123,132

( π )

,acon- tradiction. (ii) Assume that

π

i is still in the

(

123

,

132

)

- stackand

π

jhadleftthisstack.Again,

π

j

π

k+¹

π

kisanoc- currence of231 inout^123,132

( π )

,a contradiction.(iii) As- sumethat

π

jisstillinthe

(

123

,

132

)

-stackand

π

ihadleft thisstack.Since

π

u

< π

u+¹fork

≤

^u

≤ −

^{1,thenextsteps} oftheprocesspushsuccessivelyallentries

π

k+¹

, . . . , π

in the

(

123

,

132

)

-stack.Asabove,

π

i

π

π

_k isanoccurrenceof 231 inout^123,132

( π )

,againacontradiction.

Thecase

τ =

^{4213 canbetreatedsimilarly.}

Finally, suppose that

π

contains

[

²⁴13.

¯

Equivalently, thereare twoindicesi

<

j such that

π

1

π

i

π

j isan occur- renceof132 and

π

k

> π

1foreachi

<

k

<

j.Observethat, by choosing j minimal andi maximal(in thisorder), we canassume j

=

ⁱ

+

^1.Now,if

π

i isstillinthe

(

123

,

132

)

- stack when

π

i+1 enters, then out^123,132

( π )

contains an occurrence

π

i+¹

π

i

π

1 of 231,which is impossibledue to thesortability of

π

.Therefore

π

i isextractedbefore

π

i+¹ enters. Thismeansthat thereare twoelements

π

u

, π

v in the

(

123

,

132

)

-stack, withu

<

v (andthus

π

v above

π

u), suchthat

π

i+¹

π

v

π

u isanoccurrenceofeither123 or132.

Chooseu

,

v minimalamongstthoseindices,² sothat

π

u is still inthe

(

123

,

132

)

-stackwhen

π

i+¹ enters.Notice that

π

i+¹

< π

u (andthusu

=

^{1).Moreover,itmustbe}

π

u

< π

i, otherwise

π

i

π

u

π

1 wouldbe an occurrenceof 231 in the

(

123

,

132

)

-stack,whichisforbidden.Butthen

π

i+¹

π

u

π

1is anoccurrenceof231 inout^123,132

( π )

,acontradiction.

Conversely, suppose that

π

is not

(

123

,

132

)

-sortable.

Weshallprovethat

π

containsatleastoneofthepatterns 3214, 2314, 4213 or

[

²⁴13.

¯

By hypothesis out^123,132

( π )

contains anoccurrencebcaof231.Letb

= π

j andc

= π

k, forsomeindices j

,

k.Wedistinguishtwo cases,according whether j

<

kor j

>

k.

•

^{Supposethat j}

<

k andthus

π

j isextractedfromthe

(

123

,

132

)

-stackbefore

π

k enters.Thenthereare two elements

π

u

, π

v in the

(

123

,

132

)

-stack, with u

<

v (and thus

π

v above

π

u), such that

π

z

π

v

π

u is an oc- currence of either 123 or 132, where

π

z is the next element of the input. Notice that

π

j

≥

^min

{ π

u

, π

v

}

^, since otherwise

π

j

π

v

π

u would be an occurrence of either 123 or 132 in the

(

123

,

132

)

-stack, which is impossible. Thus

π

k

> π

j

> π

z. If

π

z

π

v

π

u is an oc- currence of 123, then

π

u

π

v

π

z

π

k isan occurrenceof

2 In other words, pick the deepest such elements πu,πv in the (123,132)-stack.

either4213,if

π

u

> π

k,or3214,if

π

u

< π

k.Finally,if

π

z

π

v

π

u isanoccurrenceof132,then

π

u

π

j

π

z

π

kisan occurrenceof2314.

•

^{Suppose instead that j}

>

k and thus

π

_k is still in the

(

123

,

132

)

-stack when

π

j enters. Observe that k

=

^{1, since}

π

1 is the last element of out^123,132

( π )

. Therefore, when

π

j enters the

(

123

,

132

)

-stack, the

(

123

,

132

)

-stackcontains theelements

π

j

π

_k

π

1,read- ingfromtoptobottom.Noticethat

π

j

> π

1,otherwise

π

j

π

k

π

1 wouldrealizean occurrenceoftheforbidden 132 insidethe

(

123

,

132

)

-stack.Moreover,foreachen- try

π

t, with k

<

t

<

j, we have

π

t

> π

1. Otherwise

π

t

π

k

π

1 wouldbeanoccurrenceof132 and

π

k would beextractedbefore

π

j,whichisimpossibleduetoour assumptions.Thus

π

1

π

k

π

j isanoccurrenceof

[

²⁴13.

¯

Thiscompletestheproof.

Proposition3.ThedistributionofthefirstelementinSort

(

123

,

132

)

is given by the Catalan triangle (sequence A009766 in [15]).

Proof. Let An

(

k

)

be the set of

(

123

,

132

)

-sortable per- mutations of length n and starting with k. Let A¹_n

(

k

)

be thesubset of An

(

k

)

consistingofthosepermutations

π = π

1

π

2

. . . π

n where any occurrence

π

1

π

i

π

of

[

^{231 with}

π

= π

1

−

^{1 canbeextendedintoanoccurrence}

π

1

π

i

π

j

π

of

[

^{3412.Set A2}n

(

k

) =

^An

(

k

)\

^An¹

(

k

)

andletk

≥

^{2.We shall} providebijections

α :

^A1n

(

k

) →

^An

(

k

−

¹

)

and

β :

^An²

(

k

) →

An−¹

(

k

)

.

Define

α :

^An¹

(

k

) →

^An

(

k

−

¹

)

by

α ( π ) = π

, where

π

is obtainedfrom

π

by swapping thetwo entries

π

1 and

π

1

−

^{1 in}

π

.Since

π ∈

^A1n

(

k

)

, itis easyto check that

π

avoids

[

²⁴13.

¯

In addition, swapping

π

1 and

π

1

−

^{1 does} notaffecttheavoidanceofthethreepatterns3214,2314, 4213,whichimplies(see Theorem 3) that

α ( π ) ∈

^An

(

k

−

1

)

.

Nextdefine

β :

^A2n

(

k

) →

^An−¹

(

k

)

by

β( π ) = π

,where

π

isobtainedfrom

π

by deletingtheentry

π

immedi- atelybeforek

−

^{1 andby decreasingby oneall entriesof}

π

greaterthan

π

.Noticethat

β( π ) ∈

^An−¹

(

k

)

.Letusnow sketchtheproofthat

β

isbijective,leavingsometechnical detailstothereader.Weshallexplicitlydefinetheinverse map of

β

. Given

π _∈

An−¹

(

k

)

,choose aninteger

asfol- lows:

•

is the minimal entryl

= π

u

> π

1, with 1

<

u

<

i, such that there isan index v with

π

v

< π

i andu

<

v

<

i,ifsuchentryexists.

•

^{Otherwise,set}

=

^n.

Thepreimage

π

isobtainedbyinserting

immediatelybe- fore

π

i

=

^k

−

^{1 andthen increasingbyone alltheentries}

π

j of

π

with

π

j

≥

.

Finally,settinga^k_n

= |

^An

(

k

) |

^{,we have that ak}n

=

^akn⁻¹

+

a^k_n−1,for2

≤

^k

≤

^{n.Since A}n

(

1

) = {

¹²³

· · ·

ⁿ

}

^{and A}n

(

n

)

is the setof length npermutations avoiding 213 andstart- ing withn,theinitial conditionsare givenbya¹_n

=

^{1 and} aⁿ_n

=

cn−¹,wherecn isthenthCatalannumber.Therefore, a^k_n generatesthewell-knownCatalantriangle(see Table1 and [6,11,14]).