MarinPezarski
InstituteofInformatis,WarsawUniversity
ul.Banaha2,02{097Warszawa,Poland
e-mail:marpemimuw.edu.pl
Abstrat. Weprovethatsorting13elementsrequires34omparisons.
Thissolvesalong-standingproblemposedbyD.E.Knuthinhisfamous
bookTheArtofComputerProgramming,Volume3,SortingandSearh-
ing.TheresultisduetoaneÆientimplementationofanalgorithmfor
ountinglinearextensionsofagivenpartialorder.Wepresentalsosome
usefulheuristis whih allowed us to derease the running timeof the
implementation.
1 Introdution
The problem of nding optimal sorting algorithms is one of fundamental and
fasinatingproblemsinthetheoryofsorting.Theproblemofndingminimum-
omparisonsortingalgorithmsisespeiallyinteresting.D. E.Knuthdevoted to
this problem a part of his famous book The Art of Computer Programming,
Volume3, SortingandSearhing.
LetS(N)betheminimumnumberofomparisonsthatwillsuÆetosortN
elements.Theinformation-theoreti lower bound tellsusthat
S(N)dlgN!e=
N :
Lester Ford,Jr. andSelmer Johnson [2℄disoveredanalgorithm,alled merge
insertion,whihnearlyandsometimesevenexatlymathesthetheoretiallower
bound.Let F
N
be thenumber of omparisonsrequired to sort N elementsby
mergeinsertion.Wehave
N =1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
N
=0 1 3 5 7 10 13 16 19 22 26 29 33 37 41 45 49 53 57 62 66 70
F
N
=0 1 3 5 7 10 13 16 19 22 26 30 34 38 42 46 50 54 58 62 66 71
One an see that S(N) =
N
= F
N
for N 11 and for N = 20;21. After
thedisoveryofmergeinsertiontherstunknownvaluewasS(12).MarkWells
disovered,arryinganexhaustiveomputersearh,thatS(12)=30[7℄.Donald
KnuthposedtheproblemofndingthenextvalueS(13)inhislassibook[3,
Chap.5.3.1, Exerise35℄.Heonjetured thatS(13)=33[4℄.Inthis paperwe
show that S(13) = 34. Our result is based on a smart implementation of the
algorithmofMarkWells.
ThispaperwaspublishedinproeedingsofESA2002,eds.R.MohringandR.Raman,
2 Preliminaries
LetU =fu
0
;u
1
;:::;u
N 1
gbeanN-elementsettobesorted.SortingU anbe
viewed asasequene ofpartially orderedsets (U;r),where r isapartial order
overU.IfU isxedwewillidentifytheposet(U;r)withr.Thesequenestarts
from thetotal disorderr
0
=f(u;u): u2 Ugand ends with alinearlyordered
set fu
i0
<:::<u
iN
1
g.Everysubsequentposetis obtainedfrom theprevious
oneas theresultofaomparisonbetweentwoelementsof U.Letr be aposet
after performing 1 omparisons and suppose that elements u
j and u
k are
beingomparedin thenextstep.W.l.o.g.weanassumethat (u
j
;u
k
)2=r and
(u
k
;u
j
)2=r. Iftheanswerto theomparisoniselement u
j
isless thanelement
u
k
thenthetransitivelosureofthesetr[f(u
j
;u
k
)gisthenewposet.Wewill
denoteitbyr+u
j u
k .
Denition1. Ifr andlareposetssuhthat rlandlisalinearorder then
lisalledalinearextension ofr.
Everyposethasatleastonelinearextension.Lete(r)denotethenumberof
linear extensions of aposet r. In partiular everytotally unordered n-element
sethasn!linearextensionsandeverylinearlyorderedsethasexatlyonelinear
extension.Everyposetranbesortedusingomparisonsonlyifomparisons
are suÆientto obtainalinearorder from r. Thefollowinglemma generalizes
theinformation-theoretilowerboundto theposets.
Lemma1. If aposetr anbesortedusing omparisonsthene(r)2
.
Foraposetrtheposetr
=f(u;v):(v;u)2rgisalledthedualposet tor.
Denition2. Twoposetsr
1 andr
2
areongruentifr
1 andr
2 orr
1 andr
2 are
isomorphi.
Itiseasytoseethat:
Lemma2. Congruentposetsneedthesamenumberofomparisonstobesorted.
Let(U;r)beaposet andlet Abeasubsetof U.The poset (A;r\AA)
willbedenotedbyrjA.Ifv2U thenr[v℄ willdenotethesetofelementswhih
are larger than v, preisely r[v℄ = fu : (v;u) 2 r^u 6= vg. Then we have
r
[v℄=fu:(u;v)2r^u6=vg.
3 The Method of Wells
Inthis setion wedesribebriey themethod whihwasused to disoverthat
thereisno29-stepsortingproedurefor12elements[7℄.
The set S
ontains posets whih are obtained after omparisons. The
funtionsearh(S
;r)returnsTRUEithesetS
ontainsaposetongruentto
S
0 :=fr
0
g,wherer
0
=f(u
0
;u
0 );(u
1
;u
1
);:::;(u
N 1
;u
N 1 )g
for:=1to
N do
S
:=;
foreahr2S
1 do
forj:=0to N 2do
fork:=j+1toN 1do
if(u
j
;u
k
)2=rand (u
k
;u
j
)2=rthen
r
1
:=r+u
j u
k
r
2
:=r+u
k u
j
ifsearh(S
;r
1
)=FALSEand searh(S
;r
2
)=FALSEthen
t
1 :=e(r
1 )
t
2 :=e(r
2 )
ift
1 2
N
andt
2 2
N
then
ift
1 t
2 then
S
:=S
[fr
1 g
else
S
:=S
[fr
2 g
Webegin from the set S
0
ontaining only one,totally unordered poset r
0 .
Instep everyposet r2S
1
isexamined foreveryunrelated pairu
j and u
k
in r in order to verify whether it an be sorted in the remaining
N
+1
omparisons. As theresult of theomparison of u
j and u
k
one anget oneof
twoposetsr
1
=r+u
j u
k andr
2
=r+u
k u
j
.Ifthenumberoflinearextensions
of r
1 (r
2
) exeeds2
N
then r
1 (r
2
) an't be sorted in the remaining
N
omparisons(byLemma1).Itfollowsthatinthisase,inordertonishsorting
in
N
+1omparisons,elementsu
j andu
k
shouldnotbeomparedinstep.
Ifthenumbersoflinearextensionsofbothr
1 andr
2
don'texeed2 N
thenwe
storeoneofthemforafurtheranalysis.Inprinipleweanhooseanyofthem.
Itdoesn'tinuenetheorretnessofthemethod.Sinetheposetwiththelarger
numberoflinearextensionsseemstobehardertosortwekeeppreiselythisone.
Tiesanbebrokenarbitrarily.ObservethatifS
ontainsaposetongruentto
oneoftheposetsr
1 orr
2
thenwedon'tneedtokeepanyofthem(byLemma2).
Thisreduessubstantiallythenumberofposetsto beonsidered.
Ifthe nalset S
N
doesn't ontainalinearlyordered set then weonlude
thatsortingN elementsrequiresmorethan
N
omparisons.Ontheotherhand
ifalinearlyorderedsetbelongstotheset S
N
itdoesn'tmeanthatitisalways
possible to sort N elements using
N
omparisons. This happens in the ase
N =13whereS
33
ontainsalinearorderand themethod fails.
4 The Funtion Sortable
Inthissetionweproposeanimprovementofthemethoddesribedintheprevi-
oussetion.Thepresentedmodiationenablestooveromewiththeasewhere
thesetS
ontainsalinearorder.Tobeginwithweproveasimplelemma.
Lemma3. If N elements an be sorted using
N
omparisons then for every
, 0
N
,the set S
ontains atleastone posetwhih an besortedusing
N
omparisons.
Proof. Indutionon.Thelemma isobviouslytruefortheinitialset S
0 .Letr
beaposetinS
1
whihanbesortedin
N
+1omparisons.Itfollowsthat
thereexists asortingproedurewhih sortsrin
N
+1steps.Supposethat
theproedureapplyingtorompareselementsu
j andu
k
instep.Hene both
r
1
=r+u
j u
k andr
2
=r+u
k u
j
anbesortedusing
N
omparisons.But
S
ontainsaposetongruenttor
1 orr
2
.HeneS
ontainsaposetsortablein
N
omparisonsbyLemma 2.
Inthesequelwewillextensivelyusethefollowingorollary.
Corollary 1. Ifforsome thesetS
doesn'tontainaposetsortablein
N
omparisons then there is no sorting proedure for N elements using
N om-
parisons.
Letusonsiderthereursivefuntionsortabledesribedbelow.Thefuntion
verieswhetheragivenposetr anbesortedusing omparisons.
booleanfuntionsortable(r;)
ife(r)2then
returnTRUE
forj:=0to N 2do
fork:=j+1toN 1do
if(u
j
;u
k
)62rand (u
k
;u
j
)62rthen
r
1
:=r+u
j u
k
r
2
:=r+u
k u
j
ife(r
1 )2
1
ande(r
2 )2
1
then
ifsortable(r
1
; 1)andsortable(r
2
; 1) then
returnTRUE
returnFALSE
Wekeepthefollowinginvariant: The parameters r and satisfy e(r) 2
.
Theinvariantensuresthatthereursionalwaysstopsbeausedereasesineah
subsequentallandthenumberoflinearextensionsisapositiveinteger.
Ife(r) 2then either e(r)= 1or e(r) = 2.In the rst ase r is alinear
order.Inthelatterase1andtherearetwolinearextensionsoftheposetr.
Denotethemr 0
andr 00
respetively.Letusonsidertheminimalpositionwhere
r 0
and r 00
dier and let u 0
and u 00
be the elements on this position in r 0
and
r 00
respetively. W.l.o.g. weanassume that u 0
<u 00
in r 0
and u 0
> u 00
in r 00
.
Makingasingleomparisononeandeterminetheproperlinearextensionand
sortr.Thereforethefuntion sortable returnsTRUEfore(r)2.
Ifthenumberoflinearextensionsofr islargerthan2,oneonsidersposets
r
1
=r+u
j u
k andr
2
=r+u
k u
j
foreah pairof unrelatedelementsu
j andu
k
inr. Ifthenumberoflinearextensionsofr
1 orr
2
exeeds2 1
thenomparing
u andu weannotsortrin omparisonsbyLemma1.Inthelatterasewe
verifyreursivelywhether r
1 andr
2
aresortablein 1omparisons. Observe
that the invariant is maintained. If both r
1 and r
2
are sortable then r is also
sortableandthefuntionreturnsTRUE.Ifthereisnopairofsortableposetsr
1
andr
2
thenthefuntion returnsFALSE.
An essential property of this method is that the funtion sortable veries
alwaysorretlywhetheritispossibletosortrin omparisons.Unfortunately
we an't use the funtion sortable diretly to verify if sorting is possible in
aseN =13 and =33 beause ofits exponentialomplexityin .But using
Corollary1weanapplythefuntionsortable toimprovethemethodofWells.
5 Useful Heuristis
Counting the linear extensions is the most time-onsuming operation in the
method desribedabove.Inordertoredue thenumberofountingoperations
weapplyafewheuristis.
Reursiveallsinthefuntionsortable shouldbeperformedinthelazyman-
ner.Weansupposethat amongtheposetsr
1 andr
2
theposetwiththelarger
numberof linearextensions isharderto sort. Thereforeapossibilitythat only
oneall will besuÆient islargerwhen thefuntion sortableis alled rstfor
the poset with the larger numberof linearextensions. But if the rst allre-
turnsTRUEthentheseond allisstillneessary.Oneanshowaposetrand
elementsu
j andu
k
suhthate(r
1 )>e(r
2 ),r
1
issortablebutr
2
isnotsortable
(usingthesamenumberofomparisons).
Foraposet (U;r) letV =fv 2U :thereexists u2 U;u6=v;(u;v)2 r or
(v;u)2rg.Wehave:
{ foru;v2UnV andw2V theposetsr+uw,r+vwareongruentandthe
posetsr+wu,r+wvareongruentaswell;
{ foru;v;x;y2UnV andu6=v,x6=ytheposetsr+uv,r+xyareongruent.
ThereforeifV =fu
0
;u
1
;:::;u
n 1
gitissuÆienttoompareonlythefollowing
pairsofelements:
{ u
j andu
k
, for0jn 2andj+1kn 1;
{ u
j andu
n
,for0j<n<N;
{ u
n andu
n+1
,forn<N 1.
FurthermoreweanstartthemethodofWellsfromtheseondstep.Allposets
obtained after the rst omparison are ongruent and set S
1
always ontains
onlyoneposet.Weanassumethatthisistheposetr
0 +u
1 u
0 .
Itisknownthat e(r
1 )+e(r
2
)=e(r)[3,7℄.Thenumbere(r)is omputedin
step before. Hene one an storee(r) and it is suÆientto ompute onlyone
value:e(r
1 )ore(r
2
).Inpratieoneof themanbealulatedfaster.Beause
ountingof linear extensions an take time proportional to their number (see
Set.6)then intuitivelythesmallervalueshouldbefasterto ompute.Wean
extensions. If jr
[j℄j+jr[k℄j jr[j℄j+jr
[k℄j then r
1
has `not worse ordering'
thanr
2
and inthisaseweompute
t
1
=e(r
1
); t
2
=e(r) t
1 :
Weompute
t
2
=e(r
2
); t
1
=e(r) t
2
in theotherase. Furthermore weknow alreadythatifV =fu
0
;u
1
;:::;u
n 1 g
andn<N 1thentheposetsr+u
n+1 u
n
andr+u
n u
n+1
areongruentand
henee(r+u
n+1 u
n
)=e(r+u
n u
n+1
)=e(r)=2.
Thenextimprovementfollowsthelemma:
Lemma4. If (x;u)2r,(v;y)2r and(v;u)2=rthen e(r+xy)e(r+uv).
Proof. Observethat r+uvand(r+xy)+uvareidential. Sineaomparison
an't inreasethenumberoflinearextensionswehave
e(r+uv)=e((r+xy)+uv)e(r+xy):
Itfollowsthatife(r+u
j u
k )>2
forsomeu
j andu
k
thene(r+xy)>2
for
everyxandy suhthat(x;u
j
)2rand(u
k
;y)2r.
6 Counting Linear Extensions
Theproblem ofountinglinearextensionsofagiven poset is#P-omplete[1℄.
This indiates that the ounting linear extensions is probably not easier than
generating them. Therefore we an't expet a polynomial-time algorithm for
ounting, but nevertheless we will show in the next setion that the method
desribedhereisthefastest existing.
Themethodisbasedonthetheoremgivenin[7℄.Inordertostatethetheorem
preiselyweneedsomenewnotion.
Denition3. Let (U;r) be a poset, D U and d 2 D. The pair (A;B) is
alledanadmissiblepartitionof D withrespettoelementdwhenthefollowing
onditionsaresatised:
{ A[B =DnfdgandA\B=;;
{ if(a;d)2r anda6=dthena2A;
{ if(d;b)2randb6=dthenb2B;
{ (b;a)2=r foralla2A andb2B.
Theorem1. Let(U;r)beaposet.
1. IfA;BU,A\B=;and(a;b)2r for a2Aandb2B then
e(rjA[B)=e(rjA)e(rjB) : (1)
2. IfA;BU,A\B=;and(a;b)2=r,(b;a)2=r for a2Aandb2B then
e(rjA[B)=e(rjA)e(rjB)
jAj+jBj
jAj
: (2)
3. IfDU andd2D then
e(rjD)= X
A;B
e(rjA)e(rjB); (3)
where thesum istakenoveralladmissiblepartitions ofD withrespettod.
Themain ideaof thealgorithm isto divide theproblem into`simpler'sub-
problems andto solvethem reursively. First,agivenposetis partitioned into
onneted omponents(in the graphsense). Next,thenumberof alllinearex-
tensions is alulated independently for eah omponent and the results are
ombined using (2). In order to ompute the number of linearextensions of a
onnetedomponentD weapplypoint(3)ofTheorem1.Wetakedsuh that
the numberof admissiblepartitions of D is the smallestpossible.It turns out
thatforsuhdthenumberofrelatedelementsinrjDisthelargestpossible.We
get aset of pairsof simplersubproblemsin this way. Eahsuh subproblemis
solvedinthereursivemannerandtheresultsareombinedusing(3).
In order to improve the performane of the above algorithm we stop the
reursionasearlyaspossible.Tothisaimwedon'tpartitionsmall omponents
D suh thatjDj5.IfjDj2thentheposetrjDis alinearorder(beauseit
is onneted)ande(rjD)=1.If D onsists of3, 4,or5elementswestorethe
numberoflinearextensionsin 3auxiliarytables.Inorderto getthenumberof
linearextension ofrjD we omputetheindex (rjD) in theappropriatetable.
IfjDj=3then
(rjD)= X
v2D
jr[v℄\Dj:
IfjDj=4then
(rjD)= X
v2D
4
(jr[v℄\Dj);
wherethefuntion
4
isdened asfollows:
i =0 1 2 3
4
(i)=0 1 4 10
ForjDj=5wehave
(rjD)= X
5
(jr[v℄\Dj;jr
[v℄\Dj);
wherethefuntion
5
isdened asfollows:
i = 0 1 2 3 4
5
(0;i)= 0 1 3 10
5
(1;i)= 0 6 8 5
5
(2;i)= 1 8 1
5
(3;i)= 3 5
5
(4;i)=10
Wehavethat ifjD
1 j=jD
2
jand(rjD
1
)=(rjD
2
)thene(rjD
1
)=e(rjD
2 ).
7 Results of Experiments
As it was already mentioned the method of Wells produes for N = 13 and
N
=33thesequeneofsetsS
1
;S
2
;:::;S
33
.ThesetS
33
ontainsalinearorder.
Thereforewean'tsayanythingabouttheexisteneofasortingproedurefor13
elementsusing33omparisons.Applyingthefuntionsortable wegetthatthere
isnoposetinS
15
whihanbesortedusing18omparisons.ByCorollary1we
havethat there isnoproedure sorting13 elementsand using33 omparisons.
SinetheFord'sandJohnson'nalgorithmsorts13elementsusing34omparisons
wegetnallythatS(13)=34.GeneratingthesetsS
1
;S
2
;:::;S
33
tookabout2
hoursandtheanalysisofS
15
took about8.5hours.
Toensureorretnessofourimplementationwehekedwhethertheprogram
andetermineproperlytheknownvaluesofS(N)forN12.Supposingweuse
at mostS(N) 1omparisons weobtainedthat there is alwaysS(N) 1
for whih the set S
is empty. This onrm that S(N) 1 omparisons isn't
enoughtosortN elements.ForN 11wehekedwhether S(N)omparisons
is suÆient. Weobtained that eah set in the sequene S
1
;:::;S
S(N)
ontains
at leastonesortableposet,whihisonsistentwithLemma 3.
Thereareotheralgorithmswhihanbeusedforomputingtheexatnum-
berof linearextensionsofagivenposet. Thealgorithm ofVaroland Rotem[8℄
runs in time O(ne(r)) and the algorithm of Pruesse and Ruskey [5℄has the
time omplexity O(e(r)). Both algorithms are designed to generate all linear
extensionsofagivenposet.PruesseandRuskeypresentalsoamodiationthat
sparestheomputationtimeifonewantsto omputeonlythenumberoflinear
extensions. The asymptoti time omplexityof the algorithm presented in the
previoussetionisn'tstillknown.Thefollowingtableomparesimplementations
ofallthreealgorithms.
n e(r) Set. 6 Varol,Rotem Puesse,Ruskey
12 2702765 0.0006 0.5 0.07
13 22368256 0.0014 3.9 0.46
14 199360981 0.0033 37 3.4
15 1903757312 0.0073 320 28
16 19391512145 0.018
21 4951498053124096 1.1
Theposetunderonsiderationwasann-elementfene.Theomputationtimeis
giveninseonds.AllalgorithmsareimplementedintheClanguageandompiled
usingthesameompilerwiththesameoptimisationoptions.Varol'sandRotem's
algorithmisverysimple.Itssoureodehasabout20lines.Thesoureodeof
Pruesse'sandRuskey'salgorithm(written byKennyWong andFrankRuskey)
isaessibleviatheInternet[8℄.Testswererun onPCwith233MHzproessor
and64MBmemory.
8 Some Aspets of Implementation
OuralgorithmsperformalotofsetoperationsonsubsetsoftheindexsetI
N
=
f0;1;2;:::;N 1g.Ourintentionwastoperformthefollowingoperationsasfast
aspossible:A[B,A\B,AnB,jAj,reatingone-elementsetfigandhoosing
anarbitraryelementfromaset AforA;B I
N ,i2I
N
.There isalsoalot of
operationsonerningsetsanning.Ourgoalwastoimplementthoseoperations
to runintimeproportionalto thesize ofagivenset.
A subset A I
N
is represented as the unsigned integer with bit i set i
i2A.Theunion,intersetionandsubtrationofsetsareimplementedaslogial
operations:A|B,A&B,A&~B.Creatingasingleelementset,hoosinganarbitrary
elementfromaset(elementswiththeminimalindiesarederived)andounting
theardinalityof aset areimplementedasmaros:pof2(a),minelement(A),
ardinality(A).Operationminelement(A)isundenedwhenArepresentsthe
empty set. It turns out that the fastest implementation of the maros an be
donebystoringtheappropriatevaluesin arrays.
#define pof2(x) pof2_tbl[x℄
#define min_element(x) min_element_tbl[x℄
#define ardinality(x) ardinality_tbl[x℄
typedef har MIN_EL_TBL_EL;
typedef har CARD_TBL_EL;
SET pof2_tbl[N + 1℄;
MIN_EL_TBL_EL min_element_tbl[1 << N℄;
CARD_TBL_EL ardinality_tbl[1 << N℄;
The tables minelementtbl and ardinalitytbl oupy 2 N
memory ells
eah.Ifitisunaeptableweanapplythefollowingimplementation.
#define HALF_N ((N + 1) >> 1)
#define HALF_SET ((1 << HALF_N) - 1)
#define min_element(x) ((x) & HALF_SET ? \
min_element_tbl[(x) & HALF_SET℄ : \
min_element_tbl[(x) >> HALF_N℄ + HALF_N)
#define ardinality(x) (ardinality_tbl[(x) & HALF_SET℄ + \
ardinality_tbl[(x) >> HALF_N℄)
MIN_EL_TBL_EL min_element_tbl[1 << HALF_N℄;
This redues memory requirements to only2 dN =2e
ellsand slow down the set
operationsonlybyaonstant.Theimplementationofloop`foreaha2A'use
themarosdenedabove.
while (A) {
a = min_element(A);
A ^= pof2(a);
...
}
Aposetis representedin thetables r[0::N 1℄and r
[0::N 1℄wherer[j℄
andr
[j℄arethesetsofindiesofallelementslargerandsmallerthanu
j
respe-
tively. Theongruene ofposets is hekedusing hash funtions and Ullman's
graphisomorphismalgorithm[6℄.Theompletesoureodeanbedownloaded
from[9℄.
9 Aknowledgement
IwouldliketothankKrzysztofDiksforhisomments,suggestionsandenour-
agement.
Referenes
1. Brightwell, G.,Winkler,P.:CountingLinearExtensions.Order8(1991)225{242
2. Ford, L., Johnson, S.: A Tournament Problem. Amerian Mathematial Monthly
66(1959)387{389
3. Knuth,D.E.: TheArtofComputerProgramming,Vol.3.SortingandSearhing,
2nded.Addison-Wesley(1998)
4. Knuth, D. E., Kaehler, E. B.: An Experiment in Optimal Sorting. Information
Proessing Letters1(1972)173{176
5. Pruesse,G.,Ruskey,F.:GeneratingLinearExtensionsFast.SIAMJournalonCom-
puting23(1994)373{386
6. Ullman, J.R.:An Algorithmfor SubgraphIsomorphism.Journal ofthe ACM23
(1976)31{42
7. Wells,M.:ElementsofCombinatorialComputing.PergamonPress(1971)
8. TheCombinatorialObjetServer:http://www.theory.s.uvi.a/~os
9. Author'sHomePage:http://mimuw.edu.pl/~marpe
