On the rate of decrease in logical depth Theoretical Computer Science

Contents lists available atScienceDirect

Theoretical Computer Science

www.elsevier.com/locate/tcs

On the rate of decrease in logical depth

L.F. Antunes

^a

^,

¹

, A. Souto

^b

^,

^c

^,

²

, P.M.B. Vitányi

^d

^,

^e

^,∗,

³

aUniversityofPorto&CRACS/INESC-TEC,RuadoCampoAlegre,1021,4169-007Porto,Portugal

bLaSIGE,FaculdadedeCiências,UniversidadedeLisboa,PortugalCampoGrande,1749-016Lisboa,Portugal

cDepartamentodeInformática,FaculdadedeCiências,UniversidadedeLisboa,CampoGrande,1749-016Lisboa,Portugal dCWI,SciencePark123,1098XGAmsterdam,TheNetherlands

eUniversityofAmsterdam,TheNetherlands

a rt i c l e i n f o a b s t ra c t

Articlehistory:

Received15March2017

Receivedinrevisedform26July2017 Accepted19August2017

Availableonline25August2017 CommunicatedbyL.M.Kirousis

Keywords:

Logicaldepth Kolmogorovcomplexity Compression

The logical depth with significance b of a string x is the shortest running time of a programforxthat canbecompressedbyatmostbbits.Anotherdefinition isbasedon algorithmicprobability. Wegiveasimplenew prooffortheknownrelationbetweenthe twodefinitions.Wealsoprovethefollowing:Givenastringwecanconsiderthemaximal decreaseinlogicaldepthwhenthesignificance parameterincreasesby 1.Thereexists a sequenceofstringsoflengthsn=¹,2,. . .,suchthatthismaximaldecreaseasafunction ofnrisesfasterthananycomputablefunctionbutnotasfastastheBusyBeaverfunction.

Thisholdsalsoforthecomputationtimesoftheshortestprogramsofthesestrings.

©^{2017ElsevierB.V.Allrightsreserved.}

1. Introduction

Thelogicaldepthisrelatedtocomplexitywithboundedresourcesandmeasuresthetradeoffbetweenprogramsizesand runningtimes.Computingastringxfromoneofitsshortestprogramsmaytakeaverylongtime,butcomputingthesame stringfromasimple“print(x)”programoflengthabout|^x|^{bitstakesverylittletime.}

Aprogrampforx,i.e.suchthatU

(

p

)

=^{xwhereU isauniversalTuringmachine,oflargerlengththanagivenprogram} qforxmayrequirelesscomputationtimethanqdoes.Thisneednotalwaysbethecase,asalongerprogrammightsimply performsomepointlessredundantsteps.

In generalwe associatelonger computation times withshorterprograms for x.As a consequence,one may raise the questionofhowmuchtimecanbesavedbycomputingagivenstringfromalongerprogram.

*

Correspondingauthor.

E-mailaddresses:antunes.lfa@gmail.com(L.F. Antunes),andrenunosouto@gmail.com(A. Souto),Paul.Vitanyi@cwi.nl(P.M.B. Vitányi).

1 This work was supported by “Project *NanoSTIMA: Macro-to-Nano Human Sensing: Towards Integrated Multimodal Health Monitoring and Analytics*/NORTE-01-0145-FEDER-000016”whichisfinancedbytheNorthPortugalRegionalOperationalProgramme(NORTE2020),underthePORTUGAL 2020PartnershipAgreement,andthroughtheEuropeanRegionalDevelopmentFund(ERDF).

2 ThisworkwassupportedbyLaSIGEResearchUnit,ref.UID/CEC/00408/2013andtheFCTprojectConfidentPTDC/EEI-CTP/4503/2014,tSQIG–Security and Quantum InformationGroup,the Instituto deTelecomunicações(IT) Research Unit, ref. UID/EEA/50008/2013, and the FCT Post-doc scholarship SFRH/BPD/76231/2011duringwhichthemajorpartoftheworkwasdone.

3 CWIistheNationalResearchInstituteforMathematicsandComputerScienceintheNetherlands.

http://dx.doi.org/10.1016/j.tcs.2017.08.012 0304-3975/©^{2017ElsevierB.V.Allrightsreserved.}

1.1. Previouswork

The above question was first considered by C. Bennett in [1]. The minimum time to compute a string by a b- incompressibleprogram was calledthe logicaldepthatsignificanceb ofthe string considered.Bennett also providedvari- ationsofthisdefinitionandstudiedtheir basicpropertiesandrelations.Amoreformal treatment,aswellasan intuitive approach,wasgivenin[4],Section 7.7.

1.2. Results

Section2introducesnotations,definitionsandresultsneededintheremainder.Section3presentstwoversionsoflogical depthandgivesasimplenewproofofquantitativerelations betweenthem.Section 4showsthat slightvariationsonthe significancelevelmaycausedrasticvariationsoflogicaldepth.Section5presentsconclusions.

2. Preliminaries

We usestring fora finitebinary string. Stringsare denotedby lower caseletters,such as x, y and z. Thelength ofa stringx(thenumberofoccurrencesofbitsinit)isdenotedby|^x|^{,andtheemptystringby .Thus,}|

|=^{0.Thenthstring} inthelexicographiclength-increasingorderisviewedalsoasthenaturalnumbernandviceversa.

Computability,resource-boundedcomputationtime,self-delimitingstrings,big-Onotation,and(prefix)Kolmogorovcom- plexityarewell-knownandtheproperties,notations,aretreatedin[4].⁴ OriginallyKolmogorovcomplexitywasintroduced in1965in[2]andtheprefixversionin1974in[3].Thelength ofa self-delimitingversionofastringoflengthnwillbe n+^2logn+^{1 where}log denotesthelogarithmbase 2.Thatis,theself-delimitingversion ofxisx=^1||x||0|^x|^xwhere||^x||

denotesthelengthof|^x|^.Restrictingthecomputationtimeresourceisindicatedbyasuperscriptgivingtheallowednumber ofsteps,usuallydenotedbyd.

We choose a referenceoptimaluniversalprefixTuringmachine and call it U. Let x

,

y be strings. The prefixKolmogorov complexity K

(

x|^y

)

ofxwithauxiliary yisdefinedby

K

(

x

|

^y

) =

^min

p

{|

^p

| :

^U

(

p

,

y

) =

^x

}.

IfxisastringoflengthnthenK

(

x|^y

)

≤ⁿ+^O

(

logn

)

.ThenotationU^d

(

p

,

y

)

=^xmeansthatU

(

p

,

y

)

=^{xwithindsteps.The} d-time-boundedprefixKolmogorovcomplexity K^d

(

x|^y

)

isdefinedby⁵

K^d

(

x

|

^y

) =

^min

p

{|

^p

| :

^Ud

(

p

,

y

) =

^x

} .

Iftheauxiliarystring yistheemptystring ,thenweusuallydropit.Similarly,wewriteU

(

p

)

forU

(

p

, )

.Thestringx^∗is ashortestprogramforxifU

(

x^∗

)

=^xandK

(

x

)

= |^x∗|^.Astringxisc-incompressibleif|^x∗|≥ |^x|−^{c anditis}c-compressibleif

|^x∗|≤ |^x|−^c.

Define Q

(

x

)

=

p:U(p)=x2^−|p|.In[3]L.A.LevinprovedintheCodingTheoremthat(see[4]Theorem 4.3.3fordetails):

−

^logQ

(

x

) =

^K

(

x

) +

^O

(

1

)

3. Differentversionsoflogicaldepth

Thelogicaldepthasdefinedin[1]forastring(thefinitecase)comesintwoversions:onebasedonthecompressibility ofprograms ofprefix Turing machinesandtheother usingthe ratiobetweenalgorithmic probabilitieswithandwithout timelimits.

Thealgorithmicprobabilityversionisbasedontheso-calleda prioriprobability[4]anditstime-boundedversion:

Q

(

x

) =

U(p)=^x

2^−|p|

,

Q^d

(

x

) =

U^d(p)=x

2^−|p|

.

4 ThereadermaybelessfamiliarwiththeprefixTuringmachine.ItisaTuringmachinewithaone-wayread-onlyprogramtape,anauxiliarytape,one ormoreworktapesandanoutputtape.Alltapesarelinearanddividedincellscapableofcontainingoneoutofafinitesetofsymbols.Initiallythe programtapeisinscribedwithaninfinitesequenceof0’sand1’sandtheheadisscanningtheleftmostcell(theprogramtapeissemi-infinite).Whenthe computationterminatesthesequenceofbitsscannedistheprogram.Foreveryfixedcontentsoftheauxiliarytapethesetofprogramsforsuchamachine isaprefixcode(noprogramisaproperprefixofanotherprogram).AnotherconsequenceisthatifthecomputationofaprefixTuringmachinetakesd stepsthenforitsprogrampholdsthatd≥ |^p|^{.TheprefixKolmogorovcomplexityisbasedontheprefixTuringmachinesimilartothe(plain)Kolmogorov} complexitybasedonthe(plain)Turingmachine.

5 Sincewedealwithrunningtimesofcomputationsthefollowingcanhappen.TwodifferentreferenceoptimaluniversalprefixTuringmachinesmay havedifferentcomputationtimesforthesamecombinationofinput,auxiliarystring,andoutput.Itcanalsobethecasethattheyhavedifferentsetsof programs.LetUandUbetwooptimaluniversalprefixTuringmachinesinthestandardenumerationofprefixTuringmachines.Foreveryauxiliaryyand everyprogramqthereisaprogrampwith|^q|= |^p|+^O(1)suchthatU^d(q,y)=^Ud(p,y)forintegersd,dandd=^g(d)withgacomputablefunction.

Therefore,althoughUandUhavethesamelengthofshortestprogramsforastringx(withO(1)precision,thatis,uptoaconstantadditiveterm),the time-limitedprefixKolmogorovcomplexityofastringxmaydifferbyanon-constantadditiveterm.

Definition1.Letxbeastringandb anonnegativeinteger.Thelogicaldepth,version 1,ofxatsignificancelevel

ε

₌2^−bis depth⁽_ε¹⁾

(

x

) =

min

d

:

^Qd

(

x

)

Q

(

x

) ≥ ε

.

The definition oflogical depthbased onthe prefix Kolmogorov complexity version is basedon the minimal time the referenceoptimalprefixTuringmachineneedstocomputexfromaprogramwhichisb-incompressible.

Definition2.Letxbeastringandb anonnegativeinteger.Thelogicaldepth,version 2,ofxatsignificancelevelb,is

depth⁽_b²⁾

(

x

) =

^min

d

:

^p

∈ {

⁰

,

1

}

^∗

∧

^Ud

(

p

) =

^x

∧ |

^p

| ≤

^K

(

p

) +

^b

,

theleastnumberofstepstocomputexbyab-incompressibleprogram.

Remark1.Therelationofalmostequalitybetweenthosedefinitionsisknown[4,Theorem 7.7.1]basedonthemoreinfor- mal [1,Lemma 3].As far aswe knowthat isthe onlyexisting proof. We give asimple newproof basedon algorithmic probability. 3

Theorem1.Letx beastringandb

,

d positiveintegers.Then

(i) depth⁽¹⁾

2^−b

(

x

)

=^{d impliesdepth(}_b²₋⁾_O(1)

(

x

)

=^d,and

(ii) depth⁽_b²⁾

(

x

)

=^{d impliesdepth(}₂¹₋⁾b

(

x

)

≤^{d forsomeb}≤^b+^K

(

b

)

+^O

(

1

)

.

Proof. (i):Let Q^d

(

x

)

≥^2−bQ

(

x

)

withd least.Letc be thegreatest integersuch that all programs computing xwithin d steps are c-compressible.Therefore every p such that U^d

(

p

)

=^{xsatisfies U}

(

r

)

=^{pforsome r.Tocompute string xby U} using program r requires that r isconcatenated withan extra program q of constant length c≥^{0 to restart U after it} has computed p. Then U^d

(

U

(

rq

))

=^xand |^r|≤ |^p|−^c−^{c.Forevery p there are possiblymore than onesuch r.Let P} be the set ofthese p’s and R be theset ofthe r’s. Hence

r∈R2^−|r|≥

p∈P2^{−(|p|−c−c)}. Thatis,the referenceoptimal universal Turing machine computes x withprobability atleast 2^c+cQ^d

(

x

)

≥^2c+c−bQ

(

x

)

fromthe programs in R.Then Q

(

x

)

≥^2c+c−bQ

(

x

)

+^2−bQ

(

x

)

(the left-hand side comprises all halting programs for x no matter what is the running time andtheright-handside comprisesalowerbound onthecontributionoftheprogramsin R plusthecontributionof the programs in P). Thismeans that 2^c+c−b

<

1 and thereforec+^c−^b

<

0,that isc+^c

<

b. Among theprograms in P thereis aprogram whichisc-incompressible (otherwise c is notgreatest asassumedabove)andsince c+^c

<

b itis

(

b−^c

)

=

(

b−^O

(

1

))

-incompressible.

(ii): Assumethatdistheleastnumberofstepsto computex byab-incompressible program.Bywayofcontradiction let Q^d

(

x

) <

2^−BQ

(

x

)

with B=^b+^K

(

b

)

+^e

>

0 foralarge enoughconstante.Letanauxiliary distribution R bedefined asfollows.Computablyenumeratetheset P ofallprograms psuchthat U^d

(

p

)

=^{x.Let R}

(

x

)

=

p∈P2^B−|p|.Then⁶ R

(

x

)

= 2^BQ^d

(

x

) <

Q

(

x

)

.Thefunction RgivenB islowersemicomputable.Since Q isasemimeasureandR

(

x

) <

Q

(

x

)

forallxthe function Risasemimeasure.A program p∈^{P adds2−|p|+B} probabilitytoR

(

x

)

andpistherefore

(

B−^O

(

1

))

-compressible given B,andhence

(

B−^K

(

B

)

−^O

(

1

))

-compressible.Itremainstoboundtheconstantterm.Forsufficientlylargee,

B

−

^K

(

B

) −

^O

(

1

) =

^b

+

^K

(

b

) +

^e

−

^K

(

b

+

^K

(

b

) +

^e

) −

^O

(

1

)

≥

^b

+

^K

(

b

) +

^e

−

^K

(

b

+

^K

(

b

)) −

^K

(

e

) −

^O

(

1

)

>

b

.

The first inequality follows from K

(

u+^v

)

≤^K

(

u

)

+^K

(

v

)

+^O

(

1

)

, andthe second inequality holds since K

(

b+^K

(

b

))

≤ K

(

b

,

K

(

b

))

+^O

(

1

)

≤^K

(

b

)

+^O

(

1

)

and K

(

e

) <

e

/

2 for e large enough. Hence all programs that compute xin d steps are

(

b+¹

)

-compressiblecontradictingtheassumption.Therefore Q^d

(

x

)

≥^2−BQ

(

x

)

. 2

Remark2.Withb

,

dasinthetheoremaslightmodificationintheproofyields

1

2^b+K(b)+O(1)

≤

^QdU

(

x

)

QU

(

x

) ≤

¹

2^b−O(1)

,

where(i)correspondstotherightinequalityand(ii)totheleftinequality. 3

6 WeassumethatR(x)>0.IfR(x)=^{0 thenthereisnoprogrampsuchthatUd}(p)=^{xand(ii)isvacuouslycorrect.}

Remark3. Notice thatit is possibleto use K

(

d

)

instead of K

(

b

)

in theabove proof by changing theconstruction ofthe semiprobabilityasfollows:knowingdgenerateallprograms pcomputingxwithindstepsandletthesemiprobabilitiesbe proportionalto2^−|p|andthesumbelessthan Q

(

x

)

.Inthisway,K

(

b

)

inTheorem 1canbeimprovedtomin{^K

(

b

),

K

(

d

)

}+ O

(

1

)

. 3

4. Thegraphoflogicaldepth

Slightchangesofthesignificancelevelb candrasticallychangethevalueofthelogicaldepth.

Lemma1.Let

φ

bedefinedby

φ (

n

) =

^max

|^x|=ⁿmin

d

{

^d

:

^Ud

(

x^∗

) =

^x

}.

Then

φ

isnotcomputableandgrowsfasterthananycomputablefunction.

Proof. Ifa function

φ

as inthe lemma were computable, then foran x of length n we could run U for

φ (

n

)

steps on anyprogram oflength n+^O

(

logn

)

. Among thoseprograms that halt within

φ (

n

)

steps, we could selecttheoneswhich outputx.Subsequently,wecouldselectfromthatsetaprogramofminimumlengthfromwhichxcanbecomputed.Sucha programhaslengthK

(

x

)

.ThiswouldimplythatK wouldbecomputable.ButthefunctionK isincomputable:contradiction.

Therefore

φ

cannot be computable.Sincethisholds foreveryfunction majoring

φ

, thefunction

φ

must growfasterthan anycomputablefunction. 2

Definition3.Thebusybeaverfunction B B:N→N^isdefinedby B B

(

n

) =

^max

{

^d

: |

^p

| ≤

ⁿ

∧

^Ud

(

p

) < ∞} .

Thisis a variantin termsofnumber ofsteps fromthe original Busy Beaverfunction in[5].The following resultwas mentionedinformallyin[1].

Lemma2.Therunningtimeofaprogramp oflengthn thathaltsisatmostB B

(

n

)

.Therunningtimeofashortestprogram(shortest inprefixKolmogorovcomplexity)forastringx oflengthn isatmostB B

(

n+^O

(

logn

))

.

Proof. ThefirststatementofthelemmafollowsfromDefinition 3.Thesecondstatementfollowsfromtheobservationthat K

(

x

)

≤ⁿ+^O

(

logn

)

forevery xoflengthn. 2

Theorem2.Thefunction

f

(

n

) =

^max

|x|=n,0≤b≤n

{

^x

:

^depth(_b²⁾

(

x

) −

^depth(_b²₊⁾₁

(

x

)}

growsfasterthananycomputablefunctionbutnotasfastastheBusyBeaverfunction.

Proof. Let

φ

betheincomputablefunctionofLemma 1.Foreveryn≥^{1 selectx}noflengthnsuchthatU^{φ (n)}

(

x^∗_n

)

=^xn.Since a shortestprogram can be compressedby at most O

(

1

)

bits we have depth⁽_O²₍⁾₁₎

(

x

)

=

φ (

n

)

.The optimal referenceprefix Turingmachine U can compute xfromits self-delimiting version xof lengthn+O

(

logn

)

usingan O

(

1

)

-bitprogram in time g

(

n

)

withg acomputablefunction.

For every n we can consider the finite sequence Lⁿ₀

,

Lⁿ₁

, . . . ,

Lⁿ_{n−K(x)+O(logn)} with Lⁿ_m=^depth(m²⁾

(

xn

)

−^depth(m²+⁾¹

(

xn

)

(0≤^m≤ⁿ−^K

(

n

)

+^O

(

logn

)

), and

φ (

n

)

=n−^K(x)+^O(logn)

i=0 Lⁿ_i. Then, the average

(φ (

n

)

−^g

(

n

))/(

n−^K

(

n

)

+^O

(

logn

))

is incomputable(asitgrowsfasterthananycomputablefunction),andthereforethefunction f definedby

f

(

n

) =

^max

m

{

^Lmn

:

⁰

≤

^m

≤

ⁿ

−

^K

(

x

) +

^O

(

logn

)}

alsogrowsfasterthananycomputablefunction,since f

(

n

)

≥

(φ (

n

)

−^g

(

n

))/(

n−^K

(

n

)

+^O

(

logn

))

. 2 5. Conclusion

In[1]oneversionoflogicaldepthisbasedoncompressionandoneisbasedonalgorithmicprobability.Theseareclosely related[1,4].Wegaveanewproofofthisrelationbasedonincompressibilityandalgorithmicprobability.Forastringxthe logicaldepthwithsignificanceb istheminimumnumberofstepsdina computationofx froma program pthat isless than b bits longer than K

(

p

)

. This b is calledthe significance. There is a sequence of strings such that for each string

changingthesignificanceparameterbyonly1canchangethedepthbyalargeamount.Asafunctionofthepositionofthe stringinthesequencethisamountgrowsfasterthananycomputablefunctionbutnotasfastastheBusyBeaverfunction.

The computationtimesofshortestprograms cansimilarlyrisefasterthan anycomputable functionbutnotasfastasthe BusyBeaverfunction.

Acknowledgements

We thankananonymous personforcomments andsuggestingthenewproof ofTheorem 1andthereferees forcom- ments.

References

[1]C.Bennett,Logicaldepthandphysicalcomplexity,in:R.Herken(Ed.),TheUniversalTuringMachine: AHalf-CenturySurvey,OxfordUniversityPress, 1988,pp. 227–257.

[2]A.N.Kolmogorov,Threeapproachestothequantitativedefinitionofinformation,Probl.Inf.Transm.1 (1)(1965)1–7.

[3]L.A.Levin,Lawsofinformationconservation(non-growth)andaspectsofthefoundationofprobabilitytheory,Probl.Inf.Transm.10(1974)206–210.

[4]M.Li,P.M.B.Vitányi,AnIntroductiontoKolmogorovComplexityandItsApplications,Springer,NewYork,2008.

[5]T.Rado,Onnon-computablefunctions,BellSyst.Tech.J.41 (3)(1962)877–884.