Randomized algorithms and non-uniform lower bounds
Sylvain Perifel Cours MPRI, 2010–2011
Note: parts of this text are inspired by the book of Arora and Barak Computational Complexity: a modern approach.
Randomized algorithms: the program can toss a coin instructionrand with independent tosses.
Can randomness speed up computation?
Some examples:
• Thanks to randomness, for some problems the worst case can be reduced to the average case:
– Quicksort(deterministic worst case: Ω(n2); randomized: in all cases, expected costO(nlogn));
– Problem: in a word x ∈ {a, b}? containing as many a’s as b’s, find the position of an a.
Deterministic worst case: 'n/2 trials; randomized (test positions at random): Pr(itrials) = 2−i ⇒expected number of trialsP+∞
i=1i2−i=O(1).
• Problem: compute the volume of a (bounded) convex body in Rn given as a black box for membership.
This can be done with error with a randomized algorithm running in time poly(n,1/) (Dyer, Frieze, Kannan 1991).
However, nodeterministicpolynomial-time algorithm can give an upper bound and a lower bound satisfying sup/inf≤(cn/logn)n for some fixedc >0 (B´ar´any and Fredi, 1987).
Randomness seems to speed up computations. . . However:
Generic techniques for derandomization: d-wise independence, methods of conditional proba- bilities and expectations, pessimistic estimators. . .
Ad-hoc derandomizations: Primes (AKS 2002),Undirected reachability(Reingold 2004). . . Almost all “natural” problems having polynomial-time randomized algorithms turn out to have polynomial-time deterministic algorithms.
Can every problem be derandomized?
Goal of this course: formalize this question, draw links with other areas of computational com- plexity, be convinced that the answer is “yes”.
Anobstacle: problemInteger nullity(make a picture, explain the ideas why inBPPand why not yet inP)
How to derandomize? Make formal an old idea: a hard problem appears random to a machine with limited computational power.
1 Probabilistic classes
Notation: ifAis a language andxa word, [x∈A] = 1 if x∈A, 0 otherwise.
Definition(BPP) – A languageAis in the classBPPif there exist a polynomialp(n) and a deterministic polynomial-time Turing machine (TM)M(x, y) with|y|=p(|x|) such that for allx, Prr(M(x, r) = [x∈ A])≥2/3.
Definition(RP) – A languageAis in the classRPif there exist a polynomialp(n) and a deterministic polynomial-time TMM(x, y) with|y|=p(|x|) such that for allx,x∈A⇒Prr(M(x, r) = 1)≥2/3 and x6∈A⇒Prr(M(x, r) = 0) = 1 (i.e. ∀r, M(x, r) = 0).
Remark: RP⊆BPP
Proposition – The error inRP can be made≤2−poly(n).
Proof. LetA∈RPand M andpthe corresponding TM and polynomial. Letq(n) be a polynomial: we show that the error can be reduced to 2−q(n). LetM0(x, r0), withr0=r1, . . . , rq(n)and|ri|=p(n), be the following machine: simulate successivelyM(x, r1), . . . , M(x, rq(n)); if one of the result is 1, then acceptx, otherwise rejectx. Ifx6∈A, then it is rejected. Ifx∈A, then Prr0(M0(x, r0) = 0)≤(1/3)q(n)≤2−q(n).
The same is true forBPP: repeatc.p(n) times (for some constantc) and take the majoritary answer.
Chernoff bounds for the analysis.
Definition – Integer nullity(IN):
Input: an arithmetic circuitC (with gates + and×) computing an integerN from the constant−1.
Question: isN equal to 0?
Proposition – INis in coRP(that is,cIN∈RP).
Proof. The idea is to evaluateC modulomfor a randomly chosen integer m.
Suppose N 6= 0. Let n be the number of gates of C. Then |N| ≤ 22n, therefore N has at most 2n prime factors. Choose a number m∈[2,2n2] at random. By the prime number theorem, there are
≥ c2n2/n2 primes in this interval (for some absolute constant c > 0). Hence, Pr(mis prime) ≥c/n2 and, if m is prime, Pr(mdividesN) ≤ 2n/(c2n2/n2) = (n2/c)2n−n2. Therefore Pr((N mod m) = 0) = Pr((N modm) = 0|mprime).Pr(mprime) + Pr((Nmodm) = 0|mnot prime).Pr(mnot prime)≤ (n2/c)2n−n2+(1−c/n2)≤1−d/n2for somed >0. By choosing Θ(n2) modulimi: Pr(V
i((Nmodmi) = 0))≤1/3.
If nowN = 0, then (Nmodmi) = 0 for alli. We deduce thecoRPalgorithm: choose Θ(n2) integers mi ∈ [2,2n2] at random. Evaluate C modulo mi. If N mod mi 6= 0 for some of them, then reject,
otherwise accept.
Proposition – RP⊆NP.
Proof. Let A∈RPand M the corresponding machine. LetM0 be the following nondeterministic TM:
M0(x) guesses r of size p(n) and simulatesM(x, r). If there is an accepting path then Prr(M(x, r) = 0)<1, therefore x∈A. Otherwise,∀r, M(x, r) = 0 andx6∈A.
Reminder: A∈Σp2 if there exists a polynomialp(n) and a languageB ∈Psuch thatx∈A ⇐⇒ ∃y∈ {0,1}p(n)∀z∈ {0,1}p(n),(x, y, z)∈B. Or, equivalently, Σp2=NPNP.
Proposition – BPP⊆Σp2.
Proof. Let A ∈ BPP and M be the corresponding TM with error <2−n. Denote by m = nO(1) the number of random bits used. For eachx, letSx={r∈ {0,1}m : M(x, r) = 1}. Thenx∈A⇒ |Sx| ≥ (1−2−n)2mandx6∈A⇒ |Sx| ≤2−n2m.
For a set S ⊆ {0,1}m and u∈ {0,1}m, denote byS+uthe set {u+v : v ∈S} where + is the vector addition modulo 2.
If|S| ≤2−n2mthen for all vectors u1, . . . , um,S
i(S+ui)6={0,1}mbecausem|S|<2m. If |S| ≥ (1−2−n)2m, let us prove that Pr(S
i(S +ui) = {0,1}m) > 0, where the ui are taken independently at random. For a fixed v ∈ {0,1}m, we have Pru(v 6∈ S
i(S +ui)) = Pru(V
i(v 6∈
(S+ui))) = (2−n)m= 2−nm. Hence Pru({0,1}m 6=S
i(S+ui)) ≤2−n, therefore there exist ui such that{0,1}m=S
i(S+ui).
Now, x∈A can be rewritten as the following Σp2 statement:
∃u1, . . . , um,∀v,(v∈[
i
(S+ui)).
2 Circuits and advices
Picture. Why circuits: in order to “look inside the structure of the computation”.
Definition(Boolean circuits) – A Boolean circuit is a DAG whose vertices have indegree 0, 1 or 2 and are called gates. Gates of indegree 0 are called inputs and are labelled byx1, . . . , xn; gates of indegree 1 are labelled by¬; gates of indegree 2 are labelled by ∨or ∧. Exactly one gate has outdegree 1: this gate is called the output. The size of the circuit is the number of vertices.
Lemma – The number of circuits of size sis≤(3s)2s.
Proof. Let us first count the number of circuits withninputs. Order the gates such that thenfirst gates are the input gates. Each non-input gate has at most 2 inputs among s−1, which makes ≤ (s−1)2 possibilities, and has a label among{∧,∨,¬} (3 possibilities). There ares−nsuch gates, therefore the number of circuits is at most (3(s−1)2)s−n≤(3(s−1)2)s.
Now, taking into account an arbitrary number of inputs, the number of circuits is≤s×(3(s−1)2)s≤
(3s)2s.
Definition – Ifx∈ {0,1}n andC a circuit withninputs, the value of C(x) is the value of the output gate, where the value of a gate is defined inductively:
• the value of thei-th input gate isxi;
• the value of a¬gate is the negation of the value of its input;
• the value of a∧(resp. ∨) gate is the conjunction (resp. disjunction) of the values of its inputs.
Fixed input size Deciding a language A ⊆ {0,1}? requires a family (Cn) of circuits, Cn having n inputs (nonuniformity). The language recognized by (Cn) is{x∈ {0,1}? : C|x|(x) = 1}.
Definition(P/poly) – The classP/polyis the set of languages recognized by a family of polynomial-size circuits (i.e. there exists a polynomialp(n) such that|Cn| ≤p(n)).
Remark: P/polyis an uncountable class!
Definition – A family (Cn) of circuits is calleduniformif there exists a TM which, on inputn, computes the encoding ofCn in time polynomial inn.
Proposition – The classPis the set of languages recognized by auniformfamily of circuits of polynomial size.
Proof. IfAhas a uniform family of circuits: M(x) builds the circuitC|x|and simulates C|x|(x).
In the other direction, ifA∈Pthen the machine can be simulated by circuits of polynomial size and
the construction is uniform.
Proposition –P/polyis the set of languages recognized by a polynomial-time TM with polynomial-size advice: i.e. A∈P/poly iff there exists a polynomialp(n), a sequence (an) of words of sizep(n) and a languageB ∈Psuch thatx∈A ⇐⇒ (x, a|x|)∈B.
Proof. If A ∈ P/poly with circuits Cn, then we give as advice the circuit Cn: the language B is {(x, C) : C(x) = 1} ∈P.In the other direction, you build a circuit simulating a machineM forB (level iof the circuit corresponds to the content of the tape at stepi) and hard-wire the advicean. Proposition – BPP⊂P/poly.
Proof. Let A∈ BPP. There exists a machineM(x, y) with |y|=|x|O(1) such that for all x∈ {0,1}n, Prr(M(x, r)6= [x∈A])<2−n. Therefore Prr(∃x∈ {0,1}n, M(x, r)6= [x∈A])<1, hence there exists r0suitable for everyx(i.e. ∀x, M(x, r0) = [x∈A]). It is enough to giver0as advice to the machineM.
Picture of complexity classes L,P,NP,BPP,RP,Σp2,PH,PSPACE,EXP,NEXP,P/poly, . . . Question: do exponential-time problems have polynomial-size circuits? I.e. “EXP⊂P/poly?”
Proposition – For allc > 0, if d > c then there exists A ∈DTIME(2nd) such thatA does not have circuits of sizenc.
Proof. Let us fixn and defineA=n. Letx0, x1, . . . be the words of {0,1}n in lexicographic order. We will successively decide whetherxi∈Afori= 0,1, . . .
LetV0be the set of all circuits of sizenc. We decide thatx0∈Aiff more than half of the circuits of V0rejectx0. LetV1be the set of circuits ofV0that are correct onx0, that is,C(x0) = [x0∈A]. Then we decide thatx1∈Aiff more than half of the circuits ofV1rejectx1. LetV2be the set of circuits ofV1that are correct onx1, that is,C(x1) = [x1∈A]. We go on like this: xi∈Aiff more than half of the circuits ofVi rejectxi, and Vi+1 is the set of circuits ofVi that are correct onxi, that is,C(xi) = [xi∈A].
Then |Vi+1| ≤ |Vi|/2. Furthermore, by the lemma, the number of circuits of size nc is at most (3nc)2nc, hence fori >2nclog(3nc), we haveVi=∅: we then decide thatxi6∈A.
SupposeA=n is recognized by a circuitCof sizenc: leti0 be the (unique)isuch thatC∈Vi\Vi+1. ThenC(xi)6= [xi∈A], a contradiction.
Let us now show thatA∈DTIME(2nd). Each step requires to simulate at most (3nc)2nc circuits of size nc, which requires time ≤ nc(3nc)2nc, and memorize which ones give the minority answer. As a whole, the 2nclog(3nc) steps take timeO((3nc)2nc+3) =O(2nd) ford > c.
3 Unpredictability implies derandomization
Questions: “BPP=P?”, “BPP6=EXP?”
Definition(Unpredictability) – Letm:N→N. A functionf :{0,1}?→ {0,1}?ism(n)-unpredictable if
• for allx∈ {0,1}?,|f(x)|=m(|x|) and
• for alln, for every circuitC of size≤m(n)2and for every i∈[0, m(n)−1], we have:
Prz∈{0,1}n:r=f(z) C(r1, . . . , ri) =ri+1
<1/2 + 1/(10m(n)).
Remark: The identity map id :{0,1}n→ {0,1}n isn-unpredictable.
Theorem (Yao) – If f : {0,1}? → {0,1}? is m(n)-unpredictable and computable in time t(n), then BPTIME m(u(n))
⊆DTIME 2u(n)(t(u(n)) +m(u(n)))
for any time-constructible functionu(n).
Proof. For simplicity, define N = u(n). Let A be an algorithm for a language L ∈ BPTIME m(N) . The idea is to replace the random bits r∈ {0,1}m(N) ofAbyf(z), wherez runs through all strings of
lengthN, and to take the majority answer. This amounts to enumerate allz∈ {0,1}N and for each of them, to computef(z) and to simulateA(x, f(z)), which gives a deterministic algorithmD running in time 2N(t(N) +m(N)).
Let us now show that this algorithm is correct. Suppose by contradiction that D makes a mistake on somex0∈ {0,1}n, that is,
Prz∈{0,1}N(A(x0, f(z)) = [x6∈L])≥1/2, whereas, by definition ofA, we have:
Prr∈{0,1}m(N)(A(x0, r) = [x6∈L])<1/3.
We will construct a circuitCof size ≤m(N)2 such that there existsi∈[0, m(N)2−1] satisfying Prz∈{0,1}N:r=f(z)(C(r1, . . . , ri) =ri+1)>1/2 + 1/(10m(N)).
Let us first design a probabilistic algorithmB, depending on x0 and on [x0 ∈L], before turning it into a circuit. Let z∈ {0,1}N and r=f(z)∈ {0,1}m(N): on inputr1, . . . , ri, B chooses r0i+1, . . . , r0m(N) at random and outputs the bitai+1 defined by:
ai+1=r0i+1 ifA(x0, r1. . . riri+10 . . . rm(N)0 ) = [x06∈L]
ai+1= 1−ri+10 otherwise.
The intuition is that, since Amakes a mistake onx0 with bitsf(z), it should have more propensity to output the wrong answer ifri+10 =ri+1. The running time of B is O(m(N)). We want to show that Prz,r0(ai+1=ri+1)>1/2 + 1/(10m(N)). Letpi = Prz,r0:r=f(z)(A(x0, r1. . . rir0i+1. . . r0m(N)) = [x06∈L]):
thenp0<1/3 andpm(n)≥1/2. We have:
Prz,r0(ai+1=ri+1) = Prz,r0:r=f(z) A(x0, r1. . . rir0i+1. . . r0m(N)) = [x06∈L]∧ri+10 =ri+1
+ Prz,r0:r=f(z) A(x0, r1. . . rir0i+1. . . r0m(N)) = [x0∈L]∧ri+10 6=ri+1
= Pr A(x0, rr0) = [x06∈L]|r0i+1=ri+1
Pr(r0i+1=ri+1)+
1−Pr A(x0, rr0) = [x06∈L]|ri+10 6=ri+1
Pr(r0i+16=ri+1)
But Pr(r0i+1 =ri+1) = Pr(ri+10 6=ri+1) = 1/2 and we have Pr A(x0, rr0) = [x0 6∈ L]| r0i+1 =ri+1
= Pr A(x0, r1. . . ri+1ri+20 . . . rm(N)0 ) = [x06∈L]
=pi+1, hence:
= 1/2
1 +pi+1−Pr A(x0, rr0) = [x06∈L]|r0i+16=ri+1
. Therefore
Pr A(x0, rr0) = [x06∈L]|ri+10 6=ri+1
= 1 +pi+1−2Pr(ai+1=ri+1).
Furthermore,
pi = Pr A(x0, rr0) = [x06∈L]
= 1/2
Pr A(x0, rr0) = [x06∈L]|r0i+1=ri+1
+ Pr A(x0, rr0) = [x06∈L]|r0i+16=ri+1
= 1/2 pi+1+ 1 +pi+1−2Pr(ai+1=ri+1)
= 1/2 +pi+1−Pr(ai+1=ri+1) hence
Pr(ai+1=ri+1) = 1/2 +pi+1−pi. SincePm(N)−1
i=0 (pi+1−pi) =pm(N)−p0>1/6, there exists isuch that pi+1−pi >1/ 6m(N) , hence Prz,r0(ai+1=ri+1)>1/2 + 1/ 10m(N)
.
Therefore there exists a choice r00 of r0 such that Prz(ai+1 =ri+1)>1/2 + 1/(10m(N)). Now, our circuitCmerely simulatesB(x0, r00): its size ism(N)2(since the running time ofB ism(N)) and
Prz∈{0,1}N:r=f(z) C(r1, . . . , ri) =ri+1
>1/2 + 1/ 10m(N) ,
a contradition with the unpredictability off.
Corollary 1–
• If, for some >0, there exists a 2n-unpredictable functionf :{0,1}?→ {0,1}? inDTIME(2O(n)), thenBPP=P.
• If, for some >0, there exists a 2n-unpredictable functionf :{0,1}?→ {0,1}? inDTIME(2O(n)), thenBPP⊆ ∪kDTIME(2(logn)k).
• If there is α > 0 such that for allc > 0, there exists annc-unpredictable function fc :{0,1}? → {0,1}? inDTIME(2αn), thenBPP⊆ ∩DTIME(2n).
Proof.
• For eachk, consideru(n) =k/logn: thenm(u(n)) =nk. Letα >0 be such thatf ∈DTIME(2αn).
By the theorem,BPTIME(nk)⊆DTIME(nk/nαk/nk).
• For each k, consider u(n) = (klogn)1/: then m(u(n)) = nk. By the theorem, BPTIME(nk) ⊆ DTIME(2(logn)O(1)).
• Fixandk. Takeu(n) =n andc=k/. Then by the theorem BPTIME(nk)⊆DTIME(2O(n)).
4 Average case hardness implies unpredictability
Definition(Average-case and worst-case hardness) – Letf :{0,1}n → {0,1}andρ∈[0,1].
• Theρ-average-case hardness off, denotedHρavg(f), is the largestssuch that for every circuitC of size ≤s, Prx∈{0,1}n(C(x) =f(x))< ρ.
• The worst-case hardness of f isHwrs(f) =H1avg(f).
• The average-case hardness off isHavg(f) = max{s : H1/2+1/savg (f)≥s}, that is, the largestssuch that for every circuit Cof size ≤s, Prx∈{0,1}n(C(x) =f(x))<1/2 + 1/s.
• Iff :{0,1}? → {0,1}thenHρavg(f),Havg(f) andHwrs(f) are the functions depending onndefined accordingly.
Remark: Forf :{0,1}?→ {0,1}, Hwrs(f)≥Havg(f).
Exercise: Show that most functions f :{0,1}?→ {0,1} have exponential average-case hardness (that is,Havg(f) = 2Ω(n)).
Remark:
• NP6⊂P/poly iffHwrs(SAT) =nω(1).
• SinceBPP⊂P/poly, iff ∈BPPthenHavg(f) =nO(1).
Theorem(Nisan, Wigderson 1988) – Lets(n) be a function such that there existsd:N→Ncomputable in time 2n satisfyingd(n)≤n/48 and 22d(n)< s(p
nd(n)/12). Suppose there existsf :{0,1}?→ {0,1}
in DTIME(2αn) (for some constant α ≥ 2) such that Havg(f) ≥ s(n). Then there exists a 2d(n)/2- unpredictable functiong∈DTIME(2αn).
Proof idea: use the hard function f : {0,1}? → {0,1} on “almost disjoint” sets of inputs to get an unpredictable functiong:{0,1}?→ {0,1}?. For the proof we need some lemmas.
Definition (combinatorial design) – Let m and d be integers. A family (I1, . . . , IN) of subsets of {1, . . . , n} is an (m, d)-design if
• m≥2dandn= 12m2/d;
• for alli,|Ii|=m;
• |Ii∩Ij| ≤dfori6=j.
Remark: The parametersm, n, das above satisfyn≥48d(sincem≥d) andn≥24m(sinced≤m/2).
Lemma – Letm≥2dandn= 12m2/d. IfI1, . . . , Ikare arbitrary subsets of sizemof{1, . . . , n}, where k <2d/2, then there existsJ ⊂ {1, . . . , n} such that|J|=mand for alli,|J∩Ii| ≤d.
Proof. Build a random subset J of {1, . . . , n} as follows: pick every element x ∈ {1, . . . , n} with probability 2m/n < d/(6m). Then E(|J|) = 2m and by Chebyshev’s inequality: Pr(|J| ≥ m) ≥ Pr(||J| −E(|J|)| ≤m)≥1−Pr(||J| −E(|J|)|> m)>1−Var(|J|)/(m2) = 1−n(2m/n)(1−2m/n)/m2>
1−2/m >9/10 form≥20.
Furthermore, for every fixed i, Pr(|J ∩Ii| > d) < md
(d/(6m))d < (md/(d!))(d/m)d/6d ≤ 1/2d because d! > (d/3)d by Stirling, hence (dd/d!)/6d < 3d/6d = 1/2d. Therefore Pr(∃i,|J ∩Ii| > d) <
2d/2/2d < 1/10 (for d large enough) and Pr(∀i,|J ∩Ii| ≤ d) > 9/10. Altogether, Pr (∀i,|J ∩Ii| ≤ d)∧ |J| ≥m
>4/5>0, hence there existsJ such that∀i,|J∩Ii| ≤dand|J| ≥m. Removing arbitrary
elements fromJ to get|J|=mpreserves the property.
Lemma – For allm≥2dandN ≤2d/2, an (m, d)-design (I1, . . . , IN) exists and can be constructed in time<224m2/d.
Proof. A greedy algorithm will work. Choose the first setI1 arbitrarily, e.g. I1={1, . . . , m}. Then at each subsequent step, find by exhaustive search in{1, . . . ,12m2/d}a subset Ii+1 satisfying|Ii+1|=m and for allj≤i,|Ij∩Ii+1| ≤d. Such a set exists by the preceding lemma. The time needed to find one is 12md2/d
≤212m2/d. The whole algorithm runs in time 2d/2212m2/d <224m2/d. Definition (Nisan-Wigderson generator) – Let f : {0,1}m → {0,1} and (I1, . . . , IN) be an (m, d)- design. The function fNW : {0,1}12m2/d → {0,1}N is defined by fNW(x) = f(x|I1)f(x|I2). . . f(x|IN), where x|Ii is xi1xi2. . . xim if Ii = {i1 < i2 < . . . < im}. (The reference to the design is omitted for brevity) This definition extends to functionsf :{0,1}?→ {0,1}.
Proof (of the theorem). Let us fix n and define m= p
nd(n)/12, so that 12m2/d =n. By the first assumption, m ≥ 2d. Let (I1, . . . , I2d/2) be an (m, d)-design and let g = fNW. Then g : {0,1}n → {0,1}2d/2. Furthermore, g∈DTIME(2αn) since it is enough to build the design (time 224m2/d = 22n ≤ 2αn) and to computef on each subset (time 2d/2.2αm<2αn).
Suppose by contradiction thatgis not 2d/2-unpredictable, that is, there exist a circuitCof size≤2d andi∈[0,2d/2−1] such that
Prz∈{0,1}n:r=g(z)(C(r1, . . . , ri) =ri+1)≥1/2 + 1/(10.2d/2).
We will obtain a contradiction with the hardness off. Sincerj=f(z|Ij), we have:
Prz∈{0,1}n C(f(z|I1), . . . , f(z|Ii)) =f(z|Ii+1)
≥1/2 + 1/(10.2d/2).
Callz0 them variablesz|Ii+1,zj0 the ≤dvariables z|Ij∩Ii+1, z00 then−m variablesz|[1,n]\Ii+1 and z00j the≥m−dvariablesz|Ij\Ii+1. Then
Prz0∈{0,1}m,z00∈{0,1}n−m C(f(z10, z001), f(z20, z200), . . . , f(zi0, z00i)) =f(z0)
≥1/2 + 1/(10.2d/2).
Hence there exists a fixedz00 such that
Prz0∈{0,1}m C(f(z01, z001), f(z20, z002), . . . , f(z0i, zi00)) =f(z0)
≥1/2 + 1/(10.2d/2).
Now, since z00 is fixed, f(zj0, zj00) depends only on the ≤ d variables zj0 and has therefore a circuit of size d2d (the circuit merely computes the truth table). Combined withC, this gives a circuitD of size id2d+ 2d≤2d/2d2d+ 2d such that
Prz0∈{0,1}m(D(z0) =f(z0))≥1/2 + 1/(10.2d/2).
This is a contradiction with the hardness off as soon as 2d/2d2d+ 2d< s(m). Sincem=p
nd/12, it is enough that 22d(n)< s(p
nd/12), which is true by assumption.
Corollary 2–
• If there exists f ∈DTIME(2O(n)) such thatHavg(f)≥2n for some >0, thenBPP=P.
• If there existsf ∈DTIME(2O(n)) such thatHavg(f)≥2nfor some >0, thenBPP⊆ ∪kDTIME(2(logn)k).
• If there isα >0 such that for allc >0 there existsfc∈DTIME(2αn) such thatHavg(fc)≥nc, then BPP⊆ ∩>0DTIME(2n).
Proof.
• Wlog ≤ 1. For d(n) = (2/50)n, the function s(n) = 2n satisfies the assumptions of the theorem: s(p
nd/12) = s((/√
600)n) = 2(2/
√600)n > 2(2/25)n = 22d(n). Hence there exists a 2n/100-unpredictable functiong∈DTIME(2O(n)). By Corollary 1, this impliesBPP=P.
• Wlog < 2. For d(n) = 1/2(n/12)/2, the function s(n) = 2n satisfies the assumptions of the theorem: there is η ∈]0,1[ such that s(p
nd/12) =s(ηn1/2+/4) = 2ηn/2+
2/4
>22n/2 = 22d(n). Hence there exists a 2O(n/2)-unpredictable functiong∈DTIME(2O(n)). By Corollary 1, this implies BPP⊆ ∪kDTIME(2(logn)k).
• Ford(n) = (c/4) logn, the functions(n) =ncsatisfies the assumptions of the theorem: s(p
nd/12) = ((c/48)nlogn)c/2> nc/2= 22d(n). Hence there exists annc/8-unpredictable functiong∈DTIME(22αn).
By Corollary 1, this impliesBPP⊆ ∩DTIME(2n).
5 Worst case hardness implies average case hardness
Idea: consider f :{0,1}n → {0,1} as a word of {0,1}2n and suppose you have a good error-correcting codeE. Then, if one can computeE(f) correctly on a fraction of its inputs (say, 3/4), we can compute (recover)f exactly. Therefore, iff is hard in the worst case, then E(f) is hard in the average case.
5.1 Error-correcting codes
Let Σ be a finite set.
Definition(Fractionnal Hamming distance) – Forx, y∈Σm, the fractionnal Hamming distance ofx, y is ∆(x, y) =|{i : xi6=yi}|/m.
Definition(Error-correcting code) – Forδ∈[0,1], an error-correcting code with distanceδis a function E: Σn→Σm such thatx6=y⇒∆(E(x), E(y))≥δ.
This definition generalizes to functionsE : Σ?→Σ?satisfying|x|=|y| ⇒ |E(x)|=|E(y)|.
Remark: Over {0,1}, polynomial-sized codes exist for every distance δ < 1/2 but not for δ = 1/2, exponential-sized codes exist for δ = 1/2, and no code exists for δ > 1/2 (this is not true over Σ is
|Σ|>2).
Remark: IfE : Σn→Σm is a code with distanceδand x∈Σn, one can recover xgiven anyz∈Σm satifsying ∆(z, E(x))< δ/2: this is the nearesty∈Im(E).
Forx, y∈ {0,1}n, the inner product ofx, yisxy=Pn
i=1xiyimod 2.
Definition (Walsh-Hadamard code) – If x ∈ {0,1}n, let WH(x) = z0z1. . . z2n−1, where zj = xj (here, j is seen as its binary encoding over {0,1}n). We see WH as a function from {0,1}? to {0,1}? where|WH(x)|= 2|x|.
Lemma – The functionWHis a code with distance 1/2.
Proof. Let x, y ∈ {0,1}n be distinct on bit i. For all u ∈ {0,1}i−1 and v ∈ {0,1}n−i−1, either xu0v6=yu0vor xu1v6=yu1v, thereforexj6=yj for one half of thej.
For the definition of the Reed-Muller code, we need a lemma. By “individual degrees”, we mean the degrees for each variable.
Lemma – Supposek divides dand d/k ≤ |Σ|. For any set S ⊆ Σ of size 1 +d/k, the application φ defined by φ(p) =f :x∈Sk 7→p(x) is a bijection between the set of k-variate polynomialsp: Σk →Σ of individual degrees≤d/k (and thus of total degree≤d) and the set of functions f :Sk →Σ.
Proof. Let us show by induction on k that φ is indeed a bijection. It is enough to show that each f determines uniquely a polynomialp.
For k= 1, by interpolation d+ 1 values characterize our polynomialp. For k >1: p(x1, . . . , xk) = Pd/k
i=0qi(x1, . . . , xk−1)xik. The case k = 1 shows that for each s∈ Sk−1, qi(s) is uniquely determined.
Now, by induction, eachqi is uniquely determined.
Definition(Reed-Muller code) – Let Σ be a finite field andk≤dbe integers such thatd <|Σ| andk dividesd. Ak-variate polynomial
p(x1, . . . , xk) = X
d1≤d/k,...,dk≤d/k
cd1,...,dkxd11. . . xdkk
of individual degrees≤d/k(and thus of total degree≤d) over Σ is characterized by the list of its values on all inputs from{σ0, . . . , σd/k}k, whereσi ∈Σ are arbitrary distinct elements.
The function RM: Σ(1+d/k)k →Σ|Σ|k is defined on such a list characterizing a polynomialp, by the list of values thatptakes over Σk, that is,RM(p) = (p(a1, . . . , ak))a1,...,ak∈Σ.
Remark: This is not the usual definition of the Reed-Muller code: usually, the input is the list of coefficients ofp. The present version prevents us from doing interpolation, which would not be efficient enough for decoding.
From now on, Σ is a finite field.
Lemma – The functionRMis a code with distance 1−d/|Σ|.
Proof. Let p6=q be two k-variate polynomials of degree ≤dover Σ: then p−q is a nonzerok-variate polynomial of degree≤d.
Claim (Schwartz-Zippel lemma) - If r is a nonzero k-variate polynomial of degree dover Σ, then Pra∈Σk(r(a)6= 0)≥1−d/|Σ|
The proof is by induction on k. For k = 1, there are at most droots and the claim follows. For k > 1: considerr(x1, . . . , xk) =P
iri(x2, . . . , xk)xi where ri are (k−1)-variate polynomials of degree
≤d−i. Sincer is nonzero, there is i such that ri is nonzero: take i0 to be the maximal such i. By induction hypothesis, Pra2,...,ak(ri0(a2, . . . , ak)6= 0) ≥1−(d−i0)/|Σ|, and ifri0(a2, . . . , ak) 6= 0 then r(x1, a2, . . . , ak) is a nonzero univariate polynomial of degree i0. Therefore Pra1,...,ak(r(a1, . . . , ak) 6=
0)≥(1−(d−i0)/|Σ|)(1−i0/|Σ|)≥1−d/|Σ|. The claim is proven.
Thereforep6=qon a fraction≥1−d/|Σ|of the inputs Σk, which proves the lemma.
Definition – Suppose that |Σ|is a power of 2 (this will be the case in our applications).
• A code E : Σn →Σm can be seen as a function E :{0,1}nlog|Σ| →Σm by encoding each input symbol from Σ using log|Σ| symbols from{0,1}. Similarly, a codeF : {0,1}log|Σ|→ {0,1}tcan be seen as a functionF : Σ→ {0,1}t.
• If E : Σn → Σm and F : {0,1}log|Σ| → {0,1}t are two codes, their concatenation is the code F◦E:{0,1}nlog|Σ|→ {0,1}mt defined by (F◦E)(x) =F(z1). . . F(zm), wherez=E(x)∈Σm.
Lemma – IfE: Σn →Σm has distanceδE andF :{0,1}log|Σ|→ {0,1}thas distanceδF, thenF◦E has distanceδEδF.
Proof. Let x, y∈Σn be distinct. Then E(x) andE(y) differ on at least δEm positions. Ifi is such a position (i.e. E(x)i6=E(y)i), thenF(E(x)i) andF(E(y)i) differ on at least δFt positions. As a whole, (F◦E)(x) and (F◦E)(y) differ on at leastδEmδFt positions.
Lemma – The concatenationWH◦RM:{0,1}(1+d/k)klog|Σ|→ {0,1}|Σ|k+1of the Reed-Muller code and the Walsh-Hadamard code is a function computable in time polynomial in|Σ|k.
Proof. Using Lagrange interpolation, the coefficients of a polynomialpcan be recovered from the list of its values onSk in time polynomial in|Σ|k. Then it is easy to evaluatepon every point from Σk in time
polynomial in|Σ|k.
Suppose, in order to build an average-case hard function from a worst-case hard one, that we plan to apply the idea mentioned at the beginning of this section. Since no binary code has distance<1/2, it seems that we cannot recover more than 1/4 of the errors, which is not enough. This issue is solved by
“list decoding”. The following lemma shows that there is only a small number of candidates. Note that this lemma is given as an illustration, but it is not useful in the sequel.
Lemma (Johnson bound, 1962) – If E : {0,1}n → {0,1}m is an error-correcting code with distance 1/2−, then for everyz∈ {0,1}m, there are at mostk= 1/(2) vectorsz1, . . . , zk ∈E({0,1}n)⊆ {0,1}m such that ∆(z, zi)≤1/2−√
(for everyi).
Proof. Definek vectorsai∈Rm byaij = 1 ifzij=zj, andaij =−1 otherwise. Letw=P
iai. Then for alli,P
jaij ≥2m√
because ∆(z, zi)≤1/2−√
(they coincide on at least (1/2 +√ )m positions), therefore P
iwi =P
i
P
jaij ≥ 2km√
. Hence, by the following claim, hw, wi =P
iwi2 ≥ (2km√
)2/m≥4k2m.
Claim- IfPm
i=1xi≥α≥0, thenPm
i=1x2i ≥α2/m.
The proof is by induction on m. Obvious for m = 1. For m > 1: Pm−1
i=1 xi ≥ α−xm and Pm
i=1x2i =x2m+Pm−1
i=1 x2i ≥x2m+ (α−xm)2/(m−1) by induction. But x2m+ (α−xm)2/(m−1) = (mx2m+α2−2αxm)/(m−1)≥α2/mbecause m2x2m+α2−2mαxm= (α−mxm)2 ≥0. This proves the claim.
Furthermore, since E is a code with distance 1/2−, fori0 6=iwe have ∆(zi0, zi)≥1/2−(they coincide on at most (1/2 +)m positions), i.e. Pm
j=1aijaij0 ≤ 2m (zji = zji0 iff aijaij0 = 1). Hence hw, wi=Pk
i=1hai, aii+P
i6=jhai, aji ≤km+ 2k2m.
Therefore 4k2m≤km+ 2k2m, that is,k≤1/(2).
As a preparation to the local list decoding of the Reed-Muller code, we need some lemmas. The first lemma shows that the list in the Johnson bound can be efficiently recovered in the case of the Reed-Muller code wherek= 1:
Lemma (Sudan, 1996) – There is a polynomial-time algorithm which, given an integerd < m/4 and m pairs (a1, b1), . . . ,(am, bm) of elements of Σ, returns the list of all degree d polynomials g : Σ→ Σ such that the number of indices i for whichg(ai) =bi is >2√
dm. This list contains at most p m/d polynomials.
Proof. SupposeQ(x, y) is a nonzero bivariate polynomial over Σ such thatQ(ai, bi) = 0 for everyi, and degx(Q)≤√
mdand degy(Q)≤p
m/d. Ifg is in the list, then:
• Q(x, g(x)) = 0 because the univariate polynomialQ(x, g(x)) is of degree≤√
md+dp
m/d= 2√ md and vanishes on>2√
mdpoints.
• g(x)−y is a factor ofQ(x, y): considerQ0(y) =Q(x, y)∈(Σ[x])[y], then g(x) is a root ofQ0(y), thereforey−g(x) dividesQ0(y), that is, there existsR0∈(Σ[x])[y] such thatQ0(y) = (y−g(x))R0(y):
in other words, there existsR∈Σ[x, y] such thatQ(x, y) = (y−g(x))R(x, y). The number of such factors is≤degy(Q)≤p
m/d.
Therefore it is enough to find such a polynomial Q(x, y) and to factorize it using, e.g., Berlekamp’s algorithm. In order to findQ: write down themlinear equations that the coefficients must fulfill. There are (1 +√
md)(1 +p
m/d)> mcoefficients, therefore the system has a nonzero solution which can be
found by Gaussian elimination.
The next two lemmas show that from a corrupted version of a polynomial, one can efficiently recover the initial polynomial. The first concerns univariate polynomials, whereas the second is about multivari- ate polynomials. Note that the polylog-time bound (in the size of the input, which is exponential inn) is required since we will apply this algorithm to an input of exponential size.
Lemma (Berlekamp and Welch, 1986) – There is a polynomial-time algorithm that, given an integer d <|Σ|and a list (a1, b1), . . . ,(am, bm) ofm > dpairs of elements from Σ such that there exists a degree d polynomial p: Σ → Σ satisfying p(ai) =bi for at least t > m/2 +d/2 numbers 1≤i ≤m, returns the list of coefficients of p. (Remark that pis unique because it is of degree≤dand specified ont > d points.)
Proof. The algorithm proceeds as follows.
• Find a degreem−t+dpolynomialA(x) and a degreem−tnonzero polynomialB(x) such that A(ai) =biB(ai) for all 1≤i≤m. Such polynomials are guaranteed to exist since forB one can take a degree≤m−tnonzero polynomial vanishing on the≤m−tpointsai for whichp(ai)6=bi, and thenA=pBis of degree≤m−t+d.
To findAandB, it is enough to solve a linear system with 2m−2t+d+2 unknowns (the coefficients of Aand B) andm equations (one for eachi): this can be done in polynomial time by Gaussian elimination.
• CallQthe quotient of the Euclidean division ofAbyB, and outputQ. SinceA(x)−p(x)B(x) is of degree≤m−t+d < m/2 +d/2< tand is zero on≥t points, we haveA(x) =p(x)B(x), hence Q(x) =p(x).
Definition – Ify is a string, an algorithmA is said to have random access toy (denoted byAy) if it has an oracle which, on inputi, answers the bityi.
Lemma (Reed-Muller local decoder) – Letd <(|Σ| −1)/3. There exists a probabilistic algorithm D running in time poly(|Σ|, k, d) such that, iff : Σk →Σ agrees with a degreedpolynomialP on a 17/18 fraction of the inputs, then Pr(Df(x) =P(x))≥5/6.
Proof. The algorithmD works as follows on inputx:
• Choosez∈Σkrandomly and consider the lineLz={x+tz : t∈Σ}. IfX ⊂Σkis the set of inputs on whichf andP differ, then letN =|Lz\ {x} ∩X|: we haveE(N) =P
t∈Σ\{0}Prz(x+tz∈X)≤ (|Σ|−1)/18 therefore, by Markov inequality, Prz(N >(|Σ|−1)/3)<(|Σ|−1)/18/((|Σ|−1)/3) = 1/6.
Hence, with probability>5/6, the number of points inLz\ {x}on whichf andP coincide is at least 2(|Σ| −1)/3>(|Σ| −1)/2 +d/2.
• Queryf on all the|Σ| −1 points ofLz\ {x}to obtain the set of points{(t, f(x+tz)) : t∈Σ\ {0}}.
• If Q(t) =P(x+tz), then Qis a univariate polynomial of degree ≤d. Run the Berlekamp-Welch procedure on the points {(t, f(x+tz)) : t ∈ Σ\ {0}}to recover a polynomial Q0 (supposed to equal Q).
• OutputQ0(0).
With probability>5/6,Lz\ {x}contains more than (|Σ| −1)/2 +d/2 points on whichf andP coincide, hence the Berlekamp-Welch procedure outputsQ, therefore the algorithm outputsQ(0) =P(x).
The following two lemmas concern random curves of degree 3. In the sequel, when we speak of polynomialsq: Σ→Σk (curves parametrized byt∈Σ), we use the producttu= (tu1, . . . , tuk) ift∈Σ andu∈Σk,u= (u1, . . . , uk).
Lemma – Let S ⊆Σk be a fixed subset and let x ∈Σk be fixed. Let q(t) =x+tu+t2v+t3w be a random degree 3 polynomial wheret ∈Σ and u, v, w ∈Σk, defining a curve Q=q(Σ). Then for all t∈Σ\ {0}, Pr(q(t)∈S) =|S|/|Σ|k. Furthemore, for all t6=t0 ∈Σ, the events q(t)∈S and q(t0)∈S are independent.
Proof. First suppose thatt6= 0 andt0 6= 0. Let us first show that, ify, y0∈Σk are fixed, then the events q(t) =y and q(t0) =y0 are independent. We haveq(t) =y ⇐⇒ u= (y−x−t2v−t3w)/t, therefore the choice ofvandwis free but thenuis fixed, hence Pr(q(t) =y) = 1/|Σ|k and similarly forq(t0) =y0. Furthermore, (q(t) =y∧q(t0) =y0) ⇐⇒ (tu+t2v+t3w=y−x∧t0u+t02v+t03w=y0−x) ⇐⇒ (u= (x−t2v+t3w)/t∧(t0t2−tt02)v+ (t0t3−tt03)w=t0(y−x)−t(y0−x)): wcan be chosen freely but then uandvare fixed, therefore this happens with probability 1/|Σ|2k= Pr(q(t) =x)Pr(q(t0) =y).
Now, Pr(q(t)∈S) =P
y∈SPr(q(t) =y) because these events are disjoint, =|S|/|Σ|k, similarly for Pr(q(t0)∈S). On the other hand, Pr(q(t)∈S∧q(t0)∈S) =P
y,y0∈SPr(q(t) =y∧q(t0) =y0) (disjoint events) =|S|2/|Σ|2k.
Now, if one of t or t0 is zero, say t = 0, then either x ∈ S and Pr(q(t) ∈ S) = 1, or x 6∈ S and Pr(q(t)∈S) = 0: in both cases, the events q(t)∈S andq(t0)∈S are independent.
Lemma – LetS⊆Σk be a fixed set such that Pry(y∈S)≥, where <1/2, and letx∈Σk be fixed.
Letq(t) =x+tu+t2v+t3wbe a random degree 3 polynomial wheret∈Σ andu, v, w∈Σk, defining a curveQ=q(Σ). Then Pr(|Q∩S| ≥2|Σ|/15)≥1−100/(|Σ|).
Proof. Since q is a degree 3 polynomial, every y ∈ Σk cannot have more than 3 preimages. Hence
|Q∩S| ≥(P
t∈Σ\{0}xt)/3 where xt = 1 if q(t) ∈ S and xt = 0 otherwise. By the preceding lemma, this is a sum of pairwise independent random variables, where Pr(xt= 1)≥. We have: E(|Q∩S|)≥ (|Σ| −1)/3, Var(|Q∩S|)≥(|Σ| −1)(1−)/9 and by Chebyshev’s inequality: Pr(|Q∩S| ≤2|Σ|/15)≤ Pr(||Q∩S| −E(|Q∩S|)| ≥(|Σ| −1)/3−2|Σ|/15). But since|Σ| ≥2, (|Σ| −1)/3−2|Σ|/15≥ |Σ|/30, hence: Pr≤Pr(||Q∩S| −E(|Q∩S|)| ≥|Σ|/30)≤(|Σ| −1)(1−)/(9(|Σ|/30)2)≤100/(|Σ|).
For our purpose, we need extremely efficient decoding algorithms.
Definition (local list decoder) – Let E : Σn → Γm be a code and let 0 < η < 1. A probabilistic algorithm D is called a local list decoder for E handling η errors in time t with advice of size sif, for every x∈Σn andz ∈ Γm satisfying ∆(E(x), z)≤η, there existsa ∈ {0,1}?, |a| ≤ s(a binary advice of sizes) such that for every i≤n, D on input (a, i) and with random access to z, runs in time t and outputsxi with probability ≥2/3.
Remark:
• In the sequel, we will consider local list decoders working in time polylogarithmic inmwith advice of size logarithmic inn.
• IfF is the set of functionsf : Σk→Σ, then the Reed-Muller code can be considered as taking its values intoF.
• In the following proposition, we want to local-list decode Reed-Muller code: the integer a(advice) must depend only on f and must therefore be the same for allx∈Σk.
Proposition(local list decoding of the Reed-Muller code) – If|Σ| ≥d2 then the Reed-Muller code has a local list decoder handling 1−30p
d/|Σ|errors in time poly(|Σ|, k) with advice of size (k+ 1) log|Σ|.
More precisely, there is a probabilistic algorithmDrunning in time poly(|Σ|, k) such that, iff : Σk → Σ agrees with a degreedpolynomialpon a fraction 30p
d/|Σ|of the inputs, then there is (a, b)∈Σk×Σ such that for allx∈Σk, Pr(Df(x, a, b) =p(x))≥2/3.
Proof. Let= (2p
d/|Σ|)1/2. Let us first propose a probabilistic algorithm, working on ≥1− of the x∈Σk, which has a probability of success≥1−.
1. Choose at random r ∈ Σ (r 6= 0) and v, w ∈ Σk, and define u = (a−x−vr2 −wr3)/r and q(t) =x+tu+t2v+t3w(fort∈Σ), so that the degree 3 curveQ=q(Σ) satisfiesq(0) =xand q(r) =a.
2. Query f on all the points ofQto obtain the pairs (t, f(q(t))) for allt∈Σ.
3. Run Sudan’s algorithm to obtain a list g1, . . . , gp of all degree 3dpolynomials such that gi(t) = f(q(t)) for at least 4p
d|Σ|pointst∈Σ. (Note that Sudan’s algorithm returns polynomials which coincides with at least 2p
3d|Σ|<4p
d|Σ|points, but we can ignore those who agree on<4p d|Σ|
points)
4. If there is a unique isuch thatgi(r) =bthen outputgi(0). Otherwise halt (no output).
In order to prove that a correct pair (a, b) exists, we will use a probabilistic argument. Fixx∈Σk, take a∈Σk at random andb=p(a). The random choices are thena, r, v andw. Then the first step of the algorithm amounts to choose a random curveQsuch thatq(0) =x, and the last step amounts to choose a random r ∈ Σ and check if gi(r) = p(q(r)). We claim that the probability that gi = p◦q is high.
Indeed, thanks to the lemma, ifS ={y : f(y) =p(y)}, then by assumption Pry(y∈S)≥30p d/|Σ|, therefore by the lemma, Pra,r,v,w(|Q∩S| ≥4p
d/|Σ|)≥1−4/p
d|Σ|. Therefore, p◦q lies in the list g1, . . . , gp with probability ≥1−4/p
d|Σ|. Furthermore, the probability that no other polynomial gi satisfies gi(r) = p(q(r)) is high: gi and p◦q, as distinct polynomials of degree ≤ 3d, only agree on
≤ 3d points, therefore Pra,r,v,w(gi(r) = p(q(r))) ≤ 3d/|Σ|. As a whole, since there are ≤ p
|Σ|/(3d) polynomials gi, we have Pra,r,v,w(iunique st gi(r) = p(q(r))) ≥ 1−p
|Σ|/(3d)3d/|Σ| = 1−p 3d/|Σ|.
Thus with probability≥1−4/p
d|Σ| −p
3d/|Σ| ≥1−2p
d/|Σ|= 1−2 (for sufficiently large d) we identify uniquelyp◦qand outputp◦q(0) =p(x). Hence Pra,r,v,w(Df(x, a, b) =p(x))≥1−2.
LetAbe the random event (depending ona) Prr,v,w(Df(x, a, b) =p(x))≥1−. Then Pra,r,v,w(Df(x, a, b) = p(x)) = Pra,r,v,w(Df(x, a, b) = p(x)|a ∈ A)Pr(A) + Pra,r,v,w(Df(x, a, b) = p(x)|a 6∈ A)(1−Pr(A)) ≤ Pr(A) + (1−)(1−Pr(A)) = 1−+Pr(A). Hence 1−+Pr(A)≥1−2, that is, Pr(A)≥1−. Hence, a randomais valid forxwith probability 1−.
Since the argument is valid for an arbitraryx∈Σk, we have: PrxPra(ais valid forx) = PraPrx(ais valid forx)≥ 1−. Therefore there exists a specifica∈Σk which is valid for a fraction≥1−of the inputs x∈Σk.
It remains to get the result on all inputsx∈Σk: it is enough to apply the Reed-Muller local decoder in order to recover the polynomialpwhich was computed correctly on≥1− >17/18 of the inputs. As
a whole, the probability of success is≥1−(+ 1/6)≥2/3.
The next proposition is given without proof.
Proposition(local list decoding of the Walsh-Hadamard code) – For every >0, the Walsh-Hadamard codeWH:{0,1}n→ {0,1}2n has a local list decoder handling (1/2−) errors in time poly(n,1/) with advice of sizeO(log(n/)).
