HAL Id: hal-02382117
https://hal.archives-ouvertes.fr/hal-02382117
Preprint submitted on 27 Nov 2019
HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
heuristic algorithms over finite fields
Joris van der Hoeven, Grégoire Lecerf
To cite this version:
Joris van der Hoeven, Grégoire Lecerf. Sparse polynomial interpolation. Exploring fast heuristic algorithms over finite fields. 2019. �hal-02382117�
Exploring fast heuristic algorithms over finite fields
JORIS VAN DERHOEVENabc, GRÉGOIRELECERFbd
a. CNRS (UMI 3069, PIMS) Department of Mathematics
Simon Fraser University 8888 University Drive Burnaby, British Columbia
V5A 1S6, Canada
b. CNRS, École polytechnique, Institut Polytechnique de Paris Laboratoire d'informatique de l'École polytechnique (LIX, UMR 7161)
1, rue Honoré d'Estienne d'Orves Bâtiment Alan Turing, CS35003
91120 Palaiseau, France
c. Email:vdhoeven@lix.polytechnique.fr d. Email:lecerf@lix.polytechnique.fr Work in progress, draft version of November 27, 2019
Consider a multivariate polynomial f∈K[x1, . . . ,xn]over a fieldK, which is given through a black box capable of evaluating fat points inKn, or possibly at points inAn for anyK-algebraA. The problem of sparse interpolation is to express fin its usual form with respect to the monomial basis. We analyze the complexity of various old and new algorithms for this task in terms of boundsDandTfor the total degree of f and its number of terms. We mainly focus on the case whenKis a finite field and explore possible speed-ups under suitable heuristic assumptions.
1. I
NTRODUCTIONConsider a polynomial functionf:Kn→Kover a fieldKgiven through a black box capable of evaluating f at points in Kn. The problem of sparse interpolation is to recover the representation of f∈K[x1, . . . ,xn]in its usual form, as a linear combination
f=
i1, . . . ,in
fi1, . . . ,inx1i1⋅ ⋅ ⋅xnin (1) of monomials. The aim of this paper is to analyze various approaches for solving this problem, with our primary focus on the case whenKis a finite field. We will survey and synthesize known algorithms, but we will also present a few new algorithms, together with improved complexity bounds for some important special cases.
We explore various methods under heuristic conditions that we expect to fairly reflect average behavior in practice. We preferred a relaxed and intuitive style of exposition to mathematically precise theorems with rigorous proofs.
1
Efficient algorithms for the task of sparse interpolation go back as far as to the eighteen's century and the work of Prony [41]. The first modern version of the algorithm is due to Ben Or and Tiwari [8]. This method was swiftly embraced in computer algebra [11,28, 32,34,36,37,39]; for early implementations, we refer to [13,14]. There has been a regain of interest for the problem during the last decade, both from a theoretical perspective [2, 3,4,15,16, 29,30,33] and from the practical point of view [25,26,27,31,35]. We also mention the survey paper [43] by Roche on the more general topic of computations with sparse polynomials.
1.1. Complexity considerations
Throughout this paperdwill stand for the total degree of f andtfor the number of non- zero terms in (1). Whenever available, the uppercase charactersDandTrepresent upper bounds fordandt. We will also writeLfor the number of ring or field operations inK that are required in order to evaluate f.
The complexity analysis of sparse interpolation has to be carried out with a lot of care, due to the large variety of cases that can occur:
• What kind of complexity/evaluation model do we use?
∘ Do we count the number operations inKor the number of bit operations?
∘ Are we interested in theoretic (asymptotic) or practical complexity?
∘ Are divisions allowed for the evaluation of f and how do we count them?
∘ Are we only allowed to evaluate f at points inKnor also at points inKˆnfor certain extension rings or fieldsKˆ?
• What kind of coefficient fieldKdo we use?
∘ A field from analysis such asℂ.
∘ A discrete field such asℚor a finite field𝔽q.
∘ Fields with roots of unity𝜔of large smooth order inK.
• The univariate case (n= 1)versusthe multivariate case (n> 1).
• Informally speaking, there are three levels of “sparsity”:
∘ Weakly sparse: total degreesdof the orderO(logt).
∘ Normally sparse: total degreesdof the ordertO(1).
∘ Super sparse: total degrees of orderdwithlogt=o(logd).
We also notice that almost all general algorithms for sparse interpolation are proba- bilistic of Monte Carlo type. Indeed, without furthera priori knowledge about f, such as its support or its number of terms, the mere knowledge of a finite number of eval- uations of f only allow us to guess plausible expressions for f.
In this paper, we will be mostly interested in the practical bit complexity of sparse interpolation over finite fieldsK= 𝔽q. Sparse interpolation over the rational numbers can often be reduced to this case as well, in which caseq=pis a well chosen prime number that fits into 32 or 64 bits and such thatp−1admits a large smooth divisor; see section6.5.
We analyze the complexities of specializations of existing algorithms to the finite field case and also present a few new algorithms and tricks for this specific setting. Due to the large number of cases that can occur, we will not prove detailed complexity bounds for every single case, but rather outline how various ideas may be used and combined to reduce the practical complexity.
From our practical perspective, it is important to take into account logarithmic factors in complexity bounds, but it will be convenient to ignore sublogarithmic factors. For this reason, we use thead hocnotation
𝜑 =O♭(𝜓) ⇔⇔⇔⇔⇔⇔⇔⇔⇔⇔⇔⇔def⇔⇔⇔⇔⇔⇔⇔⇔ 𝜑 =O(𝜓 (log 𝜓)o(1)(log log (TDq))O(1)) for any functions𝜑, 𝜓.
We will also writeMK(d)for the bit cost of multiplying two polynomials of degreed overKand abbreviateMq(d) ≔M𝔽q(d). For instance, the naive multiplication algorithm yieldsMq(d) =O♭(d2log2q). For our complexity analyses we will give priority to the asymptotic complexity point of view and use the well known [20, 44] boundMq(d) = O♭(dlogdlogq).
1.2. Overview of the paper
Many of the challenges concerning sparse interpolation already arise in the univariate case when n= 1. As we will see in section 7.1, the multivariate case can actually be reduced to the univariate one using the technique called “Kronecker segmentation”, even though other approaches may be more efficient. For this reason, a large part of the paper is devoted to methods for interpolating a univariate black box function f(x).
We distinguish three major types of algorithms:
• Cyclic extension methods (section4).
• Geometric progression methods (section5).
• FFT based methods (section6).
For the first two types of methods we mostly review existing algorithms, although we do propose some new variants and optimizations. The third FFT based method is new, as far as we are aware. For each of the three methods, an importantleitmotif is to evaluate f(x)moduloxr−1for one or more suitable ordersr, after which we reconstruct f(x)from these modular projections.
Cyclic extension methods directly evaluate f over the cyclic extension ring K[x]/(xr−1). This has the advantage that r can be freely chosen in a suitable range.
However, the evaluation of f over such a large cyclic extension induces a non-trivial overhead in the dependency of the complexity onL.
Geometric progression methods rather evaluate f at a sequence 1, 𝜔, . . . , 𝜔2T−1 of pairwise distinct elements inK(or inside an extension ofKof modest degrees). IfK= 𝔽q is a finite field, then𝜔necessarily has an orderrthat dividesq−1(orqs−1when working over an extension of degree s). Although the evaluations of f become more efficient using this approach, the recovery of f(x) modulo xr−1 from f(1), f(𝜔), . . . , f(𝜔2T−1) requires extra work. The cost of this extra work highly depends on the kind of ordersr that can be taken as divisors ofq−1 (orqs−1for small s). Theoretically speaking, the existence of suitable ordersris a delicate issue; in practice, they always tend to exist as long asD=TO(1); see sections2,6.3and6.4for some empirical evidence.
Geometric progression methods allow us to take r much larger than T, but they involve a non-trivial cost for recovering f(x)moduloxr−1from f(1),f(𝜔),...,f(𝜔2T−1).
IfL=o((logT)3), then this cost may even dominate the cost of the evaluations of f. In such situations, an alternative approach is to evaluate f at1, 𝜔, . . . , 𝜔r−1and to recover f(x)moduloxr−1using one inverse DFT of lengthr. However, this puts an even larger strain on the choice ofr, since it is important that T⩽r=O(T)for this approach to be
Method Complexity bound Conditions
Cyclic extension O♭(LTlogDlog(qT)) H1andlogD=O(T) Geometric progression O♭ (L+ (logT)3)T loglogDT 3log(qT) H1,H2, andD=TO(1) FFT O♭ (L+ logT)T loglogDT 2log(qT) H1,H2, andD=TO(1)
Table 1. Heuristic complexity bounds for the three main approaches to sparse interpolation.
efficient. Moreover, the recovery of f(x)from its reductions modulo xr−1 for several ordersr of this type is more delicate and based on a probabilistic analysis. Yet, suit- able ordersragain tend to exist in practice as long asD=TO(1).
The expected complexities of the best versions of the three approaches are summa- rized in Table1. These bounds rely on two types of heuristics:
H1. Forr≪d, the exponents of f are essentially randomly distributed modulor.
H2. Fors=O(logD/logT), the numberqs−1admits a large smooth divisor.
We will present a more precise version ofH1in section4. The heuristicH2will be made precise in sections5.6and6.2and numeric evidence is provided in sections2,6.3and6.4.
The last section7is dedicated to the interpolation of multivariate polynomials. We start with the well known strategies based on Kronecker segmentation (section7.1) and prime factorization (section7.2). For sparse polynomials in many variables, but with a modest total degreed, we also recall the inductive approach by packets of coordinates in section7.3. If T<q, then geometric progression and FFT based methods should be particularly favorable in combination with this inductive approach, since one can often avoid working over extensions ofKin this case. We conclude section7with a few algo- rithms for special situations.
2. P
RELIMINARIES ON FINITE FIELDSOne remarkable feature of the finite field𝔽qwithqelements is that everya∈ 𝔽qsatisfies the equationaq=a. In particular, for any sparse polynomial f as in (1) and with coeffi- cients in𝔽q, the polynomial f takes the same values as
i1, . . . ,in
fi1, . . . ,inx1i1rem(q−1)⋅ ⋅ ⋅xninrem(q−1),
forx1,...,xn∈ 𝔽q, where “rem” stands for the remainder of a Euclidean division. In other words, the exponents of f are only determined moduloq−1, so we may assume without loss of generality that they all lie in{0, . . . ,q−2}and that the total degreedof f satisfies d⩽n(q−2).
On the other hand, in the case that our black box function f can be evaluated not only over𝔽q, but also over field extensions𝔽qs(this is typically the case if f is given by an expression or a directed acyclic graph (dag)), then the exponents in the expression (1) can be general non-negative integers, but the above remark shows that we will crucially need to evaluate over extensions fields𝔽qswiths> 1in order to recover exponents that exceedq.
More generally, if we choose to evaluate f only at points(a1, . . . ,an) ∈ 𝔽qssuch that a1,...,anarer-th roots of unity, then we can only hope to determine the exponents modulor in the expansion (1). In that case,rmust divide the orderqs−1of the multiplicative group
s 2s−1 3s−1 5s−1
1 2 22
2 3 23 23⋅ 3
3 7 2 ⋅ 13 22⋅ 31
4 3 ⋅ 5 24⋅ 5 24⋅ 3 ⋅ 13
6 32⋅ 7 23⋅ 7 ⋅ 13 23⋅ 32⋅ 7 ⋅ 31
8 3 ⋅ 5 ⋅ 17 25⋅ 5 ⋅ 41 25⋅ 3 ⋅ 13 ⋅ 313
12 32⋅ 5 ⋅ 7 ⋅ 13 24⋅ 5 ⋅ 7 ⋅ 13 ⋅ 73 24⋅ 32⋅ 7 ⋅ 13 ⋅ 31 ⋅ 601
16 3 ⋅ 5 ⋅ 17 ⋅ 257 26⋅ 5 ⋅ 17 ⋅ 41 ⋅ 193 26⋅ 3 ⋅ 13 ⋅ 17 ⋅ 313 ⋅ 11489
24 32⋅ 5 ⋅ 7 ⋅ 13 ⋅ 17 ⋅ 241 25⋅ 5 ⋅ 7 ⋅ 13 ⋅ 41 ⋅ 73 ⋅ 6481 25⋅ 32⋅ 7 ⋅ 13 ⋅ 31 ⋅ 313 ⋅ 601 ⋅ 390001 30 32⋅ 7 ⋅ 11 ⋅ 31 ⋅ 151 ⋅ 331 23⋅ 7 ⋅ 112⋅ 13 ⋅ 31 ⋅ 61 ⋅ 271 ⋅ 4561 23⋅ 32⋅ 7 ⋅ 11 ⋅ 31 ⋅ 61 ⋅ 71 ⋅ 181 ⋅ 521 ⋅ ⋅ ⋅ 36 33⋅ 5 ⋅ 7 ⋅ 13 ⋅ 19 ⋅ 37 ⋅ 73 ⋅ 109 24⋅ 5 ⋅ 7 ⋅ 13 ⋅ 19 ⋅ 37 ⋅ 73 ⋅ 757 ⋅ 530713 24⋅ 33⋅ 7 ⋅ 13 ⋅ 19 ⋅ 31 ⋅ 37 ⋅ 601 ⋅ 829 ⋅ ⋅ ⋅
Table 2.Prime factorizations of2s−1,3s−1, and5s−1for various small smooth values ofs.
s 1299743s−1 a(s)
1 2 ⋅ 649871 2
2 26⋅ 32⋅ 4513 ⋅ 649871 23⋅ 3
3 2 ⋅ 7 ⋅ 13 ⋅ 397 ⋅ 649871 ⋅ 6680137 2
4 27⋅ 32⋅ 52⋅ 193 ⋅ 4349 ⋅ 4513 ⋅ 40253 ⋅ 649871 24⋅ 3 ⋅ 5
6 26⋅ 33⋅ 72⋅ 13 ⋅ 31 ⋅ 397 ⋅ 4513 ⋅ 649871 ⋅ 6680137 ⋅ 18164844799 23⋅ 32⋅ 7 8 28⋅ 32⋅ 52⋅ 172⋅ 73 ⋅ 193 ⋅ 241 ⋅ 4349 ⋅ 4513 ⋅ 40253 ⋅ 649871 ⋅ 298090889 ⋅ 941485217 25⋅ 3 ⋅ 5 12 27⋅ 33⋅ 72⋅ 13 ⋅ 31 ⋅ 193 ⋅ 397 ⋅ 4349 ⋅ 4513 ⋅ 40253 ⋅ 649871 ⋅ 6680137 ⋅ 387205657 ⋅ ⋅ ⋅ 24⋅ 32⋅ 5 ⋅ 7 ⋅ 13 16 29⋅ 32⋅ 52⋅ 172⋅ 73 ⋅ 193 ⋅ 241 ⋅ 4349 ⋅ 4513 ⋅ 40253 ⋅ 649871 ⋅ 3955153 ⋅ 298090889 ⋅ ⋅ ⋅ 26⋅ 3 ⋅ 5 ⋅ 17 24 28⋅ 33⋅ 52⋅ 72⋅ 13 ⋅ 172⋅ 31 ⋅ 73 ⋅ 193 ⋅ 241 ⋅ 397 ⋅ 4349 ⋅ 4513 ⋅ 40253 ⋅ 649871 ⋅ ⋅ ⋅ 25⋅ 32⋅ 5 ⋅ 7 ⋅ 13 30 26⋅ 33⋅ 72⋅ 11 ⋅ 13 ⋅ 31 ⋅ 61 ⋅ 71 ⋅ 271 ⋅ 397 ⋅ 701 ⋅ 881 ⋅ 1171 ⋅ 2411 ⋅ 4513 ⋅ 649871 ⋅ ⋅ ⋅ 23⋅ 32⋅ 7 ⋅ 11 ⋅ 31 36 27⋅ 34⋅ 52⋅ 72⋅ 13 ⋅ 31 ⋅ 37 ⋅ 109 ⋅ 193 ⋅ 397 ⋅ 757 ⋅ 1657 ⋅ 4349 ⋅ 4513 ⋅ 40253 ⋅ 649871 ⋅ ⋅ ⋅ 24⋅ 33⋅ 5 ⋅ 7 ⋅ 13 ⋅ 19 ⋅ 37
Table 3.Prime factorizations ofqs−1anda(s)forq= 1299742and various values ofs.
of𝔽qs. In addition, as we will recall in sections5.1and5.2below, several important tools such as polynomial root finding and discrete logarithms admit faster implementations if we can takerof the formr=r1r2withr1=O(T)and wherer2is smooth. Sometimes, primitive roots of unity of such ordersralready exist in𝔽q. If not, then we need to search them in extension fields𝔽qswiths> 1as small as possible.
Let us briefly investigate the prime factorization ofqs−1for variousqand smalls. As observed in [21], the numberqs−1typically admits many small prime divisors whensis itself smooth. This phenomenon is illustrated in Table2for small values ofq. For prac- tical purposes, givenT, it is easy in practice to find a smallsand divisorsr=r1r2| (qs−1) such thatr1≈Tandr2is smooth.
For larger q, we may need to resort to larger extension degrees s in order to find appropriate ordersr. A natural question is whetherqs−1is guaranteed to have a non- trivial smooth divisor for largeqand a fixed value ofs. This leads us to introduce the following guaranteed lower bound:
a(s) ≔ lim
q0→∞gcd (qs−1 :q⩾q0). (2)
In Table3, we have shown the prime factorizations ofqs−1anda(s)forq= 1299743and various small smooth values ofs. Hereq= 1299743 was chosen such that (q−1)/2is also prime. For the practical applications in this paper, the table suggests that it remains likely that suitable ordersrcan still be found whenever needed, and thata(s)is usually quite pessimistic, even for large values ofq. Let us finally mention that the sequencea(s) coincides with Sloane's integer sequenceA079612; seehttps://oeis.org/A079612.
3. G
ENERAL OBSERVATIONSAs already mentioned in the introduction, most algorithms for sparse interpolation are probabilistic of Monte Carlo type. We notice that it is easy to check (with high proba- bility) whether a candidate sparse interpolation f∗of f is correct. Indeed, it suffices to evaluate f−f∗at random sample points and check whether the result vanishes. Deter- ministic algorithms exist but with higher complexities; see for instance [22,42].
Many algorithms for sparse interpolation require extra information, such as boundsT⩾tandD⩾dfor the number of terms and the total degree of f. Furthermore, several algorithms are only able to guess some of the terms offwith high probability, but not all of them. In this section, using ideas from [3], we show how to turn such “partial”
algorithms for sparse interpolation into full-blown ones. Provided that the character- istic of K is zero or sufficiently large, we also show how to upgrade an interpolation method modulo(x1r−1, . . . ,xnr−1)into a general algorithm, following [30].
3.1. From partial to full interpolation
Assume that we have an algorithm for “partial” sparse interpolation that takes a black box for f as input, together with bounds T and D for t and d. The algorithm should always return a sparse polynomialf∗of total degree at mostDand with at mostTterms.
Moreover, for some fixed constant0 < 𝜆 < 1, ift⩽Tandd⩽D, then f−f∗should contain at most𝜆tterms, with high probability. Ift>Tord>D, then we allow f∗to be essentially arbitrary under the above constraint that f∗has at mostTterms of degree⩽D. Then we may use the following strategy for arbitrary sparse interpolations:
Algorithm 1
Input:a polynomial black box function f(x1, . . . ,xn) Output:the sparse interpolation f∗of f
1. Let f∗≔ 0be an initial approximation of f.
2. Set initial bounds T≔ 1 andD≔ 1for the number of terms and total degree.
3. Determine the approximate sparse interpolation𝛿∗to𝛿 ≔f−f∗usingT andDas bounds for the number of terms and the total degree of𝛿.
4. Set f∗≔f∗+ 𝛿∗.
5. If f=f∗with high probability, then return f∗.
6. Whenever appropriate, increaseTand/orD, and reset f∗≔ 0.
7. Return to step 3.
In step 6, the precise policy for increasingTandDmay depend on the application.
We typically doubleT whent is suspected to exceedT and we doublelogDwhendis suspected to exceedD. In this way, the boundsTandlogDare at most twice as large as the actual valuestandlogd, and the running time is essentially a constant times the running time of the approximate sparse interpolation with boundsTandD.
However, for this “meta complexity bound” to hold, it is crucial in step 3 that the sparse approximation f∗can be evaluated efficiently at the sample points used during the sparse interpolation (the naive evaluation of a polynomial withtterms atΘ(t)points would take timeΘ(t2), which is too much). Fortunately, this is the case for all sparse interpolation strategies that will be considered in this paper.
When do we suspectTorDto be too low in step 6? In the case ofT, a natural idea is to test whether the number of terms in f∗or𝛿∗exceeds a fixed constant portion ofT.
This strategy assumes that𝛿∗be essentially random whenTis too small (if the number of terms of𝛿∗is much smaller than T whenever t>T, then we might need more than Θ(logt)iterations before the algorithm converges).
The handling of exponents and degree bounds is more involved. When interpolating over a finite field, all non-zero evaluation points are roots of unity, so the exponents can only be determined modulo a certain integerr(or even modulo a submodule ofℤn). If the characteristic ofKis sufficiently large, then the exponents can be computed directly:
see the next subsection. Otherwise, we need to recombine reductions with respect to several moduli: see section4below. This also provides a natural test in order to check whetherd⩽D. Indeed, it suffices to compute the sparse interpolation of ffor one or more extra moduli and check whether the results agree with our candidate interpolation.
3.2. Supersparse interpolation in large characteristic
Assume that we have an algorithm that allows us to compute the sparse interpolation of f moduloIr= (x1r−1,...,xnr−1)for given modulir. Assume also that we have access to the program that computes f, typically in terms of a straight-line program. IfcharK= 0 orcharK> max (degx1f,..., degxnf), then let us show how to derive an algorithm for the sparse evaluation of f.
It is well known that Baur–Strassen's technique of “automatic differentiation” [7]
allows us to evaluate the gradient(∂f/∂x1, . . . , ∂f/∂xn)using at most 4L operations inK. Using5L+noperations, this provides an algorithm for the simultaneously eval- uation of f,g1, . . . ,gnwithgi=xi(∂f/∂xi) fori= 1, . . . ,n. With a small overhead, this next allows us to jointly compute the sparse interpolations of f,g1, . . . ,gnmoduloIr.
Now assume thatcx1e1⋅⋅⋅xnenis a term of f such that for any other termc′x1e1′⋅⋅⋅xnen′of f, we have(e1modr, ... ,enmodr) ≠ (e1′ modr,. .. ,en′ modr); we say that this term “does not collide” modulor. Then the sparse interpolation of f moduloIrcontains the non-zero termcx1e1modr⋅⋅⋅xnenmodr. Moreover, giveni∈ {1,...,n}withei≠ 0, the sparse interpolation of gi moduloIralso contains the non-zero termeic x1e1modr⋅ ⋅ ⋅xnenmodr. This allows us to retrieveeithrough one simple division(eic)/c.
Furthermore, if the modulusrwas picked at random withr>t, then there is a high probability that a fixed non-zero proportion of terms in f do not collide modulor. Com- bined with Algorithm1, this yields an algorithm for obtaining the sparse interpolation of f. This strategy for sparse interpolation was first exploited by Huang [30].
Remark.For simplicity, we consider sparse interpolation of polynomials over fieldsKin this paper. In fact, the algorithms also work for vectors of polynomials inK[x1, . . . ,xn]𝜈, by considering them as polynomials with coefficients inK𝜈. We implicitly used this fact above when saying that we “jointly” compute the sparse interpolation of f,g1, . . . ,gn
moduloIr.
3.3. Conclusion
In summary, we have shown how to reduce the general problem of sparse interpolation to the case when
1. we have bounds for the number of terms and the total degree, and
2. we only require an approximate sparse interpolation (in the sense of section3.1).
4. U
NIVARIATE INTERPOLATION USING CYCLIC EXTENSIONS One general approach for sparse interpolation of univariate polynomials over gen- eral base fieldsKwas initiated by Garg and Schost [15]. It assumes that the black box function f can be evaluated over any cyclic extension of the form K[x]/(xr−1). The evaluation off=c1xe1+ ⋅ ⋅ ⋅ +ctxet (3)
atxinside such an extension simply yields
f[r]≔c1xe1remr+ ⋅ ⋅ ⋅ +ctxetremr.
In absence of “collisions”ei=ejmodulorfori≠j, this both yields the coefficients of fand its exponents modulor. By combining the evaluations for various moduli, it is possible to reconstruct the actual exponents using Chinese remaindering.
Throughout this section, we assume that we are given boundsDandTfor the degreed and the number of terms t of f. Garg and Schost's original algorithm was determin- istic under these assumptions. However, their algorithm was not designed to be efficient in practice. In the past decade, many variants have been proposed. Roughly speaking, they all follow the same recipe that we summarized in Algorithm2 below. The vari- ants mainly differ in the precise way recombinations are done in step 3.
Algorithm 2
Input:a black box polynomial f(x), a degree boundD, a sparsity boundT Output:a partial sparse interpolation f∗of f
1. Determine suitable modulir1, . . . ,rl>Twithlcm (r1, . . . ,rl) >D.
2. Evaluate f atxinK[x]/(xri−1)fori= 1, . . . ,l.
3. Determine matching terms in the expansions of f[r1], . . . ,f[rl]that are likely to be the reductions of the same termcixeiin the expansion of f. 4. Return the sum f∗of all termscixeias above.
4.1. Complexity analysis
For all matching strategies that have been considered so far, the cost of steps 1 and 3 is dominated by the cost of step 2. If the evaluation of fonly involves ring operations, then the running time of Algorithm2is therefore bounded byO((MK(r1) + ⋅ ⋅ ⋅ +MK(rl))L).
The moduli rk are usually all of the same order of magnitude rk≈T𝜈 for some small 𝜈 ⩾ 1that depends on the matching strategy. Then we may takel=O(logD/logT), so the cost simplifies toO(LMK(T𝜈) logD/logT). For finite fieldsK= 𝔽q, this cost becomes O♭(LT𝜈logDlogq). For the design of matching strategies, it is therefore important that we can take𝜈as small as possible.
Remark. The above analysis can be refined by maintaining separate countsLadd,Lmul, andLdivfor the numbers of additions (or subtraction), multiplications, and divisions that are necessary for one evaluation of T. Then the cost of Algorithm 2 over𝔽qbecomes O♭(T𝜈logq(Ladd+LmullogT+Ldiv(logT)2)).
Remark.The complexity analysis may need to be adjusted somewhat ifDis so large that we run out of suitable moduliri. If our matching strategy requires prime numbers of the order ofTc, then this happens whenlogDexceeds approximately the same orderTc. In that case, we need to replaceT by an appropriate power oflogDin our complexity bounds. Alternatively, if the characteristic of K is sufficiently large, then we may fall back on the strategy from section3.2.
4.2. Survey of existing variants based on cyclic extensions
4.2.1. Determining the exponents using Chinese remaindering
Garg and Schost's original algorithm from [15] uses prime numbers p for the moduli rk. Assuming that f[p] admits t terms, the algorithm is based on the observa- tion that the projection of the polynomial(z−e1) ⋅ ⋅ ⋅ (z−et) modulo p coincides with (z−e1remp) ⋅⋅⋅ (z−etmodp). This allows for the recovery ofE= (z−e1) ⋅⋅⋅ (z−et)through Chinese remaindering, by working with a sufficient number of primes. It then suffices to determine the zerose1, . . . ,etof Eand to recoverci as the coefficient of xeirempin f[p]
fori= 1, . . . ,t.
However, this strategy is very sensitive to collisions, and requiresp≫T2in order to ensure with high probability that f[p]admits exactly tterms. In other words, it forces us to take𝜈 ⩾ 2in the complexity analysis. Garg and Schost's original algorithm is actu- ally deterministic and usesO˜(L T4(logD)2)operations inK. The derandomization is achieved by usingΘ(T2logD)different primesp.
4.2.2. Composite moduli
Another matching strategy for step 3 of Algorithm2has been proposed by Arnold, Gies- brecht, and Roche [3]. The idea is to pick rk=p0pk for k= 1, . . . ,l, where p0, . . . ,pl are primes withp0≍Tandp1, . . . ,pl≈T𝜖for some𝜖 > 0(so that𝜈 = 1 + 𝜖). Fori= 1, . . . ,tand k= 1, . . . ,l, there then exists a fixed non-zero probability such that the term cixeiremp0 of f[p0]matches a termcixeiremrkof f[rk]. Let𝒦ibe the set of indiceskfor which we have a match. For some fixed constantc> 0, we then have lcm (rk:k∈ 𝒦i) >Tc𝜖l with high probability. By takingl>logD/(c𝜖 logT)in step 1, this implieslcm(rk:k∈ 𝒦i) >D. With high probability, this allows us to reconstruct those termscixeisuch thatei≠ejmodulop0
for allj≠i. The sum of these terms gives the desired approximation f∗of f for which a fixed non-zero proportion of terms are likely to be correct.
4.2.3. Diversification
Giesbrecht and Roche proposed yet another matching strategy [16] which is based on the concept of “diversification”. The polynomial f is said to bediversifiedif its coefficientsci
are pairwise distinct. Assuming thatK is sufficiently large, it is shown in [16] that the polynomial f(𝛼x) is diversified with high probability for a random choice of 𝛼 ∈K∗. Without loss of generality, this allows us to assume that f is diversified.
In step 3 of Algorithm2, we then match a termc xeof f[rk]with a termc′xe′of f[rk′] if and only ifc=c′. Giesbrecht and Roche's original algorithm usesl=O(logD)moduli r1, . . . ,rlof sizeO˜(T2logD). Consequently, their algorithm for sparse interpolation uses O˜(L T2(logD)2) operations in 𝔽q. As we will see below, their probabilistic analysis is actually quite pessimistic: in practice, it is possible to take r1, . . . ,rl=O˜(T) as long as logD=O(T).
4.3. An optimized probabilistic algorithm based on diversification
Let us now focus on the design and analysis of a probabilistic algorithm that exploits the idea of diversification even more than Giesbrecht and Roche's original method from [16].
Given a diversified polynomial f, together with boundsDandTfor its degreedand its number of termst, our aim is to computeΘ(t)terms of f, with high probability. Our algorithm uses the following parameters:
• A constant𝜏 ≍ 1.
• Ordersr1< ⋅ ⋅ ⋅ <rlthat are pairwise coprime, withrk≈ 𝜏Tfork= 1, . . . ,l.
• The minimal number𝜈 ∈ {1, . . . ,l}such thatlcm (r1, . . . ,r𝜈) =r1⋅ ⋅ ⋅r𝜈>D.
The precise choice of𝜏 andlwill be detailed below; the parameter𝜏 and the ratiol/𝜈 should be sufficiently small for the algorithm to be efficient, but𝜏andl/𝜈should also be sufficiently large for our algorithm to succeed with high probability.
We now use Algorithm3below in order to compute an approximate sparse interpo- lation of f. It is a variant of Algorithm2with a matching strategy that has been detailed in steps 2, 3, and 4. Each individual termcxeof f is reconstructed from only a subset of its reductions moduloxr1−1, . . . ,xrl−1.
Algorithm 3
Input:a diversified black box polynomial f(x)and parameters as above Output:an approximate sparse interpolation f∗of f
1. Compute f[rk]=f rem (xrk−1)fork= 1, . . . ,l.
2. Let f∗≔ 0.
3. LetCkbe the set of all coefficients that occur once in f[rk], fork= 1, . . . ,l.
4. For eachc∈C1∪ ⋅ ⋅ ⋅ ∪Cldo:
If𝒦 ≔ {k:c∈Ck}is such thatlcm (rk:k∈ 𝒦) >D, then:
Determine the unique exponent e<Dsuch that c xeremrk occurs in f[rk]for everyk∈ 𝒦, and set f∗≔f∗+cxe.
5. Return f∗.
4.4. Analysis of the expected number of correct terms
How to ensure that a non-zero portion of the terms of f∗can be expected to be correct?
In order to answer this question, we make the following heuristic hypothesis:
Hred. Fork= 1, . . . ,l, the modular reductions of exponentseiremrkfori= 1, . . . ,tare uni- formly distributed in{0,...,rk−1}. The distribution associated torkis independent of the one associated tork′wheneverk≠k′.
Such a heuristic is customary in computer science, typically when using hash tables.
According toHred, the probability that a fixed termcxedoes not collide with another term modulorkis
(1−rk−1)T−1 ≈ (1−(𝜏T)−1)T−1 ⩾ (1−(𝜏T)−1)T ⩾ eTlog(1−(𝜏T)−1) ⩾ e−1/𝜏.
Setting𝜖 ≔ 1−e−1/𝜏, this probability tends to1−𝜖for largeT. The probability thatc xe collides with another term modulorkfor exactly𝜅values ofkis therefore bounded by
l
𝜅 𝜖l−𝜅(1−𝜖)𝜅
and this bound is sharp for largeT. Consequently, the probability that we cannot recover a term feixeiin step 4 from its reductions moduloxrk−1fork= 1, . . . ,lis at most
P(𝜈;l, 1−𝜖) =
i<𝜈
li 𝜖l−i(1−𝜖)i, (4)
and this bound is sharp for largeT.
The probability (4) has been well studied; see [5] for a survey. Whenever a≔ 𝜈/l> 1−𝜖,
Chernoff's inequality [5, Theorem 1] gives us P(𝜈;l, 1−𝜖) ⩽ exp −l alog a
1−𝜖 + (1−a) log 1−a
𝜖 .
Let𝜂 < 1be a positive real parameter. In order to ensureP(𝜈;l, 1−𝜖) < 𝜂it suffices to have l alog a
1−𝜖 + (1−a) log 1−a
𝜖 > log(1/𝜂).
Now thanks to [40, Lemma 2(a)] we have alog a
1−𝜖 + (1−a) log 1−a
𝜖 ⩾ 2 (a−(1−𝜖))2, so it suffices to ensure that
(𝜈−l(1−𝜖))2>log(1/𝜂)
2 l. (5)
Now let us take𝜈 =c(1−𝜖)lwithc> 1, after which (5) is equivalent to (c−1)2>log(1/𝜂)
2l(1−𝜖)2. (6)
For fixed𝜂and largel(i.e. for largeD), it follows that we may takecarbitrarily close to1.
Summarizing, we may thus takel≈ 𝜈/(1−𝜖)in order to recover an average of at least (1−𝜂)tcorrect terms, where𝜂can be taken arbitrarily close to0:
PROPOSITION1. AssumeHredand let𝜏,𝜖, l, r1,...,rl, and𝜈be parameters as above. Given𝜂 < 1, assume that𝜈 =c(1−𝜖)l, where c satisfies(6). Then Algorithm3returns at least(1−𝜂)t correct terms of f on average.
4.5. Probabilistic complexity analysis
Let us now analyze the running time of Algorithm3. Takingl≈ 𝜈 /(1−𝜖), the cost of step 1 is proportional to
𝜏
1−𝜖 = 𝜏 e1/𝜏,
and𝜏 e1/𝜏reaches its minimum valueeat𝜏 = 1. This means that the total complexity is best when𝜏 is close to1. In other words, this prompts us to take𝜏 = 1,r1≈ ⋅ ⋅ ⋅ ≈rl≈T, 𝜈 ≈logD/logT, andl≈𝜈/(1−e−1). For this choice of parameters, we obtain the following heuristic complexity bound:
PROPOSITION2. Assume thatHredandlogD=O(T). Given0 < 𝜂 < 1and a diversified poly- nomial f of degree d⩽D and with t⩽T terms, there exists a Monte Carlo probabilistic algorithm which computes at least(1−𝜂)t terms of f in time
O♭(LTlogDlogq).
Proof. We take ri to be the i-th smallest prime numbers that is larger thanT, so that 𝜈 =O(logD/logT) is the smallest number with r1⋅ ⋅ ⋅r𝜈>D. We also take 𝜏 = 1, 𝜖 = 1−1/e, and letl=O(logD/logT)be smallest such that (6) is satisfied forc= 𝜈/(l(1−𝜖)).
Combining [6] and [1], we may computer1, . . . ,rlin timeO♭(TlogD).
Now the running time of step 1 of Algorithm3is O((Mq(r1) + ⋅ ⋅ ⋅ +Mq(rl))L).
Withl=O(logD/logT), this cost simplifies to
O♭(LTlog(qT) logD/logT) =O♭(LTlogqlogD).
Step 3 may be obtained by sorting the coefficients of the f[rk], in time O(lTlogTlogq) =O(TlogDlogq).
Using fast Chinese remaindering, step 4 takes timeTO˜(log 𝜈 logD) =O♭(TlogD). □ Remark.Ifq≫T2, then f(𝛼x)is diversified with high probability for a random choice of 𝛼∈𝔽q∖{0}: see [16]. In the range whereq=Θ(T)andq=O(T2), it is possible to work with a slightly weaker assumption: we say that f isweakly diversifiedif{c1,...,ct}is of sizeΘ(t).
Ifq= Θ(T), then the polynomial f(𝛼x)is still weakly diversified, for a random choice of𝛼 ∈ 𝔽q∖ {0}. If f is only weakly diversified andt′ ≔ {1 ⩽i⩽t: ∀j≠i,ci≠cj}, then our analysis can be adapted to show that Algorithm3returns about(1−𝜂)t′correct terms of f on average. Finally, in the range whereq=o(T), we need to work over a field exten- sion𝔽qswithqs⩾T, which leads to an additional arithmetic overhead ofO♭(logT/logq).
Remark. Let us show that with high probability, the polynomial f∗returned by Algo- rithm 3 only contains correct terms of f (although it does not necessarily contain all terms). For this, we make the additional hypothesis that the coefficients of f are essen- tially random non-zero values in𝔽q(which is typically the case after a change of variables
f(x)↝f(𝛼x), where𝛼 ∈ 𝔽q∖ {0}is random).
Now assume that some coefficientcin step 4 gives rise to a termcxethat is not in f. Then for everyk∈ 𝒦, there should be at least two terms in f that collide modulorkand for which the sum of the corresponding coefficients equalsc. The probability that this happens for a fixedk∈ 𝒦is bounded by(q−1)−1and the probability that this happens for allk∈ 𝒦 is bounded by(q−1)−𝜈 ′, where𝜈 ′ ≈ 𝜈is minimal withrl−𝜈 ′+1⋅ ⋅ ⋅rl>D.
4.6. Estimating the number of terms t
For the algorithms in this section, we assumed that a boundTfortwas given. It turns out that a variant of our probabilistic analysis can also be used for the efficient computation of a rough estimate fort. This yields an interesting alternative to the naive doubling strategy described in section3.1.
Let us still assume that Hredholds. We will also assume that colliding terms rarely cancel out (which holds with high probability if qis sufficiently large). This time, we compute f(x) rem (xB−1)forB≔ 𝛼t, where𝛼 < 1is to be determined, and letNbe the number of terms in this remainder. When randomly distributingtballs overBboxes, the probability that none of the balls lands in a particular box is(1−1/B)t. Consequently, the expected number of boxes with no ball is(1−1/B)tB, whence
B−N≈ 1−1 B
tB≈ e−
1 𝛼B.
It follows that
1
𝛼≈ log B B−N , and thus
t=B
𝛼≈Blog B
B−N . (7)
By doublingBuntilB−N≳ B, we may then use the approximation (7) as a good candi- date fort. Notice that we haveB−N≈ B whenB≈ 2t/logt.
4.7. Conclusion
Cyclic extension methods for sparse interpolation are attractive due to their generality and the possibility to derive deterministic complexity bounds. However, even their most efficient probabilistic versions suffer from the overhead of arithmetic operations in cyclic extension algebrasK[x]/(xr−1).
The matching strategy based on diversification leads to the best practical complexity bounds, as shown in section4.5. AssumingHred,logD=O(T), andq= Θ(T), we have given a Monte Carlo algorithm for sparse interpolation of complexityO♭(LTlogDlogT).
The case when q=o(T) can be reduced to this case using a field extension of degree O(logT/logq). Assuming onlyHredand logD=O(T), we thus obtain a probabilistic algorithm that runs in time
O♭(LTlogDlog (qT)). (8)
5. U
NIVARIATE INTERPOLATION USING GEOMETRIC PROGRESSIONS Prony's method is one of the oldest and most celebrated algorithms for sparse interpola- tion of univariate polynomials. It is based on the evaluation of f at points in a geometric progression. Since there are many variants, we keep our presentation as general as pos- sible. As in the previous section, assume thatf=c1xe1+ ⋅ ⋅ ⋅ +ctxet (9)
and that we know boundsDandTfor the degree and the number of terms of f. Algorithm 4
Input:a black box polynomial f(x), a degree boundD, a sparsity boundT Output:the sparse interpolation f∗of f
1. Find a suitable element𝜔 ∈Kwith multiplicative orderr⩾ max (D, 2T), while replacingKby an extension if necessary.
2. Evaluate f(1),f(𝜔),f(𝜔2), . . . ,f(𝜔2T−1).
3. Compute a minimalt⩽T, a monicΛ ∈K[z]of degreet, and anN∈K[z]
of degree<t, such that the following identity holds moduloO(z2T):
k∈ℕ
f(𝜔k)zk =
1⩽i⩽t
ci
1−𝜔eiz = N(z)
Λ(z). (10)
4. Find the roots𝜔−eiofΛ ≔zt+ Λt−1zt−1+ ⋅ ⋅ ⋅ + Λ0∈K[z].
5. Compute the discrete logarithms of the roots to base𝜔to discover the exponentse1, . . . ,etof f as in (9).
6. Computece1,...,cetfrom f(1),...,f(𝜔t−1)ande1,...,et, using linear algebra.
It is well known that steps 3 and 6 can be performed in timeO(MK(T) logT), through fast Padé approximation [10, 38] in the case of step 3, and using a transposed version of fast multipoint interpolation [9, 11] for step 6. If K= 𝔽q, then this bound becomes O♭(T(logT)2logq). The efficiency of steps 4 and 5 highly depends on the coefficient fieldK. In the remainder of this section, we will discuss this issue in detail in the case whenK= 𝔽qis a finite field.
5.1. Root finding
Finding the roots of a univariate polynomial over a finite field is a well-known and highly studied problem in computer algebra. The most efficient general purpose algorithm for this task is due to Cantor–Zassenhaus [12]. It is probabilistic and computes the roots ofΛin timeO♭(T(logT)2(logq)2).
In [17,18], several alternative methods were designed for the case whenr=r1r2with r1=O(T)andr2⩽TO(1)smooth (in the sense that𝜋 =O♭(1)for each prime factor𝜋ofr2).
The idea is to proceed in three steps:
1. We first compute the r2-th Graeffe transform Gr2(Ω) of Ω, whose roots are the r2-th powers of the roots ofΩ. This step can be done in timeO♭(T(logT)2logq) by [17, Proposition 5].
2. We next compute the roots ofGr2(Ω)through an exhaustive evaluation at allr1-th roots of unity. This step takes timeO♭(TlogTlogq).
3. We finally lift these roots back up to the roots of Ω. This can be done in time O♭(T(logT)2logqlogr2) =O♭(T(logT)3logq), using g.c.d. computations.
Altogether, this yields a sparse interpolation method of costO♭(T(logT)3logq).
The backlifting of single roots can be accelerated using so-called “tangent Graeffe transforms”. The idea is to work over the ringR≔K[𝜖]/(𝜖2)instead ofK. Then𝛼 ∈K is a root of a polynomialP(z) ∈K[z]if and only if𝛼 + 𝜖 ∈Ris a root of the polynomial P(z−𝜖) ∈R[z]. Now if we know a single root(𝛼 + 𝜖)r2= 𝛼r2+r2𝛼r2−1𝜖of Gr2(Ω(z−𝜖)), then we may retrieve𝛼using one division of𝛼r2byr2𝛼r2−1and one multiplication byr2 (note thatr2is invertible in𝔽qsincer2dividesq−1). In other words, the backlifting step can be done in timeO♭(Tlogq), usingO(T)operations in𝔽q.
However, this method only works for single roots𝛼. When replacingΩ(z)andΩ(z−𝜆) for a randomly chosen 𝜆 ∈ 𝔽q, the polynomial Gr2(Ω(z−𝜆)) can be forced to admit a non-trivial proportion of single roots with high probability. However, these roots are no longer powers of𝜔, unless we tookr=q−1. Assuming thatr=q−1and using sev- eral shifts𝜆, it can be shown [17, Proposition 12] that the tangent Graeffe method yields a sparse interpolation method of complexityO♭(TlogT(logq)2).
5.2. Discrete logarithms
The discrete logarithm problem in abelian groups is a well-known problem in compu- tational number theory. Ifr is smooth, then Pohlig–Hellman's algorithm provides an efficient solution; it allows step 5 of Algorithm4to be performed in timeO♭(Tlogrlogq).
Under the assumption that we may taker=TO(1), this bound reduces toO♭(TlogTlogq).
Again, the same bound still holds ifr=r1r2withr1=O(T)andr2smooth. Indeed, in that case, we may tabulate the powers 𝜔0, 𝜔r2, 𝜔2r2, . . . , 𝜔(r1−1)r2. This allows us to efficiently determine the discrete logarithms of𝜔e1r2, . . . , 𝜔etr2with respect to𝜔r2, which yields the exponentse1, . . . ,et modulo r1. We next use Pohlig–Hellman's algorithm to computee1, . . . ,et.
