1Introduction A S P I G C D

(1)

IRREDUCIBILITY AND G^REATEST C^OMMON D^IVISOR ALGORITHMS FOR SPARSE POLYNOMIALS

Michael Filaseta Mathematics Department University of South Carolina

Columbia, SC 29208 USA

Andrew Granville

Département de Mathématiques Université de Montréal

Montr´eal QC H3C 3J7 Canada

Andrzej Schinzel Institute of Mathematics Polish Academy of Sciences

ul. ´Sniadeckich 8, 00-950 Warsaw, Poland

1 Introduction

Letf(x) = P_r

j=0a_jx^d^j ∈ Z[x]with eacha_j nonzero and withd_r > d_r₋₁ > · · ·> d₁ > d₀ = 0.

For simplicity, we refer to the degreed_r off(x)asn. Observe thatr+ 1is the number of terms off(x). For convenience, we suppose bothn > 2andr >0. The heightH, as usual, denotes the maximum of the absolute values of thea_j.

The lattice base reduction algorithm of A. K. Lenstra, H. W. Lenstra, Jr., and L. Lovasz [7] gives a factoring algorithm forf(x)that runs in time that depends polynomially onlogH andn. This clearly serves also as an irreducibility test for f(x). One problem we address in this paper is the somewhat different issue of describing an irreducibility algorithm for sparse polynomials, that is whereris small compared ton. We view the input as being the list ofr+ 1coefficientsajtogether with the list ofr+ 1exponentsd_j. With this in mind, the input is of sizeO¡

r(logH+ logn)¢ . We give an algorithm for this problem that runs in time that is polynomial inlogn (but note that the dependence onrandlogH in our arguments is not polynomial).

Forf(x) ∈ Q[x], we definef˜=xⁿf(1/x). We say thatf(x)is reciprocal if f(x) = ±f˜(x).

Otherwise, we say thatf(x)is nonreciprocal. We note that f(x) is reciprocal if and only if the

2000 Mathematics Subject Classification: 11Y16, 12Y05, 68W30, 11C08, 11R09

The first author was supported by the National Science Foundation and the National Security Agency and the second author by the Natural Sciences and Engineering Research Council of Canada.

(2)

condition f(α) = 0 for α ∈ C implies that α 6= 0 and f(1/α) = 0. Our methods require the additional assumption thatf(x)is nonreciprocal. We establish the following.

Theorem A. There is a constant c₁ = c₁(r, H) such that an algorithm exists for determining whether a given nonreciprocal polynomial f(x) ∈ Z[x]as above is irreducible and that runs in timeO¡

c₁logn(log logn)²log log logn¢ .

The result relies heavily on some recent work by E. Bombieri and U. Zannier described by the latter in an appendix of [11]. Alternatively, we can make use of [1], work by these same authors and D. Masser, which describes a new simplified approach to the previous work. The other main ingredients are the third author’s application of the work of Bombieri and Zannier, given originally in [10], and an improvement on the the first and third authors’ joint work in [4].

The constant c₁ can be made explicit. We note though that c₁ depends on some effectively computable constants that are not explicitly given in the appendix of [11] or in [1]. We therefore do not address this issue further here.

The algorithm will give, with the same running time, some information on the factorization of f(x)in the case thatf(x)is reducible. Specifically, we have the following:

(i) Iff(x)has a cyclotomic factor, then the algorithm will detect this and output anm ∈ Z⁺ such that the cyclotomic polynomialΦ_m(x)dividesf(x).

(ii) Iff(x)does not have a cyclotomic factor but has a non-constant reciprocal factor, then the algorithm will produce such a factor. In fact, the algorithm will produce a reciprocal factor off(x)of maximal degree.

(iii) Otherwise, iff(x)is reducible, then the algorithm outputs a complete factorization off(x) as a product of irreducible polynomials overQ.

The algorithm for Theorem A will follow along the lines given above. First, we will check iff(x) has a cyclotomic factor. If it does, the algorithm will produce m as in (i) and stop. If it does not, then the algorithm will check iff(x)has a non-cyclotomic non-constant reciprocal factor. If it does, then the algorithm will produce such a factor as in (ii) and stop. If it does not, then the algorithm will output a complete factorization off(x)as indicated in (iii).

Our approach to (i) will allow us to obtain additional information about the complete set of cyclotomic factors off(x). In particular, we are able to describe, in the same running time given for the algorithm in Theorem A, the factor off(x)which has largest degree and only cyclotomic divisors. Details are given in the next section.

Our approach can be modified to show that iff(x)∈Z[x]is nonreciprocal and reducible, then f(x)has a non-trivial factor inZ[x]containingO(c₂)terms wherec₂ =c₂(r, H). We note that the results of [9] imply that iff(x)also does not have a reciprocal factor, then every factor off(x)in Z[x]containsO(c₂)terms.

In the case thatf(x)∈Z[x]is reciprocal, one can modify our approach to obtain some information on the factorization off(x). Define the nonreciprocal part off(x)to be the polynomialf(x) removed of its irreducible reciprocal factors inZ[x]with positive leading coefficients. Then in the case that f(x) is reciprocal, one can still determine in time O(c₁(logn(log logn)²log log logn) whether the nonreciprocal part of f(x) is irreducible. Furthermore, in this same time, one can

(3)

determine whetherf(x)has a cyclotomic factor and, if so, an integermfor whichΦ_m(x)divides f(x).

In addition, we address the problem of computing the greatest common divisor of two sparse polynomials. For nonzero f(x) and g(x) in Z[x], we use the notation gcd_Z(f(x), g(x)) to denote the polynomial inZ[x]of largest degree and largest positive leading coefficient that divides f(x) and g(x) in Z[x]. Later in the paper, we will also make use of an analogous definition for gcd_Z(f, g)wheref andg are inZ[x₁, . . . , x_r]. In this case, we interpret the leading coefficient as the coefficient of the expressionxê₁¹xê₂². . . xê_r^r withe₁ maximal, thene₂ maximal givene₁, and so on. Our main result for the greatest common divisor of two sparse polynomials is the following.

Theorem B. There is an algorithm which takes as input two polynomials f(x)andg(x)inZ[x], each of degree≤ nand height≤ Hand having ≤r+ 1nonzero terms, with at least one off(x) and g(x)free of cyclotomic factors, and outputs the value of gcd_Z(f(x), g(x)) and runs in time O¡

c₃logn¢

for some constantc₃ =c₃(r, H).

Our approach will imply that iff(x), g(x)∈Z[x]are as above withf(x)org(x)not divisible by a cyclotomic polynomial, then gcd_Z(f(x), g(x))has O(c₄)terms where c₄ = c₄(r, H). The same conclusion does not hold if one omits the assumption that eitherf(x)org(x)is not divisible by a cyclotomic polynomial. The following example, demonstrating this, was originally noted in the related work of the third author [12]. Letaandbbe relatively prime positive integers. Then

gcd¡

x^ab−1,(x^a−1)(x^b−1)¢

= (x^a−1)(x^b−1)

x−1 .

In connection with Theorem B, we note that D. A. Plaisted [8] has shown that computing gcd_Z(f(x), g(x))for general sparse polynomialsf(x) andg(x)in Z[x]is at least as hard as any problem in NP. On the other hand, his proof relies heavily on considering polynomialsf(x)and g(x)that have cyclotomic factors. By contrast, our proof of Theorem B will rest heavily on the fact that one off(x)org(x)does not have any cyclotomic factors.

Our proof of Theorem A will rely on Theorem B. In fact, Theorem B is where we make use of the work of Bombieri and Zannier already cited. It is possible to prove Theorem A in a slightly more direct way, for example by making use of Theorem 80 in [11] instead of Theorem B and Theorem 1 below. This does not avoid the use of the work of Bombieri and Zannier since Theo- rem 80 of [11] is based on this work. We have chosen the presentation here, however, because it clarifies that parts of the algorithm in Theorem A can rest on ideas that have been around for over forty years. In addition, we want the added information given by (i), (ii) and (iii) above as well as Theorem B itself.

To aid in our discussions, we have used letters for labelling theorems that establish the existence of an algorithm and will refer to the algorithms using the corresponding format. As examples, Algorithm A will refer to the algorithm given by Theorem A, and Algorithm B will refer to the algorithm given by Theorem B. Also, we make use of the notationO_r,H¡

w(n)¢

to denote a function with absolute value bounded by w(n) times a function ofr and H. Thus, the running time for Algorithm A and Algorithm B can be expressed asO_r,H¡

logn(log logn)²log log logn¢ and O_r,H(logn), respectively.

(4)

2 The Proof of Theorem A

We begin with the following result which improves on the main result in [4].

Theorem C. There is an algorithm that has the following property: givenf(x) = Pr

j=0a_jx^d^j ∈ Z[x] of degree n > 1 and with r + 1 > 1 terms, the algorithm determines whether f(x) has a cyclotomic factor in running time O_r,H¡

logn(log logn)²log log logn¢

, where H denotes the height of f(x). Furthermore, with the same running time, if f(x) is divisible by a cyclotomic polynomial, then the algorithm outputs a positive integermfor whichΦ_m(x)dividesf(x).

Proof. We begin as in the proof of Theorem 2 of [4] and initially give an argument for the existence of an algorithm as in the theorem with running time O_r,H¡

(logn)²¢

. We then explain how the algorithm can be sped up to produce the running time given in the statement of the theorem.

We describe and make use of Theorem 5 from [2]. For k a positive integer, define γ(k) = 2 +P

p|k(p−2). Following [2], we call a vanishing sum S minimal if no proper subsum ofS vanishes. We will be interested in sumsS =Pt

j=1a_jω_j wheretis a positive integer, eacha_j is a nonzero rational number and eachω_j is a root of unity. We refer to the reduced exponent of such anSas the least positive integerkfor which(ω_i/ω₁)^k = 1for alli∈ {1,2, . . . , t}. Theorem 5 of [2] asserts then that ifS =Pt

j=1a_jω_j is a minimal vanishing sum, thent ≥ γ(k)wherek is the reduced exponent ofS. Also, note that Theorem 5 of [2] implies that the reduced exponentkof a minimal vanishing sum is necessarily squarefree.

To explain our algorithm, suppose first that f(x)has a cyclotomic factorΦ_m(x), and that we can write f(x) = P_s

i=1f_i(x) where each f_i(x) is a nonzero polynomial divisible byΦ_m(x), no twof_i(x)have terms involvingx to the same power, ands is maximal. Observe that each f_i(x) necessarily has at least two terms. Setting ζ_m = e²^πi/m, we see that each f_i(ζ_m) is a minimal vanishing sum. For each i ∈ {1,2, . . . , s}, we write f_i(x) = x^bⁱg_i(x^eⁱ)where g_i(x) ∈ Z[x], b_i ande_i are nonnegative integers chosen so thatg_i(0) 6= 0and the greatest common divisor of the exponents appearing ing_i(x)is1. Theng_i(ζ_m^eⁱ)is a minimal vanishing sum with reduced exponent m_i = m/gcd(m, e_i). Necessarily, we haveg_i(ζ_m_i) = 0andm_i is squarefree. Also, if t_i denotes the number of nonzero terms ofg_i(x), we have

t_i ≥γ¡ m_i¢

= 2 +X

p|mi

(p−2),

which implies each prime divisor ofm_iis≤t_i. Define

M_i ={`∈Z⁺ : Φ_`(x)|g_i(x), `is squarefree, andγ(`)≤t_i}. In particular,m_i ∈M_i. In other words,

(1) m

gcd(m, e_i) ∈Mi for alli∈ {1,2, . . . , s}. We have not explained how we can writef(x) =Ps

i=1f_i(x)as above. In particular, even if we knowmexists withΦm(x)dividingf(x), we do not know whatmis. We circumvent this issue by considering every possible partition of the set{0,1, . . . , r}as a disjoint union of setsJ₁, J₂, . . . , J_s with each setJ_icontaining at least two elements. For each partition, we consider the polynomials

f_i(x) =X

j∈Ji

a_jx^d^j =x^bⁱg_i(x^eⁱ), 1≤i≤s,

(5)

where as before b_i and e_i are nonnegative integers chosen so that g_i(0) 6= 0 and the greatest common divisor of the exponents appearing ing_i(x)is1. Definingt_iandM_ias above, depending on the partition of {0,1, . . . , r}, we see then that if f(x) is divisible by someΦ_m(x), then there is a partition for which (1) holds. On the other hand, if (1) holds for some positive integer m and some partition of{0,1, . . . , r}as above, then we havef_i(ζ_m) = 0for eachi ∈ {1,2, . . . , s}, which implies f(ζ_m) = 0 and hence Φ_m(x) | f(x). Thus, (1) holding for some m and some partition of {0,1, . . . , r}as above is a necessary and sufficient condition forf(x)to be divisible by a cyclotomic polynomial.

With the above in mind, we describe the algorithm for determining whetherf(x)has a cyclotomic factor, give further justification that the algorithm works and give a proof that its running time is as claimed. The algorithm is as follows. We go through every partition of the set{0,1, . . . , r} into disjoint non-empty setsJ₁, J₂, . . . , J_s with each setJ_i containing at least two elements. Ob- serve that there areO_r(1) such partitions. For each such partition and eachi ∈ {1,2, . . . , s}, we setu = u(i)to be the element ofJ_i for whichd_u is minimal. In terms of our definition of f_i(x) andg_i(x), this meansb_i =d_u ande_i is the greatest common divisor of the degrees of the terms of the polynomialf_i(x)/x^d^u. We computee_i by taking the greatest common divisor of the numbers d_v −d_u wherev ∈ J_i. In terms of the complexity of the algorithm, givenJ_i, determining d_u can be done in O_r(logn) bit operations and computing e_i takes at most O_r¡

(logn)²¢

bit operations (cf. the discussion of Euclid’s algorithm in [3, p. 79]). We can in fact obtain a running time of O_r¡

logn(log logn)²log log logn¢

using a recursive gcd computation for large integers [3, p. 428]

leading to the running time stated in Theorem C, but for the moment we use the O_r¡

(logn)²¢ estimate. The number of these computations that are needed as we vary over the partitions of {0,1, . . . , r}and vary over the sets J_i making up the partitions isO_r(1). The computations have therefore thus far taken at mostO_r¡

(logn)²¢

bit operations.

Next, for each partition J₁, J₂, . . . , J_s of {0,1, . . . , r} as above, we compute the sets M_i as follows. Observe thatt_i is the number of elements ofJ_i and is necessarily≤ r+ 1. Thus, we can construct a list of the`that are squarefree positive integers and such thatγ(`)≤t_i in timeO_r(1).

For each such`, we want to check ifΦ_`(x)dividesg_i(x). An algorithm that works well here and in more generality as well is given as Algorithm A in [4]. For our purposes, we can simply take each terma_vx⁽^d^v⁻^d^u⁾^/eⁱ ing_i(x), where v ∈ J_i, and replace it with a_vx^d⁰^v whered⁰_v ∈ {0,1, . . . , `−1} and

d⁰_v ≡ d_v−d_u

e_i (mod `).

If we call the resulting polynomial h_i(x), then g_i(x)is divisible byΦ_`(x)if and only if h_i(x)is divisible byΦ_`(x). Observe that the degree ofh_i(x)is≤ ` ≤ (r+ 1)^r. Also, the height ofh_i(x) is ≤ (r + 1)H. Hence, one can check directly if h_i(x) is divisible by Φ_`(x) in time O_r,H(1).

The construction of each h_i(x) takes time no more than O_r,H¡

(logn)(log logn)²¢

, for ε > 0 arbitrary, where the main contribution of the time required comes from the division ofd_v−d_u by e_i above. Hence, the total time spent on constructing the variousM_ias we vary over the partitions J₁, J₂, . . . , J_sof{0,1, . . . , r}andi∈ {1,2, . . . , s}isO_r,H¡

(logn)(log logn)²¢ .

For the algorithm, we consider each partition J₁, J₂, . . . , J_s of{0,1, . . . , r}as above one at a time. We construct the numbers e_i and the sets M_i as indicated. Next, we want to determine for a fixed partition whether (1) holds for some positive integerm. In other words, we want to know

(6)

whether there is anmandm_i ∈M_i for which

(2) m=m_igcd(m, e_i) fori∈ {1,2, . . . , s}.

For a positive integerk, we use the notationνp(k)to denote the positive integerusuch thatp^ukk.

Then (2) holds if and only if each of the following is true:

• Ifp|m₁. . . m_s, thenν_p(m)≤ν_p(m_ie_i)for alliwith equality wheneverpdividesm_i.

• Ifp-m₁. . . m_s, thenν_p(m)≤ν_p(e₀), wheree₀ = gcd(e₁, . . . , e_s).

Defining

D= Y

p^tke₀ p-m₁···ms

p^t=e₀±µ Y

p^tke₀ p|m₁···ms

p^t

¶

and m₀ = gcd(m₁e₁, . . . , m_se_s)/D,

then we see that a solution to (2) exists if and only if there existm_iinM_i such that for every prime pdividing somem_i, the exact power ofpdividingm₀is the same as the exact power ofpdividing m_ie_i. Furthermore, the set ofmsatisfying (2) in this case is precisely the set ofm =m₀d, where d|D. Observe thatm₀ is the uniquemsatisfying (2) (if suchmexist) with the property that every prime divisor ofmis a divisor ofm₁m₂· · ·m_s. Furthermore, every prime divisor ofm₁m₂· · ·m_s is a divisor of m₀. We are interested in knowing whether there existm andm_i satisfying (2), so we simply restrict our attention to determining whether there existm_i inM_i such that

(3) m₀ =m_igcd(m₀, e_i) fori∈ {1,2, . . . , s}.

Recall that the numberse_i and all elements ofM_i have been computed (for eachi = 1,2, . . . , s).

Also, as the partitions vary, the number of differente_i and m_i in M_i that arise is O_r(1). We go through all these possibilities and computeP, the set of primes dividingm₁m₂· · ·ms. There are O_r(1)such primes and it takesO_r(1)time to compute them. We computee₀,Dandm₀as defined above and check whether (3) holds. Note that the second formula for D involves removing the prime divisors frome₀ that are inP, which is a fixed set of primes of size Or(1). Thus, bothe₀ andDcan be computed in timeO_r¡

(logn)²¢

. We also computem₀ and check (3) with the same bound on the running time. If anm₀ is obtained for which (3) holds, then we output thatf(x)has a cyclotomic factor, indicate that the choice ofm = m₀ is such thatΦm(x)divides f(x)and end the algorithm. If nom₀ is obtained for which (3) holds, then we output thatf(x)does not have a cyclotomic factor. As there areO_r(1) differentm₀ each of sizeO_r(n), the running time estimate is not affected by going through the variousm₀ and outputting the result. Hence, the proof of the theorem, but with running time onlyO_r,H¡

(logn)²¢

, has been explained.

We improve the running time as follows. For the algorithm above, we made use of a few different greatest common divisor computations. These were done to constructei fori ∈ {1,2, . . . , s}, to calculatee₀ = gcd(e₁, . . . , e_s)andm₀ = gcd(m₁e₁, . . . , m_se_s)/D, and to determine the value of the right-hand side of (3). As noted earlier, we can apply known algorithms for gcd computations [3, p. 428] that would allow us to reduce the running time to that required by the theorem.

However, it is also worth noting that these gcd computations can be circumvented and the required running time obtained in a different manner. We explain this approach now.

(7)

Let J₁, J₂, . . . , J_s be a partition of {0,1, . . . , r} as in the argument above. Write e_i = e⁰_ie⁰⁰_i where every prime divisor ofe⁰_i is≤ r+ 1and every prime divisor of e⁰⁰_i is> r+ 1. Recall that u=u(i)∈J_iis chosen so thatd_u is minimal. One can computee⁰_i without computinge_i from the formula

e⁰_i = Y

p≤ti

p^min^v^∈Ji^{^ν^p⁽^d^v⁻^d^u^)}.

In other words, for eachp ≤ t_i, we can calculate the minimum ofν_p(d_v −d_u)asv runs through the elements of J_i and then form the product above to get e⁰_i. As we shall see momentarily, the numberse⁰_i can be calculated in timeO_r(logn(log logn)²log log logn).

We note now that

g_i¡ x^e⁰⁰ⁱ¢

=X

v∈Ji

a_vx⁽^d^v⁻^d^u⁾^/e⁰ⁱ, so we can computeg_i¡

x^e⁰⁰ⁱ¢

without computingg_i(x),e_i ore⁰⁰_i. Define M_i⁰ ={` ∈Z⁺ : Φ_`(x)|g_i¡

x^e⁰⁰ⁱ¢

, `is squarefree, andγ(`)≤t_i}.

The setM_i⁰can be computed in the same manner that we computedM_i but withg_i(x)replaced by g_i¡

x^e⁰⁰ⁱ¢

. Thus, computingM_i⁰, given the polynomialsg_i¡ x^e⁰⁰ⁱ¢

, takes timeO_r,H¡

(logn)(log logn)²¢ . Recall that the prime divisors of e⁰⁰_i are all> r + 1 ≥ t_i. We deduce that the numbers` in the definition of M_i and M_i⁰ are relatively prime to e⁰⁰_i. It follows that M_i = M_i⁰. Thus, the above analysis allows us to computeM_i without explicitly computing the numberse_i and with running timeO_r,H¡

logn(log logn)²log log logn¢ .

Next, we address how to determine whether (3) holds. Recall thatP is the set of prime divisors ofm₁m₂· · ·ms, and note that these primes are≤ r+ 1. The prime divisors ofm₀ are precisely the primes inP. We deduce that (3) holds if and only if

(4) ν_p(m₀) =ν_p(m_i) + min{ν_p(m₀), ν_p(e_i)}

for eachi ∈ {1,2, . . . , s} and for eachp ∈ P. For each primep ∈ P, we compute the values of ν_p(e_i), fori∈ {1,2, . . . , s}, by using thatν_p(e_i) =ν_p(e⁰_i). Next, we compute

ν_p(m₀) = min

1≤i≤s{ν_p(m_i) +ν_p(e_i)}.

Then we check if (4) holds. Observe that eachν_p(m_i)is either0or1, soν_p(m_i)can be computed by a simple division. We want also a method to compute ν_p(e_i) = ν_p(e⁰_i), for i ∈ {1,2, . . . , s}. We further need to explain the computation of ν_p(d_v −d_u)to obtain e⁰_i above. For U a positive integer andpa prime≤ r+ 1, the value ofν_p(U)can be computed as follows. We compute the values ofp²^j successively forj ≥ 0by squaring until we arrive at a positive integer t for which p²^t > U. Observe thatt = O(log logU). We setk₀ = 0. Forj ∈ {1,2, . . . , t}, we successively check if p²^t⁻^j|U and, if so, setk_j = k_j₋₁ + 2^t⁻^j and replace U with U/p²^t⁻^j. Ifp²^t⁻^j - U, then we set k_j = k_j₋₁. Then k_t = ν_p(U). Using this procedure, we can compute ν_p(U) in time O_r(logU(log logU)²log log logU). The theorem follows.

Although it does not affect our main results, it is of some value to note that the running time of the algorithm can be shown to be O_r¡

logn(log logn)²log log logn + logH¢

. Indeed, the