HAL Id: hal-01165972
https://hal.archives-ouvertes.fr/hal-01165972
Submitted on 21 Jun 2015
HAL
is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire
HAL, estdestinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Distributed under a Creative Commons
Attribution - NonCommercial - NoDerivatives| 4.0 International LicenseOn the distribution of the trace in the unitary symplectic group and the distribution of Frobenius
Gilles Lachaud
To cite this version:
Gilles Lachaud. On the distribution of the trace in the unitary symplectic group and the distribution
of Frobenius. Workshop on Frobenius Distributions on Curves, David Kohel and Igor Shparlinsky,
Feb 2014, Marseille, France. pp.194-231, �10.1090/conm/663/13355�. �hal-01165972�
IN THE UNITARY SYMPLECTIC GROUP AND THE DISTRIBUTION OF FROBENIUS
GILLES LACHAUD
Abstract. The purpose of this article is to study the distribution of the trace on the unitary symplectic group. We recall its relevance to equidistribution results for the eigenvalues of the Frobenius in families of abelian varieties over finite fields, and to the limiting distribution of the number of points of curves.
We give four expressions of the trace distribution ifg= 2, in terms of special functions, and also an expression of the distribution of the trace in terms of elementary symmetric functions. In an appendix, we prove a formula for the trace of the exterior power of the identity representation.
Contents
1. Introduction 1
2. The unitary symplectic group 3
3. Weyl’s integration formula 4
4. Equidistribution 7
5. Expressions of the law of the trace in genus 2 11
6. The Vi`ete map and its image 19
7. The symmetric alcove 23
8. Symmetric integration formula 26
Appendix A. The character ring ofG 31
References 35
1. Introduction
LetG be a connected compact Lie group, andπ :G−→ GL(V) a continuous representation ofGon a finite dimensional complex vector spaceV. The map
m7→τ(m) = Traceπ(m)
is a continuous central function onG, whose values lie in a compact intervalI⊂R. Thedistribution orlaw of τ is the measure µτ =τ∗(dm) onI which is the image
2010 Mathematics Subject Classification. Primary 11G20, 11G30, 11M50, 22E45; Secondary 05E05, 11K36, 14G10, 33D80, 60B20.
Key words and phrases. Curves and abelian varieties over finite fields, distribution of the trace of matrices, equidistribution, Frobenius operator, generalized Sato-Tate conjecture, Katz-Sarnak theory, random matrices, symplectic group, Weyl’s integration formula.
1
by τ of the mass one Haar measure dm on G. That is, for any continuous real functionϕ∈C(I), we impose the integration formula
Z
I
ϕ(x)µτ(x) = Z
G
ϕ(Traceπ(m))dm.
Alternately, ifx∈R, then
volume{m∈G | Traceπ(m)≤x}= Z x
−∞
µτ.
We are especially interested here with the group G = USp2g of symplectic unitary matrices of order 2g, withπequal to the identity representation inV =C4. With the help of Weyl’s integration formula, one establishes that the distributionµτ
has adensityfτ, that is, a positive continuous function such thatdµτ(x) =fτ(x)dx.
Our main purpose is the study offτ, especially in the case g= 2 andg = 3. For instance, forg= 2, we have
fτ(x) = 1 4π
1−x2
16 4
2F1 3
2,5
2; 5; 1−x2 16
if|x| ≤4, withGauss’ hypergeometric function 2F1 (see Theorem 5.2).
Another representation of the distribution of the trace, following a program of Kohel, is realized by theVi`ete map, which is the polynomial mapping
s(t) = (s1(t), . . . , sg(t)), t= (t1, . . . , tg),
wheresn(t) is the elementary symmetric polynomial of degreen. LetIg = [−2,2]g. Thesymmetric alcove is the set
Σg=s(Ig)⊂Rg,
which is homeomorphic to the g-dimensional simplex. By a change of variables in Weyl’s integration formula, one obtains a measureαxon the hyperplane section
Vx={s∈Σg | s1=x}
such that, if|x|<2g,
fτ(x) = Z
Vx
αx(s).
As a motivation for the study of these distributions, it is worthwhile to recall that they provide an answer to the following question:
Can one predict the number of points of a curve of given genus over a finite field?
When a curveCruns over the setMg(Fq) ofFq-isomorphism classes of (nonsingular, absolutely irreducible) curves of genus g over Fq, the number |C(Fq)| seems to vary at random. According to Weil’s inequality, an accurate approximation to this number is close toq+ 1, with a normalized “error” termτ(C) such that
|C(Fq)|=q+ 1−q1/2τ(C), |τ(C)| ≤2g.
The random matrix model developed by Katz and Sarnak gives many informations on the behaviour of the distribution of τ(C) on the set Mg(Fq). For instance, according to their theory, and letting g be fixed, for every x ∈ R, we have, as q→ ∞(cf. Corollary 4.3):
|{C∈Mg(Fq) | τ(C)≤x}|
|Mg(Fq)| = Z x
−∞
fτ(s)ds+O q−1/2
.
Hence, the knowledge offτ provides a precise information on the behaviour of the distribution of the number of points of curves.
The outline of this paper is as follows. After Section 2, devoted to notation, we recall in Section 3 the Weyl’s integration formula, expressed firstly in terms of the angles (θ1. . . , θg) defining a conjugacy class, and secondly in terms of the coefficientstj= 2 cosθj. We discuss equidistribution results for a family of curves or abelian varieties over a finite field in Section 4. In Section 5 we obtain four explicit formulas for the trace distribution ifg= 2, respectively in terms of hypergeometric series, of Legendre functions, of elliptic integrals, and of Meijer G-functions. We also give the distribution of the trace for the representation of the groupSU2×SU2
inUSp4.
In the second part of the paper, we take on a different point of view by using ele- mentary symmetric polynomials, and obtain a new expression of Weyl’s integration formula. Section 6 defines the Vi`ete map, asssociating to a sequence of coordinates the coefficients of the polynomial admitting as roots the elements of this sequence, and Section 7 describes the symmetric alcove, that is, the image of the set of “nor- malized real Weil polynomials” by the Vi`ete map. By a change of variables using the Vi`ete map, we obtain in Section 8 a new integration formula, which leads to another expression for the distribution of the trace on the conjugacy classes, in the casesg = 2 andg = 3. If g = 2, we compute also the trace of ∧2π. Finally, we include an appendix on the character ring of USp2g, including a formula on the exterior powers of the identity representation.
I would like to thank David Kohel for fruitful conversations. Also, I warmly thank the anonymous referee for carefully reading this work and for its suggestions, especially regarding the appendix.
2. The unitary symplectic group
The unitary symplectic group G =USp2g of order 2g is the real Lie group of complex symplectic matrices
G=
m∈GL2g(C) | tm.J.m=J andtm.m¯ =I2g , with
J =
0 Ig
−Ig 0
. Alternately, the elements ofGare the matrices
m=
a −¯b b ¯a
∈SU2g, a, b∈Mg(C).
The torusTg= (R/2πZ)g is embedded intoGby the homomorphism
(2.1) θ= (θ1, . . . , θg)7→h(θ) =
eiθ1 . . . 0 . . . .
0 . . . eiθg
0
0
e−iθ1 . . . 0 . . . .
0 . . . e−iθg
whose imageT is a maximal torus inG. The Weyl group W of (G, T) is the semi- direct product of the symmetric groupSgingletters, operating by permutations on theθj, and of the groupN of order 2ggenerated by the involutionsθj7→ −θj. Since
every element ofUSp2ghas eigenvalues consisting ofgpairs of complex conjugate numbers of absolute value one, the quotient T /W can be identified with the set ClGof conjugacy classes of G, leading to a homeomorphism
Tg/W −−−−→∼ T /W −−−−→∼ ClG
Remark 2.1. Here is a simple description of the set ClG. Let Φ2gbe the subset of monic polynomialsp∈R[u] of degree 2g, withp(0) = 1, with roots consisting ofg pairs of complex conjugate numbers of absolute value one. Ifθ∈Tg, let
pθ(u) =
g
Y
j=1
(u−eiθj)(u−e−iθj).
The mapθ7→pθ is a bijection fromTg/W to Φ2g. Renumbering, we may assume that
0≤θg≤θg−1≤ · · ·< θ1≤π.
The mapm7→cpm(u) = det(u.I−m) induces a homeomorphism ClG −−−−→∼ Φ2g
with cpm=pθ if and only if mis conjugate to h(θ). The polynomial pθ ispalin- dromic, that is, if
pθ(u) =
2g
X
n=0
(−1)nan(θ)un, thena2g−n(θ) =an(θ) for 0≤n≤g.
3. Weyl’s integration formula
The boxXg= [0, π]gis a fundamental domain forNinTgand the mapF 7→F◦h defines an isomorphism
(3.1) C(G)◦ −−−−→∼ C(Xg)sym
from the vector spaceC(G)◦ =C(ClG) ofcomplex central continuous functionson Gto the spaceC(Xg)symof complex symmetric continuous functions onXg. Notice that the isomorphism (3.1) has an algebraic analog, namely the isomorphism (A.1) in the appendix. Let dm be the Haar measure of volume 1 on G. If F∈C(G)◦, then
Z
ClG
F( ˙m)dm˙ = Z
G
F(m)dm,
where dm˙ is the image measure on ClG of the measuredm. The following result is classical [4, Ch. 9,§6, Th. 1, p. 337], [10, 5.0.4, p. 107].
Theorem 3.1 (Weyl integration formula, I). IfF∈C(G)◦, then Z
G
F(m)dm= Z
Xg
F◦h(θ))µg(θ), with the Weyl measure
µg(θ) =δg(θ)dθ, dθ=dθ1. . . dθg, δg(θ) = 1
g!
g
Y
j=1
2 π
(sinθj)2Y
j<k
(2 cosθk−2 cosθj)2.
We call the open simplex
(3.2) Ag={(θ1, . . . , θg)∈Xg | 0< θg< θg−1<· · ·< θ1< π}
thefundamental alcove in Xg. The closure ¯Ag ofAg is a fundamental domain for Sg in Xg, and, for everyf ∈C(Xg)sym, we have
Z
Xg
f(θ)dθ=g!
Z
Ag
f(θ)dθ.
There is another way to state Weyl’s integration formula, which will be used in Section 8. LetIg= [−2,2]g. The map
(θ1, . . . , θg)7→(2 cosθ1, . . . ,2 cosθg) defines an homeomorphismXg −→Ig. Let
k(t1, . . . , tg) =h
arccost1
2, . . . ,arccostg 2
.
whereh(θ) is given by (2.1). Then the mapF 7→F◦kdefines an isomorphism C(G)◦ −−−−→∼ C(Ig)sym
whereC(Ig)sym is the space of complex symmetric continuous function onIg. For an algebraic analog, see the isomorphism (A.2) in the appendix. Let
(3.3) D0(t) =Y
j<k
(tk−tj)2, D1(t) =
g
Y
j=1
(4−t2j).
Proposition 3.2 (Weyl integration formula, II). If F∈C(G)◦, then Z
G
F(m)dm= Z
Ig
F◦k(t)λg(t)dt, wheret= (t1, . . . , tg)anddt=dt1. . . dtg, with the Weyl measure
λg(t)dt, dt=dt1. . . dtg, λg(t) = 1
(2π)gg!D0(t)p D1(t).
Proof. Ifϕ∈C(Ig), we have Z
Xg
ϕ(2 cosθ1, . . . ,2 cosθg)δg(θ)dθ= Z
Ig
ϕ(t)λg(t)dt.
Apply Weyl’s integration formula of Theorem 3.1.
As in (3.2), we call the open simplex
(3.4) Ag={t∈Ig | −2< t1< t2<· · ·< tg<2}
thefundamental alcove inIg. Then ¯Ag is a fundamental domain ofIg forSg, and iff ∈C(Ig)sym, we have
(3.5)
Z
Ig
f(t)dt=g!
Z
Ag
f(t)dt.
Examples 3.3. We have
λ2(t) = 1
4π2(t1−t2)2 q
(4−t21)(4−t22).
The maximum ofλ2 inA2is attained at the point t0= (−√
2,√
2), with λ2(t0) = 2 π2. We have also
λ3(t) = 1
48π3(t1−t2)2(t1−t3)2(t2−t3)2 q
(4−t21)(4−t22)(4−t23).
The maximum ofλ3 inA3is attained at the point t0= (−√
3,0,√
3), with λ3(t0) = 9 2π3.
Now, for the convenience of the reader, we recall some notation on the distri- bution of central functions. Let Gbe a connected compact Lie group. The Haar measure dmof volume 1 onG is a probability measure, andG becomes a proba- bility space;ipso facto, its elements become random matrices, and the functions in C(G)◦are complex random variables onG. IfF∈C(G)◦is a real random variable, whose values lie in the compact interval I⊂R, the distribution orlaw ofFis the image measureµF =F∗dmonIsuch that
Z
I
ϕ(x)µF(x) = Z
G
ϕ(F(m))dm if ϕ∈C(I), IfB is a borelian subset ofI, then
µF(B) = volume{m∈G | F(m)∈B}, and thecumulative distribution function ofFis
ΦF(x) =P(F≤x) = Z x
−∞
µF(t) = Z
F(m)≤x
dm.
Thecharacteristic function ofFis the Fourier transform ofµF(x):
fbF(t) = Z ∞
−∞
eitxµF(x) = Z
G
eitF(m)dm= Z
Xg
eitF◦h(θ)µg(θ).
This is an entire analytic function oft, of exponential type, bounded on the real line. The distributionµF has adensity ifµF(x) =fF(x)dxwith a positive function fF in L1(I). IfµF has a density, and if Fourier inversion holds, then
fF(x) = 1 2π
Z ∞
−∞
fbF(t)e−itxdt.
Conversely, iffbF∈L1(R), thenµF has a density. IfG=USp2g, notice that Weyl’s integration formulas supply the joint probability density function for the random variables (θ1, . . . , θg) and (t1, . . . , tg).
The distributionµF is characterized by the sequence of its moments Mn(F) =
Z
I
xnµF(x) = Z
G
F(m)ndm, n≥1, and the characteristic function is a generating function for the moments:
(3.6) fbF(t) =
∞
X
n=0
Mn(F)(it)n n! .
Remark 3.4. Ifπis an irreducible representation ofG, with real characterτπ, then the random variable τπ is standardized,i.e. the first moment (the mean) is equal to zero and the second moment (the variance) is equal to one.
Remark 3.5. Under suitable conditions, an expression of the density by integration along the fibers can be given. For instance, let G=USp2g, letF∈ C(G)◦ be a C∞ function, and putJ =F◦h(U), whereU is the open box ]0, π[g. IfF◦his a submersion onU, and ifx∈J, then
Vx={θ∈U | F◦h(θ) =x}
is a hypersurface. Letαxbe theGelfand-Leray differential form onVx, defined by the relation
d(F◦h)∧αx=δg(θ)dθ1∧ · · · ∧dθg. For instance,
αx= (−1)j−1(∂(F◦h)/∂θj)−1δg(θ)dθ1∧. . . dθj−1∧dθj+1· · · ∧dθg
if the involved partial derivative is 6= 0. Then the distribution is computed by slicing: since the cumulative distribution function is
ΦF(x) = Z
F◦h(θ)≤x
µg(θ) = Z x
−∞
ds Z
Vs
αs(θ), we have
fF(x) = Z
Vx
αx(θ).
See [1, Lemma 7.2] and [17, Lemma 8.5].
4. Equidistribution
LetAbe an abelian variety of dimensiong over Fq. The Weil polynomial of A is the characteristic polynomialL(A, u) = det(u.I−FA) of the Frobenius endomor- phismFAofA, and theunitarized Weil polynomial ofAis
L(A, u) =¯ L(A, q−1/2u) =
g
Y
j=1
(u−eiθj)(u−e−iθj).
This polynomial has coefficients inZ, belongs to the set Φ2gdefined in Remark 2.1, andθ(A) = (θ1, . . . θg) is thesequence of Frobenius angles ofA. We write
L(A, u) =¯
2g
X
n=0
(−1)nan(A)un,
keeping in mind that a2g−n(A) =an(A) for 0≤n≤2g, since ¯L(A, u)∈Φ2g. By associating toAthe polynomial ¯L(A, u), each abelian variety defines, as explained in Section 2, a unique class ˙m(A) in ClG, such that
L(A, u) = det(u¯ I−m(A)).˙
Let Ag(Fq) be the finite set of k-isomorphism classes of principally polarized abelian varieties of dimensiong overk. The following question naturally arises:
Asq→ ∞, and asAruns over Ag(Fq), what are the limiting distributions of the random variables a1, . . . , ag ?
In order to clarify this sentence, we look in particular to the coefficient a1, and focus on the Jacobians of curves. Let C be a (nonsingular, absolutely irreducible, projective) curve overFq. The Weil polynomial L(C, u) ofC is the Weil polyno- mial of its Jacobian, and similarly for theunitarized Weil polynomial L(C, u), the¯ sequence of Frobenius angles θ(C), the coefficientsan(C), and the conjugacy class
˙
m(C). IfFC is the geometric Frobenius ofC, then
L(C, u) = det(u.¯ I−q−1/2FC) = det(uI−m(C)).˙ Then
(4.1) |C(Fq)|=q+ 1−q1/2τ(C), whereτ(C) =a1(C), namely
τ(C) =q−1/2TraceFC= 2
g
X
j=1
cosθj, withθ(C) = (θ1, . . . , θg).
ThenKatz-Sarnak theory [10] models the behavior of the Weil polynomial of a random curve C of genusg over Fq by postulating that when q is large, the class
˙
m(C) behaves like a random conjugacy class in ClG, viewed as a probability space, endowed with the imagedm˙ of the mass one Haar measure. Here is an illustration of their results. LetR(G) be the character ring ofG(cf. the appendix) and
T(G)◦=R(G)⊗C'C[2 cosθ1, . . .2 cosθg]sym
the algebra of continuous representative central functions onG, the isomorphism coming from Proposition A.1. This algebra is dense inC(G)◦, hence, suitable for testing equidistribution on ClG. We use the following notation for the average of a complex functionf defined over a finite set Z:
I
Z
f(z)dz= 1
|Z|
X
z∈Z
f(z).
For every finite fieldk, we denote byMg(k) the finite set ofk-isomorphism classes of curves of genusg over k. The following theorem follows directly, ifg≥3, from [10, Th. 10.7.15] (with a proof based on universal families of curves with a 3K structure), and from [10, Th. 10.8.2] if g ≤ 2 (with a proof based on universal families of hyperelliptic curves).
Theorem 4.1(Katz-Sarnak). Assumeg≥1. IfCruns overMg(Fq), the conjugacy classesm(C)˙ become equidistributed in ClGwith respect to dm˙ as q→ ∞. More precisely, ifF∈T(G)◦, then
I
Mg(Fq)
F( ˙m(C))dC = Z
ClG
F(m)dm+O q−1/2
.
Theorem 4.1 means that the counting measures µg,q= 1
|Mg(Fq)|
X
C∈Mg(Fq)
δ( ˙m(C)),
defined on ClG, converges todm˙ in the weak topology of measures when q→ ∞.
Since
F◦h(θ(C)) =F( ˙m(C)),
this theorem means also that if C runs over Mg(Fq), the vectors θ(C) become equidistributed in the fundamental alcove with respect to the Weyl measure when q→ ∞.
Remark 4.2. In the preceding theorem, and the above comments, one can substitute the setAg(Fq) to the setMg(Fq) [10, Th. 11.3.10]. This is an answer to the question raised in the beginning of this section.
As discussed above, the random variableτ(C) rules the number of points on the setMg(Fq), and its law is the counting measure on the closed interval [−2g,2g]:
νg,q= 1
|Mg(Fq)|
X
C∈Mg(Fq)
δ(τ(C)) =
2g
X
x=−2g
fg,q(x)δ(x), whereδ(x) is the Dirac measure atx, with the probability mass function
fg,q(x) =|{C∈Mg(Fq) | (τ(C) =x}|
|Mg(Fq)| , defined ifx∈[−2g,2g] andq1/2x∈Z. We put now
τ(m) = Tracem, τ◦h(θ) = 2
g
X
j=1
cosθj,
form∈Gand θ∈Xg. We takeF(m) =τ(m) in Theorem 4.1, and callµτ be the distribution of the central functionτ as defined at the end of Section 3. We obtain:
Corollary 4.3. If q → ∞, the distributions νg,q of the Frobenius traces converge to the distributionµτ. More precisely, for any continuous functionϕon [−2g,2g], we have
I
Mg(Fq)
ϕ(τ(C))dC= Z 2g
−2g
ϕ(x)µτ(x) +O q−1/2
, and for everyx∈R, we have
|{C∈Mg(Fq) | τ(C)≤x}|
|Mg(Fq)| = Z x
−∞
fτ(s)ds+O q−1/2
.
Lemma 4.4. If 1≤n≤2g−1, I
Ag(Fq)
an(A)dA=εn+O q−1/2
, whereεn= 1 if nis even andεn = 0if nis odd.
Proof. As Equation (A.4) in the appendix, let τn(m) = Trace(∧nm), in such a way thatτ1=τ. By equality (A.5), we have
an(A) =τn◦h(θ(A)).
Sinceτn∈T(G)◦, we have, by Remark 4.2, I
Ag(Fq)
τn◦h(θ(A))dA= Z
G
τn(m)dm+O q−1/2
but Lemma A.6 implies that the multiplicity of the characterτ0 of the unit repre- sentation1is equal toεn, hence,
Z
G
τn(m)dm=εn. Corollary 4.5. Let u∈Candq→ ∞.
(i) If |u|< q1/2, then I
Ag(Fq)
L(A, u)dA= u2g+2−qg+1 u2−q +O
qg−12 . (ii) We have
I
Ag(Fq)
|A(Fq)|dA=qg+O qg−1 . (iii) We have
I
Mg(Fq)
|C(Fq)|dC =q+O(1).
The implied constants depend only on g.
Proof. We have
L(A, u) =qgL¯
A, q−1/2u
=
2g
X
n=0
(−1)nan(A)q(2g−n)/2un, witha0= 1 anda2g−n =an for 0≤n≤g. From Lemma 4.4, we get
q(2g−n)/2un I
Fg(Fq)
an(A)dA=εnq(2g−n)/2un+unO
q(2g−n−1)/2 for 1≤n≤2g−1, and there is no second term in the right hand side ifn= 0 and n= 2g. Now, if|u|< q1/2,
2g
X
n=0
εnq(2g−n)/2un =u2g+2−qg+1 u2−q ,
and the absolute value of the difference between this expression and I
Ag(Fq)
L(A, u)dA is bounded by
B
2g−1
X
n=1
|u|nq(2g−n−1)/2,
with B depending only ong. If|u| ≤q1/2, then |u|nq(2g−n−1)/2 ≤q(2g−1)/2, and this proves (i). If|u| ≤1, then|u|nq(2g−n−1)/2≤qg−1, hence,
I
Ag(Fq)
L(A, u)dA=qg+O qg−1 .
and this proves (ii), since|A(Fq)|=L(A,1). Since Lemma 4.4 holds by substituting Mg to Ag, (iii) is a consequence of this lemma applied to a1(C), and of formula
(4.1).
With Corollary 4.5(i), it appears as though the Frobenius angles were close in the mean to the vertices of the regular polygon with (2g+ 2) vertices, inscribed in the circle of radiusq1/2, the points±q1/2 being excluded.
C Mg
Vertical
Horizontal
2 3 5 … p
m(Cp) µg,2 µg,3 µg,5 … µg,p
Figure 1. Horizontal versus vertical distribution.
Another approach on the limiting equidistribution of Frobenius angles is the generalized Sato-Tate conjecture, see [17] for a comprehensive description. Let C be an absolutely irreducible nonsingular projective curve of genusgoverQ, andS a finite subset of prime numbers such that the reductionCp=CFp overFp is good ifp /∈S. Then one says that the groupUSp2garises as the Sato-Tate group ofCif the conjugacy classes ˙m(Cp) are equidistributed with respect to the Weyl measure ofGwhenp→ ∞. In other words, this means that ifF∈C(G)◦, then
n→∞lim I
PS(n)
F( ˙m(Cp))dp= Z
G
F(m)dm,
where PS(n) = {p∈P\S | p≤n}. The case g = 1 is the Sato-Tate original conjecture, now a theorem. Here is an example of what one expects [12] :
Conjecture 4.6 (Kedlaya-Sutherland). If EndC(JacC) = Z, and if g is odd, or g= 2, org= 6, then the groupUSp2g arises as the Sato-Tate group ofC.
The two preceding types of equidistribution are symbolically shown in Figure 1.
The sequence of prime numbers are on the horinzontal axis, and the vertical axis symbolizes the space of curves. The Katz-Sarnak approach is figured as a (horizon- tal) limit of vertical averagesµp over vertical lines, and the Sato-Tate approach is a mean performed on horizontal lines.
5. Expressions of the law of the trace in genus 2
Assume now g = 2. Our purpose is to express the density of the distribution of the trace function τ on USp4 with the help of special functions. In order to
do this, the first step is to compute the characteristic function. The density of the Weyl measure onX2 is
δ2(θ1, θ2) = 2
π2
sin2θ1sin2θ2(2 cosθ2−2 cosθ1)2. The fundamental alcove is
A2={(θ1, θ2)∈X2 | 0< θ2< θ1< π}. The maximum ofδ2 inA2 is attained at the point
θm= (αm, π−αm), where tanαm 2 =
q 2 +√
3, δ(θm) = 128 27π2. We haveτ◦h(θ1, θ2) = 2 cosθ1+ 2 cosθ2, and the characteristic function ofτ is
fbτ(t) = Z
X2
e2it(cosθ1+cosθ2)δ2(θ1, θ2)dθ1dθ2. Proposition 5.1. For every t∈R, we have
fbτ(t) = 4J1(2t)2
t2 −6J1(2t)J2(2t)
t3 +4J2(2t)2 t2 . Here,J1andJ2areBessel functions.
Proof. Let
Va(x) = 25cos2xsin2x = 8 sin2(2x) Vb(x) = 25cos2xsin2xcos 2x = 4 sin 2xcos 4x Vc(x) = 25cos2xsin2xcos22x = 2 sin24x Then
32π2δ(x, y) =Vc(x)Va(y) +Va(x)Vc(y)−2Vb(x)Vb(y).
and
Fb(t) = 2Vba(t)Vbc(t)−2Vbb(t)2. But
Vba(t) =
√2
t J1(2t), Vbb(t) = i√ 2
t J2(2t), Vbc(t) =
√2
t J1(2t)− 3
√2t2J2(2t),
and the result follows.
We now compute the momentsMn(τ) ofτ. By Proposition 5.1, the characteristic function can be expressed by ageneralized hypergeometric series[9,§9.14, p. 1010]:
fbτ(t) =1F2
3
2; 3,4;−4t2
=
∞
X
n=0
(−1)n (32)n
(3)n(4)n
22n t2n n!,
where (a)n=a(a+ 1). . .(a+n−1) is the Pochhammer’s symbol. It then follows from (3.6) that the odd moments are equal to zero. Since
(32)n
(3)n(4)n = 24
√π
(n+12)Γ(n+12) Γ(n+ 3)Γ(n+ 4) and [9, p. 897]
Γ
n+1 2
=√
π2−2n (2n)!
n! ,
we obtain
M2n(τ) = 6.(2n)!(2n+ 2)!
n!(n+ 1)!(n+ 2)!(n+ 3)! forn≥0.
One finds as expected Mihailovs’ formula, in accordance with [12, §4.1], which includes another formula forfbτ(t), and also [17, p. 126].
In what follows, four different but equivalent expressions for the distribution of τ are given.
5.1. Hypergeometric series. An expression of the densityfτ of the distribution ofτ is the following. Recall thatGauss’ hypergeometric series
2F1(a, b;c;z) =
∞
X
n=0
(a)n(b)n (c)n
zn n!
converges if|z|<1 [9, §9.1.0, p. 1005].
Theorem 5.2. If |x|<4, we have fτ(x) = 1
4π
1−x2 16
4
2F1
3 2,5
2; 5; 1−x2 16
. This theorem immediately follows from the following lemma.
Lemma 5.3. If |x|<4, we have fτ(x) = 64
5π2m(x)4I(m(x)), where m(x) = 1−x2 16, and
I(m) = Z 1
0
t2
1−t2 1−mt2
52 dt.
Moreover
I(m) = 5π 256 2F1
3 2,5
2; 5;m
.
Proof. We use a formula of Schl¨afli, see [18, Eq. 1, p. 150]. If µ and ν are real numbers, then
Jµ(t)Jν(t) = 2 π
Z π/2 0
Jµ+ν(2tcosϕ) cos(µ−ν)ϕ dϕ (µ+ν >−1).
As particular cases of this formula, we get J1(t)2= 2
π Z 4
0
J2 ut
2
du
√16−u2 J1(t)J2(t) = 2
π Z 4
0
J3
ut 2
u 4
√ du
16−u2, J2(t)2= 2
π Z 4
0
J4 ut
2
du
√16−u2. By transferring these equalities in Proposition 5.1, we obtain
cfτ(t) = 4
t2J1(2t)2− 6
t3J1(2t)J2(2t) + 4 t2J2(2t)2
= 2 π
Z 4 0
4
t2J2(ut)− 3u
2t3J3(ut) + 4 t2J4(ut)
du
√16−u2.
and since
fτ(x) = 1 π
Z ∞ 0
cfτ(t) costx dt, we have
(5.1) fτ(x) = 2 π2
Z 4 0
√ du 16−u2
Z ∞ 0
4
t2J2(ut)− 3u
2t3J3(ut) + 4 t2J4(ut)
costx dt.
We use now aformula of Gegenbauer on the cosine transform, see [15, p. 409] and [18, Eq. 3, p. 50]. Assume Reν >−1/2, u >0 and let n be an integer ≥0. If 0< x < u, then
Z ∞ 0
t−νJν+2n(ut) costx dt= (−1)n2ν−1u−ν Γ(ν)
Γ(2ν+n)(u2−x2)ν−1/2C2nν x u
, whereCnν(x) is theGegenbauer polynomial. Ifu < x <∞, this integral is equal to 0. From Gegenbauer’s formula we deduce that if 0< x < u, then
Z ∞ 0
t−2J2(ut) costx dt=1 3
(u2−x2)3/2
u2 ,
Z ∞ 0
t−3J3(ut) costx dt= 1 15
(u2−x2)5/2
u3 ,
Z ∞ 0
t−2J4(ut) costx dt=−1 30
(u2−x2)3/2 u2
12x2 u2 −2
, sinceC22(x) = 12x2−2. Transferring these relations in (5.1), we get
5π2fτ(x) = 16 Z 4
x
(u2−x2)3/2 u2
√ du 16−u2
− Z 4
x
(u2−x2)5/2 u2
√ du
16−u2 −16x2 Z 4
x
(u2−x2)3/2 u4
√ du
16−u2, and this leads to
fτ(x) = 1 5π2
Z 4 x
(u2−x2)5/2 u4
p16−u2du.
By the change of variables u= 4p
1−m(x)t2, where m(x) = 1−x2 16. we obtain
fτ(x) = 64m(x)4 5π2
Z 1 0
t2
1−t2 1−m(x)t2
52
dt,
which is the first result. Euler’s integral representation of the hypergeometric series is
2F1(a, b;c;z) = Γ(c) Γ(b)Γ(c−b)
Z 1 0
tb−1(1−t)c−b−1 (1−tz)a dt
if Rez >0, and Rec >Reb >0. From this we deduce, with the change of variables t=u2, that
I(m) = 5π 256 2F1
3 2,5
2; 5;m
,
which is the second result.
5.2. Legendre function. Another expression offτ is given by theassociated Le- gendre function of the first kind Pab(z), defined in the half-plane Rez >1. Ifa is not an integer≥1, and ifb >3/2, this function is defined by [9, Eq. 8.702, p. 959]
:
Pab(z) = 1 Γ(1−a)
z+ 1 z−1
a2
2F1
−b, b+ 1; 1−a;1−z 2
. Ifa=mis an integer and ifz >1 is real, then [9, Eq. 8.711.2, p. 960] :
Pmb (z) = (b+ 1)a
π Z π
0
z+p
z2−1 cosϕb
cosmϕ dϕ.
Ifa= 0, this is theLaplace integral.
Theorem 5.4. If |x|<4, then fτ(x) =− 64 15π
p|x|
1−x2
16 2
P21 2
x2+ 16 4x
. Proof. By Theorem 5.2, we have
F(x) = 1
4πm(x)4 2F1
3 2,5
2; 5;m(x)
. But [15, p. 51]
2F1 3
2,5 2; 5;z
= (1−z)−3/42F1 3
2,7
2; 3;−(1−√ 1−z)2 4√
1−z
and [15, p. 47]
2F1
3 2,7
2; 3;z
= (1−z)−22F1
−1 2,3
2; 3;z
. On the other hand, ifz=m(x), then
−(1−√ 1−z)2 4√
1−z =−(x−4)2 16x . By the definition of Legendre functions,
P−21 2
1 2
x 4 + 4
x
= x−4
x+ 4 4
2F1(−1 2,3
2; 3;−(x−4)2 16x ), and this implies
fτ(x) = 4 π
√x
1−x2 16
2 P−21
2
1 2
x 4 + 4
x
. Since
Pmb (z) = Γ(b+m+ 1)
Γ(b−m+ 1)P−mb (z)
ifm∈Z, we obtain the required expression.
Since2F1(a, b;c; 0) = 1, we deduce from Theorem 5.2 that fτ(x) = 1
4π
1−x2 16
4
+O(x−4)5 and hence, in accordance with [17, p. 126]:
Corollary 5.5. If |x|= 4−ε, withε→0and ε >0, then fτ(x) = ε4
64π+O(ε5).
Since
x→0lim
√xP21
2
1 2(x
4 +4 x)
=−1 π,
we deduce from Proposition 5.4 that the maximum offτ is reached forx= 0, and fτ(0) = 64
15π2 = 0.432. . .
The graph offτ is given in Figure 2 ; we recover the curve drawn in [12, p. 124].
-4 -2 2 4
0.1 0.2 0.3 0.4
Figure 2. Density of the distribution ofτ, caseg= 2.
5.3. Elliptic integrals. Another expression of fτ is given by Legendre elliptic integrals. Let
K(m) = Z π/2
0
dϕ
p1−msin2ϕ, E(m) = Z π/2
0
q
1−msin2ϕ dϕ,
be the Legendre elliptic integrals of first and second kind, respectively. The imple- mentation offτ in theMaple software gives:
Corollary 5.6. If |x|<4, then 15
64π2fτ(x) = (m2−16m+ 16)E(m)−8(m2−3m+ 2)K(m),
wherem= 1−(x2/16).
The mention of the existence of such a formula is made in [7].
5.4. MeijerG-functions. Another expression offτ is given byMeijerG-functions [9,§9.3, p. 1032]. They are defined as follows : take z in Cwith 0<|z|<1 and m, n, p, q inN. Then
Gm,np,q z
a1, . . . , ap
b1, . . . , bq
!
= 1 2iπ
Z
C
Qm
k=1Γ(s+bk) Qp
k=n+1Γ(s+ak). Qn
k=1Γ(−s−ak+ 1) Qq
k=m+1Γ(−s−bk+ 1)z−sds Here, a1, . . . , ap, b1, . . . , bq are a priori in C, and C is a suitable Mellin-Barnes contour.
Corollary 5.7. If |x|<4, fτ(x) = 6
πG x2
16
, with G(z) =G2,02,2 z
5 2, 72 0, 1
! . We have
G(z) = 1 2iπ
Z
Res=c
Γ(s)Γ(s+ 1) Γ s+52
Γ s+72z−sds, with0< c < 12.
Proof. If|z|<1, then [19, 07.34.03.0653.01]:
G2,02,2
z
a, c b,−a+b+c
=
√π
Γ(a−b)(1−z)a−b−12z14(−2a+2c−1)+bP−a+b+
1 2
−a+c−12
z+ 1 2√
z
and the left hand side is equal to zero if|z|>1. Hence, if|z|<1, G2,02,2 z
5 2, 72 0, 1
!
= 4
3(1−z)2z1/4 P−21/2 z+ 1
2√ z
,
and we apply Theorem 5.4.
Corollary 5.8. If |x|<4, then the repartition function ofτ is Φτ(x) =3x
πG x2
16
+1 2, with
G(z) =G2,13,3 z
1
2, 52, 72 0, 1, −12
! . Proof. According to [19, 07.34.21.0003.01], we have
Z
zα−1Gm,np,q z
a1, . . . , ap
b1, . . . , bq
!
dz=zαGm,n+1p+1,q+1 z
1−α, a1, . . . , ap
b1, . . . , bq,−α
! . By Corollary 5.7, a primitive offτ is
Φ0(x) = 6 π
Z
G2,02,2 x2 16
5 2, 72 0, 1
!
= 3x
πG2,13,3 x2 16
1
2, 52, 72 0, 1, −12
! ,
and Φ0(−4) =−1/2.
-4 -2 2 4 0.2
0.4 0.6 0.8 1.0
Figure 3. Repartition function ofτ
5.5. The trace in SU2×SU2. In order to present a comparison with the above results, we give here without proof the distribution of the trace of a compact semi- simple subgroup of rank 2 ofUSp4, namely, the group SU2×SU2. If
x= (x1, x2) and xi=
ai −¯bi
bi ¯ai
∈SU2, |ai|2+|bi|2= 1, i= 1,2, the map
π(x) =
a1 0 −¯b1 0 0 a2 0 −¯b2
b1 0 ¯a1 0 0 b2 0 ¯a2
defines an embedding
π:SU2×SU2 −−−−→ USp4 whose image contains the maximal torusT ofUSp4. We put
ρ(x) = Traceπ(x).
The characteristic function ofρ is the square of the characteristic function of the distribution of the trace ofSU2:
cfρ(t) = J1(2t)2 t2 .
The even moments are equal to zero, and the odd moments are M2n(ρ) =CnCn+1= 2(2n)!(2n+ 1)!
(n!)2(n+ 1)!(n+ 2)!. where
Cn= 1 n+ 1
2n n
is thenthCatalan number. One finds the sequence 1,0,2,0,10,0,70,0,588,0,5544. . . in accordance with the sequence A005568 in the OEIS [16].
Theorem 5.9. If |x|<4, the density of the distribution ofρ is fρ(x) = 1
2π
1−x2 16
2
2F1
1 2,3
2; 3; 1−x2 16
.
Corollary 5.10. If |x|= 4−ε, withε→0 andε >0, then fρ(x) = ε2
8π− ε3
64π− ε4
4096π +O(ε5).
The maximum offρ is reached forx= 0, and fρ(0) = 8
3π2 = 0.270. . . The graph offρ is given in Figure 4.
-4 -2 2 4
0.05 0.10 0.15 0.20 0.25
Figure 4. Density of the distribution ofρ.
6. The Vi`ete map and its image
Another approach of the distribution of the trace is realized by an algebraic form of Weyl’s integration formula, using symmetric polynomials. This comes from a general program developed by Kohel [13], formerly outlined by DiPippo and Howe in [6]. Ift= (t1, . . . , tg)∈Cg, consider a monic polynomial
(6.1) ht(u) = (u−t1). . .(u−tg) =ug−s1(t)ug−1+· · ·+ (−1)gsg(t) inC[u]. Here
sn(t) = X
i1<···<in
ti1. . . tik
is theelementary symmetric polynomialof degreeningvariables. The discriminant ofhtis
(6.2) discht=D0(t) =Y
j<k
(tk−tj)2.
TheVi`ete map s:Cg−→Cg is the surjective polynomial mapping s(t1, . . . , tg) = (s1(t), . . . , sg(t)),
wheret= (t1, . . . , tg), inducing a bijection Cg/Sg
−−−−→∼ Cg
which is a homeomorphism, because the map between the corresponding projective spaces is a continuous bijection between compact spaces. Hence, the Vi`ete map is open and proper. We denote by
Πg=s(Rg)
the closed subset which is the image of the Vi`ete map. Hence, (s1, . . . , sg)∈Πg if and onlyht(u) has only real roots. The induced map
Rg/Sg −−−−→∼ Πg
is a homeomorphism. Thefundamental chamber ofRg related toSg is Cg={t∈Rg | t1< t2<· · ·< tg}
and ¯Cg is a fundamental domain for Sg in Rg. We are going to show thats is a local diffeomorphism at the points of an open dense subset ofRg. For this purpose, we calculate J(s), where J(f) denotes the jacobian matrix of a polynomial map f:Cg−→Cg. Recall that the power sums
pn(t) =tn1+· · ·+tng (n≥1)
can be expressed in terms of elementary symmetric polynomials. Precisely, from Newton’s relations
pn=
n−1
X
j=1
(−1)j−1sjpn−j+ (−1)n−1nsn (n≥1), we obtain [14, p. 28] :
pn =
s1 1 0 . . . 0
2s2 s1 1 . . . 0 . . . . nsn sn−1 sn−2 . . . s1 .
This is related to a more suitable expression [5, p. 72], [2, Ch. IV, § 6, Ex. 6], obtained by Albert Girard [8] in 1629, and sometimes attributed to Waring (1762):
Proposition 6.1 (Girard’s formula). If1≤n≤g ands= (s1, . . . , sg), let vn(s) =n X
b∈Pn
(b1+b2+· · ·+bg−1)!
b1!. . . bg! ub11. . . ubgg,
whereun= (−1)n−1sn for1≤n≤g, and the summation being extended to Pn={b= (b1, . . . , bg)∈Ng | b1+ 2b2+· · ·+gbg=n}. Then
pn=vn◦s.
The mapϕ7→ϕ◦sdefines an isomorphism
s∗:Z[s1, . . . , sg] −−−−→∼ Z[t1, . . . , tg]sym.
SinceD0∈Z[t1, . . . , tg]sym, there is a polynomiald0∈Z[s1, . . . , sg] such that (6.3) d0(s(t)) =D0(t) =Y
j<k
(tk−tj)2.
Let
Ug={t∈Rg | D0(t)6= 0}, Π◦g={s∈Rg | d0(s)6= 0}.
Then Π◦g=s(Ug), and Π◦g is a dense open set of Πg. The roots of the polynomial ht∈R[u] given by (6.1) are real and simple if and only ifs(t)∈Π◦g.
Proposition 6.2. With the preceding notation:
(i) If t∈Rg, then
|detJ(s)(t)|=p
D0(t) =Y
j<k
|tk−tj|.
(ii) The map sis a local diffeomorphism at every point ofUg.
(iii) The map sis a diffeomorphism from the fundamental chamberCg toΠ◦g. Proof. Define two polynomial maps fromCg toCg:
p(t) = (p1(t), . . . , pg(t)) and v(s) = (v1(s), . . . , vg(s)).
Thenp=v◦sby Girard’s formula 6.1. If 1≤n≤g, then vn(s) = (−1)n+1nsn+vn0(s),
wherevn0(s) depends only ofs1, . . . , sn−1. This implies thatJ(v) is lower triangular, withn-th diagonal term equal to (−1)n+1n. Hence,
detJ(v) = (−1)[g/2]g!
On the other hand,
J(p) =
1 1 . . . 1
. . . . ktk−11 ktk−12 . . . ktk−1n
. . . . gtg−11 gtg−12 . . . gtg−1n
.
ThenJ(p) =D.V(t), whereD is the diagonal matrix diag(1,2, . . . , g), and
V(t) =
1 1 . . . 1
t1 t2 . . . tn
. . . . tg−11 tg−12 . . . tg−1n
is the Vandermonde matrix. Hence,
detJ(p) =g! detV(t) =g!Y
j<k
(tk−tj),
and sinceJ(p) =J(v).J(s), we get (i), which implies (ii). Then (iii) comes from the fact thatsis injective on the open subsetCg ofUg.
Thebezoutian ofhtis the matrix
B(t) =V(t).tV(t) =
p0 p1 . . . pg−1
p1 p2 . . . pg
. . . . pg−1 pg . . . p2g−2
∈Mg(R),
in such a way that detB(t) =D0(t).
Lemma 6.3. Let ht∈R[u]. The following conditions are equivalent: