Ergodic non-abelian smooth extensions of an irrational rotation

HAL Id: hal-00794134

https://hal-upec-upem.archives-ouvertes.fr/hal-00794134

Submitted on 25 Feb 2013

HAL

is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire

destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

rotation

Julien Brémont

To cite this version:

Julien Brémont. Ergodic non-abelian smooth extensions of an irrational rotation. Journal of the

London Mathematical Society, London Mathematical Society, 2010, 81 (1), pp.457-476. �hal-00794134�

Ergodic non-abelian smooth extensions of an irrational rotation

Julien Br´emont

Abstract

Above an irrational rotation on the Circle, we build optimally smooth ergodic cocycles with values in some nilpotent or solvable subgroups of triangular matrices.

1. Introduction

Let (X,B, T, µ) be an invertible dynamical system, where (X,B) is a standard Borel space,T an invertible measurable transformation ofX andµaT-invariant ergodic Borel probability measure.

LetGbe a locally compact separable group. We suppose that the topology onGis defined by some fixed metric and we writeV_ε(c) for the open ball of radiusε >0 and centerc∈G. In the sequelG isR^d or a closed subgroup ofGL_d(R).

To a measurable function ϕ:X →G one canonically associates the cylindrical or skew-product transformationT_ϕ: (x, g)7−→(T x, ϕ(x)g) onX×G. Iterates ofT_ϕ involve the cocycle (ϕ_n)_n∈Z :

ϕ_n(x) =







ϕ(Tⁿ⁻¹x)· · ·ϕ(x), n >0,

e, n= 0,

ϕ⁻¹(Tⁿx)· · ·ϕ⁻¹(T⁻¹x), n <0.

We next use the notationT ϕforϕ◦T. WhenGis abelian, the law inGis written additively and ϕn is then the Birkhoff sumϕn(x) =Pn−1

k=0ϕ(T^kx), forn≥1.

Fixing a left-invariant Haar measureλonG, the product measureµ⊗λonX×GisTϕ-invariant.

We consider in this article the existence and the construction ofϕsuch thatTϕis ergodic forµ⊗λ.

In the whole text, the ergodicity of Tϕ only refers to µ⊗λ. Such ϕ provide ergodic dynamical systems, of particular interest whenGis non-compact since the invariant measure is infinite.

The existence of ϕ:X→G with Tϕ ergodic requires that G is amenable, cf Golodets and Sinel’shchikov [9]. A problem is then to exhibit concrete examples and a large part of the literature on cocycles is devoted to expliciting classes ofϕ:X →Gsuch thatT_ϕis ergodic for specific groups and dynamical systems. The case whenG=R,X is the Circle T¹=R\Zand T is an irrational rotation R_α of angle α has received considerable attention, see for instance the bibliography in Conze [4]. Most examples are discontinuous and piecewise smooth. A non-commutative situation has recently been considered by Greschonig [10], when (X, T) = (T¹, R_α) and G is the discrete Heisenberg group N3(Z). An ergodic BV example is built, using linear combinations of interval indicator functions, for anyαwith bounded partial quotients.

When (X, T) and G have a smooth structure, it is natural to ask for smooth examples. For a large class of topological dynamical systems and amenable groups, Nerurkar [15] has first given genericity results. More precisely, among continuousϕ, the ones for whichTϕis ergodic are generic in the Baire Category sense. Producing ϕ:X →G with an a priori smoothness so that Tϕ is ergodic is often delicate and can be impossible. In the situation when (X, T) admits fast periodic approximations, Nerurkar [16] proved genericity results on ergodic cocycles with high regularity.

ForG=Rand (X, T) = (T¹, Rα), existence results, based on tower constructions, about ergodic cocycles with optimal smoothness and other properties are contained in Voln´y [20].

Let us now describe the content of this article. We fix the base to (X, T) = (T¹, R_α). When G=R^dand for a generalαwith unbounded partial quotients, the optimal regularity ofϕ:T¹→R^d

2000Mathematics Subject Classification37A20, 37E10.

with Tϕ ergodic isC¹, since C^1+ε-extensions are not ergodic if αhas Diophantine type 1. This is recalled below. The condition on αfor aC¹ regularity is minimal, since ergodicC¹-extensions are impossible ifαhas bounded partial quotients.

The purpose of this paper is to prove similar results in some non-abelian situations. More precisely, under the same hypothesis onα, we buildC¹ ergodic cocycles in the following closed subgroups of the group of invertibled×dupper-triangular matrices :

– the nilpotent non-abelian group Nd(R) of matrices with unit diagonal elements, when d≥3, – the solvable non-nilpotent groupSol3(R), whend= 3.

In these cases, the optimal regularity of an ergodic extension is thus the same as forR^d. Our strategy is semi-abstract and therefore essentially furnishes existence results. We develop via elementary Fourier Analysis a general method for building smooth ergodic cocycles by explicit formulas in terms of the sequence of convergents ofα.

2. Preliminaries

We first introduce the notion of essential value, used for considering the ergodicity of a skew- product. We next present the basic stone with which our examples are built and, in a first application, focus on the situation whenG=R^d.

2.1. Essential values ; a general lemma for abelian groups

In the general context of the beginning of the introduction, the question of the ergodicity of a skew-product can be reformulated using essential values. See Schmidt [17] or Feldman-Moore [7].

Letϕ:X →Gbe measurable.

Definition 2.1. An elementc∈G∪ {∞}is an essential value forϕ, if for any ε >0and Borel set A⊂X withµ(A)>0, there isn∈Zso thatµ(A∩T⁻ⁿA∩ {ϕn∈Vε(c)})>0.

The set of finite essential values is a closed subgroup of G, denoted by E(ϕ) in the sequel. The skew-productT_ϕonX×Gis then ergodic if and only ifE(ϕ) =G.

When G is abelian, the non-degeneracy of the cocycle along a rigid sequence provides finite essential values. Minimal translations on a torus furnish a natural situation for this to occur.

The following lemma is prop 9 in Lema´nczyk-Parreau-Voln´y [13]. See also the notion of quasi-period in [4]. Reordering arguments appearing in the literature, we give another proof.

Lemma 2.2.

LetGbe abelian andϕ:X→Gbe measurable. If(kn)→+∞verifiesT^knx→x,µ−ae, and the law ofϕk_n converges to a measureν for the vague topology, then Supp(ν)⊂ E(ϕ).

Proof of the lemma :

Starting as in [4], for a Borel setA: Z

X

|1_A−1_T−knA|dµ−→_n→+∞0. (2.1)

Indeed, this is true when 1A is replaced by a continuous function. This property is extended to functions inL¹(µ) via the invariance of the measureµ.

Following next [17], ifF is compact and such thatF∩ E(ϕ) = ø, then for allB withµ(B)>0, there isA⊂B withµ(A)>0 such that for alln∈Z,µ(A∩T⁻ⁿA∩ {ϕn∈F}) = 0. This is where the abelian character ofGis used. By (2.1), we getµ(A∩ {ϕkn∈F})→0, asn→+∞.

Via the exhaustion lemma 1.0.7 in Aaronson [1], we obtain X =∪_l≥1A_l moduloµ, whereµ(A_l∩ {ϕ_kn∈F})→0 for eachl≥1. Therefore :

µ{ϕk_n∈F} →n→+∞0. (2.2) Suppose then thatc∈Supp(ν)\E(ϕ), whereν is the vague limit of the laws ofϕ_kn. Since E(ϕ) is closed, there isε >0 such thatVε(c)∩ E(ϕ) = ø. Fix a continuous ψ:G→[0,1] so thatψ >0 onV_ε/4(c) andψ= 0 outsideV_ε/2(c). By (2.2),µ{ϕk_n ∈V¯_ε/2(c)} →0. However :

µ{ϕk_n∈V¯ε/2(c)} ≥ Z

X

ψ(ϕk_n)dµ→ Z

G

ψ dν >0.

This contradiction concludes the proof of the lemma.

Remark —A sequence (kn) as in the statement of the lemma is said “rigid”. Recall that for the vague topology a sequence of probability measures is relatively compact among Borel measures with mass less than or equal to one.

In the sequel, except for section 2.5, X is the torus T¹ identified to R/Z, with an irrational rotationT of angleαand Lebesgue measureµ.

2.2. Squashability

In most cases, we also examine the squashability of (X×G, Tϕ). This question comes after ergodicity. Following Aaronson [1,2], an infinite ergodic dynamical system (Y, S, ν) is squashable, if there exists a non-singular non-measure-preserving transformationQcommuting withS. In this case, since ν◦Q⁻¹ is S-invariant and equivalent to ν, there is a constant c such thatν◦Q⁻¹= cν, with therefore c6= 1. The dynamical system (Y, S, ν) is completely squashable if anyc6= 1 is possible.

Squashability is an obstruction to the existence of a generalised law of large numbers, ie of a measurable Ψ :{0,1}^N→R⁺ such that ν(A) = Ψ((1A(Sⁿω))_n≥1), ν−ae, for all measurable set A. Indeed, reproducing the proof in [1], takeAwith positive finite measure andωsuch thatωand Qωareν-typical. A contradiction is obtained as follows :

ν(A) = Ψ((1A(SⁿQω))) = Ψ((1A(QSⁿω))) = Ψ((1_Q⁻¹_A(Sⁿω))) =ν(Q⁻¹A) =cν(A).

In our setting we have the following result from [1] (proposition 8.4.1) : Proposition 2.3.

LetX =T¹ with an irrational rotation T of angleαand Lebesgue measure µ. LetGbe a locally compact second countable group andϕ:X →Gbe measurable. IfTϕis ergodic andQ:X×G→ X×Gis non-singular withQ◦Tϕ=Tϕ◦Q, then there existβ∈R, a continuous surjective group endomorphismw:G→Gand a measurablef :X→Gso that :

Q(x, y) = (x+β, f(x)w(y))andϕ(x+β) =T f(x)w(ϕ(x))f⁻¹(x), for allx∈X,y∈G.

The proof in [1] is given when Gis abelian. Using the notations of [1], it remains valid in general when taking the translationsQg(x, y) = (x, yg) to show that F⁻¹.F◦Qg isTϕ-invariant.

2.3. Smooth ergodic cocycles inR^d

In the present paragraphG=R^d. The sequence of convergents of αis denoted by (p_n/q_n) and the partial quotients by (a_n). Recall that they obey the recursive relations :







p_n+1=a_n+1p_n+p_n−1 andq_n+1=a_n+1q_n+q_n−1,n≥0, p−1= 1, p0= 0, q−1= 0, q0= 1.

We sayαhas bounded partial quotients (bpq), ifan≤Cfor some constantC >0. We write ((x)) for the distance fromxtoZ. Recall also that 1/(qn+qn+1)≤((qnα))<1/qn+1.

The constructions that appear in this article are essentially variations on a single example.

Consider positive integers (uk)_k≥1withP

1/uk<∞and a subsequence (vk)_k≥1of the (qn). Define : f(x) =X

k≥1

1

u_k sin(2πvkx).

Introducing a sequence of times (t_n), we computef_tn : f_tn(x) = ImX

k≥1

e^2iπvkx uk

e^2iπvktnα−1 e^2iπvkα−1

=X

k≥1

sin(πv_kt_nα)

uksin(πvkα)sin[πv_k(2x+ (t_n−1)α)]

=X

k<n

+X

k=n

+X

k>n

=An+Bn+Cn. (2.3) An existence result, classical when d= 1, is the following :

Proposition 2.4.

i) Letαhave nonbpq. Choose a strictly increasing sequence(θ(k))_k≥1so thata_θ(k)+1≥k³. Set : f = (fⁱ)1≤i≤d, wherefⁱ(x) =X

k≥1

sin(2πq_θ(dk+i)x) k²q_θ(dk+i) . Thenf isC¹ andT_f :T¹×R^d →T¹×R^d is ergodic and non-squashable.

ii) Let α have bpq. Fix a decreasing functionε such that ε(h)→+∞, ash→0⁺, and choose a strictly increasing sequence(θ(k))_k≥1 satisfyingθ(k)≥k² andε(1/q_θ(k))≥k. Set :

f = (fⁱ)_1≤i≤d, wherefⁱ(x) =X

k≥1

sin(2πq_θ(dk+i)x) q_θ(dk+i) .

Then hε(h) is a modulus of continuity for f and Tf:T¹×R^d →T¹×R^d is ergodic and non- squashable.

Proof of the proposition :

i) We fix 1≤j≤dand consider the rigid sequence (t_j,n)_n≥1= (n²q_θ(dn+j))_n≥1. For each 1≤i≤d, we focus on the decomposition (Aⁱ_n, Bⁱ_n, C_nⁱ) associated tofⁱ and given by (2.3).

Using that q_θ(k+1)/q_θ(k)→+∞, ask→+∞, we have for a generic constantC >0 :

|Aⁱ_n| ≤CX

k<n

n²((q_θ(dn+j)α))

k²((q_θ(dk+i)α)) ≤Cn²qθ(d(n−1)+i)+1

q_θ(dn+j)+1 ≤C n²

a_θ(dn+j)+1 →_n→+∞0,

|C_nⁱ| ≤CX

k>n

n²q_θ(dn+j)

k²q_θ(dk+i) ≤C q_θ(dn+j) qθ(d(n+1)+i)

≤ C

a_θ(dn+j)+1 →_n→+∞0,

due to the assumption onθand since in each case the argument ofθin the numerator is strictly less than that in the denominator. NextBⁱ_n=w_nⁱ sin[πq_θ(dn+i)(2x+ (t_j,n−1)α)], with the expression :

wⁱ_n= sin(πn²qθ(dn+j)qθ(dn+i)α)

n²q_θ(dn+i)sin(πq_θ(dn+i)α) ∼n→+∞εn(i, j) πn²qθ(dn+min(i,j))((qθ(dn+max(i,j))α))

πn²q_θ(dn+i)((q_θ(dn+i)α)) , (2.4) whereεn(i, j) =±1. The point is that|wⁱ_n| →1 ifi=jandwⁱ_n→0 otherwise. As a result, ifX is a random variable with uniform law on [0,1] :

ft_j,n →_n→∞(1i=jsin(2πX))_1≤i≤d, in law.

Lemma 2.2 then implies that{0}^j−1×[−1,1]× {0}^d−j ⊂ E(f). Since this holds for all 1≤j≤d, we obtainE(f) =R^d and thusT_f is ergodic.

If T_f is squashable, by proposition 2.3 there exists a d-square invertible matrix M satisfying

|detM| 6= 1, a measurableψ:T¹→R^d andβ such that for almost-everyx∈T¹ : g(x) :=M f(x)−f(x+β) =ψ(x)−ψ(x+α).

Fix 1≤j≤dand consider (kn) so thatq_θ(dkn+j)β→ζj mod (1) andw_k^j

n→δ∈ {±1}. Then : (ft_j,kn(.), ft_j,kn(.+β))→_n→+∞δ(1i=j(sin(2πX),sin(2π(X+ζj))))_1≤i≤d, in law.

Writing (e_i)_1≤i≤d for the canonical basis ofR^d, by lemma 2.2, for allc∈[0,1] : δsin(2πc)M ej−δsin(2π(c+ζj))ej∈ E(g).

Since R^d is abelian and g is a T-coboundary, we have E(g) ={0}, cf [17]. As a result M ej is proportional toejandMjjsin(2πc) = sin(2π(c+ζj)), for allc∈[0,1]. Necessarily,Mjj =±1. Since this holds for all 1≤j≤d,M is diagonal and detM =±1, but this is a contradiction.

ii) Concerning the regularity of fⁱ, let h >0 be small so that 1/q_θ(dl+i+1)≤h <1/q_θ(dl+i), for somel≥1. Splittingf(x)−f(x+h) into P

k≤l andP

k>l, we obtain for a generic constantC :

|f(x)−f(x+h)| ≤C(hl+h)≤Chl≤Chε(1/q_θ(dl+i))≤Chε(h).

For the ergodicity and the squashability ofT_f we run the same proof with the sequence of times (t_j,n)_n≥1= (q_θ(dn+j))_n≥1, where 1≤j≤dis fixed. Sinceq_θ(k+1)/q_θ(k)→+∞, ask→+∞, we only have to focus on the new (wⁱ_n)_1≤i≤d given by :

w_nⁱ = sin(πq_θ(dn+j)q_θ(dn+i)α) q_θ(dn+i)sin(πq_θ(dn+i)α).

As in (2.4),wⁱ_n→_n→+∞0, wheni6=j. Sinceαhas bpq, letM check a_n≤M for alln≥1. Using thatq_l+1≤(M + 1)q_l, we haveq_n((q_nα))∈[1/(M + 2),1/(1 + 1/(M + 1))]. As a result :

1 πsin

π M+ 2

≤lim inf|w^j_n| ≤lim sup|w_n^j| ≤ (M+ 2)

π .

Consequently, (w^j_n)_n≥1converges to a non-zero constant along a subsequence (kn). The reasoning is then the same, using this time the sequence (tj,k_n). This completes the proof of the proposition.

Remark 1 — Results related to item i) when d= 1 can be found in [20], where the method is developed in order to produce completely squashable examples. In the context ofii), for instance, Liardet and Voln´y [14] have constructed ergodic cocycles with an absolutely continuousf. Remark 2 —When (T¹, T) admits fast periodic approximations, in the spirit of [16] or [20], then the functionf :T¹→R^d can be taken more regular :

– of class C^p, if lim supq_n+1/q^p_n= +∞, for some integer p≥1. Indeed, choose (θ(k)) with q_θ(k)+1≥k³q_θ(k)^p and define :

f = (fⁱ)1≤i≤d, withfⁱ(x) =X

k≥1

k⁻²q_θ(dk+i)^−p sin(2πqθ(dk+i)x).

ThenTf :T¹×R^d→T¹×R^dis ergodic and non-squashable. The same demonstration works, when taking (n²q_θ(dn+j)^p ) as rigid sequences.

– of classC^∞, if lim supqn+1/q^p_n= +∞, for allp≥1. Choose this time (θ(k)) satisfyingq_θ(k)+1≥ k³q^k_θ(k) and introduce :

f = (fⁱ)_1≤i≤d, withfⁱ(x) =X

k≥1

k⁻²q_θ(dk+i)^−k sin(2πq_θ(dk+i)x).

Then Tf :T¹×R^d→T¹×R^d is ergodic and non-squashable. Take (n²qⁿ_θ(dn+j)) as rigid sequences.

– real analytic, if lim sup logq_n+1/q_n= +∞. Choose (θ(n)) checking q_θ(n)+1/(q_θ(n)e^nqθ(n))→ +∞and define :

f = (fⁱ)_1≤i≤d, withfⁱ(x) =X

k≥1

κ⁻¹_k,isin(2πq_θ(dk+i)x),

where κ_k,i =q_θ(dk+i)[exp{(dk+i)q_θ(dk+i)}] and [x] is the integer part of x. Each fⁱ is real analytic, sincez7−→P

k≥1κ⁻¹_k,iexp (zq_θ(dk+i)) uniformly converges on compact sets of Cand Tf :T¹×R^d→T¹×R^d is ergodic and non-squashable, using now (κn,j) as rigid sequences.

Remark 3 —Fixingd= 1, we recall a few facts about coboundaries. Ifαhas bpq andf⁰ ∈L²(T¹), thenf is aL²-coboundary, in particular whenf isC¹. Ifαhas Diophantine type 1 andf isC^1+ε, thenf is a continuous coboundary (cf Arnold [3]). A result of Meyer (cf Herman [11], p. 187) says that for any irrationalαthere isf of classC¹ that is not a continuous coboundary.

2.4. Adequate perturbations for some BV functions

Consider the same context withd= 1 and any irrationalα. The function : h(x) =X

l≥1

q⁻¹_l2 sin(2πq_l2x)

is an example such that (hn) converges in law along a subsequence of the (qn) to a random variable whose support contains a non-empty open interval. Such a map can be used as a “good” perturbing element for a large class of BV functions.

Proposition 2.5.

Consider (a_k)_k≥1∈l¹(N)and intervals(I_k)_k≥1in T¹. Setγ=P

k≥1a_kµ(I_k)and define : f =X

k≥1

ak1I_k−γ.

Introduce the compact K={Pakuk |uk∈Z∩[−2,2]}. We have :

– If Khas empty interior, then Tf+εh:T¹×R→T¹×Ris ergodic for anyε6= 0.

– If the box-dimension of Kis <1/2, thenTf+εhis ergodic and non-squashable for any ε6= 0.

Proof of the proposition :

Since Var(1I_k) = 2, the Denjoy-Koksma inequality gives :

(1_Ik−µ(I_k))_qn(x) =m_k,n(x)− {q_nµ(I_k)} ∈[−2,2], wherem_k,n(x)∈Z∩[−2,2].

As in the proof of proposition 2.4ii), we have :

hq_n2(x) =χnsin[q_n2π(2x+ (q_n2−1)α)] +o(1),

with uniformo(1) and where the (χn) are uniformly far from 0 and±∞. Choose (θ(n))n≥1so that γn:=P

k≥1ak{qθ(n)²µ(Ik)} →γ∞ andχθ(n)→χ6= 0. Introduce the random variables : X_n(x) =εh_q

θ(n)2(x), Y_n(x) =X

k≥1

a_km_k,θ(n)2(x) andZ_n =X_n+Y_n.

LetX be the uniform random variablex7−→xon (T¹, µ). We have : X_n→εχsin(2πX), in law.

Since the (Xn) and the (Yn) are uniformly bounded, the laws of the (Zn) are tight. Let (rn) be a sequence such that (Zr_n) converges in law to some Z. Since γn →γ_∞, notice that (Zr_n, γr_n) converges in law to (Z, γ_∞).

Letε6= 0 and suppose thatT_f+εhis not ergodic. Then E(f+εh) =λZ, for someλ≥0. Fixing δ >0, introduce the open setO_δ={x∈T¹,dist(x, λZ)< δ} and its closureF_δ. By lemma 2.2 :

1 =µ(Z−γ_∞∈Oδ).

SinceOδ+γ_∞is open, using the law convergence for the first inequality and the fact thatYr_n ∈K, as well asK=−K, for the last inequality, we obtain :

1≤lim infµ(Zr_n∈Oδ+γ_∞)≤lim supµ(Zr_n∈Fδ+γ_∞)

≤lim supµ(X_rn ∈F_δ+K+γ_∞).

SinceFδ+K+γ_∞ is closed, asK is compact, using the law convergence of (Xn) we finally get : 1 =µ(εχsin(2πX)∈Fδ+K+γ∞).

The closedness of F_δ+K+γ_∞ then implies that εχ[−1,1]⊂γ_∞+K+F_δ. As K is compact, K+F_δ tends toK+λZasδ→0. Therefore :

εχ[−1,1]⊂γ_∞+∪_n∈Z(K+nλ).

However eachK+nλis a compact with empty interior, thus∪n∈Z(K+nλ) also has empty interior, but this is a contradiction. ThereforeTf+εhis ergodic.

Suppose now that dimbox(K)<1/2 andε6= 0. IfTf+εh is squashable, by proposition 2.3 there existsκ6=±1, a measurableψ:T¹→R^d andβ such that for almost-everyx∈T¹:

g(x) :=κ(f +εh)(x)−(f+εh)(x+β) =ψ(x)−ψ(x+α). (2.5) Set ˜f(x) =κf(x)−f(x+β) =κPak1I_k(x)−Pak1I_k−β(x)−γ(κ−1). Comparing with f, the compact K is replaced in ˜f by the compact κK+K, whose Hausdorff dimension is less than 2dimbox(K)<1. Thus κK+K has empty interior.

Choose (kn) so that qθ(k_n)²β →ζ mod (1). If Tg is not ergodic, then the same proof with ˜f and the perturbationε(κh(.)−h(.+β)), using the sequence (qθ(k_n)²), gives someλ≥0 so that :

εχ(κsin(2πc)−sin(2π(c+ζ)))∈γ_∞(κ−1) +∪_n∈Z(κK+K+nλ), ∀c∈[0,1].

However, the image of the left-hand side contains a non-empty open interval, since|κ| 6= 1, whereas the right-hand side has empty interior. ThereforeT_gis ergodic. This contradicts (2.5) and concludes the proof of the proposition.

2.5. An example withϕ:T²→R²

Take now X =T² with Lebesgue measure µ and G=R². The method used in the proof of proposition 2.4 furnishes for any minimal translation T by α= (α1, α2) on T² a continuous f : T²→R² of the formf(x, y) = (ϕ(x), ϕ(y)) such thatTf :T²×R²→T²×R²is ergodic and non- squashable. Indeed, it is enough to take a sequence (q_n) growing fast enough, with ((q_nα₁)) + ((q_nα₂)))→0 fast enough. Setting :

ϕ(x) =X

k≥1

q_2k⁻¹sin(2πq2k+1x),

the same proof works with the rigid sequence (q2n).

Inspired by ideas of Yoccoz [21] (see also Conze-Chevallier [5]), we build a smooth example for a particular α.

Proposition 2.6.

Letα= (α₁, α₂) = (P

k≥12^−mk,P

k≥13^−mk), wherem_n+1/m_n>ln 12/ln 2. Define : f(x, y) = (ϕ(x), ϕ(y)), whereϕ(x) =X

k≥1

k⁻²6^−mksin(2π6^mkx).

Then the translation T by α is minimal, f is of class C¹ and Tf :T²×R²→T²×R² is ergodic and non-squashable.

Proof of the proposition :

We first check that T is minimal. Consideringαⁿ=P

k≤n(2^−mk,3^−mk), the subgroup generated byαⁿ ishαⁿi= (k2^−mn, l3^−mn)_(k,l)∈Z2 andαⁿ has order 6^mn. For 0≤q≤6^mn, we have :

d(qα, qαⁿ)≤C6^mn2^−mn+1→0, so the orbit ofαis dense inT².

Set tn=n²6^mn. Since ((tnαi))≤n²6^mn2^−mn+1→0, (tn) is a rigid sequence for α= (α1, α2).

Letqn= 6^mn and considerϕn under the rotation byαi. Using the decomposition given in (2.3) :

|A_n| ≤CX

k<n

n²((qnαi))

k²((qkαi)) ≤Cn²6^mn2^−mn+1

6^mn−12^−mn →0 and|C_n| ≤C qn

qn+1

→0.

As sin(π6^mnn²6^mnα_i)/n²6^mnsin(π6^mnα_i)→1, we obtain :

ft_n(x, y) = (sin[πqn(2x+ (tn−1)α1)],sin([πqn(2y+ (tn−1)α2)]) +o(1).

Denoting by L the law of sin(2πX), where X is a uniform random variable on [0,1], we have f_tn→ L ⊗ L, in law. Lemma 2.2 gives [−1,1]²⊂ E(f). ThusE(f) =R² andT_f is ergodic.

If Tf is squashable, proposition 2.3 furnishes a 2×2 invertible matrix M with |detM| 6= 1, a measurableψ:T²→R² andβ= (β1, β2) such that for almost-everyx∈T²:

g(x) :=M f(x)−f(x+β) =ψ(x)−ψ(x+α).

We thus haveE(g) ={0}. Consider a subsequence (kn) so thatqknβ →ζ= (ζ1, ζ2) mod Z². Using the sequence (t_kn), we obtain via the same proof that for all (x, y)∈R² :

M

sin(2πx) sin(2πy)

−

sin(2π(x+ζ1)) sin(2π(y+ζ2))

= 0.

Taking y= 0, we get sin(2πx)M e1=^t(sin(2π(x+ζ1)),sin(2πζ2)). Thus M21= 0 and this gives M11sin(2πx) = sin(2π(x+ζ1)). NecessarilyM11=±1. In the same way,M12= 0 andM22=±1.

As a consequenceM is diagonal and detM =±1, but this is a contradiction.

3. Smooth ergodic cocycles in a nilpotent group

We come back to X =T¹, with an irrational rotation T of angle α and Lebesgue measure µ, and use the same notations as in section 2.3 for the sequence of best approximations ofα. We now build examples of smooth ergodic cocycles with values in the groupG=Nd(R), where :

Nd(R) =



























1 · · · 0 . .. (a^ij) ... ... . .. . .. 0 · · · 0 1













|(a^ij) = (a^ij)_1≤i<j≤d∈R^d(d−1)/2













 .

IfA∈Nd(R), we writeA= (a^ij)_1≤i<j≤d orA= (a^ij) for simplicity. Matricial indices are written upwards, down indices being for cocycles.

We suppose thatd≥3, soNd(R) is nilpotent but non-abelian. Recall that Haar measure inNd(R) is the left and right-invariant Lebesgue measure.

Theorem 3.1.

Let α have non bpq. Choose a strictly increasing (θ(n))_n≥1 satisfying a_θ(n)+1≥n³ as well as q_θ(n+1)≥q^d_θ(n)+1. We define :

(Φ) = (f^ij)_1≤i<j≤d, byf^ij(x) =X

k≥1

sin(2πq_θ(d2k+di+j)x) k²qθ(d²k+di+j)

.

ThenΦisC¹ andT_Φ:T¹×N_d(R)→T¹×N_d(R)is ergodic.

Remark 1 —For almost-everyx∈T¹, the orbit (Φn(x))_n∈Z (and in fact (Φn(x))_n≥1) is dense in Nd(R). The continuity of Φ implies that this property is true for xin a Gδ-set of full Lebesgue measure.

Remark 2 — If α has bpq and Φ is C¹ or if α has Diophantine type 1 and Φ is C^1+ε, then TΦ

cannot be ergodic. Indeed Φ¹²_n =f_n¹², for alln∈Z andTf¹² :T¹×R→T¹×Rwould be ergodic.

Howeverf¹² is in this case a coboundary.

Remark 3 — There is a similar statement for α with bpq. Fixing a decreasing ε(h)→+∞, as h→0⁺, choose (θ(n))_n≥1 withθ(n)≥n²,ε(1/q_θ(n))≥nandq_θ(n+1)≥q_θ(n)+1^d . Set :

(Φ) = (f^ij)_1≤i<j≤d, wheref^ij(x) =X

k≥1

q_θ(d⁻¹_2k+di+j)sin(2πq_θ(d2k+di+j)x).

Thenhε(h) is a modulus of continuity for Φ and T_Φ:T¹×N_d(R)→T¹×N_d(R) is ergodic. The proof is directly adapted from the one given below, as for itemii) of proposition 2.4 from itemi).

Remark 4 —We examine squashability in the special case whend= 3 below.

As N_d(R) is not abelian, lemma 2.2 does not apply. We proceed differently and first introduce some definitions. Denote byD_l(A) = (a^ii+l)_{1≤i≤d−l} thel-upper diagonal ofA= (a^ij)∈N_d(R).

Definition 3.2.

Al-upper diagonalx= (x^ii+l)_{1≤i≤d−l}∈R^d−lis essential forΦif∀ε >0,∀Borel setBwithµ(B)>

0, there isn∈Zsuch that :

µ(B∩T⁻ⁿB∩ {(D1(Φn),· · · , Dl(Φn))∈Vε(0_Rd−1,· · ·,0_Rd−l+1, x)})>0.

We writeD_l(Φ) for the set ofl-upper essential diagonals.

A generalisation of considerations by Greschonig [10] is the following.

Lemma 3.3.

The setDl(Φ) is a closed subgroup of(R^d−l,+)and :

E(Φ) =Nd(R)⇔ Dl(Φ) =R^d−l, for1≤l≤d−1.

Proof of the lemma :

We first show that Dl(Φ) is a closed subgroup of (R^d−l,+). Let xand x⁰ be in Dl(Φ). Fix ε >0 and a Borel setB withµ(B)>0. For somem∈Z:

µ(B∩T^−mB∩ {(D₁(Φ_m),· · ·, D_l(Φ_m))∈V_ε(0_Rd−1,· · ·,0_Rd−l+1, x)})>0.

LetC be the set appearing above. Sinceµ(T^mC) =µ(C)>0, there isn∈Zsuch that :

µ(T^mC∩T^m−nC∩ {(D1(Φn),· · · , Dl(Φn))∈Vε(0_Rd−1,· · ·,0_Rd−l+1, x⁰)})>0. (3.1) The following fact can be verified by carefully considering the entries of the matrix product : ifP andQin N_d(R) are such that :

(Dj(Q))_1≤j≤l∈Vε((0_Rd−j)_{1≤j≤l−1}, x⁰) and (Dj(P))_1≤j≤l∈Vε((0_Rd−j)_{1≤j≤l−1}, x),

then (D_j(QP))_1≤j≤l∈V_{M ε}((0_Rd−j)_{1≤j≤l−1}, x+x⁰), whereM is a constant depending only ond.

IfF is the set in (3.1), using the cocycle property :

T^−mF ⊂B∩T^−m−nB∩ {(D₁(Φ_m+n),· · ·, D_l(Φ_m+n))∈V_{M ε}(0_Rd−1,· · ·,0_Rd−l+1, x+x⁰)}. (3.2) Sinceµ(T^−mF) =µ(F)>0, we conclude thatx+x⁰∈ Dl(Φ). Similarly,−x∈ Dl(Φ), ifx∈ Dl(Φ).

Hence Dl(Φ) is a subgroup of (R^d−l,+). The closedness part is obvious.

We turn to the second part of the lemma. Clearly,E(Φ) =N_d(R) implies thatDl(Φ) =R^d−l, for all 1≤l≤d−1. Conversely, fix a= (a^ij)∈N_d(R),ε >0 and a Borel setB withµ(B)>0.

Sincea¹:= (aⁱⁱ⁺¹)1≤i≤d−1∈ D1(Φ), there isn1 so thatµ(B∩T⁻ⁿ¹B∩ {D1(Φn₁)∈Vε(a¹)})>0.

Let now (bl)l≥1 be dense inNd(R). As :

{D1(Φn₁)∈Vε(a¹)}=∪_l≥1{Φn₁ ∈Vε(bl), D1(Φn₁)∈Vε(a¹)},

there is b¹= (b^ij,1)_1≤i<j≤d∈N_d(R) such thatb^ii+1,1=aⁱⁱ⁺¹, for all 1≤i≤d−1, so that : µ(B∩T⁻ⁿ¹B∩ {Φn₁ ∈Vε(b¹)})>0.

LetB₂be the set appearing above. Sincea²:= (aⁱⁱ⁺²−b^ii+2,1)_{1≤i≤d−2}∈ D₂(Φ) andµ(Tⁿ¹B₂)>0, there is n₂such that :

µ(Tⁿ¹B2∩Tⁿ¹⁻ⁿ²B2∩ {(D1(Φn₂), D2(Φn₂))∈Vε(0_Rd−1, a²)})>0.

IfB₃ is the previous set, thenµ(T⁻ⁿ¹B₃) =µ(B₃)>0 and :

T⁻ⁿ¹B₃⊂B∩T⁻ⁿ¹⁻ⁿ²B∩ {(D₁(Φ_n1+n2), D₂(Φ_n1+n2))∈V_{M ε}((aⁱⁱ⁺¹)_{1≤i≤d−1},(aⁱⁱ⁺²)_{1≤i≤d−2})}, whereM is as in (3.2). The separability ofNd(R) again implies that there isb²= (b^ij,2)_1≤i<j≤d ∈ Nd(R), withb^ii+l,2=a^ii+l forl∈ {1,2} and 1≤i≤d−l, such that :

µ(B∩T⁻ⁿ¹⁻ⁿ²B∩ {Φn₁+n₂∈V2M ε(b²)})>0.

Iterating the procedure, we obtain integersn1,· · · , nd−1such that :

µ(B∩T^{−n1−n2−···−nd−1}B∩ {Φn₁+n₂+···+nd−1 ∈V_(2M)d−2ε(a)})>0.

As a result,a= (a^ij)∈ E(Φ). ThusE(Φ) =Nd(R) and this concludes the proof of the lemma.

Proof of theorem 3.1 :

Step 1. We make preliminary computations. Recall that Φ_n =Tⁿ⁻¹Φ· · ·Φ, for n≥1. We have Φ^kk_n = 1 and by definition of the matrix product, ifk < l:

Φ^kl_n =

l−k

X

m=1

X

k=u₀<···<u_m=l

X

0≤s_m<···<s₁<n

T^s1f^u0u1· · ·T^smf^um−1um.

Using the decomposition sin(2πq_θ(d2r+dk+l)x) = (e^{2iπqθ(d2r+dk+l)x}−e^{−2iπqθ(d2r+dk+l)x})/2i, write f^kl= (f^kl,+−f^kl,−)/2i. Then :

Φ^kl_n =

l−k

X

m=1

1 (2i)^m

X

δ=(δ₁,···,δ_m)∈{±1}

δ1· · ·δm

X

k=u0<···<um=l

Φ^kl,m,δ,u_n , whereu= (u_i) and :

Φ^kl,m,δ,u_n = X

0≤sm<···<s1<n

T^s1f^u0u1,δ1· · ·T^smf^um−1um,δm.

Setwr(k, l) =d²r+dk+l andvt(p) =qθ(w_p(ut−1,u_t)). Using the expression forf^kl,±, we obtain :

Φ^kl,m,δ,u_n (x) = X

p1,···,pm≥1

e^{2iπPmt=1δtvt(pt)x} Qm

t=1p²_tvt(pt)

X

0≤sm<···<s1<n

e^{2iπδ1v1(p1)s1α}· · ·e^{2iπδmvm(pm)smα}. (3.3)

Step 2. A remark, about indices, is that wr(k, l) =wr⁰(k⁰, l⁰) if and only if r=r⁰, k=k⁰ and l=l⁰, since dk+l∈[1, d²] and l∈[1, d]. Therefore, for all values of p1,· · ·, pm, the quantities (wp_t(u_t−1, ut))_1≤t≤m are pairwise distinct.

Fix (i, j) with 1≤i < j≤dand suppose that (k, l) satisfiesl−k≤j−i. In other words (k, l) lies on a lower upper-diagonal than (i, j). Impose also (k, l)6= (i, j). As a result, for allm,δ,uand for allp₁,· · · , p_m, them-tuples (w_pt(u_t−1, u_t))_1≤t≤mappearing in the expression of Φ^kl,m,δ,u_n are also distinct from anyw_n(i, j),n≥1.

Let nowt_n=n²q_θ(wn(i,j)). We prove that Φ^kl,m,δ,u_t

n →0 uniformly, asn→+∞. Let I_n be the set of (p₁,· · · , p_m) so that max_1≤t≤mw_pt(u_t−1, u_t)< w_n(i, j). Then in (3.3) at time t_n, the sum restricted toI_n^c and written S₁ satisfies, for some constantC >0 :

|S1| ≤

m

X

t=1

X

p1,···,pm≥1;w_pt(ut−1,ut)>wn(i,j)

t^m_n Qm

t⁰=1p²_t0vt⁰(pt⁰)

≤t^m_n

m

X

t=1

Y

t⁰∈{1,···,m}\{t}



 X

p_t0≥1

1 p²_t0vt⁰(pt⁰)





X

p_t≥1,w_pt(u_t−1,u_t)>w_n(i,j)

1 p²_tvt(pt)

≤Cn^2m q^m_θ(w

n(i,j))

q_{θ(wn(i,j)+1)} →_n→+∞0.

We next turn to the sumS₂, corresponding to (3.3) at timet_nwith indices (p₁,· · ·, p_m) restricted to I_n. We have :

|S2| ≤ X

p_t≥1,w_pt(ut−1,u_t)<w_n(i,j),1≤t≤m

1 Qm

t=1p²_tvt(pt) |Ψn(p₁,· · · , p_m)|, where :

Ψ_n(p₁,· · · , p_m) = X

0≤sm<···<s1<t_n

e^{2iπδ1v1(p1)s1α}· · ·e^{2iπδmvm(pm)smα}

= X

0≤sm−1<···<s1<tn

e^{2iπPm−1t=1 δtvt(pt)stα}

e^{2iπδmvm(pm)sm−1α}−1 e^{2iπδmvm(pm)α}−1

= X

ε_m=0,1

(−1)^1+εm X

0≤sm−1<···<s1<tn

e^{2iπ(Pm−1t=1 δtvt(pt)st+εmδmvm(pm)sm−1)α} e^{2iπδmvm(pm)α}−1

= X

ε_m,···,ε₂=0,1

(−1)^{m−1+Pmt=2εt} e^{2iπ[δ1v1(p1)+Pmt=2ε2···εtδtvt(pt)]tnα}−1 Qm

r=1

e^{2iπ[δrvr(pr)+Pmt=r+1εr+1···εtδtvt(pt)]α}−1, observing that everywhere δ_rv_r(p_r) +Pm

t=r+1ε_r+1· · ·ε_tδ_tv_t(p_t)6= 0, since all v_t(p_t) are pairwise distinct and, as follows easily from the hypotheses,q_θ(k+1)≥2q_θ(k) for allk≥1.

There is then an absolute constantC >0 such that in the last fraction, the absolute value of the denominator is bounded from below byCQm

r=1((v_r(p_r)α)). As a result, ifC >0 is now generic :

|S2| ≤C X

p_t≥1,w_pt(u_t−1,u_t)<w_n(i,j),1≤t≤m

1 Qm

t=1p²_tv_t(p_t)

((t_nα))Pm t=1v_t(p_t) Qm

t=1((v_t(p_t)α))

≤C((tnα)) X

pt≥1,w_pt(ut−1,ut)<wn(i,j),1≤t≤m

Qm

t=1q_{θ(wpt(ut−1,ut))+1} Qm

t=1p²_tq_θ(w

pt(ut−1,u_t))

1≤t≤mmax q_{θ(wpt(ut−1,ut))}

≤Cn²q_{θ(wn(i,j)−1)+1}q_{θ(wn(i,j)−1)} qθ(w_n(i,j))+1

≤Cn²q_{θ(wn(i,j)−1)+1}q_{θ(wn(i,j)−1)} wn(i, j)³qθ(w_n(i,j))

≤Cq_{θ(wn(i,j)−1)+1}q_{θ(wn(i,j)−1)} q^d_θ(w

n(i,j)−1)+1

→_n→+∞0.

As a result, Φ^kl,m,δ,u_t

n →0, uniformly. When summing on m, δ and u, we get that whenever l−k≤j−iand (k, l)6= (i, j), then Φ^kl_tn→0, uniformly. Concerning Φ^ij_tn, write :

Φ^ij_tn=f_t^ij_n+

j−i

X

m=2

1 (2i)^m

X

δ=(δ₁,···,δ_m)∈{±1}

δ1· · ·δm

X

i=u0<···<um=j

Φ^ij,m,δ,u_tn .

Indices (u_t−1, u_t) appearing in the right-hand sum are all distinct from (i, j). As before, all Φ^ij,m,δ,u_t

n

tend to 0 uniformly, asn→+∞. Therefore the right-hand sum tends to 0 uniformly, asn→+∞.

On the contrary, proceeding as in proposition 2.4i) :