A branching process with competition

Following Schweinsberg [55], we shall first show that beta-coalescents can describe asymp-totically the genealogy of certain population models with fixed sizeN asN → ∞. Such

population models may be viewed as variations of the Wright-Fisher model, in which exceptionally certain individuals may have a very large offsprings.

Let µbe some finite measure onN, where for every integer k, µ(k) describes the rate at which each individual in the population gives birth to k children (simultaneously).

We consider a population model in continuous time with a fixed size N at any time, in which each individual begets with rate governed by µ and independently of the other individuals. When a birth event occurs, say whenk children are born, we instantaneously eliminate uniformly at randomk individuals amongst the N+k present, so the total size of the population remains equal to N. One possible justification for the elimination of individuals in excess may be for instance competition between individuals for resources or for space. This model is a variation of that considered by Schweinsberg [55] who rather dealt with super-critical Galton-Watson processes, i.e. branching processes in discrete time.

We use a superscript N in the notation to underline the size of the population (the law of reproduction µ being fixed). We will be interested in the genealogy, and in this direction, we samplenindividuals from the generation at the present time and follow their ancestral lineages backwards in time to obtain a coalescent tree. We write thus Π^N_|[n](t) for the partition of the n-sample into families having the same ancestor at time t in the past.

Theorem 6.1 Suppose that the tail-distribution µ(k) =¯ µ({k + 1, k+ 2, . . .}) of the re-production law µ is regularly varying with index −α for some α∈(0,1)∪(1,2), i.e.

Nlim→∞

µ(bxN¯ c)

¯

µ(N) =x^α, x >0.

Then for every n ∈ N, the process (Π^N_|[n](t/µ(N¯ )) : t ≥ 0) converges in distribution as N → ∞ to a beta(2−α, α)-coalescent restricted to [n].

Remark: In the case α <1, the limiting coalescents in Schweinsberg [55] are not simple (i.e. they involve simultaneous multiple collisions). The difference with the present state-ment stems from the fact that Schweinsberg consider Galton-Watson processes in discrete time while we deal here with branching processes in continuous time.

Before tackling the proof of Theorem 6.1, we need the following elementary bounds for the distance in total variation between two probability measures on some discrete set, say {x₁, . . . , x_n+k} which occur in this setting. First, we fix an integer p and we denote by

= 1

(n+k)^p

n+k

X

i1,...,ip=1

δ_(xi

1,...,x_ip)

the uniform probability measure on{x₁, . . . , x_n+k}^p . Next, we consider ηthe probability measure η on {x₁, . . . , x_n+k}^p obtained by first removing k points uniformly at random, and then sampling with replacementp elements amongst the remaining points.

Lemma 6.1 Denote by |−η| the distance in total variation between and η, that is p-tuples such that two coordinates are equals and the others are all distinct, and byS₃ the complementary set of S₁ ∪S₂. In this direction, it will be convenient to use a slightly abusive notation, denoting by ca number depending only on p which may take different values in different expressions.

Consider first the case when (y₁, . . . , y_p)∈S₁. The probability that the subset of the k points which are removes do not contain any of the y_i is

n+k−p In particular we have the bounds

(1 +k/n)^p

wheref is an arbitrary function bounded in modulus by 1.

Next we consider the case when (y₁, . . . , y_p)∈S₂; calculations similar to those above yield

ρ(y₁, . . . , y_p) = (1 +k/n)^p

(1 +k/n)· · ·(1 +k/(n−p+ 2)),

then (1 +k/n)^p

the proof of the claim.

Recall from Section 4.3 that we denote by M₁ the space of probability measures on [0,1], and the notation to the Stone-Weierstrass theorem, the space of linear combinations of such functionals is dense in the space of continuous functions onM1.

Proof of Theorem 6.1 Imagine that each ancestor is assigned a type τ with values in [0,1], and that types are transmitted from parents to children. We write ρ^N(t) for the empirical distribution of types at timet, i.e.

ρ^N(t) = 1

where τ_i(t) denotes the type of the i-th individual at time t. It should be plain from the description of the model that (ρ^N(t) :t ≥0) is a Markov chain in continuous time on the finite subspace of M₁ consisting of linear combinations of Dirac point masses where the masses of atoms are multiples of 1/N.

We also introduce the following notation. Let m be a probability measure as above, say m=N⁻¹Pn

i=1δ_xi where x₁, . . . , x_N are the atoms of N mrepeated according to their multiplicity. For every y∈[0,1] andk ∈N, we denote by m(N, k, y) the empirical distri-butionN⁻¹PN

i=1δ_x⁰

i where thex⁰_i areN atoms obtained by sampling without replacement from the set of N +k points x₁, . . . , x_N, x_N+1, . . . , x_N+k with x_N+1, . . . , x_N+k =y.

The mathematical translation of the description of the evolution of the population model is that the infinitesimal generator L^N is given by

L^NΦ_f(m) =

It is easy to derive by an application of Lemma 6.1 that

Now we divide this quantity by ¯µ(N), which corresponds, in terms of the Markov process, to speeding up time by a factor 1/¯µ(N). Assume that the function f which appears in the definition of the functional Φf isC¹, and letN tend to∞. The assumption that the tail ¯µ is regularly varying with index −α means that the sequence of point measures

We now recognize the generator of the generalized Fleming-Viot process corresponding to the measureν(dz) =αz^−α−1(1−z)^α−1dz.

An application of the duality lemma enables us to conclude that the semi-group of Π^N_|[n](t/¯µ(N)) converges to that of a beta(2−α, α) coalescent.

In the case when the tail distribution ¯µis regularly varying with index−α withα >2, one can also prove that under a different rescaling of time, the limit genealogy exists and is described by Kingmans coalescent.