Asymptotic Gilbert–VarshamovBound

2 Bounds on the size of codes

2.10 Asymptotic bounds

2.10.6 Asymptotic Gilbert–VarshamovBound

. In particular if q=2,then when0< δ <1/2,

α2(δ)≤H₂ 1

2 −* δ(1−δ)

Theorem 2.10.7 (The Second MRRW Bound) Let0< δ <1/2. Then α2(δ)≤ min

0≤u≤1−2δ{1+g(u²)−g(u²+2δu+2δ)}

where g(x)=H2((1−√

1−x)/2).

The Second MRRW Bound is better than the First MRRW Bound, forq =2, when δ <0.272. Amazingly, ifq =2, the bounds agree when 0.273≤δ≤0.5. Exercise 142 shows that the Second MRRW Bound is strictly smaller than the Asymptotic Elias Bound whenq =2.

Exercise 140 Continuing with Exercise 139, add to the graphs drawn forq=2,q =3, and q =4 the graph of the inequality from the First MRRW Bound. (A computer graphing tool may be helpful.) In each case, for what values ofδis the First MRRW Bound stronger than the Asymptotic Singleton Bound, the Asymptotic Plotkin Bound, the Asymptotic Hamming

Bound, and the Asymptotic Elias Bound?

Exercise 141 By the Second MRRW Bound,α2(δ)≤1+g((1−2δ)²)−g((1−2δ)²+ 2δ(1−2δ)+2δ). Verify that this is the First MRRW Bound whenq =2.

Exercise 142 This exercise shows that Second MRRW Bound is strictly smaller than the Asymptotic Elias Bound whenq =2.

(a) By the Second MRRW Bound,α2(δ)≤1+g(0)−g(2δ). Verify that this is the Asymp-totic Elias Bound whenq =2.

(b) Verify that the derivative of 1+g(u²)−g(u²+2δu+2δ) is negative atu =0.

the Asymptotic Elias Bound whenq =2?

2.10.6 Asymptotic Gilbert–VarshamovBound

We now turn to the only asymptotic lower bound we will present. This bound is the asymp-totic version of both the Gilbert and the Varshamov Bounds. We will give this asympasymp-totic bound using the Gilbert Bound and leave as an exercise the veriﬁcation that asymptotically the Varshamov Bound gives the same result.

Theorem 2.10.8 (Asymptotic Gilbert–Varshamov Bound) If 0< δ≤1−q⁻¹ where q ≥2,thenαq(δ)≥1−Hq(δ).

95 2.11 Lexicodes

Proof: By the Gilbert Bound Aq(n, δn)=Aq(n,"δn#)≥qⁿ/Vq(n,"δn# −1). Since

"δn# −1≤ δn,Aq(n, δn)≥qⁿ/Vq(n,δn). Thus αq(δ)=lim sup

n→∞ n⁻¹logq Aq(n, δn)

≥lim sup

n→∞ 1−n⁻¹log_qVq(n,δn)=1−Hq(δ)

by Lemma 2.10.3.

Exercise 143 Verify that, for q≥2, the asymptotic version of the Varshamov Bound produces the lower boundαq(δ)≥1−Hq(δ) when 0< δ≤1−q⁻¹. The Asymptotic Gilbert–Varshamov Bound was discovered in 1952. This bound guar-antees (theoretically) the existence of a family of codes of increasing length whose rela-tive distances approachδwhile their rates approach 1−Hq(δ). In the next section and in Chapter 13 we will produce speciﬁc families of codes, namely lexicodes and Goppa codes, which meet this bound. For 30 years no one was able to produce any family that exceeded this bound and many thought that the Asymptotic Gilbert–Varshamov Bound in fact gave the true value ofαq(δ). However, in 1982, Tsfasman, Vl˘adut¸, and Zink [333] demonstrated that a certain family of codes of increasing length exceeds the Asymptotic Gilbert–Varshamov Bound. This family of codes comes from a collection of codes called algebraic geom-etry codes, described in Chapter 13, that generalize Goppa codes. There is, however, a restriction onFq for which this construction works: q must be a square with q ≥49.

In particular, no family of binary codes is currently known that surpasses the Asymptotic Gilbert–Varshamov Bound. In Figure 2.2 we give five upper bounds and one lower bound on αq(δ), forq =2, discussed in this section and Section 1.12; see Exercise 140. In this figure, the actual value ofα2(δ) is 0 to the right of the dashed line. Families of binary codes meet-ing or exceedmeet-ing the Asymptotic Gilbert–Varshamov Bound lie in the dotted region of the figure.

Exercise 144 Continuing with Exercise 140, add to the graphs drawn forq =3 andq =4 the graph of the inequality from the Asymptotic Gilbert–Varshamov Bound as done in

Figure 2.2 forq=2.

2.11 Lexicodes

It is interesting that there is a class of binary linear codes whose construction is the greedy construction for nonlinear codes described in Section 2.8, except that the order in which the vectors are chosen is determined ahead of time. These codes are called lexicodes [39, 57, 192], and we will show that they indeed are linear. The construction implies that the lexicodes meet the Gilbert Bound, a fact we leave as an exercise. This implies that we can choose a family of lexicodes of increasing lengths which meet the Asymptotic Gilbert–

Varshamov Bound.

The algorithm for constructing lexicodes of lengthnand minimum distancedproceeds as follows.

✲

ppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp pppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp

pppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp

pppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp

ppppppppp

ppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp

pppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppppp

✛ Singleton

Figure 2.2 Asymptotic Bounds withq=2.

I. Order alln-tuples in lexicographic order:

0· · ·000

II. Construct the classLof vectors of lengthnas follows:

(a) Put the zero vector0inL.

(b) Look for the ﬁrst vectorxof weightdin the lexicographic ordering. PutxinL.

(c) Look for the next vector in the lexicographic ordering whose distance from each vector inLisdor more and add this toL.

(d) Repeat (c) until there are no more vectors in the lexicographic list to look at.

97 2.11 Lexicodes

The setLis actually a linear code, called thelexicodeof lengthnand minimum distance d. In fact if we halt the process earlier, but at just the right spots, we have linear subcodes of L. To prove this, we need a preliminary result due to Levenshtein [192]. Ifuandvare binary vectors of lengthn, we say thatu<vprovided thatucomes beforevin the lexicographic order.

Lemma 2.11.1 Ifu,v,and w are binary vectors of length n withu<v+w and v<

u+w,thenu+v<w.

Proof: Ifu<v+w, thenuandv+whave the same leading entries after whichuhas a 0 andv+whas a 1. We can represent this as follows:

u=a0· · ·, v+w=a1· · ·.

Similarly asv<u+w, we have v=b0· · ·,

u+w=b1· · ·.

However, we do not know that the lengthiofaand the length jofbare the same. Assume they are different, and by symmetry, thati < j. Then we have

v=bx· · ·, u+w=bx· · ·,

wherebis the ﬁrstientries ofb. Computingwin two ways, we obtain w=u+(u+w)=(a+b)x· · ·,

w=v+(v+w)=(a+b)(1+x)· · ·,

a contradiction. Soaandbare the same length, giving u+v=(a+b)0· · ·,

w=(a+b)1· · ·,

showingu+v<w.

Theorem 2.11.2 Label the vectors in the lexicode in the order in which they are generated so thatc0is the zero vector.

(i) Lis a linear code and the vectorsc₂ⁱ are a basis ofL.

(ii) After c2ⁱ is chosen, the next 2ⁱ−1 vectors generated are c1+c2ⁱ, c2+c2ⁱ, . . . , c₂ⁱ₋₁+c₂ⁱ.

(iii) LetLi = {c0,c1, . . . ,c2ⁱ−1}. ThenLiis an[n,i,d]linear code.

Proof: If we prove (ii), we have (i) and (iii). The proof of (ii) is by induction oni. Clearly it is true fori =1. Assume the ﬁrst 2ⁱvectors generated are as claimed. ThenLiis linear with basis{c1,c2, . . . ,c₂i−1}. We show thatLi+1is linear with the next 2ⁱ vectors generated in

order by addingc₂ⁱ to each of the previously chosen vectors inLi. If not, there is anr<2ⁱ such thatc₂ⁱ₊r =c₂ⁱ+cr. Chooserto be the smallest such value. As d(c₂ⁱ +cr,c₂ⁱ +cj)= d(cr,cj)≥d for j <r,c₂ⁱ +cr was a possible vector to be chosen. Since it was not, it must have come too late in the lexicographic order; so

c₂ⁱ₊r <c₂ⁱ +cr. (2.46)

As d(cr+c₂ⁱ₊r,cj)=d(c₂ⁱ₊r,cr+cj)≥dfor j <2ⁱ by linearity ofLi,cr+c₂ⁱ₊r could have been chosen to be in the code instead ofc2ⁱ (which it cannot equal). So it must be that

c₂ⁱ <cr+c₂ⁱ₊r. (2.47)

If j <r, then c₂ⁱ₊j =c₂ⁱ +cj by the assumption on r. Hence d(c₂ⁱ₊r +c₂ⁱ,cj)= d(c2ⁱ+r,c2ⁱ+j)≥d for j<r. So c2ⁱ+r +c2ⁱ could have been chosen to be a codeword instead ofcr. The fact that it was not implies that

cr <c₂ⁱ₊r+c₂ⁱ. (2.48)

But then (2.47) and (2.48) with Lemma 2.11.1 imply c2ⁱ +cr <c2ⁱ+r contradicting

(2.46).

The codesLisatisfy the inclusionsL1⊂L2⊂ · · · ⊂Lk =L, wherekis the dimension ofL. In general this dimension is not known before the construction. Ifi <k, the left-most coordinates are always 0 (exactly how many is also unknown). If we punctureLion these zero coordinates, we actually get a lexicode of smaller length.

Exercise 145 Do the following:

(a) Construct the codesLi of length 5 and minimum distance 2.

(b) Verify that these codes are linear and the vectors are generated in the order described by Theorem 2.11.2.

Exercise 146 Find an ordering ofF⁵₂so that the greedy algorithm does not produce a linear

code.

Exercise 147 Prove that the covering radius ofLisd−1 or less. Also prove that the lexicodes meet the Gilbert Bound. Hint: See Exercise 132.

The lexicodeLis the largest of theLi constructed in Theorem 2.11.2. We can give a parity check matrix forLprovidedd ≥3, which is reminiscent of the parity check matrix constructed in the proof of the Varshamov Bound. IfCis a lexicode of lengthnwithd≥3, construct its parity check matrixH=[hn· · ·h₁] as follows (wherehiis a column vector).

Regard the columnshias binary numbers where 1↔(· · ·001)^T, 2↔(· · ·010)^T, etc. Leth1

be the column corresponding to 1 andh₂the column corresponding to 2. Oncehi−1, . . . ,h₁ are chosen, choosehi to be the column corresponding to the smallest number which is not a linear combination ofd−2 or fewer ofhi−1, . . . ,h1. Note that the length of the columns does not have to be determined ahead of time. Wheneverhi corresponds to a number that is a power of 2, the length of the columns increases and zeros are placed on the tops of the columnshjfor j<i.

99 2.11 Lexicodes

Example 2.11.3 Ifn=7 andd =3, the parity check matrix for the lexicodeLis H=



1 1 1 1 0 0 0

1 1 0 0 1 1 0

1 0 1 0 1 0 1



.

So we recognize this lexicode as the [7,4,3] Hamming code.

As this example illustrates, the Hamming code H3 is a lexicode. In fact, all binary Hamming and Golay codes are lexicodes [39].

Exercise 148 Compute the parity check matrix for the lexicode of length 5 and minimum distance 3. Check that it yields the code produced in Exercise 145(c).

Exercise 149 Compute the generator and parity check matrices for the lexicodes of length

6 and minimum distance 2, 3, 4, and 5.

For deeper analysis and construction of linear codes we need to make use of the basic theory of finite fields. In this chapter we review that theory. We will omit many of the proofs; for those readers interested in the proofs and other properties of finite fields, we refer you to [18, 196, 233].

3.1 Introduction

Afieldis a setFtogether with two operations:+, called addition, and·, called multiplication, which satisfy the following axioms. The setFis an abelian group under+with additive identity called zero and denoted 0; the set F^∗ of all nonzero elements of F is also an abelian group under multiplication with multiplicative identity calledoneand denoted 1; and multiplication distributes over addition. We will usually omit the symbol for multiplication and writeabfor the producta·b. The field isfiniteifFhas a finite number of elements;

the number of elements inFis called theorderofF. In Section 1.1 we gave three fields denotedF2,F3, andF4of orders 2, 3, and 4, respectively. In general, we will denote a field withq elements byFq; another common notation is GF(q) and read “the Galois field with qelements.”

If p is a prime, the integers modulo pform a field, which is then denotedFp. This is not true ifpis not a prime. These are the simplest examples of finite fields. As we will see momentarily, every finite field contains someFpas a subfield.

Exercise 150 Prove that the integers modulondo not form a ﬁeld ifnis not prime.

The finiteness ofFq implies that there exists a smallest positive integer p such that 1+ · · · +1 (p1s) is 0. The integerpis a prime, as verified in Exercise 151, and is called thecharacteristicofFq. Ifa is a positive integer, we will denote the sum ofa 1s in the field bya. Also if we wish to write the sum ofa αs whereαis in the field, we write this as eitheraα ora·α. Notice that pα=0 for allα∈Fq. The set of p distinct elements {0,1,2, . . . ,(p−1)}ofFq is isomorphic to the fieldFp of integers modulo p. As a field isomorphic toFp is contained inFq, we will simplify terminology and say thatFp is a subfield ofFq; this subfield Fp is called theprime subfieldof Fq. The fact thatFp is a subfield ofFq gives us crucial information aboutq; specifically, by Exercise 151, the field Fqis also a finite dimensional vector space overFp, say of dimensionm. Thereforeq =p^m as this is the number of vectors in a vector space of dimensionmoverFp.

101 3.2 Polynomials and the Euclidean Algorithm

Although it is not obvious, all finite fields with the same number of elements are iso-morphic. Thus our notationFq is not ambiguous;Fq will be any representation of a field withqelements. As we did with the prime subfield ofFq, if we say thatFr is a subfield of Fq, we actually mean thatFqcontains a subfield withrelements. IfFq has a subfield with r elements, that subfield is unique. Hence there is no ambiguity when we say thatFr is a subfield ofFq. It is important to note that although all finite fields of orderqare isomorphic, one field may have many different representations. The exact form that we use for the field may be crucial in its application to coding theory. We summarize the results we have just given in a theorem; all but the last part are proved in Exercise 151.

Theorem 3.1.1 LetFqbe a ﬁnite ﬁeld with q elements. Then:

(i) q =p^mfor some prime p, (ii) Fqcontains the subﬁeldFp,

(iii) Fqis a vector space overFpof dimension m, (iv) pα=0for allα∈Fq,and

(v) Fqis unique up to isomorphism.

Exercise 151 Prove the following:

(a) Ifaandbare in a ﬁeldFwithab=0, then eithera =0 orb=0.

(b) IfFis a ﬁnite ﬁeld, then the characteristic ofFis a prime pand{0,1,2, . . . ,(p−1)}

is a subﬁeld ofF.

(d) A finite fieldFof characteristic p is a finite dimensional vector space over its prime subfield and containsp^melements, wheremis the dimension of this vector space.

Exercise 152 LetFqhave characteristic p. Prove that (α+β)^p=α^p+β^pfor allα, β ∈

Fq.

Throughout this chapter we will let pdenote a prime number andq = p^m, wheremis a positive integer.

Dans le document Fundamentals of Error-Correcting Codes (Page 113-120)