Threshold Saturation for Nonbinary Spatially-Coupled LDPC Codes on the Binary Erasure Channel

(1)

Threshold Saturation for Nonbinary Spatially-Coupled LDPC Codes on the Binary

Erasure Channel

Iryna Andriyanova^† and Alexandre Graell i Amat^‡

†ETIS laboratory, ENSEA/University of Cergy-Pontoise/CNRS

‡Department of Signals and Systems, Chalmers University of Technology, Gothenburg, Sweden

Journ´ees Codage&Cryptographie Les Sept Eaux, March 25, 2014

(2)

Background

I Spatially-Coupled LDPC (SC-LDPC) codes are codes with a low-density parity-check matrix, the non-zero elements in which are located in the diagonal band [FZ99].

I (Iterative) Belief Propagation (BP) algorithm is used for decoding.

I SC-LDPC codes showoutstanding asymptotic performancefor lots of channels and communication problems.

I For the binary erasure channel (BEC), the BP threshold of a binary SC-LDPC ensemble achieves theoptimal MAP threshold of the underlying LDPC ensemble (threshold saturation)[KRU11]. Extended to BMS channels [KMRU10].

[FZ99] J. Felstrom and K. Zigangirov, “Time-varying periodic convolutional codes with low-density parity-check matrix,” IEEE Trans. Inform. Theory, vol.45, no.6, pp.2181,2191, Sep 1999.

[KRU11] S. Kudekar, T. J. Richardson, and R. L. Urbanke, “Threshold saturation via spatial coupling: Why convolutional LDPC ensembles perform so well over the BEC,” IEEE Trans. Inform. Theory, vol. 57, pp. 803–834, Feb. 2011.

[KMRU10] S. Kudekar, C. M´easson, T. J. Richardson, and R. L. Urbanke, “Threshold saturation on BMS channels via spatial coupling,” 6th Int. Symp. Turbo Codes & Iterative Inform. Process., 2010.

(3)

Background

I Spatially-Coupled LDPC (SC-LDPC) codes are codes with a low-density parity-check matrix, the non-zero elements in which are located in the diagonal band [FZ99].

I (Iterative) Belief Propagation (BP) algorithm is used for decoding.

I SC-LDPC codes showoutstanding asymptotic performancefor lots of channels and communication problems.

I For the binary erasure channel (BEC), the BP threshold of a binary SC-LDPC ensemble achieves theoptimal MAP threshold of the underlying LDPC ensemble (threshold saturation)[KRU11]. Extended to BMS channels [KMRU10].

[FZ99] J. Felstrom and K. Zigangirov, “Time-varying periodic convolutional codes with low-density parity-check matrix,” IEEE Trans. Inform. Theory, vol.45, no.6, pp.2181,2191, Sep 1999.

[KRU11] S. Kudekar, T. J. Richardson, and R. L. Urbanke, “Threshold saturation via spatial coupling: Why convolutional LDPC ensembles perform so well over the BEC,” IEEE Trans. Inform. Theory, vol. 57, pp. 803–834, Feb. 2011.

[KMRU10] S. Kudekar, C. M´easson, T. J. Richardson, and R. L. Urbanke, “Threshold saturation on BMS channels via spatial coupling,” 6th Int. Symp. Turbo Codes & Iterative Inform. Process., 2010.

(4)

Background

I An alternative proof technique based onpotential functionsfor coupled scalar and vector recursions [YJNP12].

I DE (BP decoding, vector recursion, BEC(ε)): x^(`+1)=f(g(x^(`));ε),

wheref(y)andg(x)are the variable and check node updates.

Converges tox⁽^∞⁾= (0, . . . ,0)forεbelow the BP thresholdε^BP.

Vector admissible system

I The functionsf(y;ε)andg(x)non-decreasinginyandx;

I f(y;ε)isdifferentiableinyandg(x)is twice differentiable inx;

I f(0;ε) =f(y; 0) =g(0) =0;

I f(y;ε)isstrictly increasingwithε.

[YJNP12a] A. Yedla, Y.-Y. Jian, P. Nguyen, and H. Pfister, “A simple proof of threshold saturation for coupled scalar recursions,” 7th Int. Symp. Turbo Codes & Iterative Inform. Process., 2012. Extended version:

http://arxiv.org/abs/1309.7910

[YJNP12b] —–, “A simple proof of threshold saturation for coupled vector recursions,” 2012 IEEE Inform. Theory Workshop. Extended version: http://arxiv.org/abs/1208.4080

(5)

Background

I DE (BP decoding, vector recursion, BEC(ε)):

x^(`+1)=f(g(x^(`));ε),

I f(0;ε) =f(y; 0) =g(0) =0;

(6)

Background

x^(`+1)=f(g(x^(`));ε),

I f(0;ε) =f(y; 0) =g(0) =0;

(7)

Background

x^(`+1)=f(g(x^(`));ε),

I f(0;ε) =f(y; 0) =g(0) =0;

(8)

Background

I Potential function [YJNP12b]:

U(x;ε) =g(x)Dx^T−G(x)−F(g(x);ε),

whereF andGare scalar functions that satisfyF⁰(y;ε) =f(y;ε)D, and G⁰(x) =g(x)D, withDa positive diagonal matrix.

Afixed pointof the DE (x=f(g(x);ε)) corresponds to astationary pointof the corresponding potential function (U(x;ε)).

I TheBP threshold: ε^BP= sup_ε(ε∈[0,1]|U⁰(x;ε)>0, ∀x∈ X).

I Thepotential threshold: ε^∗= supε ε∈(ε^BP,1]|∆E(ε)≥0, ∀x∈ X ,

where∆E(ε) = infx∈X \U0(x)U(x;ε)for someε,ε^BP≤ε≤ε^∗, and U0(ε) ={x∈ X |x^∞=0}is the basin of attraction forx^∞=0.

[YJNP12b] A. Yedla, Y.-Y. Jian, P. Nguyen, and H. Pfister, “A simple proof of threshold saturation for coupled vector recursions,” 2012 IEEE Inform. Theory Workshop. Extended version: http://arxiv.org/abs/1208.4080

(9)

Background

U(x;ε) =g(x)Dx^T−G(x)−F(g(x);ε),

(10)

Background

U(x;ε) =g(x)Dx^T−G(x)−F(g(x);ε),

(11)

Background

U(x;ε) =g(x)Dx^T−G(x)−F(g(x);ε),

(12)

Background

Proof of threshold saturation [YJNP12b]:

1. DefineU(x;ε)for the (uncoupled) vector recursion.

2. Derive the potential function for the coupled vector recursion.

3. Show that, below potential threshold¹, theonly fixed point of the DE is x^∞=0.

1For several systems the MAP threshold and the potential thresholdε^∗are identical.

I Proved threshold saturation of irregular LDPC codes for aSlepian-Wolf problem with erasures, joint decoding of irregular LDPC codes on an erasure multiple-access channel, andprotograph codes on the BEC.

(13)

Background

(14)

Background

(15)

Background

(16)

Background

(17)

Background

I Proved threshold saturation of irregular LDPC codes for aSlepian-Wolf problem with erasures, joint decoding of irregular LDPC codes on an erasure multiple-access channel, andprotograph codeson the BEC.

(18)

Motivation

I Threshold saturation for binary SC-LDPC codes so far.

I Nonbinary SC-LDPC codes:

I Construction method for nonbinary SC-LDPC codes [UKS11].

I Thethreshold saturationalso occurs for nonbinary SC-LDPC codes. Contrary to uncoupled ensembles, the BP threshold of nonbinary SC-LDPC codesimproves with field size and tends to the Shannon limit[PGiA13].

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.4 0.42 0.44 0.46 0.48 0.5 0.52 0.54 0.56 0.58 0.6

hBP

ε m

Figure: BP EXIT functions for nonbinary(3,6)SC-LDPC code ensembles over GF^m₂.

[UKS11] H. Uchikawa, K. Kasai, and K. Sakaniwa, “Design and performance of rate-compatible non-binary LDPC convolutional codes,” 2011. [Online]. Available: http://arxiv.org/abs/1010.0060

[PGiA13] A. Piemontese, A. Graell i Amat, and G. Colavolpe, “Nonbinary spatially-coupled LDPC codes on the binary erasure channel,”IEEE Int. Conf. Commun., ICC’2013.

(19)

Motivation

I Thethreshold saturationalso occurs for nonbinary SC-LDPC codes. Contrary to uncoupled ensembles, the BP threshold of nonbinary SC-LDPC codesimproves with field size and tends to the Shannon limit[PGiA13].

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.4 0.42 0.44 0.46 0.48 0.5 0.52 0.54 0.56 0.58 0.6

hBP

ε m

Figure: BP EXIT functions for nonbinary(3,6)SC-LDPC code ensembles over GF^m₂.

(20)

Motivation

I Thethreshold saturationalso occurs for nonbinary SC-LDPC codes.

Contrary to uncoupled ensembles, the BP threshold of nonbinary SC-LDPC codesimproves with field size and tends to the Shannon limit[PGiA13].

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.4 0.42 0.44 0.46 0.48 0.5 0.52 0.54 0.56 0.58 0.6

hBP

ε m

Figure:BP EXIT functions for nonbinary(3,6)SC-LDPC code ensembles over GF^m₂.

(21)

Contributions

In this talk

I Proof of threshold saturation for nonbinary SC-LDPC codes to the potential thresholdfor transmission over the BEC, using the technique based on potential functions.

I (dv,dc,m)and(dv,dc,m,L,w)ensembles over the general linear group.

Contributions

I We prove themonotonicity of the variable node and check node updates for nonbinary LDPC codes, and theexistence of a fixed point in the DE.

I Existence and computation ofU(x;ε): Can be obtained by finding the functionsF andGas the solution of asystem of linear equations.

I Anecessary conditionon the existence ofF andG(thusU(x;ε)), assuming diagonalD=⇒Not verifiedfor nonbinary codes!

I AsymmetricDis sufficientso thatF andG exist.

I Proof of threshold saturation.

(22)

Contributions

In this talk

Contributions

(23)

Contributions

In this talk

Contributions

(24)

Contributions

In this talk

Contributions

(25)

Contributions

In this talk

Contributions

I Anecessary conditionon the existence ofF andG(thusU(x;ε)), assuming diagonalD

=⇒Not verifiedfor nonbinary codes!

(26)

Contributions

In this talk

Contributions

(27)

Contributions

In this talk

Contributions

I AsymmetricDis sufficientso thatF andGexist.

(28)

Contributions

In this talk

Contributions

I AsymmetricDis sufficientso thatF andGexist.

(29)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

I Code symbolsα0, . . . , α2^m−1 ∈ S=GF^m2

=⇒The messages exchanged in the BP decoding areprobability vectors of length2^m,(p0, . . . ,p2^m−1), wherepi is theprobability that the symbol isαi.

I A message(p0, . . . ,p2^m−1)has dimensionk if it has2^k non-zero entries, i.e., the symbol is known to beone out of2^k possible symbols.

I For theBECthe non-zero entries of a message are all equal, and the message is equivalent to asubspace of GF^m2 of dimensionk.

I DE simplifies to exchange ofprobability vectors of lengthm+ 1,

(˜p0, . . . ,˜pm), where˜pi is theprobability that the message has dimensioni.

(30)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

I Code symbolsα0, . . . , α2^m−1 ∈ S=GF^m2 =⇒The messages exchanged in the BP decoding areprobability vectors of length2^m,(p0, . . . ,p2^m−1), wherepi is theprobability that the symbol isαi.

(31)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

(32)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

(33)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

(34)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

I x^`_◦= (x◦0^` , . . . ,x◦m^` );x◦i^` is the probability that a messagefrom variable nodesat iteration`hasdimensioni.

I y^`_◦= (y◦0^` , . . . ,y◦m^` );y◦i^` is the probability that a message from check nodesat iteration`has dimensiondimensioni.

I Atcheck nodes, the BP decoder computes thecheck node updates (sum of the subspaces corresponding to the incoming messages),

y^`_◦=g_◦(x^`_◦⁻¹).

I Atvariable nodes, the decoder computes thevariable node updates (intersection of the subspaces of the incoming messages), corresponding to the incoming messages,

x^`_◦=f_◦(y^`_◦;ε).

Background DE for nonbinary LDPC ensembles Threshold saturation Conclusion

DE for (d

v

, d

c

, m ) LDPC code ensembles on the BEC

• x^ℓ_◦= (x◦0^ℓ , . . . ,x◦m^ℓ );x◦i^ℓ is the probability that a messagefrom variable nodesat iterationℓhasdimensioni.

• y^ℓ_◦= (y◦0^ℓ , . . . ,y^ℓ◦m);y^ℓ◦i is the probability that a message from check nodesat iterationℓhas dimensiondimensioni.

Threshold Saturation for Nonbinary SC-LDPC Codes on the BEC | A. Graell i Amat and I. Andriyanova 8 / 16

(x◦0, . . . ,x◦m)

(y_◦0, . . . ,y_◦m)

(35)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

y^`_◦=g_◦(x^`_◦⁻¹).

x^`_◦=f_◦(y^`_◦;ε).

DE for (d

v

, d

c

, m ) LDPC code ensembles on the BEC

(x◦0, . . . ,x◦m)

(y_◦0, . . . ,y_◦m)

(36)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

y^`_◦=g_◦(x^`_◦⁻¹).

x^`_◦=f_◦(y^`_◦;ε).

DE for (d

v

, d

c

, m ) LDPC code ensembles on the BEC

(x◦0, . . . ,x◦m)

(y_◦0, . . . ,y_◦m)

(37)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

I Thefixed-point DE equationforx_◦=x^∞_◦ is x◦=f_◦(g_◦(x◦);ε).

Decoding issuccessfulwhen itconverges tox^∞_◦ = (1,0, . . . ,0).

I f_◦andg_◦arenot monotone! =⇒Not an admissible system.

I Does DE for nonbinary LDPC codes converge to afixed point?

Idea

Rewrite the DE equation using complementary cumulative distribution function (CCDF) vectors.

(38)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

Idea

(39)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

I f_◦andg_◦arenot monotone!

=⇒Not an admissible system.

Idea

(40)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

Idea

(41)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

Idea

(42)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

Idea

(43)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

Definition

Given a probability vectorx◦, define the CCDF vectorx= (x1, . . . ,xm), where xi=Pm

k=ix◦k. We also definexm+1= 0. Then, it follows that x◦i=xi−xi+1. Note also thatx0= 1.

I We definenew vector functionsf = (f0, . . . ,fm)andg= (g0, . . . ,gm), fi=

m

X

k=i

f◦k(y_◦;ε) =

m

X

k=i

f◦k(y⁻¹−y;ε), gi=

m

X

k=i

g_◦k(x_◦) =

m

X

k=i

g_◦k(x⁻¹−x), wherex⁻¹= (1,x1, . . . ,xm−1).

(44)

DE for (d

_v

, d

_c

, m) LDPC code ensembles on the BEC

Definition

Given a probability vectorx◦, define the CCDF vectorx= (x1, . . . ,xm), where xi=Pm

k=ix◦k. We also definexm+1= 0. Then, it follows that x◦i=xi−xi+1. Note also thatx0= 1.

I We definenew vector functionsf = (f0, . . . ,fm)andg= (g0, . . . ,gm), fi=

m

X

k=i

f◦k(y_◦;ε) =

m

X

k=i

f◦k(y⁻¹−y;ε), gi=

m

X

k=i

g_◦k(x_◦) =

m

X

k=i

g_◦k(x⁻¹−x), wherex⁻¹= (1,x1, . . . ,xm−1).

(45)

Convergence of the DE for nonbinary LDPC codes

I Using CCDF vectors, the DE equation can be written in anequivalent formas

x=f(g(x);ε). (1)

Theorem 1

The functionsf(y;ε)andg(x)are increasing inyandx. Corollary

The DE for regular nonbinary LDPC codes given by (1)converges to a fixed point.

Successful decoding

Successful decoding corresponds to convergence of the DE equation to the fixed pointx^∞=0= (0,0, . . . ,0).

(46)

Convergence of the DE for nonbinary LDPC codes

x=f(g(x);ε). (1)

Theorem 1

The functionsf(y;ε)andg(x)are increasing inyandx.

Corollary

Successful decoding

(47)

Convergence of the DE for nonbinary LDPC codes

x=f(g(x);ε). (1)

Theorem 1

Corollary

Successful decoding

(48)

Convergence of the DE for nonbinary LDPC codes

x=f(g(x);ε). (1)

Theorem 1

Corollary

Successful decoding

(49)

Potential Function

I x=f(g(x);ε)is avector admissible system.

Can define a potential function

U(x;ε) =g(x)Dx^T−G(x)−F(g(x);ε). However...

U(x;ε)with diagonalDdoes not exist! =⇒Conditions on the existence of U(x;ε).

(50)

Potential Function

I x=f(g(x);ε)is avector admissible system. Can define a potential function

U(x;ε) =g(x)Dx^T−G(x)−F(g(x);ε).

However...

(51)

Potential Function

I x=f(g(x);ε)is avector admissible system. Can define a potential function

U(x;ε) =g(x)Dx^T−G(x)−F(g(x);ε).

However...

(52)

Properties of D and calculation of U ( x ; ε)

U(x;ε) =g(x)Dx^T−G(x)−F(g(x);ε)

Theorem 4 [U(x;ε)for the (dv,dc,m)ensemble]

F(y;ε)andG(x)exist (henceU(x;ε)exists)if there exist sets of values{djs}, {ϕ_(i₁_,...,i_m₎}and{µ_(k₁_,...k_m₎}that satisfy the following equations,

(isϕ_(i₁_,...,i_s_,...,i_m₎=Pm j=1djsϕ^(j)_(i

1,...is−1,...,im)(ε) ktµ_(k₁_,...,k_t_,...,k_m₎=Pm

j=1djtγ^(j)_(k

1,...,k_t−1,...km)

, (2)

for all possiblem-tuples(i1, . . . ,im)and(k1, . . . ,km)and allisandkt varying from1tom. The coefficientsϕ’s andγ’s in (2) are given by

ϕ^(j)_(i

1,...,im)(ε) = coeff(fj(x;ε),x₁ⁱ¹· · ·xmⁱ^m), γ_(k^(j)

1,,...km)= coeff(gj(x),x₁ⁱ¹· · ·x_mⁱ^m).

Similar result for the coupled case. General resultfor coupled vector systems.

(53)

Properties of D and calculation of U ( x ; ε)

U(x;ε) =g(x)Dx^T−G(x)−F(g(x);ε)

Theorem 4 [U(x;ε)for the (dv,dc,m)ensemble]

F(y;ε)andG(x)exist (henceU(x;ε)exists)if there exist sets of values{djs}, {ϕ_(i₁_,...,i_m₎}and{µ_(k₁_,...k_m₎}that satisfy the following equations,

(isϕ_(i₁_,...,i_s_,...,i_m₎=Pm j=1djsϕ^(j)_(i

1,...is−1,...,im)(ε) ktµ_(k₁_,...,k_t_,...,k_m₎=Pm

j=1djtγ^(j)_(k

1,...,k_t−1,...km)

, (2)

for all possiblem-tuples(i1, . . . ,im)and(k1, . . . ,km)and allisandkt varying from1tom. The coefficientsϕ’s andγ’s in (2) are given by

ϕ^(j)_(i

1,...,im)(ε) = coeff(fj(x;ε),x₁ⁱ¹· · ·xmⁱ^m), γ_(k^(j)

1,,...km)= coeff(gj(x),x₁ⁱ¹· · ·xmⁱ^m).

Similar result for the coupled case. General resultfor coupled vector systems.