6.2 Numerical Examples
6.2.3 Effect of Finite n and K
We discuss the effect of finite block length n and finiteK and N on the sys-tem achievable performance through an example. For finite code length, the decoding SNR threshold g above which the post-decoding BER is vanishing is not defined (strictly speaking, it is infinite). However, in practical system design, we set a target SNR threshold gj for each class j, such that if the SINR at the input of the decoders of users in classjis abovegj, then the BER after decoding is so small that it has a negligible impact on the following de-coding stages of classesj−1, j−2, . . . ,1. A natural question is whether such a small but non-vanishing BER has a catastrophic effect, preventing the suc-cessive stripping decoder from decoding some class of users. We argue that for n sufficiently larger than K, and sufficiently small post-decoding BER, the effect of residual errors is indeed negligible. In fact, assuming random errors (if errors are correlated after decoding, we can use independent inter-leaving for each user), the expected number of incorrectly decoded symbols interfering with a user of classj is given byPL
i>jKi, whereis the residual BER. At the transition between the waterfall and the error flattening regions, LDPC codes have the property that the BER is=O(1/n) =ω/nfor some constant ω n. Turbo-codes have a similar property, where = O(1/I) and I is the size of the interleaver of the turbo encoder. This effect is called
“interleaving gain” in [96]. Hence, by setting the thresholds gj as the SNR at the transmission between waterfall and flattening, and by letting nK, then the expected number of incorrectly decoded symbols interfering with a user of class j is given by ω(PL
i>jKi)/n1.
Eventually, asensibleapproach for the design of practical CDMA systems based on successive decoding is to use the large-system optimization methods developed before, while replacing the infinite block length thresholds gj by some target SNR values chosen according to the BER vs. SNR performance of actual finite-length codes. While this argument provides only a heuristic design approach, extensive simulations show that the resulting systems are very good.
Moreover, in order to improve the robustness of the stripping decoder to residual errors,soft-stripping can be used, and successive decoding of the users from classLto class 1 can be iterated more than once. We refer to this approach as the multi-pass soft-stripping decoder, where one decoding pass consists of decoding all the users once. Soft-stripping (see for example [24]) consists of subtracting the minimum mean-square error (MMSE) estimates of
6.2 Numerical Examples 117 the signals of the already-decoded users from the signal, instead of their hard estimates. When the user decoders are symbol-by-symbol MAP (or approx-imations via Belief-Propagation, as in LDPC decoding [19, 39]), the MMSE estimate of the i-th symbol of userk,xi,kis obtained as the conditional mean
xei,k =E[x|EXTi,k]
where EXTi,k denotes the k-th decoder extrinsic information for the i-th symbol. For example, with QPSK symbols we have1
e
xi,k= 1
√2tanh(m(I)i,k/2) + j
√2tanh(m(Q)i,k /2) (6.15) where m(I)i,k and m(Q)i,k denote the extrinsic belief-propagation decoder mes-sages for the variable nodes corresponding to the bits modulated in the in-phase and quadrature components of the i-th QPSK symbol of user k [24].
If decisions are not reliable, i.e., |m(I)i,k| and/or |m(Q)i,k | are small, then soft stripping attenuates the effect of this symbol on the residual interference signal.
Fig. 6.9 shows the average BER performance of irregular LDPC codes of binary rate 1/2 whose degree sequence is shown in [9]. The BER is obtained by averaging over randomly generated parity-check matrices with the given degree distributions with maximum left degree 100, average right degree 11.
The belief-propagation decoder performs up to 200 iterations. Fig. 6.10 shows the spectral efficiency achievable by the equal-rate design with R= 1.0. The curve denoted by “optimal” corresponds to the threshold g = 0.186 dB, of ideal infinite-length QPSK random coding. The curve denoted by “practical”
corresponds to the threshold g = 0.933 dB, of the finite-length irregular LDPC code ensemble generated from the degree distribution found in [9]. It is obtained by trial-and-error as the minimum value for which the multi-pass successive decoder removes all interference in two decoding passes for block length n= 5000. This value might be different for other codes, block length and channel load. From Fig. 6.9 we observe that this threshold corresponds to BER '5·10−3 for n= 5000 and BER '5·10−4 for n = 10000.
We have simulated a system withρ= 2 bit/s/Hz, spreading factorN = 64 and K = 128 users, corresponding to the mark in Fig. 6.10. The evolution of the minimum, maximum and average users’ SINR as a function of the
1j in the complex symbol context is given byj2=−1.
0 0.5 1 1.5 2 2.5 3 10−6
10−5 10−4 10−3 10−2 10−1 100
Eb/N 0 (dB)
BER
Irregular LDPC−coded QPSK, R = 1.0 bit/s/Hz
Shannon limit
Target SNR threshold for successive decoding n = 5000
n = 10000
Figure 6.9: Average BER performance of irregular LDPCs of binary rate 1/2 over (single-user) AWGN, block lengths n= 5000 and n = 10000
0 5 10 15
0 1 2 3 4 5 6 7 8
Equal rate CDMA R=1 bit/symbol
Eb/N 0 (dB)
ρ (bit/s/Hz)
Optimal QPSK g=0.188 dB Practical QPSK g=0.933 dB
Figure 6.10: Spectral efficiency achievable by optimal and suboptimal QPSK code ensembles of rate R= 1
6.2 Numerical Examples 119
0=4.33 dB, N=64, K=128, n=5000
Successive decoder iterations
Figure 6.11: Evolution of the user SINR at the LDPC decoder input vs.
the successive decoding steps, for the multi-pass soft-stripping decoder with LDPC code length n = 5000
0 50 100 150 200 250
0=4.33 dB, N=64, K=128, n=10000
Max
Average
Min
Pass 1 Pass 2
Target SNR
Figure 6.12: Evolution of the user SINR at the LDPC decoder input vs.
the successive decoding steps, for the multi-pass soft-stripping decoder with LDPC code length n = 10000
successive stripping decoder iterations is shown in Fig. 6.11 forn= 5000 and in Fig. 6.12 forn = 10000. These curves are snapshots obtained by random generation of the noise, the spreading sequences, the information sequences and the LDPC code graphs. The successive decoder makes use of soft strip-ping. Each LDPC decoder is run for a maximum of 200 iterations, and a maximum of three interference cancellation passes. A complete interference cancellation pass corresponds to 128 decoding steps, i.e., to the decoding of allK = 128 users.
For n = 5000 we need to perform two soft-stripping successive decoding passes before all users reach their target SNR threshold. For n = 10000, since the threshold is much more conservative, a single pass is sufficient.
Equivalently, one could have lowered the threshold and achieved the same spectral efficiency at a smaller Eb/N0.
As a general remark, the gap between an optimal (infinite block length) system and a practical finite-length system depends almost entirely on the fact that finite-length codes require a significantly larger SNR (0.933 dB for n = 5000 and 0.747 dB for n = 10000) than the infinite-length decoding threshold. On the other hand, no catastrophic error propagation effect is observed when the system is designed according to the rules outlined above.
6.3 Conclusion
We have considered the optimization of a canonical coded synchronous CDMA system characterized by random spreading and QPSK signaling, in the limit of large number of users, large spreading gain, and large user code block length. Such assumptions may be regarded as “pragmatic”, in the sense that they are all motivated by actual CDMA systems. The CDMA sys-tem considered here has low complexity, as it assumes successive stripping with MMSE filters. Excellent approximations to the MMSE filters can be precomputed using the large random matrix design approach of [97], with complexity O(K2). Moreover, powerful long user codes such as LDPC codes can be decoded iteratively, with linear complexity in the block-length. Hence, the overall complexity per decoded information bit of the multiuser decoder is linear inK and constant in the code block length, i.e., comparable with the complexity of standard CDMA systems based on single-user detection and separated single-user decoding. Nevertheless, the proposed system optimiza-tion, driven by recent information-theoretic results, yields spectral efficiencies
6.3 Conclusion 121 remarkably close to the optimal (i.e., optimizing also with respect to the user signature waveforms and using Gaussian codebooks).
We have considered two special cases of the general rate and power allo-cation problem: namely, the optimization of the received SNR distribution for an equal-rate system, and the optimization of the user rate distribution for an equal-power system, subject to the successive decodability condition imposed by the stripping decoder. Both problems yield linear programs that admit closed form explicit solutions.
From a practical point of view, the equal-rate system design seems more attractive than its equal-power counterpart since it can approach optimal spectral efficiency uniformly, for all Eb/N0’s, provided that the individual users coding rate is small. Moreover, controlling the received user SNR is much easier and closer to existing power-control schemes than allocating coding rates (and channel codes) to the users.
Numerical results show that the system optimization carried out in the large-system limit and for infinite code block length can be used effectively to dimension practical systems, provided that the SNR thresholds are chosen according to the actual BER performance of the finite-length user codes.
Systems optimized according to the proposed method do not suffer from catastrophic error propagation of the successive stripping decoder even if, in general, finite-length codes have non-vanishing post-decoding BER.
APPENDIX 6.A Proof of Proposition 6.1
A necessary condition for β minimizing the objective function in (6.7) is that the constraint X
j
βj ≥ β holds with equality. Hence, without loss of generality we rewrite (6.7) in the canonical form
The dual linear program is given by [98]
whereα can be either positive or negative.
From the properties of the coefficients ai,j and bj we get immediately that A is invertible and the vector τ such that τ =A−1b has non-negative components. The vector β ∈RL+ maximizing 1Tβ and satisfying Aβ≤b is τ (this is easily shown by contradiction, since τ is the unique non-negative vectorβthat makes the inequalityAβ ≤bcomponent-wise tight). Hence, if 1Tτ < β the primal problem is infeasible. On the other hand, if1Tτ ≥β the primal problem is feasible, and a feasible point is given by (6.8). In order to show that this is indeed the desired solution, we shall assume that1Tτ ≥β and find a feasible point for the dual problem. Then, we show that the value of the dual problem at this point is equal to the value of the primal problem at (6.8).
We rewrite the inequality constraint and the objective function in the dual problem (6.17) as
ATy≥α1−γ (6.18)
and
−bTy+αβ (6.19)
6.A Proof of Proposition 6.1 123 The vector α1−γhas decreasing components. For fixed α, letKαdenote the number of positive elements of α1−γ. It is clear from (6.18) and (6.19) that the objective function is maximized by letting the last L−Kα components of y equal to zero. We introduce the following short-hand notation: for a vectorx∈RLand a matrix M∈RL×L, we letxα andMα denote theKα×1 subvector ofxformed by its firstKαcomponents, and theKα×Kα submatrix of Mformed by its first Kα rows and columns, respectively. Then, a feasible point for the dual problem is the vector π such that its first Kα components are given by
πα =α(ATα)−11α−(ATα)−1γα (6.20) and the remaining L−Kα components are equal to zero.
The value of the objective function (6.19) at this point is given by f(α) =bTα(ATα)−1γα+α β −bTα(ATα)−11α
(6.21)
It is not difficult to see thatf(α) is a continuous and piecewise linear function of α, forα≤[γ1, γL].
The assumption 1Tτ ≥ β can be rewritten as β−1TA−1b ≤ 0. Hence, forα > γs, for some 1≤s ≤L, the slope off(α) is negative, while forα < γs
the slope is positive. Therefore, the maximum of f(α) with respect to α is achieved for α = γs and, by definition, s is the minimum index in 1, . . . , L such that
Xs j=1
τj ≥ β, i.e., s = ˆL defined in (6.8). The primal objective function evaluated at the feasible point (6.8) is given by
(γ1, . . . , γs,0, . . . ,0)A−1b+γs
β−(1, . . . ,1
| {z }
s
,0, . . . ,0)A−1b
It is immediate to see that this coincides with the dual objective function f(α) evaluated at α = γs. Hence, we conclude that (6.8) is the sought
solution.
6.B Proof of Proposition 6.2
The proof follows immediately by observing that, for β ≤ bL, the program (6.13) can be reformulated as the ˆL-dimensional polymatroid program [99]
maximize
Lˆ
X
i=1
βiRi
subject to X
i∈S
βi ≤r(S), ∀S ⊆ {1, . . . ,Lˆ} β≥0
(6.22)
where the rank function r(S) is defined by r(S) =
maxX{S} i=1
∆i
where ∆i = bi − bi−1 for i = 1, . . . ,Lˆ − 1 and ∆Lˆ = β − bLˆ−1. Since (b0, . . . , bLˆ−1, β) is increasing, r(S) is submodular.
Chapter 7
Conclusions and Perspectives
Conclusion
In this thesis, we have proposed various low-complexity coding/decoding schemes to approach the capacity limits of binary-input symmetric-output channels and CDMA channels.
First, we have analyzed the belief propagation decoder of the infinite block length, systematic, random-like IRA code ensemble using density evolution, assuming transmission over a binary-input symmetric-output channel. We have tracked the evolution of message densities over the cycle-free bipartite graph, and studied the local stability condition of the fixed point, corre-sponding to zero bit error rate, of the resulting two-dimensional non-linear dynamical system.
We have then addressed the optimization of the IRA code ensemble for the class of binary-input symmetric-output channels. The code design makes use of three tools: the mutual information evolution described by the EXIT function, the reciprocal channel approximation and the non-strict convexity of mutual information on the set of binary-input symmetric-output channels.
We have proposed four methods to approximate densities involved in density evolution with a one-dimensional parameter, namely mutual information, thus yielding four low-complexity design methods of the IRA code ensemble,
125
that are formulated as linear programs. Again, local stability conditions of the fixed-points of the approximated density evolution systems have been derived. One of these one-dimensional DE approximation systems, based on Gaussian and reciprocal channel approximations, yields the same stability condition as the one under exact DE, while the exact stability condition has to be added in the optimization problem for other DE approximation methods, such as the one based and BEC a priori with reciprocal channel approximation.
We have optimized IRA codes for a wide range of rates, on the binary-input additive white Gaussian noise channel and the binary symmetric chan-nel, using the four design methods. Comparing the BER thresholds of these codes evaluated by the exact DE, it is found that the best approximations are those based on the Gaussian approximation, and the BER performances of the codes thus designed are comparable to those of the best LDPC codes of the same rate. IRA codes are then an attractive alternative due to the simplicity of their coding/encoding.
Next, we have studied the BER and WER performances of finite length regular and irregular repeat accumulate codes on the BIAWGNC in the er-ror floor region. Two approaches have been a adopted: girth maximization and minimum stopping set size maximization. The BER performances of girth-conditioned short-length regular RA codes are found to be above the performances of the random ML-decoded regular RA ensemble with uniform interleaving. They are also comparable to the performances of the best LDPC codes of the same rate. On the other hand, the BER performances of the large-block length irregular RA codes remain inferior to those of irregular LDPC codes with comparable length and graph conditioning.
After the analysis and design of IRA codes that closely approach the Shannon limit in the single user case, we have tackled the problem of ap-proaching the optimum spectral efficiency of the random CDMA channel in the large system limit. We have shown that the loss in spectral efficiency due to the use of QPSK inputs instead of Gaussian ones vanishes as the channel load becomes large.
We have then considered approaching the optimum CDMA with a low-complexity encoding/decoding setting, using capacity-achieving (approach-ing) binary error correcting codes, QPSK modulation and stripping decoder.
We have optimized the channel load distribution in two cases: the equal-rate system with equal SNR levels, and the equal-power system with non-equal rates, subject to the successive decodability condition imposed by the
127 stripping decoder. With the equal-rate system design, the spectral efficiency approaches the optimal spectral efficiency for low code rates. The equal-rate system design seems more attractive than the equal-power system design which requires a very wide range of rates to get close to the optimal. Numer-ical results have shown that these design methods can be used to dimension practical systems, and the resulting systems do not suffer from catastrophic error propagation.
Future Work
Irregular repeat accumulate codes can be seen as a special instance of irregu-lar turbo codes, with the constituent convolutional encoder having only two states. Although the threshold of the designed IRA code ensembles in this work are very close to Shannon limit and can hardly be improved any fur-ther, we believe that the finite length performance of irregular turbo codes (i.e. convolutional constituent encoders with larger memory) can bring a substantial improvement over that of irregular repeat accumulate codes for medium to large block lengths. This is a direction that should be investi-gated. It must be noted that the DE analysis of such codes presents much more difficulty than for IRA codes, because of the presence of cycles in the bipartite graph of such an irregular turbo code due to the increased memory.
Moreover, DE approximation methods 1 and 2 cannot be used, for the same reasons. However, one can still use methods 3 and 4 to design capacity-approaching irregular turbo codes, because these methods make use of the BCJR algorithm on the cycle-free graph of the convolutional code resulting from the use of hidden variables (states).
Another issue that could be investigated is the design of a low complex-ity scheme to approach the spectral efficiency of a cellular multiple access channel. In [100], Wyner proposed to model the received signal at a cell site as the sum of the noise, the signals transmitted from within the cell and a scaled version of the sum of signals from adjacent cells with scaling factor 0 ≤α≤1. However, users from adjacent cells do not cause the same inter-cell interference at the cell site, depending on how close they are from the edge of the cell. Therefore one could use different scaling factors that depend on the position of the adjacent interferer and accordingly optimize the power/rate profile of the users to maximize the spectral efficiency.
Publications
Repeat Accumulate Codes
• A. Roumy, S. Guemghar, G. Caire and S. Verd´u, Design Methods for Irregular Repeat-Accumulate Codes, IEEE International Symposium on Information Theory, ISIT 2003, Yokohama, Japan, June 29th - July 5th 2003.
• A. Roumy, S. Guemghar, G. Caire and S. Verd´u, Design Methods for Irregular Repeat-Accumulate Codes, submitted to IEEE Transactions on Information Theory, October 2002 (revised October 2003).
• S. Guemghar and G. Caire, On the Performance of Finite Length Ir-regular Repeat Accumulate Codes, 5th International ITG Conference on Source and Channel Coding, January 14-16 2004.
Coded CDMA under Successive Decoding
• G. Caire, S. Guemghar, A. Roumy and S. Verd´u,Design Approaches for LDPC-Encoded CDMA, 39th Annual Allerton Conference on Commu-nication, Control and Computing, Monticello, IL, October 3-5, 2001.
• G. Caire, S. Guemghar, A. Roumy and S. Verd´u Maximizing the Spec-tral Efficiency of Coded CDMA under Successive Decoding, IEEE Trans-actions on Information Theory, Vol. 50, Issue 1, Jan. 2004.