Machine Learning based channel codes for
the Binary Asymmetric Channel
Context
Since Shannon’s enunciation in 1948 of the channel capacity formula, i.e., the maximum rate at which two nodes can communicate over a noisy channel, error correction coding theory took on the task of building encoding and decoding algorithms which achieve this channel capacity. Major families of codes were developed throughout the years and proved to be capacity achieving such as Multi-level codes (Ten-Brink Wachsmann 1999), Turbo Codes (Glavieux and B´erou 1990), LDPC codes (Gallager 1960), and very lately, Polar codes (Arikan 2008). These codes were proved to be optimal for classical channels such as the Additive White Gaussian Channel (AWGN), the Binary Symmetric Channel (BSC) and the Binary Erasure Channel (BEC) which model respectively, for instance, thermal noise over a communication link, bit flips over a wired link, or packet losses.
For more challenging channel models such as the Binary Asymmetric Channel (BAC) which models for instance Free Space Optical (FSO) communications with On-Off Keying (OOK) or molecular communications, these classical codes are not capacity achieving. Indeed, optimal codes for the BAC were investigated in [1, 2] and it was proved that specific design criteria (probability distribution of codewords, non-linearity of the code) have to be met in order for a code to be capacity achieving for the BAC. Possible constructions were thus suggested in [2] based on modifications of polar codes and LDPC codes.
In this internship we will rather investigate the design of optimal channel codes for the BAC based on Machine Learning ML inspired algorithms, namely, deep neural networks [3]. The idea of building neural based communication blocks originated from the first set of works (see [4] and references therein) in which classical channel decoders were replaced with neural-based channel decoders which have a comparatively much lower complexity whilst maintaining quasi-optimal performances. The use of these neural based blocks was then extended to the encoder and transmitter as well, and the whole communication chain could then be viewed as an end-to-end optimized neural network [5] with a stochastic layer to model the noise, namely, an auto-encoder (cf Figure 1).
Figure 1: From block functional separation to auto-encoders
Resorting to auto-encoders in the communication chain leverages the historical block separation in digital communications, and thus allows to explore more intricate encoding and decoding schemes such as coded modulations, joint decoding and equalization as well as non-linear error correction codes. As such, they prove to be promising in the design of optimal channel codes for challenging channels such as the BAC.
Research program
In this internship we will investigate the design of auto-encoders for the BAC and compare them to the best family of codes known for this channel.
The research program will consist first in a state of the art on optimal code constructions for the BAC ( [1, 2]). Next, one of these possible codes will be selected and implemented so as to serve as a benchmark for later comparisons. Then, a state of the art on neural based communication systems will be conducted in order to understand the design of both channel decoders and overall auto-encoders for the BAC ([4, 5]). Then, a neural based channel decoder will be developed for the encoder selected as a benchmark, and compared to the optimal channel decoder (the Maximum A Posteriori decoder). Later, an overall auto-encoder will be developed and compared to the benchmark channel code. An analysis of the sensitivity of the auto-encoder to the training meta-parameters will be carried out as well as in [6, 7].
Candidate profile
Applicants should be last-year research master (or/and engineer) students. A strong background in digital communications and applied mathematics is required since the re-search assignment requires tools from information theory and error correction coding. Good communication skills in English are necessary, as well as good development skills (Python). Applications from candidates familiar with digital communications, information theory or error correction coding are particularly encouraged.
About
ISAE-Supa´ero is a leading European institute in system designs for aeronautical and space applications. The internship will take place in the Department of Electronics, Optronics and Signal processing (DEOS) of ISAE-Supa´ero. Among the department’s main research interests lies the design of “satellite communication systems with high spectral efficiency and enhanced security”, and this internship is directly related to this research activity.
• Applications (CV, cover letter) are to be addressed to: [email protected] (supervisor) and [email protected] (co-supervisor). • Application deadline: open until March 3rd 2020.
• Dates and duration: between March and October 2020 (5 to 6 months).
• Financial grant and accommodation and food services are available on the campus. • The contributions of this internship are intended for publication
References
[1] S. M. Moser, “Error probability analysis ofbinary asymmetric channels,” https://pdfs.semanticscholar.org/74b5/eb493ed7b2a7c1838cf57a44778e7b059a50.pdf, Tech. Rep.
[2] M. Mondelli, S. H. Hassani, and R. L. Urbanke, “How to achieve the capacity of asymmet-ric channels,” IEEE Transactions on Information Theory, vol. 64, no. 5, pp. 3371–3393, 2018.
[3] D. Kriesel, A Brief Introduction to Neural Networks, 2007. [Online]. Available: availableathttp://www.dkriesel.com
[4] T. Gruber, S. Cammerer, J. Hoydis, and S. ten Brink, “On deep learning-based channel decoding,” in 2017 51st Annual Conference on Information Sciences and Systems (CISS). IEEE, 2017, pp. 1–6.
[5] T. J. O’Shea and J. Hoydis, “An introduction to machine learning communications sys-tems,” arXiv preprint arXiv:1702.00832, 2017.
[6] M. Benammar and P. Piantanida, “Optimal training channel statistics for neural-based decoders,” in 2018 52nd Asilomar Conference on Signals, Systems, and Computers, Oct 2018, pp. 2157–2161.
[7] ——, “On robust neural-based decoders,” in 2019 53rd Asilomar Conference on Signals, Systems, and Computers, Nov 2019.
