High speed optical communications in silicon photonics modulators

(1)

High speed optical communications in silicon photonics

modulators

Thèse

Sasan Zhalehpour

Doctorat en génie électrique

Philosophiæ doctor (Ph. D.)

(2)

High speed optical communications in silicon

photonics modulators

Thèse

Sasan Zhalehpour

Sous la direction de:

(3)

Résumé

Les communications optiques basées sur la photonique sur silicium (SiP) sont au centre des récents efforts de recherche pour le développement des futures technologies de réseaux optiques à haut débit. Dans cette thèse, nous étudions le traitement numérique du signal (DSP) pour pallier aux limites physiques des modulateurs Mach-Zehnder sur silicium (MZM) opérés à haut débit et exploitant des formats de modulation avancés utilisant la détection cohérente. Dans le premier chapitre, nous présentons une nouvelle méthode de précompensation adap-tative appelée contrôle d’apprentissage itératif par gain (G-ILC, aussi utilisé en linéarisation d’amplificateurs RF) permettant de compenser les distorsions non-linéaires. L’adaptation de la méthode G-ILC et la précompensation numérique linéaire sont accomplies par une procédure « hardware-in-the-loop » en quasi-temps réel. Nous examinons différents ordres de modula-tion d’amplitude en quadrature (QAM) de 16QAM à 256QAM avec des taux de symboles de 20 à 60 Gbaud. De plus, nous combinons les précompensations numériques et optiques pour contrevenir surmonter les limitations de bande-passante du système en régime de transmission haut débit.

Dans le second chapitre, inspiré par les faibles taux de symbole du G-ILC, nous augmentons la vitesse de transmission au-delà de la limite de bande-passante du système SiP. Pour la première fois, nous démontrons expérimentalement un record de 100 Gbaud par 16QAM et 32QAM en transmission consécutive avec polarisation mixte. L’optimisation est réalisée sur le point d’opération du MZM et sur la DSP. Les performances du G-ILC sont améliorées par égalisation linéaire à entrées/sorties multiples (MIMO). Nous combinons aussi notre précompensation non-linéaire innovante avec une post-compensation. Par émulation de la polarisation mixte, nous réalisons un taux net de 833 Gb/s avec 32QAM au seuil de correction d’erreur (FEC) pour une expansion en largeur de bande de 20% et 747 Gb/s avec 16QAM (une expansion en largeur de bande de 7% du FEC).

Dans le troisième chapitre, nous démontrons expérimentalement un algorithme de précompen-sation numérique basé sur une table de consultation (LUT) unidimensionnelle pour compenser les non-linéarités introduites à l’émetteur, e.g. réponse en fréquence non-linéaire du MZM en silicium, conversion numérique-analogique et amplificateur RF. L’évaluation est réalisée sur un QAM d’ordre élevé, i.e. 128QAM et 256QAM. Nous examinons la diminution en complexité

(4)

de la LUT et son impact sur la performance. Finalement, nous examinons la généralisation de la méthode de précompensation proposée pour des jeux de données différents des données d’apprentissage de la table de consultation.

(5)

Abstract

Optical communications based on silicon photonics (SiP) have become a focus of the recent research for future high speed optical network technologies. In this thesis, we investigate digital signal processing (DSP) approaches to combat the physical limits of SiP Mach-Zehnder modulators (MZM) driven at high baud rates and exploiting advanced modulation formats with coherent detection.

In the first section, we present a novel adaptive pre-compensation method known as gain based iterative learning control (G-ILC, previously used in RF amplifier linearization) to overcome nonlinear distortions. We experimentally evaluate the G-ILC technique. Adaptation of the G-ILC, in combination with linear digital pre-compensation, is accomplished with a quasi-real-time hardware-in-the-loop procedure. We examine various orders of quadrature ampli-tude modulation (QAM), i.e., 16QAM to 256QAM, and symbol rates, i.e., 20 to 60 Gbaud. Furthermore, we exploit joint digital and optical linear pre-compensation to overcome the bandwidth limitation of the system in the higher baud rate regime.

In the second section, inspired by lower symbol rate G-ILC results, we push the baud rate beyond the bandwidth limit of the SiP system. For the first time, we experimentally report record-breaking 16QAM and 32QAM at 100 Gbaud in dual polarization back-to-back trans-mission. The optimization is performed on both MZM operating point and DSP. The G-ILC performance is improved by employing linear multiple input multiple output (MIMO) equal-ization during the adaptation. We combine our innovative nonlinear pre-compensation with post-compensation as well. Via dual polarization emulation, we achieve a net rate of 833 Gb/s with 32QAM at the forward error correction (FEC) threshold for 20% overhead and 747 Gb/s with 16QAM (7% FEC overhead).

In the third section, we experimentally present a digital pre-compensation algorithm based on a one-dimensional lookup table (LUT) to compensate the nonlinearity introduced at the transmitter, e.g., nonlinear frequency response of the SiP MZM, digital to analog converter and RF amplifier. The evaluation is performed on higher order QAM, i.e., 128QAM and 256QAM. We examine reduction of LUT complexity and its impact on performance. Finally, we examine the generalization of the proposed pre-compensation method to data sets other than the original training set for the LUT.

(6)

List of Tables

I.1 MZM platform properties . . . 2

(9)

List of Figures

I.1 Eye-diagrams of an OOK transmission at a) 11 Gbaud (larger eye-opening),

and b) 40 Gbaud (smaller eye-opening). . . 4

I.2 a) Transfer function of an MZM (response to large RF input is highlighted); (b) and (c) are received constellations of an experimental 256QAM transmission at 20 Gbaud with a LiNbO3 IQ modulator b) for small RF driving signal resulting

in equidistant constellation points, and c) for large RF driving signal resulting

in non-equidistant constellation points. . . 5

I.3 Nonlinear phase response of a silicon phase shifter. . . 6

I.4 E/O S21 reponse of a SiP MZM biased at zero, -0.75 V and -4 V [2]. . . 7

I.5 Block diagram of adaptive DPD techniques: a) ILA, b) DLA, and c) ILC [80]. . 11

1.1 Block diagram of iterative learning control (ILC) method. . . 21

1.2 Block diagram of DSP and experimental set-up and feedback loop for the G-ILC

predistortion method. . . 22

1.3 Received constellations at optical power of 2 dBm for 20 Gbaud/256QAM: (a) linear predistortion at TX, (b) linear predistortion at TX and linear postcom-pensation at RX, (c) G-ILC predistortion at TX, and (d) G-ILC predistortion at TX and linear post-compensation at RX. Black-red-yellow is the transition

from lowest to highest density of samples. . . 24

1.4 Received constellations at optical power of 2 dBm at 40 Gbaud/128QAM: (a) linear predistortion at TX and linear postcompensation at RX, and (b) G-ILC predistortion at TX and linear post-compensation at RX. Black-red-yellow is

the transition from lowest to highest density of samples. . . 25

1.5 BER performance versus optical received power for a) 256QAM at 20 Gbaud, and b) 128QAM at 40 Gbaud which correspond to constellation plots in Figs. 3

and 4, respectively. . . 26

1.6 Optical pre-emphasis filter response for 32QAM at 60 Gbaud. . . 27

1.7 Received 32QAM constellations at optical power of 2 dBm at 60 Gbaud. (a) linear predistortion at TX and linear postcompensation at RX, and (b) G-ILC predistortion at TX and linear post-compensation at RX. Black-red-yellow is

transition from lowest to highest density of samples. . . 27

1.8 BER performance versus optical received power for 32QAM at 60 Gbaud. . . . 28

1.9 BER for different modulation orders, M-QAM (M = 32, 64, 128 and 256) at 2 dBm optical received power before CoRx at 20 Gbaud, 40 Gbaud, and

60 Gbaud. . . 29

2.1 (a) Schematic diagram of SiP IQ modulator [114], and (b) experimental set-up

(10)

2.2 BER vs. bias voltage when using linear compensation. . . 35

2.3 High level block diagram of a signal processing (insets 32QAM at 100 Gbaud) . 36

2.4 Block diagram of the experimental set-up (grey shading) and offline DSP. . . . 36

2.5 Block diagrams of two DSP methods applied during training phase; convergence at 100 Gbaud for 16/32QAM: dashed lines using method A (upper) DSP, and

solid lines using method B (lower) DSP. . . 40

2.6 32QAM constellations (a) without and (b) with improved G-ILC. . . 41

2.7 BER sweep for 100 Gbaud DP-16/32QAM without G-ILC, but with post-compensation: DD-MMSE (×), adding memory polynomial (◦), adding MLSD

(), and adding memory polynomial and MLSD (+).. . . 43

2.8 BER vs. received power for dual polarization 16QAM and 32QAM at 100 Gbaud with nonlinear pre-compensation (trained G-ILC); post-compensation cases are linear DD-MMSE only (×), linear and nonlinear polynomial alone (◦), linear

and nonlinear MLSD alone (), and all post-compensation together (+).. . . . 45

2.9 Spectra at various points in linear compensation optimization: (a) after MZM, without any pre-compensation, (b) optical bandpass filter response with γ to control depth, (c) after OBF, and (d) after a MMSE equalizer (i.e., all linear

techniques applied). . . 46

2.10 BER vs. received power for single polarization 16QAM and 32QAM at 100 Gbaud, with nonlinear pre-compensation (trained G-ILC); post-compensation cases are linear DD-MMSE only (×), linear and nonlinear polynomial alone (◦), linear,

and all post-compensation together (+). . . 47

3.1 LUT-based predistortion block diagram where SW(n) indicates sliding window

with length n.. . . 53

3.2 DSP block diagram and experimental setup; inset is E/O S21 of SiP IQ

modu-lator for several reversed DC bias voltages [100]. . . 55

3.3 When using linear equalizers alone, constellations for a) 20 Gbaud 256QAM, b) 40 Gbaud 128QAM, and average absolute error per symbol for c) 20 Gbaud

256QAM, and d) 40 Gbaud 128QAM. . . 56

3.4 In-phase PDD vs. pattern index, for 20 Gbaud 256QAM in a) 3, b) LUT-5, and c) LUT-7, and for 40 Gbaud 128QAM in d) LUT-3, e) LUT-LUT-5, and

f) LUT-7; mean and variance are given in insets. . . 58

3.5 BER when using linear equalizer alone, and in combination with full-size LUT with memory depth n=3, 5, and 7 for a) 20 Gbaud 256QAM, and b) 40 Gbaud

128QAM. . . 60

3.6 For LUT-7, 20 Gbaud 256QAM in a) predistorted constellation, b) mean of the absolute value of the predistortion, and c) its standard deviation, and for 40 Gbaud 128QAM in d) predistorted constellation, e) mean of the absolute

value of the predistortion, and f) its standard deviation. . . 61

3.7 R-LUT for n = 3 and 20 Gbaud 256QAM: a) absolute PDD vs. pattern index with four horizontal threshold levels are set by the percentage of PDD values above the threshold, and b) for the 20% threshold, the distribution of patterns retained as a function of the center symbol of the pattern for both in-phase and

quadrature R-LUTs. . . 62

3.8 For 20% R-LUT for n = 3 and 20 Gbaud 256QAM, distribution of first and third symbol of a) patterns with middle symbol of -15, and b) patterns with

(11)

3.9 PDD of I-LUT-3 for 20 Gbaud 256QAM with pattern indexes grouped by center

symbol, for a) full-size LUT and b) 20% R-LUT. . . 64

3.10 Left axis (× markers) is BER vs. R-LUT size in percentage at 2 dBm optical received power, right axis (square markers) is R-LUT size, for a) 20 Gbaud

256QAM, b) 40 Gbaud 128QAM. . . 65

3.11 BER vs. data set PRBS order at 2 dBm optical received power; training always

on PRBS24. . . 66

3.12 20 Gbaud 256QAM predistorted constellations, ˜X, for a) LUT-3, b) LUT-5,

and c) LUT-7. . . 67

3.13 40 Gbaud 128QAM predistorted constellations, ˜X, for a) LUT-3, b) LUT-5,

and c) LUT-7. . . 67

3.14 Received constellations for linear equalizer and LUT-7 at a) 20 Gbaud 256QAM,

and b) 40 Gbaud 128QAM. . . 69

3.15 Electrical spectrum of 256QAM at 20 Gbaud a) with RC pulse shaping filter alone, b) with RC filter and MMSE pre-distortion, and c) with RC filter, MMSE

(12)

List of Abbreviation

ADC analog-to-digital converter AI artificial intelligence

ASE amplified spontaneous emission BER bit error rate

BPSK binary-phase-shift-keying

CMOS complementary metal-oxide semiconductor CoRx coherent receiver

CPR carrier phase recovery DAC digital-to-analog converter

DD-LMS decision-directed least-mean-squares DLA direct learning architecture

DP-QPSK Dual-polarization quadrature phase shift keying DPC digital post-compensation

DPD digital pre-distortion DSP digital signal processing ECL external cavity laser

EDFA erbium-doped fiber amplifier ENoB effective number of bits

(13)

ER extinction ration

FEC forward error correction FFT fast Fourier transform FIR finite-impulse-response

FMCW frequency-modulated continuous-wave FOC frequency offset compensation

FPGA field programmable gate arrays G-ILC gain based ILC

GS geometric shaping

GS-SG ground signal-signal ground ILA indirect learning architecture ILC iterative learning control

IM/DD intensity modulation and direct detection InP Indium Phosphide

ISI intersymbol interference LiDAR light detection and ranging LiNbO3 Lithium Niobate

LMS least mean square LO local oscillator LPF low-pass filter LS least square LUT lookup table

(14)

MIMO multiple input multiple output

MLSD maxmimum likelihood sequence detector MLSE maximum likelihood sequence estimation MMSE minimum mean square error

MP memory polynomial MSE mean square error

MZM Mach-Zehnder modulator OBF optical bandpass filter OBPF optical bandpass filter ODL optical delay link OOK on-off keying

PAM pulse amplitude modulation PBC polarization beam combiner PBS polarization beam splitter PC polarization controller PD photodiode

PDD pattern dependent distortion PDL polarization dependent loss PDM polarization division multiplexing PMD polarization mode dispersion PRBS pseudo random bit sequence PRBS pseudorandom binary sequence PS phase shifter

(15)

PS probabilistic shaping PSD power spectral density

QAM quadrature amplitude modulation QPSK quadrature phase shift keying RLS recursive least squares

RLUT reduced sized lookup table RTO real-time oscilloscope SDM space division multiplexing Si silicon

SiP Silicon Photonics SiP silicon photonics SNR signal-to-noise ratio TDM time division multiplexing TIA trans-impedance amplifier

VCSEL vertical-cavity surface-emitting laser VOA variable optical attenuator

WDM wavelength division multiplexed WLUT weighted lookup table

(16)

My beloved mother (my hero), father, and sister. Also, I have CrossFit to thank for waking me up at 5:30 am to do the workout and start my day early to be able to finish my Ph.D.

(17)

You have to die a few times before you can really live.

(18)

Acknowledgements

Now, I look at all the years that I have spent to make this work done, I clearly see that I could not do it without many people who contributed to it. For that I want to thank them. Firstly, I would like to thank my parents and sister, especially my mother who always believes in me. Having the support of people who love me and encourage me is like a light in my dark days. Secondly, I would like to thank my supervisor, professor Leslie Rusch. During the last four years, I faced a lot of obstacles in my research but she always encouraged me not to give up and be optimistic. She taught me to focus on the solutions rather than the problems. Her office door is always open to students to ask their questions. She always has a smile on her face and her words are so powerful that can motivate any student to turn an impossible work into a success. I am honored to work under her supervision.

I would like to thank the members of jury, Prof. Sophie LaRochelle, Prof. Paul Fortier, and Prof. René-Jean Essiambre, for taking the time to read my thesis and for valuable comments and suggestions.

Moreover, I am grateful to all my friends in COPL for their friendship and support. Siamak Amiralizadeh, Hassan Sepehrian, Amin Yekaneh, and Jiachuan Lin whom I had a lot of fruitful discussion about my work. Jiachuan Lin has spent a lot of time in the lab to patiently explain me the details of the set-up to make a better experimental work. I would like to thank Mengqi Guo. She is a very hardworking student and we have spent a lot hours in the lab to perform difficult experiments. Jiachuan Lin, Mengqi Guo and I have worked a lot of weekends and late nights to catch the deadlines.

I am grateful for my friends, Philippe Jean and Jean-Michel Vallée who helped me to translate some sections of this thesis in French. I also would like to thank my good friends, Yigit Ozan Aydin, Laurent Séguin, Mingyang Lyu, Omid Jafari, Rizan Homayoun Nejad, and Alessandro Corsi for their friendship, love and moral support.

Doing research for more than four years in the university, working late nights and weekends in the lab, facing fear of failure were not possible without the help and support of my family, supervisor, colleagues and friends. I am deeply grateful to have them in my life.

(19)

Foreword

Three chapters (one, two and three) of this thesis are based on conference and journal papers, both published and submitted. The introduction of each chapter includes additional remarks to those appearing in the papers, to aid the reading of the thesis as a whole. I was the main contributor to these papers. All publications are based on experimental work using a silicon photonic (SiP) IQ modulator designed by Hassan Sepehrian under the supervision of Wei Shi. In what follows, the responsibility and contribution of each co-author of papers used for this thesis will be explained.

Chapter 1: Sasan Zhalehpour, Jiachuan Lin, Hassan Sepehrian, Wei Shi, and Leslie A. Rusch, "Mitigating pattern dependent nonlinearity in SiP IQ-modulators via iterative learning con-trol predistortion," in Optics Express, Vol. 26, Issue 21, pp. 27639-27649, 2018. This paper demonstrates and experimentally validates a novel pre-distortion algorithm to overcome non-linear distortion in the SiP coherent detection system. A subset of results from this journal paper also presented at ECOC 2018 [1].

I developed the central DSP algorithm, i.e., adaptive iterative learning control (ILC) technique, which we used in this paper. I analyzed the captured data and investigated the performance of linear equalization compared to the ILC. Jiachuan Lin, a postdoc assigned to this project, and Hassan Sepehrian, who designed the chip, assisted me with preparation of the experimental set-up. Jiachuan Lin and I discussed options of overcoming the limitations of the SiP modulator. Wei Shi provided insight on efficient exploitation of the SiP modulator. Leslie Rusch, who was the supervisor of the project, provided overall guidance, as well as input on the DSP and analysis of results. The paper was written by me and revised by Leslie Rusch.

Chapter 2: Sasan Zhalehpour, Mengqi Guo, Jiachuan Lin, Zhuhong Zhang, Yaojun Qiao, Wei Shi, and Leslie A. Rusch, "System Optimization of an All-Silicon IQ Modulator: Achiev-ing 100 Gbaud Dual Polarization 32QAM," Journal of Lightwave Technology, in revision Sept., 2019. This paper reports the record braking transmission in SiP modulator, achiev-ing the dual polarization 32QAM at 100 Gbaud. The subset of the results includachiev-ing sachiev-ingle polarization transmission was presented at the OFC 2019 postdeadline session [2].

(20)

I developed and verified several DSP algorithms to improve performance, mostly focused at the transmitter side. Mengqi Guo, a one-year visiting PhD student from the School of Information and Communication Engineering at BUPT, helped me with the experimental set-up preparation; she contributed the nonlinear DSP algorithms for the receiver side processing. Yaojun Qiao is the supervisor of Mengqi Guo from Beijing University.

Jiachuan Lin, the postdoc in our team who subsequently joined Huawei (our industrial part-ner), assisted Mengqi Guo and me in performing the experiment and developing the DSP algorithms. Jiachuan Lin, Mengqi Guo and I had many fruitful discussions. Zhuhong Zhang is our industrial collaborator from Huawei who provided us with valuable practical advice for improving our transceiver system.

Wei Shi provided insight on efficient exploitation of the SiP modulator. Leslie Rusch, who was the supervisor of the project, provided overall guidance, as well as input on the DSP and analysis of results. The paper was written by me and revised by Leslie Rusch; Mengqi Guo provided text on the nonlinear post-processing.

Chapter 3: Sasan Zhalehpour, Jiachuan Lin, Wei Shi, and Leslie A. Rusch, "Reduced-size lookup tables enabling higher-order QAM with all-silicon IQ modulators," in Optics Express, Vol. 27, Issue 17, pp. 24243-24259, 2019. This paper experimentally validates the effectiveness of several types of lookup table (LUT) based pre-distortion algorithms for pre-compensation. The results were presented in part at OFC 2019 [3].

I ran all experiments and developed the LUT pre-distortion algorithms and verified them on the system under test. I provided an analysis of the nonlinear behavior of system, with some assistance from Jiachuan Lin and Leslie Rusch. Jiachuan Lin assisted me with preparation of the experimental set-up and experimental advice. Jiachuan Lin and I has a very productive discussion while investigating the source of silicon nonlinear distortion. Wei Shi provided guidance on operating the SiP modulator. Leslie Rusch provided overall guidance for the project as well as advice on DSP and data analysis. The paper was prepared by me and revised by Leslie Rusch.

(21)

Introduction

I.1 Motivation

Today, optical communications technology as the basis of modern network infrastructures, i.e., in data-center and cloud based platforms, has been expanding rapidly. Consequently, the demand for spectrally efficient transmission has exponentially increased. According to the Cisco’s global cloud index prediction, data center traffic due to the hungry bandwidth services, i.e., cloud applications, artificial intelligence (AI) applications and machine learning markets will be tripled between 2016 and 2021 [4;5;6]. Following the demonstration of the digital signal processing (DSP) application in coherent optical communications in 2005 (digital carrier-phase estimation) [7], coherent detection systems along with DSP algorithms have been continuously evolving to meet high demand of bandwidth at lower cost and energy consumption, particularly in short-reach transmissions. Coherent transceivers were originally developed for long-haul transmission systems [8], however, due to the high achievable performance, they have also been considered for short-reach applications, i.e., intra-data-center application. In the past decades, the expansion of optical communications has been carried out by scaling the multiplexing and modulation dimensions, device development and DSP advancement, which will be briefly discussed in the following paragraphs.

I.1.1 Modulation and mutliplexing

The coherent detection system is a promising solution for spectrally efficient transmissions. The spectral efficiency can be scaled up by optical signal modulations in the quadrature dimen-sion, i.e., M-ary quadrature amplitude modulation (M-QAM) by modulating both intensity and phase of the optical carrier [9]. There have been huge explorations in higher order QAM transmission in optical communications from quadrature phase shift keying (QPSK) to 2048-QAM single carrier transmission [10]. Other methods to further increase the transmission capacity are included: polarization division multiplexing (PDM) using two orthogonal polar-izations to transmit two independent optical signals [11], space division multiplexing (SDM) enabling multiplexing several spatially distinguishable paths (channels) in the space over a sin-gle fiber to override the maximum achievable transmission rate [12], time division multiplexing (TDM) to combine several optical or electrical signals at the transmitter side (interleaving the

(22)

Table I.1: MZM platform properties

Property LiNbO3 InP SiP

Vπ 1.5-4 V 1.5-4 V 5-6 V

Insertion loss 3-5 dB 3-5 dB > 6 dB

Bandwidth >35 GHz > 35 GHz 30-35 GHz

electro-optical effect Pockels Quantum-confined stark Plasma dispersion

footprint Large Small Small

CMOS compatibility No No Yes

Reported line rate (Max.) 600 Gbps 500 Gbps 500 Gbps

digital or optical pulses in time and transmitting simultaneously) [13], and wavelength division multiplexing (WDM) to simultaneously transmit several data streams in different independent wavelengths to increase the bandwidth of the transmission over a single fiber [11].

I.1.2 Device development

Optical external modulators are key components to achieve high performance transmission systems. The typical design of an external modulator is a Mach-zehnder modulator (MZM), which is mainly fabricated based on three platforms including Lithium Niobate (LiNbO3),

Indium Phosphide (InP), and Silicon Photonics (SiP), each of which offers different properties in terms of Vπ, insertion loss, bandwidth, electro-optical modulation, footprint, scalability,

and achievable line rate per wavelength and polarization. The Vπ indicates the modulator

voltage required to achieve a phase difference of π between two arms. Table I.1presents the summary of these platforms’ properties.

The LiNbO3 platform is the most well-known material which has been used from the first

demonstration of the intensity modulation and direct detection (IM/DD) transmission at 2.5 Gb/s [14]. LiNbO3 and InP MZMs have similar Vπ (on the order of 1.5-4 V); SiP has

higher Vπ . However, in terms of length, LiNbO3 MZM requires greater length, which causes

limitation for dense integration in contrast to InP and SiP. The insertion loss and bandwidth of LiNbO3 and InP are in the range of 3-5 dB and > 35 GHz, respectively unlike SiP, which

has higher insertion loss and lower bandwidth [15]. SiP technology offers complementary metal-oxide semiconductor (CMOS) compatibility, which makes co-integration of electronics and photonics on a single chip feasible unlike LiNbO3and InP. This makes SiP based platforms

a promising technology for the next generation of data center architectures [16].

The refractive index change is realized by the Pockels effect in LiNbO3 modulators, which

exhibits a linear behavior with applied voltage [17]. However, the electro-optical modulation in InP is achieved by the quantum-confined stark effect that has a nonlinear characteristic, and SiP modulator operates based on the plasma dispersion effect (nonlinear electro-optical modulation) [18; 19]. Up to now, maximum line rates (per polarization and wavelength)

(23)

of 600 Gb/s [20], 500 Gb/s [21], and 500 Gb/s [2] were reported for LiNbO3, InP and SiP

platforms, respectively.

I.1.3 DSP advancement

DSP algorithms simplify the compensation of the various optical and electrical impairments in the digital domain instead of exploiting expensive physical compensation methods. DSP techniques enable the current high speed optical communications [22; 23]. One of the first demonstrations of DSP for optical communications was linear pre-compensation of chromatic dispersion in 2005 [24]. Coherent transmission for dual-polarization quadrature phase shift keying (DP-QPSK) at 40 Gb/s was presented in 2007 from Nortel/Ciena where the DSP algorithms were used to mitigate chromatic dispersion, polarization mode dispersion (PMD) and polarization dependent loss (PDL) [8]. Nowadays, due to the availability of advanced digital processing devices, compensation of different types of impairments, i.e., intersymbol interference (ISI) [25], IQ imbalance [26], time misalignment [27], polarization demultiplexing [28], and phase noise [29;30] can be accomplished by DSP techniques. Furthermore, pushing the performance beyond the transceiver’s limit is feasible through implementing different types of DSP algorithms, for instance, pulse shaping filters, i.e., Nyquist pulse shaping [31; 32], probabilistic, and geometric pulse shaping (i.e., non-square QAM constellations) [33; 34], which provide higher spectral efficiency. In the next section, we will introduce some of the common impairments that restrict the performance, followed by a brief review of the current DSP techniques to overcome these challenges.

I.2 System impairments

Prior to reviewing DSP techniques used to enhance the spectral efficiency, we explain the most common impairments limiting the end-to-end performance of such systems. The impair-ments resulting from bandwidth limited systems using higher order modulation formats can be categorized in two main subgroups, including linear and nonlinear impairments. In this background section we treat each impairment in isolation. The thesis addresses compensation of the combined effects of linear and nonlinear impairments whose interactions are complex and difficult to model.

I.2.1 Linear impairments

Linear distortion can be caused by the bandwidth limitation in components, i.e., digital-to-analog converter (DAC), RF amplifiers, analog-to-digital converter (ADC), electro-optical external modulator, and photodetectors. In addition, the transmitter I/Q skew and crosstalk in polarization multiplexing introduce linear distortion; these effects are more pronounced at higher baud rates [35]. Modulators could be considered as the bottleneck of the system in terms of 3 dB bandwidth. As mentioned in sectionI.1.2, SiP MZMs have narrower bandwidth

(24)

(a)

(b)

Figure I.1: Eye-diagrams of an OOK transmission at a) 11 Gbaud (larger eye-opening), and b) 40 Gbaud (smaller eye-opening).

compared to other materials, i.e., LiNbO3. Therefore, achieving higher baud rate in SiP

MZMs is more challenging.

Operating the system beyond the 3 dB bandwidth results in ISI impairment, as the transceiver introduces memory to the channel. ISI leads to closer eye opening. It consequently results in a lower tolerance margin to the amplified spontaneous emission (ASE) noise and therefore, higher probably of error occurrence [36]. The ISI impact due to the limited bandwidth (a SiP MZM with 3 dB bandwidth of 22 GHz) on the eye-opening is shown in Fig. I.1a,b where on-off keying (OOK) signal is transmitted at 11 Gbaud (Fig. I.1a) and 40 Gbaud (Fig.I.1b). IQ imbalance caused by non-equal power split in IQ modulator branches and/or MZM arms, i.e., fabrication error in the 3 dB splitter in the IQ modulator, unequal responsivity of photode-tectors, or trans-impedance amplifier (TIA) in the receiver module, results in finite extinction ratios (ER) [37]. It also severely reduces the performance of a higher order QAM due to much smaller decision regions.

I.2.2 Nonlinear impairments

Nonlinear distortion is one of the major causes of performance limitation, particularly at higher bit rates. Increasing the driving signal, scaling up the baud rate, and improper adjustment of the component operating point can all push the system into a nonlinear regime. We discuss the common nonlinear impairments in SiP communications in the following.

The modulator response is inherently nonlinear due to the MZM sinusoidal transfer function. When driven with high peak-to-peak RF signal compared to the Vπ of the MZM [37] this

(25)

L2 Input RF Ou tp u t L1 L3 L2 > L1 = L3 L3 L2 L1 L1 = L2 = L3 (a) -10 0 10 In-Phase -15 -10 -5 0 5 10 15 Quadrature (b) -10 0 10 In-Phase -15 -10 -5 0 5 10 15 Quadrature (c)

Figure I.2: a) Transfer function of an MZM (response to large RF input is highlighted); (b) and (c) are received constellations of an experimental 256QAM transmission at 20 Gbaud with a LiNbO3 IQ modulator b) for small RF driving signal resulting in equidistant constellation

points, and c) for large RF driving signal resulting in non-equidistant constellation points.

nonlinear effect is heightened. FigureI.2a illustrates the transfer function of an MZM operating at the quadrature point. Although the driving signal on the x-axis has equidistant levels (L1=

L2= L3), the modulated signal on the y-axis suffers from unequal power levels (L2> L1= L3).

This asymmetric power distribution will produce asymmetric eye openings. The impact of the LiNbO3 transfer function nonlinearity is shown experimentally in FigI.2b,c; b) 256QAM

back-to-back transmission at 20 Gbaud, where the MZM is operated in the linear region (small RF driving signal) and c) 256QAM constellation in the same scenario but larger RF driving signal (nonlinear response of the MZM).

Furthermore, the phase shifters in a SiP MZM have nonlinear behavior (voltage-dependent characteristic), which leads to nonlinear phase and attenuation in response to the DC voltage [19]. The nonlinear phase response of a reversed bias SiP phase shifter is illustrated in Fig.I.3.

(26)

-10 -8 -6 -4 -2 0 Voltage 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6

Phase shift [rad/mm]

Phase

Figure I.3: Nonlinear phase response of a silicon phase shifter.

The mathematical presentation of a refractive index and attenuation change of a SiP phase shifter at 1.55 µm has been explained in [38].

The small signal frequency response is shown in Fig. I.4. We see it varies with the operating DC bias. The 3 dB bandwidth of the SiP MZM in Fig. I.4is ∼22 GHz at zero bias, ∼26 GHz at -0.75 V, and ∼34 GHz at -4 V. In theory, we could increase the bias voltage to have higher bandwidth (Fig. I.4). However, changing the bias voltage also changes the voltage required for a given phase. So despite increasing bandwidth, our performance could decrease due to reduced modulation efficiency (for a given drive voltage swing).

Consider what happens when the drive voltage swing is large. The frequency response would appear to vary as the voltage swings. This implies the modulator bandwidth changes dynam-ically at large driving signal. This phenomenon is dominant when driving the system beyond the nominal bandwidth of the SiP MZM; it becomes more noticeable at higher order QAM. Quantization noise due to the limited resolution of the DAC has a nonlinear impact on the performance. Operating the DAC at high baud rate regimes, i.e., 100 Gbaud and beyond gets more challenging since, the DAC shows small effective number of bits (ENoB) at higher sampling rate and baud rate. The small ENoB combined with limited bandwidth restricts the dynamic range of the DAC. These effects can also lead to a sharp roll-off in the spectrum. Together these effects limit the overall efficiency of the digital pre-distortion in the DAC. Therefore, transmitting higher modulation order will be problematic. The quantization noise has been extensively studies in the literature [39;40;41].

Other sources of distortion include in the RF amplifier nonlinearity while driving in the satu-ration region, square-law detection in photodetectors, and phase noise that further affect the end-to-end performance.

(27)

0 10 20 30 40 Freq. [GHz] -10 -8 -6 -4 -2 0 2 S21 [dB] _{-4 V} -0.75 V 0 V

Figure I.4: E/O S21 reponse of a SiP MZM biased at zero, -0.75 V and -4 V [2].

In the presence of all above mentioned distortions, achieving higher spectrally efficient trans-mission (i.e., higher order modulation) is more challenging. The low cost advantage of SiP is most immediately of interest for short-distance applications. Therefore we concentrate on these links where impairments from the modualtor will dominate performance. In particular, fiber degradations are not discussed in this thesis.

I.3 DSP techniques to compensate impairments

Digital compensation techniques implemented at the transmitter side are denoted as digital pre-distortion (DPD). Digital compensation techniques at the receiver side (by off-line DSP process) are called digital post-compensation (DPC). The DPD algorithm mostly deals with the limited bandwidth of components, low resolution of the DAC, and nonlinear characteristic of the components at the transmitter side [42;43]. The DPC algorithms compensate residual linear and nonlinear distortions.

The reason for implementing DSP compensation algorithms at the transmitter side rather than applying all digital compensation at the receiver can be explained as follows; the trans-mitted signal impairments are combined with the distortion in the channel (e.g., DAC or MZM) and the receiver side impairment (e.g., fiber nonlinearity). While, employing receiver side post-compensation, the noise is usually enhanced by DPC algorithms, which makes the compensation less effective. The noise enhancement at the receiver side will have less impact when the DPD algorithm is implemented at the transmitter side. Therefore, the DPD acts complementary to the DPC algorithm to enable the high data rate [44;43].

An MZM and its drivers are often modeled by cascading a memoryless nonlinearity with a linear filter [45]. The linear filter captures the bandwidth limited frequency response. The nonlinear characteristic of the MZM, along with drivers, is assumed memoryless. The inverse

(28)

of this model is then the cascade of the inverse of the linear filter followed by the inverse of the instantaneous nonlinearity [46].

In our approach we do not assume a specific underlying model, but we do adopt the general approach of cascading linear and nonlinear DPD. The DSP methods used to develop the DPD are discussed in the following sections.

I.3.1 Linear digital pre-distortion techniques

Linear pre-distortion applies the inverse of the system transfer function to the signal. For an unknown system model, we can estimate this inverse experimentally. We transmit a known binary-phase-shift-keying (BPSK) signal at high signal-to-noise ratio, and apply the fast Fourier transform (FFT) to the transmitted and received signals. The ratio of the trans-mitted and received transforms is the estimate of the inverse transfer function. The inverse FFT (IFFT) of the this estimate yields a finite-impulse-response (FIR) filter to pre-distort the signal; it can be applied to any modulation format [47; 48; 49]. This is known as the zero-forcing equalizer.

An alternate solution to estimate the FIR filter coefficients is based on the stochastic gradient approach. For instance, a decision-directed least mean squares (DD-LMS) equalizer could be used. This is a minimum mean squared error (MMSE) equalizer. In our work, either approach can be used depending on the experimental situation.

The estimation of linear DPD is usually done in single polarization to avoid polarization crosstalk [44;50;51]. This DPD is then applied to each polarization.

I.3.2 Nonlinear digital pre-distortion techniques

The nonlinear DPD algorithms can be categorized in adaptive and non-adaptive forms. Non-adaptive methods estimate parameters (model coefficients, lookup table entries, etc.) during a characterization phase. The adaptive DPD methods use a feed-back loop, either in the real-time or off-line processing, to perform coefficient estimation. In chapters 1 and 2 we adopt an adaptive approach, while in chapter 3 we use a non-adaptive technique.

Non-adaptive nonlinear digital pre-distortion techniques

There are various types of non-adaptive DPD techniques including the Volterra series, lookup table, and constellation shaping, which are briefly discussed in the following.

I Volterra series

A well-known model to describe system nonlinearity is the Volterra series [52; 46]. The Volterra series imposes huge computational complexity to the DPD processing load due to the large set of coefficients. The special case of the Volterra series, known as the memory

(29)

polynomial (MP), has fewer coefficients and is more popular [53;54]. The MP model is built based on the diagonal kernels in the Volterra series [55;56].

The coefficient estimation is performed directly by least squares (LS) solution, or adaptive estimation. The adaptive algorithms i.e., least mean square (LMS) and recursive least squares (RLS), are carried out on a sample-by-sample basis, which minimizes storage requirements at the cost of slow convergence [57].

A modified Volterra based DPD with reduced dimensions is proposed in [58;59] to com-pensate the impairments caused by the MZM. In [60], an MP model is used to numerically pre-distort the transmitted signal in a SiP MZM under the assumption of large driving signal in comparison with the Vπ. An LS solution is used to estimate the coefficients.

II Lookup table

The lookup table (LUT) is an alternative solution to mitigate the nonlinear distortion A LUT is composed of a table with pattern address indexes and a correction vector. The vector is a correction term applied to the amplitude of the center symbol of each distinguishable pattern [61; 62; 63; 64; 65; 66]. The practical implementation of LUT based DPD in a field programmable gate array (FPGA) is presented in [22;67]. Several modified LUT schemes have been presented so far, i.e., the weighted LUT (WLUT) by adding some weights for better compensation to the pattern [68], scaled LUT for only the outer constellation points [69], and reduced sized LUT (RLUT) based on evaluating the pattern dependent error [70]. The LUT method will be investigated thoroughly in chapter 3.

III Constellation shaping

Constellation shaping, including geometric shaping (GS) and probabilistic shaping (PS), can be applied to improve the system robustness. Rectangular M-QAM modulations are popular, however, the nonlinearity affects higher power symbols disproportionately. Constellation shaping is used to improve the noise tolerance, i.e., lower bit error rate for a given power level. In a typical rectangular QAM constellation, all points are equiprobable and nearest neighbors are equidistant. In contrast, in the GS-QAM, the distribution of constellation points is non-uniform and in PS-QAM the probability of symbols varies, i.e., via PS-QAM more inner points are transmitted than the outer ones [71;72]. The regular (square) and star-shaped 16QAM performance are compared with and without PS in [71]. Moreover, it is shown that PS-star-16QAM outperforms PS-square-16QAM. In [73], constellation shaping is used to demonstrate the PS-64QAM at 24.6 Tb/s over 10,285 km transmission. These solutions could improve performance in nonlinear systems.

Adaptive nonlinear digital pre-distortion techniques

DPD techniques based on behavioral models, i.e., Volterra series and PM, are used to obtain the inverse nonlinear transfer function of the system under test. The complexity of the models

(30)

is high, so that practically we are limited to a small number of coefficients for our DPDs. There are two approaches for adaptation systems using a behavioral model: direct and indirect learning architectures [57]. To estimate the pre-distorted signal without the structure of a behavioral model, we can use iterative learning control. We briefly review these techniques in this section.

I Direct and indirect learning architectures

Two well-known approaches to adaptively estimate the coefficients of a DPD function, i.e., Volterria series, are indirect [74] and direct [75] learning architectures (ILA, DLA). The block diagrams of ILA and DLA adaptation scheme are shown in Fig. I.5a and b, respectively, where x(n) and y(n) denote the input and output signals. The error generated while performing adaptation is e(n), and yd(n)is the desired output.

In both approaches, the coefficients can be extracted by using several techniques, e.g., LS method from a set of measured input and output signals, or adaptive LMS/RLS algorithms on a sample-by-sample basis [57]. Alternatively, after an initial estimation, coefficients can be updated by a method such as "damped Newton algorithm" [76]. In the ILA, the post-inverse filter’s coefficients are first estimated in the training phase through minimizing the error signal (in the feedback path). Then, the estimated coeffi-cients are used in the function of the DPD as presented in Fig. I.5a. The pre-distorter filter is an identical copy of the post-inverse filter. However, in the DLA approach, the transfer function of the system under test is estimated by minimizing the error between the actual output and desired output signals. The transfer function estimate is then used for pre-distorting the transmitted data. The DLA based approach requires higher com-putational complexity than the ILA one and also it suffers from slower convergence speed [77].

An ILA approach is used to estimate the MP based DPD model for an MZM in [78]. The applied ILA has a form of inverse control of the nonlinear MZM transfer function (an LS solution is used to extract the coefficients during adaptation). A similar approach, using MP based DPD and an ILA approach for adaptation, is used as a nonlinear DPD for I/Q imbalance and nonlinear distortion from a DAC, drivers and an MZM in [79]. The proposed method was experimentally validated for high order QAM (e.g., DP-64QAM) at 56 Gbaud.

II Iterative learning control

Unlike DLA and ILA approaches that need a DPD model (behavioral model), the iterative learning control (ILC) estimates a non-unique pre-distorted signal without recourse to a DPD model. Rather than finding coefficients in a model, a particular data pattern is replaced by a new input sequence so the system response appears linear. Knowledge of

(31)

System under test pre-distorter y(n) x(n) Post-inverse e(n) u(n) ū(n) C o p y o f P o st -in ver se (a) System under test Pre-distorter Adaptive algorithm y(n) yd(n) e(n) x(n) u(n) (b) Xk Xk+1 Yk ek System under test Learning Controller D (c)

Figure I.5: Block diagram of adaptive DPD techniques: a) ILA, b) DLA, and c) ILC [80].

the transmission system could possibly be used to speed up convergence to the best new input sequence.

The ILC technique has recently been introduced for RF power amplifier linearization [80;81]. The ILC is a well-known method in control theory [82] to improve the accuracy of the tracking and transient response in the repetitive discrete/continuous systems, i.e., mechanical control on the motion of robots in a production line [83]. The block diagram of the ILC is illustrated in Fig.I.5c, where Xk, Yk, and ek denote the input, output and

error vectors at the kth _{iteration, respectively. The learning controller block is the main}

module of the ILC scheme, which defines the adaptation policy.

In some digital communications systems, the ILC method is used to improve the dynamic range of a signal by eliminating the nonlinear distortion. After removing the nonlinear and linear distortion, a figure of merit based on the residaul hardware limitation could be used [84]. For instance, the dynamic range and resolution of an RF power amplifier

(32)

transmission is characterized after linearization by the ILC in [84]. In another application in optical systems [85], the ILC is employed to control the laser drive voltage for frequency-modulated continuous-wave (FMCW) light detection and ranging (LiDAR) systems. The controlled drive voltage by the ILC results in linear laser frequency sweep.

The adaptation of the ILC is performed on a repeated training data set. In the context of digital communications, the performance evaluation is restricted to a chosen data set. Once a given input and inverse sequence are found, a behavioral model could be fit to this data. The model, e.g., a Volterra series, would then be used to transform random data before transmission. However, complex modulation formats, which contains many transitions between different output levels, need ILC adaptation on large data sets to obtain an accurate DPD model for random data. The ILC method used in the SiP coherent detection system in our work will be discussed in chapter 1 and 2 in detail.

I.3.3 Linear digital post-compensation techniques

The most common linear DPC algorithms are feed-forward adaptive equalizers in which the coefficients can be updated in blind or training mode. In the blind mode so-called decision directed, the coefficients adaptation is performed based on inaccurate decision on the samples but enough to estimate the coefficients of the equalizer [86]. There are two criteria including zero forcing and mean square error (MSE). The MSE has higher noise tolerance. One of the most popular MSE based equalizers is the DD-LMS, which combats the linear distortion [87].

I.3.4 Nonlinear digital post-compensation techniques

While operating the system beyond the 3 dB bandwidth of the components, the MSE based equalizers strongly amplify the power of higher frequency components compared to the lower ones to overcome the bandwidth limitation (low-pass filtering effect of the limited bandwidth). However, it also enhances the noise power at higher frequencies (colored noise), which leads to lower signal-to-noise ratio (SNR) for the compensated signal. A well-known DPC algorithm to deal with this problem is a digital noise-whitening post-filter cascaded by a maximum a posteriori (MAP), maximum likelihood sequence estimator (MLSE), or maximum likelihood sequence detector (MLSD). The post-filter is used to control and alter the frequency response of the signal by adding a controlled amount of ISI distortion to the signal [2; 88;87; 89;90]. For instance, a post-filter and an MLSE are used to overcome the impairment at the receiver side to present WDM transmission of 12 channels (120 Gbaud PDM-16QAM per channel) over 1200 km in [90]. We will adapt this approach in chapter 2.

The previously discussed nonlinear DPD algorithms including Volterra based equalizer, trun-cated Volterra based equalizer (e.g., MP model), and LUT are also used at the receiver side to mitigate the nonlinear distortions. For instance, I/Q imbalance and the crosstalk for 16QAM transmission are compensated by the Volterra based DPC in [91]. Since, the nonlinearity

(33)

in-troduced by the photodetector is assumed to be estimated by a second order nonlinear term, a Volttera based DPC with the order of two is used for compensation in [87]. The frequency domain Volterra based DPC is used for fiber nonlinearity compensation in [92]. The joint MP equalizer (DD-LMS adaptation is used for MP’s coefficients estimation) and post-filter with MLSD is presented to overcome the severe ISI distortion [93]. The LUT based DPC has been extensively deployed on the receiver side [94; 95; 96; 97]. In [95], MAP detector using a LUT is employed to make the decision on the recovered symbols. The MAP detector uses the euclidean distance metric between the received pattern and the LUT entities to make the decision. In [96], a LUT based DPC is utilized for pulse amplitude modulation (PAM) trans-mission, which combats nonlinear transient of a vertical-cavity surface-emitting laser (VCSEL) and also bandwidth restriction of the components.

I.4 Thesis Outline

The main focus of this work is to address the high spectrally efficient transmission while ex-ploiting silicon (Si) single drive MZM. In order to achieve high bit rates in a single-carrier coherent detection system, we develop several DSP based compensation techniques to over-come impairments in both digital and optical domains. The performance bottlenecks in high spectrally efficient SiP optical system are included in both linear and nonlinear distortions. The development of compensation techniques in this work is concentrated at the transmitter side. All the proposed techniques in this thesis are validated through experimental work. We focus our effort on back-to-back coherent optical communications system. Thus, we do not consider the distortion caused by fiber transmission. Overall, this thesis can be categorized in three sections.

Chapter 1 addresses the limitations of the linear equalizer in an all-Silicon coherent detec-tion system, followed by presenting a novel adaptive pre-distordetec-tion technique denoted as gain based ILC (G-ILC). The G-ILC combined by linear equalizer is experimentally employed to overcome the distortion for higher order QAM. Motivated by our results for single polarization transmission in chapter 1, we improve the performance of G-ILC technique in chapter 2 to achieve the dual-polarization 32QAM 100 Gbaud transmission. It is shown that nonlinear post-compensation techniques alone are not sufficient to suppress the distortion. In chap-ter 3, we present a non-adaptive pre-distortion algorithm based on a one-dimensional LUT to realize higher order QAM, i.e., 128QAM and 256QAM in an all-Silicon coherent detection system. We investigate the inherent nonlinearity of the SiP MZM combined by other sources of distortions through different types of the LUT.

(34)

I.4.1 Objectives and contributions in Chapter 1

In Chapter 1, we demonstrate for the first time the G-ILC pre-distortion technique to deal with the limitations of all-silicon transceiver, i.e., nonlinear distortion in limited bandwidth scenarios (G-ILC was previously used in RF amplifier nonlinear compensation). The G-ILC is an adaptive nonlinear pre-distortion that is experimentally evaluated in combination with a linear equalizer in quasi-real time with hardware in the loop. The G-ILC is well-known in control theory unlike in optical communications. We describe the theory and implementation requirements based on our experimental set-up. We examine several M-QAM modulation formats (M = 16, 32, 64, 128 and 256) in different baud rates, i.e., 20 Gbaud, 40 Gbaud, and 60 Gbaud. We show that G-ILC can offer remarkable improvements in terms of power sensitivity that is not attainable by the linear equalizer alone. Finally, we exploit combined optical pre-emphasis with the digital pre-distortion to demonstrate 32QAM at 60 Gbaud using a DAC with limited bandwidth of almost 20 GHz. The main contributions can be summarized as follows.

• For the first time, we apply the G-ILC (previously used in RF amplifier linearization) for digital pre-distortion of high-speed optical communications whose performance is limited by nonlinear behavior.

• For the first time, we experimentally demonstrate the quasi real-time adaptation of G-ILC at high baud rate (40 Gbaud) and M-QAM modulation (M = 16, 32, 64, 128 and 256).

• We demonstrate the effectiveness of G-ILC to enable high spectral efficiency in a band-width limited system.

• We achieve 4.4 dB margin at 20% FEC overhead for 20 Gbaud 256QAM using G-ILC, among other results.

In Chapter 2, inspired by the improvement achieved through the G-ILC technique in Chap-ter 1, we present a record breaking line rate of 1 Tb/s. We push the baud rate beyond the bandwidth limit of the SiP MZM, i.e., 34 GHz, to report dual polarization 32QAM back-to-back transmission at 100 Gbaud (20% FEC overhead). The optimization is carried out in both MZM operation and DSP side. The SiP MZM optimization is performed in terms of the modulation efficiency versus available bandwidth through adjusting the DC bias voltage of the modulator. The G-ILC adaptation is optimized by means of a linear multiple input multiple output (MIMO) equalizer to suppress the noise impact in quasi-real time hardware in the loop. We combine our pre-distortion with several linear and nonlinear post-compensation algorithms to meet the FEC threshold targets (7% and 20%), i.e., DD-MMSE, polynomial,

(35)

and a post-filter with MLSD. We report the net rate of 747 Gb/s with 16QAM and 833 Gb/s with 32QAM through the dual polarization emulation. In this work, two different DACs were used (due to availability of equipment). In the single polarization experiment, we used a DAC with the 3 dB bandwidth of almost 40 GHz and resolution of 8 bits with a sharp roll-off of frequency response after the 3 dB bandwidth. In the dual polarization experiment, we used another DAC with 3 dB bandwidth, of almost 40 GHz and resolution of 6 bits but less sharp roll-off frequency response. Different strategies are considered for combined digital linear and optical pre-emphasis with respect to these two DACs. The main contributions in this chapter are as follows.

• We overcome many experimental challenges at 100 Gbaud in applying pre- and post-compensations for dual polarization 32QAM optical modulation with an all-silicon MZM. • We combine multiple techniques to achieve a line rate of 1 Tb/s in a severely bandwidth limited system including: operating point (operation bias voltage, RF swing), G-ILC pre-processing, optical filter, and digital post-processing.

• We reduce residual mean square error in G-ILC adaptation by enhanced noise suppres-sion during training.

• We experimentally quantify contributions of each compensation technique, both singly and in combination, in improving BER.

In Chapter 3, we experimentally examine higher order QAM, i.e., 128QAM and 256QAM, with a one-dimensional LUT to overcome the nonlinear distortion. The evaluation is performed at 20 Gbaud and 40 Gbaud to compensate different sort of nonlinear distortion, i.e., PDD and voltage dependent frequency response of the SiP MZM. We investigate the statistics of pre-distorted signal at different LUT size. We address the complexity of the LUT in terms of practical implementation and we introduce a reduced complexity LUT (R-LUT). We evaluate the performance of the R-LUT in terms of the trained LUT size. Finally, we validate the generality of the proposed pre-distortion method on data sets other than the original training set that was used for populating the LUT. Although, phase noise has an impact on the performance of higher order QAM, it is not studied in this thesis. The main contributions of this chapter are as follows.

• We compensate the nonlinear distortion of a SiP transmitter using one-dimensional LUT with advanced modulation formats (128QAM and 256QAM) at high baud rates. • We examine the nature of of transmitter induced nonlinearity (SiP MZM nonlinear

(36)

• We introduce a reduced-size LUT (R-LUT) for higher order QAM, where 50% complexity reduction provides good margin for BER.

• We validate experimentally the LUT effectiveness when varying the training and valida-tion data sets.

(37)

Chapter 1

Mitigating pattern dependent

nonlinearity in SiP IQ-modulators via

iterative learning control predistortion

1.1 Résumé

Les modulateurs Mach Zehnder sur silicium, contrairement à ceux opérant sur niobate de lithium, sont affectés par des comportements non-linéaires dépendant de la constellation, et cela au-delà d’une simple interférence inter-symbole. Nous avons démontré expérimentalement une nouvelle méthode de pré-distorsion reposant sur une technique de contrôle par appren-tissage itératif (CAI) qui permet de surmonter cette problématique grâce à une adaptation quasi-temps-réel sur du matériel dans la boucle (hardware-in-the-loop). Les performances en taux d’erreurs binaires sont comparées à celles des solutions linéaires pour différents niveaux de modulation M-QAM et de vitesses de transmission. Nous avons démontré une modulation 256QAM à 20 Gbaud, une performance n’ayant jamais été réalisée avec seulement compensa-tion linéaire. Pour une modulacompensa-tion 128QAM à 40 Gbaud, nous avons amélioré la sensibilité de puissance de 4.4 dB. En combinant la compensation optique avec la technique de CAI, nous avons amélioré la sensibilité de puissance par environ 5 dB pour une modulation 32QAM à 60 Gbaud.

1.2 Abstract

Silicon based Mach Zehnder modulators, unlike lithium niobate, suffer from nonlinear pattern-dependent behavior beyond simple intersymbol interference. We experimentally demonstrate a novel predistortion method based on the iterative learning control (ILC) technique to ad-dress this issue using quasi-real-time adaptation with hardware-in-the-loop. We compare bit error rate performance to that of linear solutions at several M-QAM modulation levels and

(38)

baud rates. We demonstrate 256QAM at 20 Gbaud which linear compensation alone cannot achieve. For 40 Gbaud 128QAM, we improve power sensitivity by 4.4 dB. We combine optical compensation with ILC to improve power sensitivity by ∼ 5 dB for 60 Gbaud 32QAM.

1.3 Introduction

The volume of data processed in data centers is rising exponentially. Videos and other files uploaded via social media, must be stored and processed in the cloud. To keep up with the expanding need for bandwidth, communications technology in data centers must be updated at low cost and low energy consumption. Silicon photonic (SiP) technology can provide the required bandwidth, at smaller footprint, lower cost, and higher power efficiency than other technologies. The cost can be especially attractive when SiP subsystems are integrated with electrical circuitry on a single chip [98;99].

The capacity of SiP technology has been demonstrated for various types of modulation. Single carrier demonstrations include 84 Gbaud 16QAM and 70 Gbaud 32QAM with BER below the 20% forward error correction (FEC) threshold [100]. Wavelength-division multiplexing (WDM) demonstrations with a silicon photonic transmitter have focused on low symbol rate (four-channels each at 28 Gb/s) [101]. The receiver design complexity of single carrier and multi-carrier are similar, hence single carrier demonstrations are sufficient for proof of concept [102] at aggressive rates/modulation orders; we also focus on single carrier demonstrations. Increasing symbol rate (baud rate) and modulation order are the path to increasing capacity, but they can pose technical challenges which limit end-to-end performance. The symbol rate is constrained by the electrical bandwidth limitations of digital-to-analog converters (DAC), RF amplifiers, and electro optical (E/O) converters used with external modulators. Pushing to higher baud rate despite limited bandwidth leads to intersymbol interference (ISI), as the transmitter introduces memory to the channel. The combination of ISI and nonlinearity causes the sampled received signal to cluster (ISI effect) at constellation points that are not aligned on a rectangular grid (nonlinearity); this is known as constellation warping. [103]. These combined effects (ISI and nonlinear) create distortion that cannot be entirely compensated by a linear equalizer [104].

One source of nonlinearity is the Mach Zehnder modulator (MZM) transfer function when driven at high peak-to-peak RF voltage. SiP modulators suffer from an additional source of nonlinearity. Unlike lithium niobate devices, phase shifters in SiP modulators are not linear in applied voltage [19]. This leads to the nonlinear pattern dependent distortion in signals generated by SiP.

To mitigate pattern dependent distortion, a wide range of linear and nonlinear predistortion methods have been reported. In [79], a Volterra series was used to model a predistorter.

(39)

An indirect learning architecture (ILA) method was used to estimate the coefficients of that model in the presence of transmitter I/Q skew and nonlinear effects. The ILA method first estimates the parameters of a post-distortion module. The post-distortion module is placed in a parallel feedback path with the pre-distortion module. In order to force the pre-distortion and post-distortion modules to behave identically, an adaptation takes place to minimize the error between outputs of pre- and post-distortion modules.

The ILA has also been used with a memory polynomial model (reduced complexity vis-à-vis a Volterra series) [105]. In order to avoid the two step method with ILA estimation of coefficients, a direct learning architecture (DLA) approach was proposed in [106]. The DLA uses the input signal and the calculated error to estimate predistorter coefficients.

Lookup tables (LUT) [62] are another well-known technique to mitigate pattern-dependent distortion in high data rate transmission [102], as well as nonlinear distortion introduced by transmitter components. A LUT with memory depth of L for M-QAM signaling has a predistortion for each of ML _{unique symbol sequences. Typically, sliding windows of L}

symbols from the recovered received signal are exploited to build LUT entries [62]. The common problem associated with LUT is the size of the table. At high M, even moderate memory length L can lead to a large table size [95].

Recently an iterative learning control (ILC) method, well known in control theory applications, was proposed to linearize power amplifiers [80; 107]. Unlike other adaptive methods, such as (ILA/DLA), no estimate or identification of a predistorter model is necessary. The ILC approach finds a non-unique input signal which leads to the desired symbol sequence output [108]. While not required, knowledge of the system model can improve ILC convergence speed. In essence, the ILC method iteratively finds the optimum input signal resulting in the desired output, thus linearizing the system.

An ILC approach reduces complexity and offers faster and more robust adaption of a wide variety of nonlinear systems with memory. The ILC is typically used for repetitive processes. To extend this technique to a random data sequence, we need a mechanism to produce the predistorted version of the sequence in real-time. Having knowledge of the ideal predistortion for a training sequence, several nonlinear solutions (e.g., polynomial model with memory) can be used to obtain a predistorter module. Our paper focuses on the ILC adaptation. We apply the ILC method to linearize a SiP modulator using advanced modulation formats with coherent detection.

This paper is organized as follows. Section 1.4 describes the ILC method [80] in greater de-tail as applied to SiP IQ modulators. Section 1.5 presents the measurement set-up for SiP IQ modulator hardware-in-the-loop experiments to adaptively find the optimum predistorted driving signal. To the best of our knowledge, this is the first time that ILC-based predistor-tion is used to linearize an optical system and extends our preliminary results in [1]. The

(40)

measurement results for various baud rates and modulation orders are presented in section1.7

and discussed. We offer concluding remarks in section 1.8.

1.4 Iterative learning control (ILC)

We exploit the ILC method in the time domain, illustrated in Fig. 1.1, where Xk is the

transmitted data block and Ykis the received data block at the kthiteration in the adaptation.

The desired received data block is a fixed, known training sequence D. The received signal is normalized to have average symbol power equal to that of the desired sequence D (i.e., to standard IQ coordinates ±1±1i, etc). The ILC iteratively transforms the transmitted data, X_k so that Y_k converges to D. The transmitted data is updated at each iteration using error block, ek, formed by the difference of Yk and D. The learning controller updates the

transmit data Xk+1 for the next iteration. To calculate the error block, the received data

block, Yk, must be time synchronized after optical frequency offset compensation and carrier

phase recovery with the training sequence D, as detailed in section 1.5.

One of the first ILC methods was formulated by Arimoto in control theory [108; 82]. He defined the ILC learning controller block (ILC law) as a recursive function called F (·)

X_k+1 = F (X_k, e_k) (1.1)

Several functions can be used for F (·), depending on the application [108]. In [80] the linear, gain-based and Newton ILC laws were investigated for RF power amplifier linearization. In this work, we employ a modified version of the gain-based law, G-ILC, to compensate pattern dependent distortion introduced by a SiP IQ modulator. Let the input signal at iteration k be given by the n-dimensional column vector Xk∈ <n×1 where

X_k= h

xk(1) · · · xk(n − 1) xk(n)

iT

(1.2) where xk(i) is the predistorted version of the ith symbol in the training sequence D. The

error column vector ekis the difference of Yk and D. We define Γk∈ <n×n as a learning gain

matrix Γk =diag h G (xk(1)) · · · G (xk(n)) iT (1.3) where the function G is an appropriate gain vector for the transmit signal, Xk. We choose

the sample-by-sample gain function

G (xk(i)) =

yk(i)

D (i) 2

(1.4) With these definitions, (1.1) can be expressed as

(41)

Xk+1 Xk Yk ek

System

under test

Learning

Controller

D

Figure 1.1: Block diagram of iterative learning control (ILC) method.

where α is the step size. We adopt a variable step size to enhance the convergence speed, changing once or twice during adaptation depending on the amount of distortion. For the case of the identity matrix, i.e., Γk = In ∈ <n×n, we have a linear ILC, which is a special case

of G-ILC, given by

X_k+1= X_k+ αe_k (1.6)

We tested and compared the choice of Γk = In vs. nonlinear Γkdefined in eq. (1.4) and (1.5).

We found convergence speed was greatly enhanced for the nonlinear version of Γk. In the case

of higher modulation order, i.e., 256QAM, our choice of G-ILC (nonlinear Γk) has sufficient

degrees of freedom to combat nonlinear effects, while exhibiting stable convergence. Linear Γk (Γk = In) could not achieve minimal error in any reasonable time. The predistortion

adaptation can be initialized with a linear equalizer such as a minimum mean square error (MMSE) equalizer.

1.5 Experimental set-up

We run the G-ILC adaptation as hardware-in-the-loop in quasi-real-time using offline digital signal processing (DSP). Figure 1.2 gives the experimental set-up. The system under test illustrated in Fig.1.1would ideally be the SiP modulator alone. However, in our back to back experiment, the system under test becomes the entire transmitter (DAC, driver, modulator) as well as the receiver used to access the optical signal at the modulator output. Therefore, all components in Fig.1.2, from the DAC to the real-time oscilloscope (RTO), are essentially part of the system under test. At the transmit side, we use a pseudo random bit sequence (PRBS) of order 19 to generate a M-QAM signal with Gray mapping where M = 32, 64, 128 or 256. A 5000-symbol subset of the QAM signal is conserved as the training sequence D. We repeat the training sequence enough times to fill DAC memory (32 k of 8 bit samples). To initialize the adaptation, the first iteration applies a 200-tap MMSE equalizer for the training sequence D. The MMSE equalizer compensates linear distortion and initiates the G-ILC for faster and more stable convergence. The MMSE predistortion X1 is used in the