## Abstract

Residual IQ skew in a coherent transmitter severely degrades the performance of long-haul coherent optical communication systems. The impairment is particularly detrimental for a high baud-rate system using quadrature amplitude modulation (QAM). Furthermore, Nyquist pulse shaping increases the spectral efficiency for WDM systems. However, sharp roll-off of Nyquist pulse shaping further reduces the tolerance to residual IQ skew. Thus, certain trade-offs between spectral efficiency and roll-off factor should be made to improve the tolerance of residual IQ skew. We experimentally studied this trade-off and determined the optimal roll-off factor, channel spacing, receiver bandwidth, and equalizer length. The results serve as a guideline for high baud-rate coherent WDM systems.

© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

## 1. Introduction

The amount of Internet’s data has been exponentially growing in the past decade. To meet the never-ending demand of Internet traffic, latest optical communication systems serving the long-haul backbone of Internet is moving towards high baud-rate advanced modulation format. For example, each carrier can support data rate beyond 400 gigabit per second (Gb/s) per wavelength [1, 2]. Besides increasing the modulation symbol rate [3], higher data rate per wavelength is achieved through coherent detection and high-spectral-efficiency modulation formats, e.g., polarization division multiplexed (PDM) quadrature amplitude modulation (QAM) [4, 5]. For coherent optical transmitter, the timing skew between the in-phase (I) and quadrature (Q) tributaries severely degrades the system performance. The impairment is particularly detrimental for high baud-rate coherent system using advanced QAM format. In [6], the theoretical study shows that to limit the penalty of signal to noise ratio (SNR) at bit error rate (BER) of 10^{−2} within 1dB, the skew needs to be smaller than 26% of the symbol period for quadrature phase shift keying (QPSK, or 4-QAM) modulation format, and 11% of the symbol period for 16-QAM modulation format. Intrinsically, high symbol rate will lead to small symbol period.

There are two broad groups of methods to estimate and compensate the skew. One group of methods are to measure the IQ skew during initial power-up and compensate it accordingly. This can be done through the destructive interference using BPSK data pattern [7], or the genetic algorithm with clock tone amplitude being the fitness function [8], or the analysis of the image spectrum [9]. Once the skew is measured, a finite impulse filter (FIR) can be used to compensate the skew [10]. During the calibration process, other impairments for coherent IQ transmitter can be measured and compensated as well. In [11], through an innovative cooperative coevolution genetic algorithm with modified clock tone amplitude being the fitness function, the time skew among tributaries, the bias voltage for Mach-Zehnder modulator (MZM), the phase imbalance between I tributary and Q tributary, and the amplitude imbalance among tributaries can be all compensated. In [12], the power level of destructive interference using BPSK data pattern is determined by the power imbalance between tributaries. Thus the power imbalance can be measured and compensated together with the skew.

One limitation is that those calibration methods rely on specific data pattern and cannot be performed with live traffic. Since skew drift over temperature and life, the performance of pre-calibrated coherent transponder could degrade. For coherent transmitter, the industrial standard body, Optical Internetworking Forum (OIF), has specified the maximum IQ skew variation over temperature and life as 2ps [13]. For coherent receiver, OIF has specified the maximum IQ skew variation over temperature and life as 2ps [14]. In the worst case scenario, the IQ skew can drift up to 4ps for a coherent link. Thus a periodic re-calibration may be necessary to compensate those skew drift. This requirement poses a challenge for commercial long-haul dense wavelength division multiplexing (DWDM) system.

The other group of methods are to estimate and compensate the IQ skew in the coherent receiver through digital signal processing (DSP). This can be done through the complex 4 × 2 multi-in multi-out (MIMO) adaptive equalizer [15, 16], or the frequency domain estimation based on the principle of Godard phase detection [17] and the time domain compensation [18], or the widely linear equalization [19], or the extra 2 × 2 butterfly adaptive equalizer specifically for transmitter impairment compensation [20]. Here the challenge is that the MIMO equalization is done in the receiver. The impairment introduced during the transmission through fiber (for example, nonlinear distortion) and the impairment introduced by phase noises (for example, equalizer enhanced phase noise (EEPN)) will limit the compensation capabilities for the transmitter’s IQ skew. Furthermore, the skew from coherent transmitter is mingled together with the skew from the coherent receiver. It is hard to estimate the skew of coherent transmitter alone and compensate it.

In summary, the first group of methods can be used to calibrate the initial static skew. Then, the high-baud rate QAM system can be designed with the tolerance to the IQ skew drifts (residual skew) in place for robustness. Thus, how to improve the tolerance to residual skew is critical for data rate beyond 400Gb/s application, which is the main subject of this paper. It is also possible to use the second group of methods to monitor the residual skew through DSP. The approach of designing the system with IQ skew drift in mind and the approach to monitor residual IQ skew using the DSP method are complementary with each other, and not mutually exclusive. Both approach can be implemented together in the system and they can improve the robustness against IQ skew drift.

Nyquist pulse shaping is widely used to increase the spectral efficiency for DWDM systems [21, 22]. The roll-off factor (*ROF*) for Nyquist pulse shaping has a significant impact on the tolerance to residual skew. In addition, the channel spacing between adjacent transponder (*CS*), the bandwidth of coherent receiver (*RB*) and the length of adaptive equalizer (*EL*) in coherent DSP should be optimized based on *ROF* as well. A detailed understanding of the interplay among those parameters is needed. A laudable goal would be to have a general guideline to specify the maximum skew allowance, especially in the high-spectral-efficiency scenario.

The paper is organized as following: in section 2, we first propose to improve the tolerance to the residual skew at certain trade-off of the spectral efficiency. We then show the simulation results which demonstrate the improvement on the skew tolerance with large *ROF*. In addition, we discuss other benefits of pulse shaping using large *ROF*, like high tolerance to nonlinear distortion and small implementation penalty with limit number of filter taps. In section 3, we first construct a coherent optical transponder capable of transmitting data rate beyond 400Gb/s. We then systematically characterize the influence of residual skew on bit error rate (BER) for different symbol rates and modulation formats. Finally, we conduct comprehensive study to optimize *ROF, CS, RB,* and *EL* given certain residual IQ skew. In section 4, we reveal a few application scenarios for our proposed method. Furthermore, we discuss the methodology to apply our proposed technique. Our results serve as a guideline for system specification for high baud-rate coherent WDM systems. In section 5, we provide the conclusion.

## 2. Principle and theoretical analysis

Figure 1 shows the block diagram of coherent IQ transmitter, which is composed by the analog coherent optics (ACO) and the DSP application specific integrated circuit (ASIC). An important block within the DSP is the finite impulse response (FIR) filter. The FIR filter is *T _{s}*/2 spaced, where

*T*is the symbol period. There are multiple functionalities of the FIR filter: to overcome the bandwidth limitation coming from the components on the data path; to perform Nyquist pulse shaping; to compensate the impairment like the skew and the imbalance. The output from FIR can be expressed as

_{s}*x*is the input signal to FIR filter,

*h*is the impulse response of the FIR filter, and

*N*is the total number of the taps of FIR filter.

The output from FIR filter is converted from digital domain to analog domain through a high speed digital-to-analog (DAC) converter. The analog electrical signal first goes through the traces on the radio-frequency (RF) print circuit board (PCB) and pluggable interface (if applicable). The electrical signal is then boosted by the linear RF amplifiers and finally applied to the MZM of the particular tributary.

As shown in [11], the output from the coherent IQ transmitter can be expressed as

*ϕ*is the phase of the carrier, $\overrightarrow{X}$and $\overrightarrow{Y}$are the unit vector along the X polarization and Y polarization. The ${\tau}_{IQ}^{X},{\tau}_{IQ}^{Y},{\tau}_{XY}^{}$are the skew between XI and XQ tributary, the skew between YI and YQ tributary, and the skew between X polarization and Y polarization.

*S(t)*is the modulated signal for a particular tributary. There are two components for each skew: the static skew and the dynamic skew. The static skew is introduced by the length difference of the trace in the PCB. The residual dynamic skew is introduced by the variation in the trace length over temperature and life. As discussed in Section 1, the static skew can be firstly measured through pre-calibration method or DSP method, and secondly compensated by FIR filter. However, it is challenging to measure the dynamic skew with the live traffic. Thus, there is a great need to improve the tolerance to the residual dynamic skew.

Nyquist pulse shaping is ubiquitously used to increase the spectral efficiency for wavelength division multiplexing (WDM) systems. Equation (3) below shows the spectral response of raise cosine (RC) filter [6], where *ROF* is the roll-off factor of raised cosine filter, *ω* is the angular frequency, *T _{s}* is the period of symbol.

*H*=

_{RRC}(ω)*H*. The advantage of using RRC filter instead of RC filter is as following: both the coherent transmitter and the coherent receiver can implement RRC filter in the digital domain; thus, the coherent transmitter and the coherent receiver have a matched filter; this reduces ISI and improves SNR.

_{RC}(ω)^0.5*ROF* plays an important role for the tolerance to residual skew. When *ROF* is 0.1, its frequency response is almost rectangular shape, which could allow near-symbol rate-equivalent spacing with adjacent channels with negligible crosstalk. However, small roll-off factor inevitably reduces eye width, thus aggravates the system’s skew tolerance. When *ROF* is 1, its frequency response is raise cosine or root raise cosine shape. It causes strong crosstalk among DWDM channels. A larger *CS* is needed to minimize crosstalk, which reduces the spectral efficiency. Thus, certain trade-off between spectral efficiency and roll-off factor should be made to improve the tolerance of residual IQ skew.

We propose to dynamically adjust *ROF* based on residual skew as needed. Here we consider a scenario where all DWDM channels are running at data rate beyond 400Gb/s using 16-QAM at 64GBd. Standard for flex-grid optical line system specifies the channel spacing (*CS*) at multiple times of 12.5GHz [23]. To accommodate 64 gigabaud (GBd) signal, a 75-GHz channel spacing is required, which is approximately 1.17 times of symbol rate. Initially, the skews of all transmitters are well calibrated. The *ROF* can be set close to zero to minimize ISI, as shown in Fig. 2(a). With IQ skew of some channels drifts over time and temperature, one can adjust the roll-off factors of the drifted channels to improve their tolerance, as shown in Fig. 2(b). The *CS* between the drifted channel and the adjacent channels needs to be increased to accommodate wider spectrum due to higher *ROF*. In addition, the central wavelength of adjacent channels may be adjusted to accommodate the larger *CS* and the guard band between channels may be reduced.

To achieve the dynamic adjustment of *ROF*, one needs to monitor the transmitter’s IQ skew. As discussed in Section 1, the coherent receiver can monitor the transmitter’s IQ skew through MIMO equalization [15–20]. However, the impairment during the transmission limits the monitoring accuracy. Thus it is desirable to monitor the transmitter’s IQ skew before the signal is launched into the fiber. To achieve this, one can develop a coherent optical channel monitor (OCM). Essentially this OCM is composed of components in the receiver path of a coherent transponder, namely 90 degree optical hybrid, balanced photo diodes, linear trans impedance amplifier, integrated tunable laser assembly (ITLA) and high-speed analog-to-digital converter (ADC). The coherent OCM can be placed in the output of DWDM multiplexer. The ITLA can be tuned to the wavelength of a particular channel under testing. Then, the optical signal is coherently detected and the data is stored in the memory of ADC. Finally, an offline DSP algorithm can process the data to extract the transmitter’s IQ skew information using the algorithm developed in [15–20]. Other parameters, like chromatic dispersion (CD), differential group delay (DGD) and frequency offset, can be monitored as well. This coherent OCM can be also deployed at each reconfigurable optical add drop module (ROADM). With an optical switch to change the input to coherent OCM, one coherent OCM can monitor multiple channels on multiple output ports of the ROADM serially.

Figure 3 illustrates the simulated eye diagrams with 16-QAM signal passing through a raised cosine filter and a root raised cosine filter at different roll-off factors. Here, we first generate a 16-QAM signal with unlimited bandwidth. The number of samples per symbol period is eight. We then pass this signal through a RC filter or RRC filter with different *ROF*. The bandwidth of transmitting signal is then limited by the RC or RRC filter. Finally, we plot the eye diagram overlapping eight thousand symbols. As seen in Fig. 3(a) for RC filter, the height of the eye opening remains roughly unchanged over different *ROF*. But the eye width increases with large *ROF*, indicating better tolerance to the residual skew. As seen in Fig. 3(b) for RRC filter, the eye height is large for a small *ROF*, which indicates small inter symbol interference (ISI). Meanwhile, the eye width is narrow, indicating small tolerance to residual IQ skew. For a large *ROF*, the eye height is reduced, indicating large ISI. Meanwhile, the eye opening is wide, leading to large tolerance to the residual IQ skew. One can also notice that the eye width of RRC filter is wider than that of RC filter at the same *ROF*. This indicates that the Nyquist pulse shaping with RRC filter provides better tolerance to the residual skew. As shown above, the simulation results agree well with the theoretical analysis above.

In addition to the better tolerance to the residual skew, there are other benefits to use a larger *ROF*. It is well known that a pulse with a wider spectrum has a smaller peak-to-average power ratio (*PAPR*). And the nonlinear distortion like self phase modulation (SPM) and cross phase modulation (XPM) is proportional to the peak power [24]. This issue is more severe for high-order *M*-ary QAM modulation format. The larger the *M* is, the larger the *PAPR* is. By performing Nyquist pulse shaping using a larger *ROF*, the *PAPR* is reduced. Thus, the signal exhibits stronger tolerance to nonlinearity distortion, leading to approximately 20% increasing in transmission distance [25].

Another consideration for Nyquist pulse shaping is the required filter length. As seen from Eq. (3), the RC / RRC filter has a sharp transition from pass to stop in the frequency domain when the *ROF* is small. This requires a large number of taps for the FIR filter in the time domain. We perform the simulation to study the impact of filter length on Q^{2} factor for RRC filter. Here, we first generate a 16-QAM signal with unlimited bandwidth. The number of samples per symbol period is two. We then pass this signal through a RRC filter with different *ROF*. Next, we add AWGN to the signal so that the energy per bit to noise power spectral density ratio, usually referred as *E _{b}/N_{0}*, reaches a particular value. Finally, we pass the noise-loaded signal through a matched RRC filter, demodulate the QAM signal, and count the BER. Figure 4(a) plots the Q

^{2}factor versus

*ROF*at different

*E*with the filter length being 21. It is clearly that there is a Q

_{b}/N_{0}^{2}penalty when

*ROF*is equal to 0.1 due to the limit length of FIR filter. Next, we fix the

*ROF*at 0.1 and change the filter length. Figure 4(b) plots the Q

^{2}factor versus filter length under different

*E*. Clearly when the filter length is increased to 41, which spans over 20 symbol period, the penalty in Q

_{b}/N_{0}^{2}factor is reduced to almost zero. Thus, to implement a RRC filter with negligible penalty, the required filter length for

*ROF*value of 0.1 is twice the required filter length for

*ROF*value of 0.2. The large number of taps required for small

*ROF*increases the latency, the power consumption and the ASIC complexity.

As discussed above, a careful optimization on *ROF* should be carried out with the consideration of skew tolerance, nonlinear tolerance and implementation penalty. In addition, receiver bandwidth (*RB*) of optical coherent receiver should also be optimized based on *ROF*. On one hand, optical receiver can induce ISI into signal if receiver bandwidth is not sufficient. On the other hand, inter-channel crosstalk and additive white Gaussian noise (AWGN) can be induced if receiver bandwidth is too wide. The length of adaptive equalizer in coherent DSP (*EL*) also influences the performance. Thus it should be carefully optimized as well.

## 3. Results and discussion

Figure 5 depicts the experimental setup to investigate the relationship between skew, roll-off factor and channel spacing. The coherent transmitter is composed by a high-speed DAC (92G samples/s), a tunable laser, a dual-polarization IQ modulator, and an automatic bias control. We first generate pseudo random binary sequence (PRBS), then apply Gray coding to map the QAM signal. Next, the signal is convoluted with a FIR filter. The coefficients of FIR filter are set to compensate the limited bandwidth, generate transmitter skew, and perform RRC pulse shaping with various *ROF*. The coherent receiver is composed by optical front-end and a high-speed real-time oscilloscope, serving as high-speed ADC (160G samples/s). Offline DSP is used to recover the signal and count the BER. In Fig. 5, we also show the recovered constellation diagram for 16-QAM and 64-QAM at 64GBd. The DSP processing steps are similar to that described in [6]: static equalization with a matched RRC filter implemented in time domain and a filter implemented in frequency domain to compensate the chromatic dispersion; adaptive equalization to perform polarization de-multiplexing and compensation of polarization mode dispersion; time recovery using Gardner’s method; frequency offset estimation; carrier phase recovery; and symbol estimation and recovery.

The amplified spontaneous emission (ASE) source is used to emulate adjacent aggressor channels. The spectral shape and the central frequencies of the aggressor channels can be dynamically adjusted by a wavelength selective switch (WSS). A high-resolution optical spectrum analyzer (Finisar WaveAnalyzer, 10pm resolution) measures the spectrum of the channel under testing. The spectrum is then down-sampled to 100-pm resolution and loaded into WSS (Finisar WaveShaper) to emulate aggressor channel, as shown in Fig. 6. WSS also acts as a variable optical attenuator so that the power level of aggressors is adjusted as the same as the channel under testing. One can note that there is a slight difference between the spectral shape of the central channel under testing and the aggressor channels generated by WSS. This is due to the limited resolution of WSS.

The approach to emulate the adjacent aggressor channel using the spectral-shaped ASE source has been demonstrated as a valid and cost-effective approach. As shown in [26], the spectral shaped ASE source can be used to emulate the wide band channel loading. At the transmitter’s output, the probability density function (PDF) of the 16-QAM signal contains three distinct peak. On the other hand, the PDF of the ASE source in a form of the Rayleigh distribution. But when the 16-QAM signal transmits through the fiber, the chromatic dispersion will lead to the pulse broadening. The PDF of 16-QAM signal will then have a shape of Rayleigh distribution, matching the PDF of ASE source. In [27], it is shown that in the linear regime where OSNR is the limiting factor for BER, the BER value obtained with the real 16-QAM signal being the aggressor channel and the BER value obtained with spectral-shaped ASE source are very close.

Figure 7 displays the recovered eye diagrams for 400Gb/s 16-QAM signal at 64GBd symbol rate. From the eye diagram, one can notice the 4ps IQ skew from the shift between eye centers of two tributary. As seen, the larger the roll-off factor *ROF* is, the wider the eye width is. This agrees with the result shown in Fig. 3, confirming that the skew tolerance is indeed improved with the larger roll-off factor.

Figure 8 shows the penalty of Q^{2} factor as a function of the transmitter at different scenarios. The penalty is calculated against the reference point at 0ps. We use the penalty of Q^{2} factor as the comparison metrics so that the other factors like implementation penalty can be removed. In Fig. 8(a), we plot the influence of the symbol rate. As expected, the higher the symbol rate is, the large the penalty due to IQ skew is. In Fig. 8(b), we plot the influence of the modulation format. As expected, high order modulation format suffers more from residual IQ skew. For 64-QAM, the skew tolerance is less than 2ps. Outside this range, the coherent receiver cannot lock. In Fig. 8(c), we plot the influence of IQ skew versus the influence of XY skew which is the time skew between two polarizations. The penalty from IQ skew is much larger than that from XY skew. The reason is that XY skew is similar to polarization mode dispersion (PMD), which can be largely compensated by 2 × 2 butterfly filter used for PMD compensation. Even the penalty from the XY skew is relatively small, one still needs to minimize the XY skew. Usually, the 2×2 butterfly filter has a limited number of taps. Large XY skews reduces the capacity of the adaptive equalizer to combat PMD. More severely, when XY skew is in the order of half symbol period, the clock reference disappears. Therefore, the DSP in the coherent receiver can experience the outage [28]. Also as expected, the IQ skew in *x*-polarization has the same effect as the IQ skew in *y*-polarization. In Fig. 8(d), we plot the influence of roll-off factor. As expected, the larger the roll-off factor is, the higher the tolerance to IQ skew is.

Optimization of *RB* and *CS* are studied with zero residual IQ skew. Here, the *RB* is adjusted through the low-pass filter implemented in the high-speed oscilloscope. The filter is a brick-wall anti-aliasing filter with a sharp roll-off near the bandwidth setting. We normalize both *RB* and *CS* against the symbol rate. Figure 9(a) shows the result with small *ROF*, where the spectral shape is rectangular. When the *RB* is larger than certain value, further increase of *RB* does not influence Q^{2} factor; when *RB* is smaller than certain value, a sharp penalty incurs due to cutoff of signal content. For *ROF* value of 0.1, the optimal *CS* value is 1.1 with a small decrease of Q^{2} factor. With large *ROF*, the spectral shape is raised-cosine. As shown in Fig. 9(b), when *RB* is larger than certain value, further increase of the *RB* degrades Q^{2} factor. There are two possible reasons for this degradation. One is the white Gaussian noise introduced when RB is increased. The other is the additional crosstalk from adjacent channels. From the experimental result, the optimal *CS* value is 1.3 times of symbol rate For *ROF* value of 1. Further increase of *CS* brings marginal benefit at the cost of spectral efficiency.

From Fig. 9(a) and 9(b), we determine the optimal *CS* given zero residual IQ skew. Optimal *RB* can be determined from Fig. 9(c) by further comparison of four scenarios. When *ROF* is equal to 0.1, at the optimal *CS* value of 1.1, there is no significant difference between *RB* value of 1 and *RB* value of 1.25. This can be understood as that at *RB* value of 1, all spectral content of signal is captured given the rectangular shape of the spectrum. So *RB* value of 1 is an optimal setting for *ROF* value of 0.1. When *ROF* is equal to 1, at the optimal *CS* value of 1.3, *RB* value of 1 performs noticeably better than *RB* value of 1.25. If the receiver bandwidty is too wide, the crosstalk from neighboring channel will appear, leading to certain penalty. So *RB* value of 1 is an optimal setting for *ROF* value of 1 as well.

The influence of equalizer length (*EL*) of the coherent receiver is summarized in Fig. 10. The adaptive equalizer within the DSP on the receiver side is mainly used to perform polarization de-multiplexing and compensate PMD. Meanwhile, it can compensate the limited bandwidth of coherent receiver through equalization. Here we implement a blind equalization using the radius directed equalization (RDE). The equalizer is implemented in time domain with a *T _{s}*/2 spacing [29]. With no residual skew, the Q

^{2}factor does not change significantly after

*EL*is larger than 21 when the bandwidth limitation is fully equalized. With 5ps residual skew, the Q

^{2}factor shows improvement when

*EL*is increased. When

*EL*is increased from 21 to 43, the Q

^{2}factor improves approximately 0.6dB. This shows that adaptive equalizer can compensate IQ skew to a small degree, but the increased complexity in DSP outweighs the improvement on Q

^{2}factor. A 21-tap equalizer is an ideal choice.

Figure 11 displays the contour plot of BER versus *ROF* and *CS* with different residual IQ skew. The measurement is done in back-to-back scenario without OSNR loading. The Certain BER can be achieved within a region of *ROF* and *CS*. As shown in the figures, BER is less than 1E-3 in the green color, while in the blue region, BER is larger than 1E-3, but still less than 1E-2. For 2-ps residual skew, *ROF* is increased from 0.1 to 0.5 with 1E-3 BER requirement. The *CS* is increased from 1.1 times of symbol rate to 1.3 times of symbol rate, translating to approximately 18% reduction in spectral efficiency. For 4-ps residual skew, the BER degradation is even more severe. A tight control of skew is necessary to guarantee performance of the coherent transponder at the data rate beyond 400Gb/s. For application where spectral efficiency is not critical, it would be beneficiary to choose large roll-off factor for better tolerance to residual skew of coherent transponder.

For single-carrier 400Gb/s application, both 64GBd 16-QAM and 45GBd 64-QAM are the potential modulation formats [30]. Clearly, 64GBd 16-QAM is more tolerable to residual IQ skew from those results above, as shown in Fig. 8(b). Also, for potential 600Gb/s application using 64GBd 64-QAM format, the skew tolerance is less than 2ps. Outside this range, the coherent receiver cannot lock.

## 4. Applications

As described in section 3, in presence of residual IQ skew, modulation format and roll-off factor of individual channel, system channel spacing, receiver bandwidth, and equalizer length should be optimized to achieve best system performance. Current coherent optical communication system offers much flexibility to address the issues above. For coherent transmitter, each channel can be flexibly configured, which includes modulation format, symbol rate, roll-off factor, wavelength, etc. In the coherent receiver, the wavelength of the local oscillator, the receiver bandwidth, and the DSP algorithms can also be adaptively tuned based on the change of coherent transmitter. After coherent detection, real-time DSP is used to de-multiplex *X* and *Y* polarizations, compensate CD and PMD, and track phase change and frequency offset. The flexibility can be easily accessible through the software controlled network (SDN).

There are many application scenarios for the proposed techniques. The first applicable scenario is skew drift. Initially all coherent transponders have their skew calibrated. A small *ROF* ensures highest spectral efficiency. Later, the timing skew of some transponders drifts over life. Increasing *ROF* improves the skew tolerance at the trade-off spectral efficiency. The second applicable scenario is system upgrading. Originally, the coherent transponders carry 200Gb/s traffic using skew-tolerant QPSK format. For example in one implementation, a CFP2-ACO (CFP2 form factor analog coherent optics) module contains the optical front-end and a DSP ASIC resides in the host card. They are connected through a pluggable interface. Next, the system is upgraded to 400Gb/s traffic using 16-QAM format. The optical front-end is the same for 200Gb/s QPSK and 400Gb/s 16-QAM. So the CFP2-ACO module can still be used and the DSP ASIC can be upgrade. This reduces the upgrading cost for the whole system. However, from the experimental result presented in this work, the 400Gb/s 16-QAM is skew-sensitive. Increasing *ROF* improves the skew tolerance at small cost of spectral efficiency. But the gain of spectral efficiency from upgrading to high-order QAM format is much more significant. The third applicable scenario is to allocate enough IQ-skew margin for each channel, before optical modules are put in service. Initially, IQ skew can be calibrated to zero. With the knowledge of maximum IQ skew drift over life and temperature, certain of amount of skew margin would be allowed in system pre-configuration by applying greater roll-off factor and channel spacing. In addition, other benefits of Nyquist spectral shaping with a large roll-off factor include high tolerance to nonlinearity and small number of taps required.

There are two methods to apply the change of roll-off factor and the required adjustment of channel spacing. From Fig. 8(a), skew tolerance could be increased by using lower symbol rate. Furthermore, the channel spacing is increased between the channels with the lower symbol rate. Here the central wavelength of each channel remains the same. Thus when IQ skew deteriorates, downgrading the symbol rate and increasing roll-off factor would ensure the optimum system performance, as long as total capacity meets requirement.

The other method is to configure channel wavelength and roll-off factor. Here, the symbol rate remains the same. The required increasing of the channel spacing can be met by tuning the central wavelength. In this method, when system reports IQ degradation, the SDN controller would send request to the transponder to adjust channel roll-off factor and channel wavelength. The wavelength and roll-off factor could be fine-tuned at small size without impacting the in-flight traffic.

## 5. Conclusion

Residual dynamic IQ skew in coherent transmitter is particularly detrimental for high baud-rate QAM coherent optical communication system. The use of Nyquist pulse shaping to increase the spectral efficiency further reduces the tolerance to the residual IQ skew. Thus there is a trade-off between spectral efficiency and roll-off factor under different residual IQ skew.

In this work, the impact of Nyquist pulse shaping on the tolerance to residual IQ skew was comprehensively investigated. We optimized the roll-off factor, the channel spacing, the receiver bandwidth and the equalizer length given certain residual IQ skew. We further discussed the potential application scenarios and the methodologies to apply this technique. The results serve as guideline for high baud-rate coherent long-haul optical communication systems.

## Acknowledgments

The authors gratefully acknowledge Dr. Xuan He for the fruitful discussion on the work. The authors also gratefully acknowledge vigorous encouragement and sturdy support on innovation from Dr. Domenico Di Mola at Juniper Networks.

## References and links

**1. **K. Roberts, S. Foo, M. Moyer, M. Hubbard, A. Sinclair, J. Gaudette, and C. Laperle, “High Capacity Transport 100G and Beyond,” J. Lightwave Technol. **33**(3), 563–578 (2015). [CrossRef]

**2. **J. Wei, Q. Cheng, R. Penty, I. White, and D. Cunningham, “400 Gigabit Ethernet using advanced modulation formats: performance, complexity, and power dissipation,” IEEE Commun. Mag. **53**(2), 182–189 (2015). [CrossRef]

**3. **X. Chen, S. Chandrasekhar, S. Randel, G. Raybon, A. Adamiecki, P. Pupalaikis, and P. J. Winzer, “All-Electronic 100-GHz Bandwidth Digital-to-Analog Converter Generating PAM Signals up to 190 GBaud,” J. Lightwave Technol. **35**(3), 411–417 (2017). [CrossRef]

**4. **J. Cai, H. G. Batshon, M. Mazurczyk, H. Zhang, Y. Sun, O. V. Sinkin, D. Foursa, and A. N. Pilipetskii, “64QAM Based Coded Modulation Transmission over Transoceanic Distance with > 60 Tb/s Capacity,” in *Optical Fiber Communication Conference Post Deadline Papers*, OSA Technical Digest (online) (Optical Society of America, 2015), paper Th5C.8.

**5. **W. Peng, T. Tsuritani, and I. Morita, “Transmission of High-Baud PDM-64QAM Signals,” J. Lightwave Technol. **31**(13), 2146–2162 (2013). [CrossRef]

**6. **M. Faruk and S. Savory, “Digital Signal Processing for Coherent Transceivers Employing Multilevel Formats,” J. Lightwave Technol. **35**(5), 1125–1141 (2017). [CrossRef]

**7. **Y. Yue, B. Zhang, Q. Wang, R. Lofland, J. O’Neil, and J. Anderson, “Detection and alignment of dual-polarization optical quadrature amplitude transmitter IQ and XY skews using reconfigurable interference,” Opt. Express **24**(6), 6719–6734 (2016). [CrossRef] [PubMed]

**8. **C. M. Diniz, F. D. Ros, R. T. Jones, and D. Zibar, “Time Skew Estimator for Dual-Polarization QAM Transmitters,” in *European Conference on Optical Communication* (2017), paper P1.SC3.41. [CrossRef]

**9. **H. Chen, X. Su, Z. Tao, T. Oyama, H. Nakashima, T. Hoshida, and K. Kato, “An Accurate and Robust In-phase/Quadrature Skew Measurement for Coherent Optical Transmitter by Image Spectrum Analyzing,” in *European Conference on Optical Communication* (2017), paper P1.SC3.35. [CrossRef]

**10. **T. Tanimura, S. Oda, T. Tanaka, T. Hoshida, Z. Tao, and J. Rasmussen, “A simple digital skew compensator for coherent receiver,” in European Conference on Optical Communication (2009), p. 7.3.2.

**11. **J. Diniz, F. Da Ros, E. Da Silva, R. Jones, and D. Zibar, “Optimization of DP-MQAM Transmitter Using Cooperative Coevolutionary Genetic Algorithm,” J. Lightwave Technol. **36**(12), 2450–2462 (2018). [CrossRef]

**12. **Y. Yue, Q. Wang, B. Zhang, A. Vovan, and J. Anderson, “Detection and compensation of power imbalances for DP-QAM transmitter using reconfigurable interference,” Proc. SPIE **10130**, 101300M (2017).

**13. ** Optical Internetworking Forum, “Implementation Agreement for Integrated Polarization Multiplexed Quadrature Modulated Transmitters for Metro Applications,” www.oiforum.com/wp-content/uploads/OIF-PMQ-MTX-01.0-IA_final.pdf

**14. ** Optical Internetworking Forum, “Implementation Agreement for Micro Intradyne Coherent Receivers,” https://www.oiforum.com/wp-content/uploads/OIF-DPC-MRX-02.0.pdf

**15. **M. Paskov, D. Lavery, and S. Savory, “Blind Equalization of Receiver In-Phase/Quadrature Skew in the Presence of Nyquist Filtering,” IEEE Photonics Technol. Lett. **25**(24), 2446–2449 (2013). [CrossRef]

**16. **R. Rios-Müller, J. Renaudier, and G. Charlet, “Blind Receiver Skew Compensation and Estimation for Long-Haul Non-Dispersion Managed Systems Using Adaptive Equalizer,” J. Lightwave Technol. **33**(7), 1315–1318 (2015). [CrossRef]

**17. **J. Diniz, E. da Silva, M. Piels, and D. Zibar, “Joint IQ skew and chromatic dispersion estimation for coherent optical communication receivers,” in *Advanced Photonics 2016 (IPR, NOMA, Sensors, Networks, SPPCom, SOF), OSA Technical Digest (online) (Optical Society of America**,**2016**),* paper SpTu2F.2.

**18. **N. Stojanovic and X. Changsong, “An Efficient Method for Skew Estimation and Compensation in Coherent Receivers,” IEEE Photonics Technol. Lett. **28**(4), 489–492 (2016). [CrossRef]

**19. **E. da Silva and D. Zibar, “Widely Linear Equalization for IQ Imbalance and Skew Compensation in Optical Coherent Receivers,” J. Lightwave Technol. **34**(15), 3577–3586 (2016). [CrossRef]

**20. **C. Fludger and T. Kupfer, “Transmitter Impairment Mitigation and Monitoring for High Baud-Rate, High Order Modulation Systems,” in *European Conference on Optical Communication* (2016), paper Tu.2.A.2.

**21. **H. Chaouch and M. Filer, “Analog Coherent Optics for Long Haul Datacenter Regional Networks,” J. Lightwave Technol. **36**(2), 372–376 (2018). [CrossRef]

**22. **J. Wang and Z. Pan, “Generate Nyquist-WDM Signal Using a DAC With Zero-Order Holding at the Symbol Rate,” J. Lightwave Technol. **32**(24), 4085–4091 (2014).

**23. **International Telecommunication Union (ITU-T) standard G.694.1, “Spectral grids for WDM applications: DWDM frequency grid.”

**24. **D. Rafique and A. D. Ellis, “Nonlinear penalties in long-haul optical networks employing dynamic transponders,” Opt. Express **19**(10), 9044–9049 (2011). [CrossRef] [PubMed]

**25. **B. Châtelain, C. Laperle, K. Roberts, M. Chagnon, X. Xu, A. Borowiec, F. Gagnon, and D. V. Plant, “A family of Nyquist pulses for coherent optical communications,” Opt. Express **20**(8), 8397–8416 (2012). [CrossRef] [PubMed]

**26. **D. J. Elson, L. Galdino, R. Maher, R. I. Killey, B. C. Thomsen, and P. Bayvel, “High spectral density transmission emulation using amplified spontaneous emission noise,” Opt. Lett. **41**(1), 68–71 (2016). [CrossRef] [PubMed]

**27. **D. J. Elson, G. Saavedra, K. Shi, D. Semrau, L. Galdino, R. Killey, B. C. Thomsen, and P. Bayvel, “Investigation of bandwidth loading in optical fibre transmission using amplified spontaneous emission noise,” Opt. Express **25**(16), 19529–19537 (2017). [CrossRef] [PubMed]

**28. **D. Zibar, A. Bianciotto, Z. Wang, A. Napoli, and B. Spinnler, “Analysis and dimensioning of fully digital clock recovery for 112 gb/s coherent polmux QPSK systems,” in European Conference on Optical Communication (2009), p. 7.3.4.

**29. **I. Fatadin, D. Ives, and S. Savory, “Blind Equalization and Carrier Phase Recovery in a 16-QAM Optical Coherent System,” J. Lightwave Technol. **27**(15), 3042–3049 (2009). [CrossRef]

**30. ** Optical Internetworking Forum, “Flex Coherent DWDM Transmission Framework Document,” http://www.oiforum.com/wp-content/uploads/OIF-FD-FLEXCOH-DWDM-01.0.pdf.