## Abstract

We theoretically and experimentally investigate a time-domain digital pre-equalization (DPEQ) scheme for bandwidth-limited optical coherent communication systems, which is based on feedback of channel characteristics from the receiver-side blind and adaptive equalizers, such as least-mean-squares (LMS) algorithm and constant or multi- modulus algorithms (CMA, MMA). Based on the proposed DPEQ scheme, we theoretically and experimentally study its performance in terms of various channel conditions as well as resolutions for channel estimation, such as filtering bandwidth, taps length, and OSNR. Using a high speed 64-GSa/s DAC in cooperation with the proposed DPEQ technique, we successfully synthesized band-limited 40-Gbaud signals in modulation formats of polarization-diversion multiplexed (PDM) quadrature phase shift keying (QPSK), 8-quadrature amplitude modulation (QAM) and 16-QAM, and significant improvement in both back-to-back and transmission BER performances are also demonstrated.

© 2014 Optical Society of America

## 1. Introduction

With the advent of high speed digital-to-analog converter (DAC), signal generation based on DAC becomes an attractive method due to the simple configuration and flexible signal generation capability, and it has been attracting a great deal of interest in recent years for the transmission of 100G and beyond [1–8]. DAC for signal generation allows the software-defined optics (SDO) with arbitrary waveform generation, which can be used for signal software switch in different modulation formats [1–6]. On the other hand, it also allows the digital signal processing (DSP) at the transmitter side (Tx) with pre-compensation or pre-equalizations [3–10]. One of the first electronic pre-equalization technologies for chromatic dispersion was demonstrated in 5,120 km transmission for a 10 Gb/s Differential Phase Shift Keying (DPSK) system [9]. In order to achieve high speed signal generation, the industrial research communities have made great effort to increase the bandwidth and sample rate of DAC.

However, the 3-dB analog bandwidth of state-of-the-art DACs is still much less than the half of its sample rate, which means that the generated signals suffer the distortions caused by the bandwidth limitation, especially for generated signals with high baud rate. Meanwhile, when it operates at high baud rate, other opto-electronic devices, such as the electrical drivers and modulators which work beyond their specified bandwidth, can further suppress the signal spectrum. Due to such cascade bandwidth narrowing effect, the system performance is seriously degraded by inter-symbol interference (ISI), noise and inter-channel crosstalk enhancement [5–8]. Electrical domain pre-equalization for the bandwidth-limitation impairments, which is a well-known technique in optical communication, has been widely utilized in recent publications [5–8, 10–17]. The first pre-equalization of the filtering penalty for the 43 Gb/s optical DQPSK signals is reported in [10] using DAC. In previous works [5–8], zero-forcing frequency domain equalizations are carried out to pre-equalize the linear band-limiting effects. The inverse transfer function of DAC and other opto-electronic devices is measured by calculating the fast Fourier transform (FFT) of both transmitted and received binary data using a known training signal sequence. However, from the perspective of system implementation, such approach may not be easily utilized in current 100G or 400G systems since an additional DSP block at the receiver needs to be developed to deal with the channel estimation. Furthermore, to remove the fluctuation caused by signal and noise randomness, more than 100 measurements are required to be carried out for averaging [6–8]. Also, a strict time-domain synchronization is also required in [6]. On the other hand, to increase the measurement accuracy in the high frequency region, the spectrum of the De Bruijin binary PSK signal needs special process with pre-emphasized [7].

Alternatively, a time-domain pre-equalization method can be a good solution [11–17]. In most band-limited systems, adaptive equalizers are used for ISI equalization at the receiver side (post-equalization). Theoretically, in a zero-ISI system, the receiver-side linear adaptive equalizers approach the channel inverse within the sampling rate of equalization taps, to compensate the bandwidth limitation [11–24]. In fact, this channel inverse is also effective for pre-equalization. It is worth noting that, we use the channel here describing the transfer function of the transmitter hardware, but not the outside plant of the transmitter. The linear equalizers used in the receiver side is a good tool for channel estimation, which has been employed in digital cable television (CATV) and wireless transmission system for pre-equalizations [11–14]. Significant signal BER gain can be obtained using this method. Numerical results in [15] also show that the pre-equalization outperforms post equalization-only in band-limited system with narrowband filtering impairments. In optical coherent communication system, such as constant modulus algorithm (CMA), multi-modulus algorithm (MMA) or decision-directed least-mean-squares (DD-LMS), these adaptive equalizers’ transfer function is naturally modeled with the inverse Jones matrices of the channel [24–26]. However, when there is no polarization mode dispersion (PMD), the frequency response of these adaptive equalizers is just the inverse transfer functions of the channel. With this feature in mind, one can simply get the inverse of channel transfer function for pre-equalization. Since the adaptive equalizers are blind to the data pattern, different with the scheme used in [5, 6], there is no need to do data pattern alignment. Only clock recovery is need for symbol synchronization. Compared to the prior arts, the proposed method, featuring no additional DSP, no precise symbol alignment, is advantageous for system implementations. Using this scheme, recently we have demonstrated the improved performance for DAC generated signals [16]. We also realized a band-limited 480-Gb/s dual-carrier PDM-8QAM transmission and have demonstrated the BER performance improvements by experimental results in [17].

In this paper, we extend our study by both theoretical analysis and experimental demonstration on this digital pre-equalization (DPEQ) scheme for bandwidth-limited signals in optical communication systems. The linear pre-equalization is based on the receiver-side blind and adaptive equalizers for channel estimation, such as least-mean-squares (LMS) algorithm and constant or multi- modulus algorithms (CMA, MMA). Based on the proposed channel estimation scheme, we theoretically and experimentally study the DPEQ performances under different implementation conditions, such as filtering bandwidth, taps length and OSNR. For bandwidth-limited systems, improved bit-error ratio (BER) performance can be obtained by using DPEQ compared with post-equalization only scheme. As a proof of the concept, the performance improvements by DPEQ are demonstrated by both simulation and experiment results. By using the high speed 64GSa/s DAC with pre-equalization, both improved back-to-back and transmission BER performances of 40-Gbuad polarization-diversion multiplexed (PDM) quadrature phase shift keying (QPSK), 8-quadrature amplitude modulation (QAM) and 16-QAM are obtained.

## 2. The principle of digital time-domain pre-equalization based on receiver-side adaptive equalizer

#### 2.1 The principle of DPEQ

Figure 1 shows the principle of the proposed pre-equalization by leveraging the estimated inverse channel given by the receiver-side adaptive equalizer, which comes with two stages. The first stage is the channel estimation, where a data X(t) without pre-equalization is transmitted and passes through the channel H(t). The channel is band-limited with a channel response of H(t), which cause the signal distortion with ISI due to the narrow filtering effect. In a coherent optical system, such channel response H(t) represents an end-to-end transfer function taking the analog bandwidths of the DAC, the driver, the modulator at the transmitters, and that of the photo-detector and the analog-to-digital convertor (ADC) at the receivers into consideration. Again, it is worth noting that, we use the channel here describing the transfer function of the transmitter hardware, but not the outside plant of the transmitter.

Assuming the noise n(t) is additive white Gaussian noise (AWGN), then the received signal can be expressed as

The received signal Y(t) suffers distortions with ISI caused by bandwidth limitation. As analyzed in [21–23], to compensate the channel distortion, we can employ a linear filter with adjustable taps to equalize the received signalwhere Z(*t*) is the equalized signal. Q(

*t*) is the impulse response of an adaptive linear equalizer for channel equalization. Q(

*t*) can be implemented using either zero-forcing or minimum-mean- square-error (MMSE) criterions. In practice, MSE-criterion equalizers are better and are more widely used since the zero-forcing equalizers may result in noise amplification. Here, we choose Q(

*t*) as a MSE-criterion based equalizer using stochastic gradient algorithms, such as CMA and LMS [24]. These two algorithms are widely adopted in optical coherent communication system. Since CMA- and LMS-based equalizers take both ISI and noise into account in the filter taps updating, the final goal is to find the optimal filter Q(t) having minimum MSE. In our case, we can minimize the noise in order to obtain the exact channel response. Assuming the noise

*n*(

*t*) is small and negligible, we can remove the noise part in Eq. (2) and haveTherefore, it is clear that we have

*Q*(

*t*)

*= H*(

*t*)

*if*

^{−1}*Z*(

*t*)

*= X*(

*t*). In this case,

*H*(

*t*)

**Q*(

*t*) = 1. It shows that, the equalizer is the channel inverse when the noise is small. More specifically, in practice, the ISI is limited to a finite number of samples in real channels. Therefore, the channel equalizer is approximated by a finite duration impulse response (FIR) with symbol-spaced or fractionally spaced taps.

Considering the common CMA or DD-LMS equalizer in coherent optical communication system, in which four *T*/2-spaced FIRs are used as the channel equalizer with *T*-spaced updating and detector loop. The error function is approaching zero during the convergence, therefore, ISI can approach zero at the *T* sampling points when noise is negligible.

Since only *T* sampling points are calculated during the updating and convergence, the output symbols of equalizer Q(t) after convergence (ISI is 0 at the T sampling points) can be expressed as

*X*is a Nyquist pulse-shaping criterion filter, and comparing Eqs. (3) and (4), we haveIn frequency domain, the response of the equalizer

_{N}(t)*Q(t)*can be expressed as

Here the *H(f)* is the frequency response of the bandwidth-limited channel. It shows that the frequency response of the equalizer is the inverse of the channel response H(*f*) within the Nyquist bandwidth when the noise is negligible. In this way, for the *T*/2-spaced DD-LMS with *T*-spaced detection and updating loop, the channel response within Nyquist bandwidth can be estimated, otherwise, it approaches 1. Therefore, a time-domain pre-equalization method can be employed based on the receiver-side adaptive equalizer. For optical coherent transmission system, one can simply records the FIR tap coefficients the output of those commonly-used linear equalizers (such as CMA, CMMA and DD-LMS equalizers), and feedback that information to the transmitter for pre-equalization.

Note that the analysis above excludes the factor of noise, which, however, represents in real systems. On the other hand, as analyzed in [21–23], the performance of channel estimation by adaptive filter is significantly affected by the filter taps length and also the OSNR in channel. Therefore, considering all the factors, the channel inverse calculated by adaptive filter taps is a function of channel response, noise level, taps length, which can be expressed as

where*N*is the AWGN power spectrum density and L is the taps length. These factors should be considered in practical implementation.

_{0}As analyzed in [21–23], the benefit of the pre-equalization can be proved by the smaller MSE of the recovered signal compared with post-equalization-only case in symbol detection systems. Assuming filter taps is long enough and the step size is small enough, one can obtain an optimal filter taps, which has the minimum MSE based on the MMSE criterion algorithms as

The minimum MSE achievable by a linear equalizer using above optimal filter is given byWe can see that, the minimum MSE determined by the noise power and also the channel response. The minimum MSE can be very large even with linear equalizations when the H(*f*) is small, which means the bandwidth of channel is significantly limited. For pre-equalization case, assuming the channel response is exactly estimated as Eq. (7), the bandwidth limitation can be fully compensated. In this way, the new channel response including the pre-equalization is

*H*. Therefore, the minimum MSE can be given by

_{pre}(f) = Q(f)H(f) = 1Therefore, the minimum MSE of pre-equalization case is smaller than post-equalization only case when the channel is bandwidth limited with narrow filtering effect. Lager gain can be obtained using the pre-equalization for narrower filtering bandwidth. Equation (8) gives a qualitative analysis of the factors that affect the estimation results. The estimated channel response is determined by the OSNR, taps length and bandwidth. The required OSNR or taps length can be different under different BER tolerance and different channel response. One should adjust and optimize these factors for practical use.

#### 2.2 Implementation of DPEQ for coherent optical system

Figure 2 shows the principle of channel estimation for the proposed DPEQ at the transmitter. The linear pre-equalization is based on the receiver-side blind and adaptive equalizers for channel estimation. We first use the DAC to generate the mQAM data without pre-equalization for channel estimation. Since the bandwidth limitation impairment is mainly caused by the DAC, the electrical drivers, the modulator, the receiver-side PDs and the analog-to-digital converter (ADC), only single-polarization signal is used to avoid the polarization crosstalk. One continuous-wave (CW) lightwave external-cavity-laser (ECL) is used as both the signal source and the local oscillator (LO) for the self-homodyne coherent detection. In this case, the traditional post-equalization methods for polarization demultiplexing, e.g., CMA and DD-LMS, are actually the channel equalizers for the bandwidth limitation impairment, which can be used for channel estimation. The amplitude frequency response of these equalizers is the inverse transfer function of the channel. The DD-LMS loop, which is after CMA for pre-convergence, consists of four complex-valued, N-tap, finite-impulse-response (FIR) filters for equalization. After convergence, these FIR filters achieved the steady state. After normalization and frequency symmetrization, the time domain FIR for pre-equalization can be regenerated. It is worth noting that, although the m-QAM data is used as training sequence, we do not need to know the symbol information. The only information needed for the training sequence is the modulation formats. On the other hand, as analyzed in section 1, in order to get the highly accurate channel response, we should keep the noise negligible. Thus, the channel estimation can be implemented under high OSNR condition using back to back (BTB) measurement. Since the channel estimation is based on the commonly-used linear equalizers in regular coherent receiver-side DSP, the proposed method, featuring no additional DSP, is promising for system implementation.

## 3. Numerical simulation results

To investigate the performance of DPEQ for band-limited signal in optical coherent system and the impact of different implementation conditions, such as the filtering bandwidth, OSNR, receiver-side adaptive equalizer tap length, step value and modulation formats, a numerical simulation model is firstly setup to emulate various operating situations. The system setup configuration consists of both DSP units and electro and optical components at transmitter (Tx) and receiver (Rx) sides for the DPEQ.

#### 3.1 System model setup

Figure 3 shows the numerical simulation model setup for DPEQ based on receiver-side adaptive equalizer using commercial optical simulation software. The input data consists of 4 components (Real and imaginary data in X and Y polarizations). These data components drive two I/Q modulators for signal modulation. The data components are generated by using Tx DSP without and with DPEQ as shown in Fig. 3. For data without DPEQ, the data components are generated by mQAM mapping. For DPEQ case, a time-domain pre-equalization is applied by convolution with a FIR, which is the inverse of channel response. The I/Q modulators are ideal with two parallel Mach-Zehnder modulators, which are both biased at the null point and driven at linear region. The phase difference between the upper and the lower branch of the I/Q modulator is controlled at π/2. After modulation, the two polarization signals are combined and transmitted together.

As a proof of concept, we use two pairs of low-pass filters (totally 4 LPFs) after data generation to simulate the bandwidth limitation effect. The LPFs are identical, which are general 3-order Bessel low pass filter with adjusted 3-dB bandwidth. Except the LPFs, other components, such as the driver and I/Q modulator, are ideal without any bandwidth limitation. The electrical drivers are identical and ideal without noise figure. One external cavity laser (ECL) is used as the light source with the linewidth of 100 kHz. The AWGN is added after the polarization multiplexing with different OSNR value.

For the receiver-side, the polarization and phase diversity coherent detection is employed using one polarization diversity optical hybrid and four balanced photo detectors (PDs). One ECL is used as local oscillator (LO) at the same frequency of ECL used in the Tx and linewidth of LO is also 100 kHz. The balanced PDs are based on PIN model with 1 A/W responsivity and 10 × 10^{−12} A/(Hz)^{0.5} thermal noise figure. The bandwidth of the PDs is wide enough with 1x baud rate pass-band. The analog to digital converter (ADC) is ideal with 2x sampling rate. After that, the receiver-side DSP is applied based on the regular optical digital coherent DSP blocks [24–26], including clock recovery, the channel equalization and polarization demultiplexing based on T/2 CMA and DD-LMS and carrier recovery. The bit-error-ratio (BER) is measured after the equalizations and decision. In simulation, the data baud rate is set at 32-Gbaud. Errors are counted over 1 million bits, and OSNR is loaded with 0.1-nm noise resolution bandwidth in the simulation model.

For pre-equalization, the implementation is based on the scheme proposed in Section 2.2, where the channel estimation is based on the filter taps of adaptive equalizer (CMA or DD-LMS) under the high OSNR, single-polarization, self-coherent detection conditions. Since the four LPFs are identical, the channel estimation for X polarization. is the same with Y polarization. We can use the same FIR for the four port Xi, Xq, Yi and Yq data pre-equalization. As analyzed in Section 2.1, the channel estimation performance is a function of channel bandwidth, tap length, and OSNR. Therefore, in the following simulations, we will study the impact of these key factors.

#### 3.2 DPEQ performance under different LPF bandwidth

We first study the DPEQ performance of the 32-GBaud PDM-QPSK signals under different LPF bandwidth and compared with those in the post-EQ only case. To simulate the bandwidth limitation effect, we set the LPFs under different one-sided 3-dB electrical bandwidth (EBW). To obtain the channel response, we use the channel estimation method proposed in Section 2.2 based on the DD-LMS.

Figure 4(a) shows the transfer function of the LPF with
7-GHz EBW. Figure 4(b) shows the frequency response of CMA/
DD-LMS and also the regenerated FIR for pre-equalization. The ideal channel inverse of 7-GHz
EBW LPF is also plotted in Fig. 4(b). The DD-LMS taps
response is calculated under 45-dB OSNR and the taps length is 33. The updating step size is
set at 5 × 10^{−4}. From Fig.
4(b), we can see that the taps response matches well with the ideal channel inverse
using our proposed channel estimation method. Therefore, we can use the generated FIR for
pre-equalization based on the response of DD-LMS. For QPSK signal, the response of DD-LMS is
the same with that of CMA since the constellations of QPSK are within the same modulus. Figures 4(c) and 4(d)
shows the magnitude and phase frequency response of DD-LMS taps. We can see that, the phase of
the DD-LMS taps is linear. The FIR response shape is liked “M” in Fig. 4(b). It is determined by the adaptive equalization
scheme used. The T/2-spaced FIR taps are used; therefore, the frequency response can cover a 2
Nyquist frequency range (from −32 to 32GHz for a 32GBuaud signal). However, since only T
sampling points are calculated during the updating and convergence, zero–ISI needs only
to be achieved at the sampling points and thus the response of the FIR approaches the channel
inverse only within the Nyquist bandwidth.

Figure 5 shows the signal spectrum of the 32-GBaud QPSK signals under different stages without and with pre-equalization. The spectrum of ideal 32-Gbaud QPSK signal without bandwidth limitation is shown in Fig. 5(a). The spectrum of the 32-Gbaud QPSK signal under 7-GHz LPF is shown in the Fig. 5(b). Figures 5(c) and 5(d) shows the signal with pre-equalization using time-domain FIR shown in Fig. 4(b) before and after the 7-GHz LPF. We can see that, the high frequency components within the Nyquist bandwidth is enhanced after pre-equalization.

Figure 6(a) shows the back-to-back (BTB) BER results of the 32-GBaud PDM-QPSK signal without and with DPEQ under different filtering different filtering bandwidth (7-GHz and 9-GHz EBW). Without pre-equalization, the system performance suffers degradation caused by ISI under significant bandwidth limitation as analyzed in Eqs. (9)–(11). There are about 4.5-dB and 2.5-dB OSNR improvements for signal generation using proposed DPEQ for 7-GHz and 9-GHz filtering, respectively. Insets (i) and (ii) show the eye diagrams of recovered signal without and with DPEQ for 7-GHz EBW filtering under the OSNR of 16 dB.

Figure 6(b) shows the OSNR penalty at BER of 1 × 10^{−2} for 32-GBaud PDM-QPSK without and with DPEQ under different filtering bandwidth from 4 GHz to 16 GHz. Inset (i) shows frequency response of generated FIR based on the estimated channel inverse using DD-LMS under different filtering bandwidth. We can see that, the OSNR penalty increase with the decrease of LPF bandwidth. The OSNR penalty can be larger than 6-dB when the EBW of LPF smaller than 6 GHz. However, the OSNR penalty in the DPEQ case is still lower than 2 dB even though the EBW is as small as 4 GHz. When using the proposed DPEQ method, the OSNR penalty is lower than 0.5 dB for EBW of LPF lager than 6 GHz. From the results, we can see that, large OSNR improvement can be obtained by using the proposed DPEQ method. These simulation results are in good agreement with the theoretical analysis in Section 2.1.

#### 3.3. The impact of adaptive equalizer taps length

As analyzed in Section 2, the inverse channel calculated by adaptive filter is affected by the tap length. In practice, the taps length should sufficiently large enough so that the equalizer spans the length of ISI. However, since the FIR has a finite length, the linear equalizer cannot completely eliminate ISI. However, as the length of taps *L* increase, the residual ISI can be reduced. Therefore, the inverse channel estimation can be more accurate with longer tap length.

Figures 7(a)–7(d) show the frequency response of the DD-LMS taps compared with the ideal channel inverse of the LPF with 7-GHz EBW with different taps length in channel estimation. For the 5- taps DD-LMS, the response difference is even larger than 5 dB at the frequency of ± 16 GHz ( ± 0.5 Baud rate). We can see that, with the tap length increases, the frequency response of DD-LMS matches the ideal channel inverse closer and closer, especially at the high-frequency response within the Nyquist bandwidth.

The pre-equalization performance of the FIR generated by different length of DD-LMS tap under different channel filtering bandwidth is also studied as shown in Fig. 6(e). Here, the DD-LMS taps response is calculated at 45-dB OSNR and the taps updating step size is set at 5 × 10^{−4}. The tap length is only changed for channel estimation and DPEQ FIR generation. The OSNR after pre-equalization is kept at 14 dB. For BER measurement, the post equalization for all cases is the same with 33-taps DD-LMS. Therefore, the performance differences are only influent by the channel filtering bandwidth and the channel estimation tap length. We can see that, for 7-GHz channel filtering, the required DD-LMS taps length should larger than 13 for channel estimation and DPEQ FIR generation. For 9-GHz, 9 taps DD-LMS is enough for channel estimation. However, for 6-GHz filtering, the required taps length should be larger than 17. That is, for narrower channels, the required tap length for channel estimation at the adaptive equalizer would be larger. This is because the tap length should be large enough so that the equalizer can cover the length of ISI in order to obtain accurate channel estimation.

#### 3.4. The impact of OSNR on channel estimation

As analyzed in Eqs. (6) and (9) of Section 2.1, we know that the noise should approach 0 to get the exact inverse channel by using an adaptive equalizer. However, in practice, the noise always exists and cannot be removed. Therefore, the frequency response of adaptive equalizer by MMSE method always has difference with the ideal channel inverse. In this section, we study the impact of the OSNR on channel estimation in simulation.

Figure 8(a) shows the frequency response of generated FIR
based on the DD-LMS taps at different OSNR for channel estimation. As analyzed in Eq. (9), the frequency
response of the adaptive equalizer is the inverse of noise-loaded band-limited channel added
with noise power. Therefore, as shown in Fig. 8(a), the
frequency response of FIR under low OSNR is far away from the ideal channel inverse due to the
large noise power, especially for high-frequency components with in Nqyuist bandwidth. Here, we
keep the DD-LMS with a tap length of 33 and the updating step size of 5 ×
10^{−4}. When the channel estimation is under OSNR of 13dB, the frequency
response difference compared with ideal channel inverse is larger than 6 dB at the frequency of
± 16 GHz ( ± 0.5 Baud rate). The difference decreases with the increase of OSNR
for channel estimation. These results are in good agreement with analysis in Section 2.1. Figure 7(b) shows the BER performance of DPEQ based on
channel estimation at different OSNR for different channel filtering bandwidth. The BER of
32-GBaud PDM-QPSK signal with DPEQ is measured at a fixed OSNR of 14 dB. Figure 7 reveals that, first, the DPEQ performance is improved with the
increasing of OSNR for channel estimation. It can be proved by the results in Fig. 7(b) and also the analysis in Section 2, since the
response of DPEQ approaches the ideal channel inverse with the decreasing of noise. Second, the
noise power has more significant influence on the DPEQ performance when estimating narrower
channels. We can see that, for 6-GHz filtering bandwidth, the gain of DPEQ is the highest by
increasing the OSNR for channel estimation from 13 to 30 dB. Therefore, higher OSNR is required
for channel estimation under narrower filtering channel bandwidth.

#### 3.5The pre-equalization for Nyquist and super-Nyquist WDM cases

In above simulation and analysis, the ideal signals are without any spectrum shaping for WDM transmission. For practical use, one may apply nearly rectangular Nyquist for a more tight WDM system. In the Nyqusit WDM (N-WDM) system, the carrier frequency spacing is equal to the signal baud rate to form a zero guard-band frequency-division multiplexing. On the other hand, recently, the super-Nyquist WDM (SN-WDM) transmission has also attracted lots of research interest [27–31]. In the SN-WDM system, one can use narrow spectrum shaping to multiplexing the signals in frequency domain with carrier spacing less than baud rate. To realize the SN-WDM, instead of using pre-equalization to compensate the narrow filtering, one can use additional DSP in the receiver side to suppress the enhanced noise and crosstalk after regular channel post-equalization with the multi-symbol detection, such as maximum-likelihood sequence detection (MLSD). The performance of MLSD with quadrature duo-binary (QDB) processing in filtered optical communication systems, especially for the SN-WDM systems has been previously investigated in [27–31]. Here, we compare the performances with or without pre-equalization using different processing techniques in both single channel and WDM under different carrier spacing.

Using the same implementation of DPEQ for coherent system descripted in section 2.2, we can also do the pre-equalization for Nyquist spectrum shaped signals. Using the simulation setup in Fig. 3, we send the Nyquist signal with roll-off of 0 for channel estimation. Fig. 9 shows the spectrum of 32-GBaud Nyquist QPSK signals under different stages without and with pre-equalization. The spectrums of ideal 32-Gbaud Nyquist QPSK signal before and after the 7-GHz LPF are shown in Figs. 9(a) and 9(b). Figures 9(c) and 9(d) shows the signal with pre-equalization using time-domain FIR shown before and after the 7-GHz LPF, respectively.

Figure 10(a) shows the frequency response of the DD-LMS taps under the 45-dB OSNR and with tap length of 33. From Fig. 10(a), we can see that the taps response also matches well with the ideal channel inverse using our proposed channel estimation method. Therefore, we can use the generated FIR for pre-equalization based on the response of DD-LMS.

Figure 10(b) shows the BTB BER results of the 32-GBaud Nyquist QPSK signals versus OSNR in single channel and N-WDM case with or without DPEQ. The carrier spacing is kept at 32-GHz in N-WDM case. The roll-off factor is 0. We also measure the BER of N-WDM signals under the 7-GHz narrow filtering without DPEQ but using the previously reported QDB processing with MLSD as a comparison. Several conclusions can be made from Fig. 10(b). First, without pre-equalization, the system performance suffers degradation caused by ISI under significant bandwidth limitation as analyzed above. There are about 4.5-dB OSNR improvements for signal generation using proposed DPEQ for 7-GHz filtering at the BER of 1x10^{−3}, with Nyquist spectrum shaping as shown in Fig. 9. Second, for the N-WDM case, there is about 0.5-dB penalty due to the channel crosstalk for the signals with DPEQ. Since the signals without DPEQ are greatly suppressed at high spectrum components, the crosstalk is smaller in N-WDM case. Negligible OSNR penalty is observed for the N-WDM signal without DPEQ. However, the BER performance N-WDM signal with DPEQ is still better than that without DPEQ. About 4-dB OSNR improvement are obtained by using DPEQ in N-WDM cases. Finally, compared with the results using QDB processing with MLSD, the DPEQ still shows better performances in N-WDM case with 1-dB OSNR improvement. Since the QDB processing with MLSD using additional DSP to equalize the ISI, the BER performance is better than that using only regular QPSK DSP in narrow filtering case. About 3-dB OSNR improvement can be observe under the 7-GHz narrow filtering in N-WDM case as shown in Fig. 10(b). However, it is worth noting that the QDB processing with MLSD requires additional DSP with a considerable increase in computational complexity [27–31].

The BER results of WDM signals under narrow filtering with different carrier spacing are shown in Fig. 10(c). The signal baud rate is kept at 32-GBaud, with roll-off factor of 0 and the carrier spacing is changed from 38 to 24-GHz. As shown in Fig. 10(c), the BER performance of pre-equalized signal drops quickly as the channel spacing narrows down with more severe inter-channel interference (ICI) brought by neighboring channels. Therefore DPEQ schemes are not best choice for the situation that strong ICI exists, when the channel spacing is smaller than the baud rate in super-Nyquist WDM cases. For carrier spacing larger than and equal to 32GHz (WDM or N-WDM cases), the DPEQ shows the best performances under both 7-GHz and 9-GHz narrow filtering. However, for signals in SN-WDM case with carrier spacing less than 32-GHz, the QDB processing with MLSD shows the best performance compared under the narrow filtering. Therefore, the QDB processing with MLSD shows better ICI tolerance capability for SN-WDM cases and it is widely used in previously SN-WDM reports [27–31].

## 4. Experiment results

As a proof of concept, Fig. 11 shows the experimental
setup of the 40-Gbaud PM-QPSK/8QAM/16QAM generation based on high speed DAC with adaptive
pre-equalization, transmission and coherent detection in a 50-GHz WDM grid. Eight tunable external cavity lasers (ECLs) ECL1 to ECL8 as 8 channels are used in our
system with the linewidth less than 100 kHz and the output power of 14.5 dBm. The
carrier-spacing of ECLs is 50 GHz. Before the independent in-phase and quadrature (I/Q)
modulation, the odd and even channels are implemented with two sets of polarization-maintaining
optical couplers (PM-OCs). The QPSK/8QAM/16QAM signals with 40-GBaud are generated by a 64-GSa/s
DAC. The 3-dB analog bandwidth of the DAC is about 11.3-GHz. As shown in Fig. 8, the inphase (I) and quadrature (Q) data are generated by the Tx DSP
blocks. The transmitted data is firstly mapped to *m*-QAM (*m*:
4/8/16), then up-sampled to 2 Sa/symbol. The pre-equalization is implemented to for the
up-sampled data to compensate the bandwidth limitation impairment caused by the DAC, the
drivers, the I/Q modulator and the ADC. In our case, the DPEQ is implemented and the FIR is
obtained by the adaptive scheme as shown in Fig. 2. The
polarization multiplexing of the signal is realized by the polarization multiplexer, which
comprises a PM-OC, an optical delay line to provide a delay of 150 symbols, and a polarization
beam combiner (PBC) to recombine the signal. The even and odd channels are modulated and
polarization multiplexed individually. Then, they are combined by a 3-dB optical coupler. At the
receiver side, the polarization and phase diversity coherent detection is employed. Here, the
linewidth of receiver-side local oscillator (LO) is around 100kHz. A digital oscilloscope with
the sample rate of 80GSa/s and bandwidth of 30GHz is used for analog-to-digital conversion (ADC)
before offline processing.

The results of channel estimation and pre-equalization are shown in Fig. 12.First, we use the 64-GSa/s DAC generate the QPSK data without pre-equalization for channel estimation to get the channel transfer function as shown in Fig. 2. Since the bandwidth limitation impairment is mainly caused by the DAC, the electrical drivers, the modulator and the ADC, only single- polarization signal is used to avoid the polarization crosstalk. Self-homodyne coherent detection is applied using the same CW lightwave (ECL1) as both the signal source and the LO source. Figures 12(a) and 12(b) show the frequency response of 33-taps FIR filter Hxx in DD-LMS and the regenerated FIR for DPEQ, which indicates the inverse transfer functions of the channel. It is worth noting that, the same source is used in channel estimation only for convenience and also easy operation, since no frequency offset is needed for channel estimation. However, it is not the central to this technique, since the frequency offset can be compensated before the channel estimation. Other parts of the experiment, such as back to back and transmission performance measurement are done using different lasers

Figure 13 shows the back-to-back (BTB) BER performance
versus the OSNR for single carrier 40-Gbaud PM-QPSK, PM-8QAM, and PM-16QAM signal with and
without pre-equalization, respectively. Here, 33-taps FIR filters are used. We can see that, about 3.5dB, 2.5dB and 1.5dB OSNR
improvement can be obtained at the BER of 1 × 10^{−3} by DPEQ based on
DD-LMS method for the 40-Gbaud PM-QPSK/8QAM/16QAM signals, respectively. As a comparison, we
also plot the theoretical BER curves in these figures. The BER results of signals with a
previously reported frequency domain pre-equalization (FD Pre-eq) method [5, 6] are also plotted. We can see that,
the results of the proposed DPEQ has similar performance with the frequency domain method, since
both scheme use the signals with very high OSNR for channel estimation. On the other hand, we
also observe the OSNR penalties compared with the theoretical performances. About 1.5, 2.5 and
4.5-dB OSNR penalties are observed for signals with pre-equalization compared with the
theoretical curves. Since higher modulation formats require higher OSNR and they are more
sensitive to the ISI, the pre-equalization shows less effective for the high modulation
formats.

Figure 14(a) shows the experiment results of BTB BER results of 40-Gbaud PM-QPSK/8QAM/16QAM with adaptive pre-equalization versus the tap length N of Hxx used in DD-LMS. Here, we keep QPSK as the training data modulation format and the OSNR of these signals measured is 16, 21.5 and 25-dB for 40-Gbaud PM-QPSK, 8QAM and 16QAM. From Fig. 13, we can see that, QPSK is less sensitive to the tap length, and 9 taps is sufficient for pre-equalization. However, higher modulation formats 8QAM and 16QAM requires more taps for pre-equalization.

Figure 14(b) shows the transmission BER performance with and without pre-equalization for the 8 channels WDM PM-QPSK/8QAM/16QAM signals. The combined 8 channels WDM signals are launched into a re-circulating transmission loop, which consists of 5 spans of 85-km conventional SMF-28 with average loss of 18.5 dB and chromatic dispersion (CD) of 17 ps/km/nm, loop switches (SWs), optical coupler (OC), and Erbium-doped fiber amplifier (EDFA)-only amplification without optical dispersion compensation. In the loop, we place one wavelength-selected switch (WSS) programmed as an optical band-pass filter to suppress the ASE noise. The BER of WDM PM-QPSK, 8QAM, and 16QAM signals without and with pre-equalization versus the transmission distance is shown in Fig. 14(b). Longer distance can be achieved for signal with pre-equalization. Without DPEQ, the system performance is seriously degraded by ISI, noise enhancement and inter-channel crosstalk due to the bandwidth limitation and filtering effect. The system performance can be improved by reducing these bandwidth limiting impairments using the proposed DPEQ scheme.

Finally, we measure the DPEQ performances for the Nyquist signals in N-WDM cases. The 40GBuad Nqyuist signal with roll-off factor of 0 is generated by the 64GSa/s DAC for test. Figures 15(a) and 15(b) shows the FFT spectrum of the Nyquist signals after coherent detection and ADC sampling without and with proposed DPEQ, respectively. As shown in Fig. 15(b), we can see that the Nyquist signals are pre-compensated with flat spectrum shape with DPEQ. Figure 15(c) shows the BTB BER results versus OSNR for the Nyquist 40GBaud PDM-QPSK signals in SC and N-WDM cases without and with DPEQ. Again, improved OSNR performances can be observed by using the DPEQ scheme. Therefore, the proposed DPEQ method is also effective for the Nyquist signals in N-WDM case.

## 5. Conclusions

We theoretically and experimentally investigate the time-domain DPEQ scheme for bandwidth-limited signals in optical coherent communication systems. Based on the proposed channel estimation scheme, we theoretically and experimentally study the DPEQ performances under different implementation conditions, such as filtering bandwidth, taps length, and OSNR. For bandwidth-limited systems, improved BER performance can be obtained by using DPEQ compared with post-equalization only scheme. As a proof of the concept, the performance improvements by DPEQ are demonstrated by both simulation and experiment results.

## Acknowledgments

This work was partially supported by NNSF of China (No. 61325002, 61250018, No. 61177071), “863” projects under grants 2012AA011303 and 2013AA010501, The National Key Technology R&D Program (2012BAH18B00), Key Program of Shanghai Science and Technology Association (12dz1143000)), and the China Scholarship Council (201206100076).

## References and links

**1. **J. Cai, H. Zhang, H. G. Batshon, M. Mazurczyk, O. Sinkin, Y. Sun, A. Pilipetskii, and D. Foursa, “Transmission over 9,100 km with a capacity of 49.3 Tb/s using variable spectral efficiency 16 QAM based coded modulation,” in *Proc. OFC* (2014), Postdeadline Papers, paper Th5B.4. [CrossRef]

**2. **G. Bosco, V. Curri, A. Carena, P. Poggiolini, and F. Forghieri, “On the performance of Nyquist-WDM terabit superchannels based on PM-BPSK, PM-QPSK, PM-8QAM or PM-16QAM subcarriers,” J. Lightwave Technol. **29**(1), 53–61 (2011). [CrossRef]

**3. **Y. Gao, J. C. Cartledge, A. S. Karar, S. S.-H. Yam, M. O’Sullivan, C. Laperle, A. Borowiec, and K. Roberts, “Reducing the complexity of perturbation based nonlinearity pre-compensation using symmetric EDC and pulse shaping,” Opt. Express **22**(2), 1209–1219 (2014). [CrossRef] [PubMed]

**4. **J. Wang, C. Xie, and Z. Pan, “Generation of spectrally efficient Nyquist-WDM QPSK signals using digital FIR or FDE filters at transmitters,” J. Lightwave Technol. **30**(23), 3679–3686 (2012). [CrossRef]

**5. **Z. Dong, X. Li, J. Yu, and N. Chi, “6 × 144-Gb/s Nyquist-WDM PDM-64QAM generation and transmission on a 12-GHz WDM Grid equipped with Nyquist-band pre-equalization,” J. Lightwave Technol. **30**(23), 3687–3692 (2012). [CrossRef]

**6. **Z. Dong, X. Li, J. Yu, and N. Chi, “6 × 128-Gb/s Nyquist-WDM PDM-16QAM generation and transmission over 1200-km SMF-28 with SE of 7.47 b/s/Hz,” J. Lightwave Technol. **30**(24), 4000–4005 (2012). [CrossRef]

**7. **X. Zhou, J. Yu, M.-F. Huang, Y. Shao, T. Wang, L. Nelson, P. Magill, M. Birk, P. I. Borel, D. W. Peckham, R. Lingle, and B. Zhu, “64-Tb/s, 8 b/s/Hz, PDM-36QAM transmission over 320 km using both pre- and post-transmission digital signal processing,” J. Lightwave Technol. **29**(4), 571–577 (2011). [CrossRef]

**8. **X. Zhou, L. E. Nelson, P. Magill, B. Zhu, and D. W. Peckham, “8x450-Gb/s,50-GHz-spaced,PDM-32QAM transmission over 400km and one 50GHz-grid ROADM,” in *Proc. OFC* (2011), paper PDPB3. [CrossRef]

**9. **D. McGhan, C. Laperle, A. Savchenko, C. Li, G. Mak, and M. O’Sullivan, “5120 km RZ-DPSK transmission over G652 fiber at 10 Gb/s with no optical dispersion compensation,” in Proc. OFC (2005), PDP27. [CrossRef]

**10. **T. Sugihara, T. Kobayashi, Y. Konishi, S. Hirano, K. Tsutsumi, K. Yamagishi, T. Ichikawa, S. Inoue, K. Kubo, Y. Takahashi, K. Goto, T. Fujimori, K. Uto, T. Yoshida, K. Sawada, S. Kametani, H. Bessho, T. Inoue, K. Koguchi, K. Shimizu, and T. Mizuochi, “43 Gb/s DQPSK pre-equalization employing 6-bit, 43 GS/s DAC integrated LSI for cascaded ROADM filtering,” in *Proc. OFC* (2010), Paper PDPB6. [CrossRef]

**11. **D. L. Hershberger, “Full channel adaptive equalization for DTV transmitters,” in *The Broadcast Engineering Conf. Proc. of the NAB* (2002), 223–230.

**12. **E. Coersmeier and E. Zielinski, “Adaptive pre-equalization in analog heterodyne architectures for wireless LAN,” in *Conf. Proc. of the IEEE RAWCON* (2002), T2.3, 107–110. [CrossRef]

**13. ** Data Over Cable Service Interface Specification (DOCSIS), Proactive Network Maintenance Using Pre-equalization (2011), http://www.cablelabs.com/wp-content/uploads/specdocs/CM-GL-PNMP-V01-100415.pdf

**14. **E. Coersmeier and E. Zielinski, “Comparison between different adaptive pre-equalization approaches for wireless LAN,” in *Conf. Proc. of the IEEE International Symposium on Personal, Indoor and Mobile Radio Communications* (2002), 1136–1140. [CrossRef]

**15. **J. Pan, P. Isautier, and S. E. Ralph, “Digital pre-shaping for narrowband filtering impairment compensation in superchannel applications,” in *Proc. of Advanced Photonics* (2013), paper JT3A.1.

**16. **J. Zhang and H. Chien, “A novel adaptive digital pre-equalization scheme for bandwidth limited optical coherent system with DAC for signal generation,” in *Proc. OFC* (2014), paper W3K.4. [CrossRef]

**17. **J. Zhang, H. Chien, Z. Dong, and J. Xiao, “Transmission of 480-Gb/s dual-carrier PM-8QAM over 2550km SMF-28 using adaptive pre-equalization,” in *Proc. OFC* (2014), paper Th4F.6. [CrossRef]

**18. **J. Yang, J.-J. Werner, and G. A. Dumont, “The multimodulus blind equalization and its generalized algorithms,” IEEE J. Sel. Areas Commun. **20**(5), 997–1015 (2002). [CrossRef]

**19. **D. Godard, “Self-recovering equalization and carrier tracking in two-dimensional data communication systems,” IEEE Trans. Commun. **28**(11), 1867–1875 (1980). [CrossRef]

**20. **C. R. Johnson, P. Schniter, T. J. Endres, J. D. Behm, D. R. Brown, and R. A. Casas, “Blind equalization using the constant modulus criterion: a review,” Proc. IEEE **86**(10), 1927–1950 (1998). [CrossRef]

**21. **S. U. H. Qureshi, “Adaptive EQUALIZATION,” Proc. IEEE **73**(9), 1349–1387 (1985). [CrossRef]

**22. **J. G. Proakis, *Digital Communications IV* (McGraw Hill, 2001).

**23. **S. Haykin, *Adaptive Filter Theory IV* (Prentice Hall, 2001).

**24. **S. J. Savory, “Digital filters for coherent optical receivers,” Opt. Express **16**(2), 804–817 (2008). [CrossRef] [PubMed]

**25. **I. Fatadin, D. Ives, and S. J. Savory, “Blind equalization and carrier phase recovery in 16-QAM optical coherent system,” J. Lightwave Technol. **27**(15), 3042–3049 (2009). [CrossRef]

**26. **X. Zhou, J. Yu, and P. Magill, “Cascaded two-modulus algorithm for blind polarization de-multiplexing of 114-Gb/s PDM-8-QAM optical signals,” in *Proc. OFC* (2009), paper OWG3.

**27. **N. Alic, M. Skold, P. A. Andrekson, M. Karlsson, and S. Radic, “Benefits of joint statistics in MLSD-equalized transmission,” IEEE Photon. Technol. Lett. **21**(8), 495–497 (2009). [CrossRef]

**28. **N. Alic, M. Karlsson, M. Sköld, O. Milenkovic, P. A. Andrekson, and S. Radic, “Joint statistics and MLSD in filtered incoherent high-speed fiber-optic communications,” J. Lightwave Technol. **28**(10), 1564–1572 (2010). [CrossRef]

**29. **N. Alic, “Advances in MLSD-Equalized Incoherent Transmission,” in *Proc. of OFC* (2010), paper OThT1.

**30. **J. Li, Z. Tao, H. Zhang, W. Yan, T. Hoshida, and J. C. Rasmussen, “Spectrally efficient quadrature duobinary coherent systems with symbol-rate digital signal processing,” J. Lightwave Technol. **29**(8), 1098–1104 (2011). [CrossRef]

**31. **J. Zhang, Z. Dong, H.-C. Chien, Z. Jia, Y. Xia, and Y. Chen, “Transmission of 20× 440-Gb/s super-Nyquist-filtered signals over 3600 km based on single-carrier 110-GBaud PDM QPSK with 100-GHz grid,” in *Proc. OFC* (2014), Postdeadline Papers, Th5B. 3.