## Abstract

We propose a stochastic bit error ratio estimation approach based on a statistical analysis of the retrieved signal phase for coherent optical QPSK systems with digital carrier phase recovery. A family of the generalized exponential function is applied to fit the probability density function of the signal samples. The method provides reasonable performance estimation in presence of both linear and nonlinear transmission impairments while reduces the computational intensity greatly compared to Monte Carlo simulation.

©2010 Optical Society of America

## 1. Introduction

Digital signal processing (DSP) based coherent detection quadrature-phase-shift-keying (QPSK) is a key candidate for long-haul optical transmission systems [1,2]. Without relying on a complex and costly optical phase locked loop, digital carrier phase recovery (CPR) is instead performed together with flexible electrical equalization for various transmission impairments [2]. Very few methods are currently available to estimate the bit error ratio (BER) of coherent systems with digital CPR. A closed-form analytical solution for the BER has been reported [3]. However, it is restricted to systems with only linear impairments. For nonlinear optical fiber transmission of such systems, the noise is no longer Gaussian distributed and it is difficult to obtain analytical BER expression to evaluate the system performance. Instead, time-consuming Monte Carlo (MC) simulations have to be employed to give accurate performance estimation.

In this paper we propose a stochastic BER estimation approach based on a statistical analysis of the probability density function (PDF) of the recovered signal phase after CPR. The tail extrapolation method based on the generalized exponential function (GEF) offers a reasonable estimation of the actual sample statistics near threshold. Compared to MC, our method greatly reduces the computational intensity, which is feasible to coherent optical fiber systems with both linear and nonlinear degradations.

## 2. Theory

The symbol error rate (SER) of a digital communication system can be written as [5]

where*S*is the range of signal value in which error occurs and $f(x)$stands for the PDF of the statistical variable for decision. For phase-modulated format, the decision is made according to the phase information of the signals. The transmission impairments result in the phase rotation. The phase noise (PN) of the transmitted pulse is chosen as the statistical variable for evaluating the system performance. As shown in Fig. 1 , the symbol position in the complex plane for QPSK is given by

The bit-to-symbol assignment is calculated by converting the binary value of ${n}_{t}$ to Gray-code {00,01,11,10}. Due to the phase ambiguities in CPR algorithm, one should differentially encode ${n}_{t}$ by

Here *k* is the symbol index. Same as${n}_{t},$
${n}_{d}\in \left\{0,1,2,3\right\}.$ After coherent detection and sampling the *k*-th QPSK symbol passed to the DSP unit can be expressed as${X}_{k}=\mathrm{exp}(i{\theta}_{k}).$ Here ${\theta}_{k}={\theta}_{d,k}+{\varphi}_{k}.$
${\theta}_{d,k}=({n}_{d,k}+1/2)\cdot \pi /2$ is the nominal phase for QPSK.${\varphi}_{k}$ is the corresponding phase impairment given by

For DSP based coherent detection, the scheme of block feed-forward carrier phase recovery is usually applied [4]. If the phase offset between two adjacent symbols for m-PSK is$2\pi /m,$ the modulation can be eliminated by raising the input signal${X}_{k}$to the *m*-th power. In the case of QPSK $m=4.$ For a block of *N* symbols, the common phase estimator for all samples in the block is given as

The recovered phase information is determined by

where $\lfloor x\rfloor $is the biggest integer$\le x.$ $\Delta {\theta}_{k}={\theta}_{k}-{\widehat{\varphi}}_{est}$ is the PN associated with the phase discrimination. The factor of $1/m$in Eq. (5) introduces an*m*-fold phase ambiguity which is eliminated by employing the differential encoding of Eq. (3) and the corresponding decoding

Note that the distribution of $\Delta {\theta}_{k}$ for the *k*-th symbol is denoted by the PDF$f(\Delta {\theta}_{k}).$ We divide the whole phase range$-\pi ~\pi $into four regions:$-\pi /4~\pi /4,$
$\pi /4~3\pi /4,$
$-3\pi /4~-\pi /4$ and the rest. Therefore the error occurs when the PN $\Delta {\theta}_{k}$ and $\Delta {\theta}_{k-1}$ of the neighboring symbols fall into different regions. Denoting the PN combinations when the error occurs as $S,$ the SER equals

The separation of variables in Eq. (8) is only valid when neglecting the correlation between the consecutive symbols due to the fact that ASE-contaminated phase noise of different symbols are independent. The assumption is acceptable as we consider inter-symbol interference (ISI) coming from intra-channel fiber nonlinearities and electrical filtering in both the transmitter and the receiver in the following PDF fitting.

For Gray-coded QPSK signals, when ${n}_{d,k}$and ${n}_{d,k-1}$ are in neighboring regions ($\left|{n}_{d,k}-{n}_{d,k-1}\right|\ne 2$) one symbol error corresponds to one bit error and this situation is denoted as *S*
_{1}. Otherwise ($\left|{n}_{d,k}-{n}_{d,k-1}\right|=2$), one symbol error corresponds to two bit errors, which is denoted as *S*
_{2}. As a result, BER is calculated as follows.

As for pattern-dependent ISI, taking the nearest-neighbors interactions into consideration [6], the PDFs of the four symbols are given as a sum of elementary distributions.

*j*indicates the sampled symbol and indexes $i,k$ are for the nearest neighbors. $i,j,k\in ${0, 1, 2, 3}. Here we use the GEF to fit the phase noise of the received signals, which provides a smoothed estimator of the PDF in the analytical form. The GEF is a family of the generalization of the Gaussian function which is defined as [7]

Among which, $\Gamma (\cdot )$ is the gamma function and *μ* is the mean value of the distribution. *ν* is related to the distribution’s variance ${V}_{\nu}$ by ${V}_{\nu}=2{\sigma}^{2}\Gamma (3/\nu )/\Gamma (1/\nu ).$The parameters $\nu ,$
$\mu ,$and *σ* may be different for various symbol patterns. Through GEF fitting with maximum likelihood estimation, the BER is calculated according to Eqs. (9)–(11).

## 3. Simulation results

To verify our proposed BER estimator, we simulate a 10Gbaud NRZ-QPSK system via commercial software VPItransmissionMaker7.5. The linewidths of the transmitter laser and the LO are set to 1.0 MHz. The fiber link consists of ten spans of 80 km standard single-mode fiber (SSMF) with attenuation of 0.21dB/km, dispersion of 16ps/nm/km and nonlinear coefficient of 1.32/W/km. An erbium-doped fiber amplifier (EDFA) compensates for the loss in each span. At first, inline EDFA is assumed noiseless and ASE noise is added in front of the receiver to set different optical signal-to-noise ratios (OSNRs) with a resolution of 0.1nm. Fiber nonlinearities are intensified by setting high launch power of 3.0 dBm. We choose a De Bruijn sequence of 65536 symbols to obtain statistical samples for nonlinear interaction. The block length for phase estimation is optimized according to the principle in [3].

Figure 2(a) and 2(b) show the constellations for both linear and nonlinear transmission by switching off and on the nonlinearity, respectively. Without loss of generality, we set the OSNR to be 10 dB. Figure 3(a) and 3(b) show the PDFs of the PN after phase estimation for different symbols, corresponding to Fig. 2(a) and 2(b). The squares stand for the results obtained directly from Monte Carlo simulation. The circles are based on GEF-ISI fitting with Eqs. (10) and (11), which offers a reasonable fitting of MC results. For comparison we also include Gaussian distribution fitting in the triangles with ISI taken into account. In comparison with GEF-ISI, Gaussian-ISI performs more poor in the tail, which is the most important region to calculate the BER.

Figure 4 compares the performance of different BER estimators in both linear and nonlinear transmission. For MC simulation, the BER is calculated with more than 100 errors counted. GEF and Gaussian fitting with and without ISI consideration are demonstrated. The analytical results based on the solution from [3] are also included. Figure 4(a) and 4(b) show that in both linear and nonlinear cases Gaussian approximation underestimates the BER significantly. For both GEF and Gaussian, the PDF fitting with ISI provides better BER estimation, since it accounts for signal interactions coming from electrical filtering in linear case and intra-channel nonlinearities in nonlinear case. For instance, in Fig. 4(b) GEF-ISI offers substantial improvements over GEF only, indicating that intra-channel nonlinearity induced ISI must be taken into account. For linear transmission GEF-ISI performs slightly better than the analytical method. Both of them agree well with MC simulation. Without surprising, for nonlinear transmission the analytical method significantly departures from MC and makes an unacceptable BER underestimation. Meanwhile, the BERs calculated from GEF-ISI fitting are quite close to that from MC. The difference between them keeps within one order of magnitude.

Note that the results of GEF-ISI fitting are quite stable for different ASE seed settings. To estimate the BER as low as 10^{−5}, MC simulation needs a sequence of 65536 symbols together with hundreds of ASE realizations to count 100 errors. For GEF-ISI, 65536 symbols with one ASE realization is more than enough for a reasonable BER estimation, which decreases the computational intensity by at least two orders of magnitude compared to MC simulation. However, it should be noted that a BER underestimation always exists, which, we believe, is attributed to the approximation in Eq. (8) and the inaccuracy of GEF distribution for PDF fitting in the nonlinear regime.

So far we have added ASE noise only before the receiver. To verify the feasibility of GEF-ISI fitting procedure in systems with nonlinear phase noise, inline signal-noise interaction should be considered. Here we simulate 80 spans transmission with −6.0 dBm launch power. The noise figure of the inline EDFA is set to 7.0 dB, resulting in an OSNR of 9.1dB. Figure 5 shows the constellations for both linear [Fig. 5(a)] and nonlinear transmission [Fig. 5(b) and 5(c)]. Figure 5(b) only considers nonlinear signal-signal beating by adding ASE at the receiver end. In contrast, Fig. 5(c) includes signal-ASE beating, i.e. nonlinear phase noise, by adding distributed ASE at each EDFA. Corresponding to Fig. 5(c), Fig. 6 shows the PDFs of the PN after phase estimation for different symbols with nonlinear phase noise. Compared to the linear case, nonlinear transmission yields a worse BER. Nonlinear phase noise further degrades system performance. The BER is calculated by MC and GEF-ISI fitting, respectively, for Fig. 5(a)–5(c). For GEF-ISI, 8192 symbols are used to fit the PDF. Compared to MC, GEF-ISI slightly underestimates the BER, indicating good agreement between these two methods.

Figure 7 shows BER estimation for different launch powers in the case of nonlinear phase noise. The system parameters are the same as those of Fig. 5(c). For GEF-ISI, 8192 symbols are more than enough to get a reliable BER estimation. However, if a sequence of 2048 symbols is used, noticeable deviations are observed between MC and GEF-ISI. In our approach, 64 symbol patterns are considered to take ISI into account. GEF-ISI fitting is performed for each pattern. Therefore, the PDF accuracy of GEF-ISI should be guaranteed by enough pattern samples. For a sequence of 2048 symbols, there are only 32 samples for each symbol pattern. The BER estimator performs worse due to inaccurate PDF fitting. In contrast, a sequence of 8192 symbols provides 128 samples for each symbol pattern, which yields an accurate BER estimation.

We also test the required sequence length for our proposed BER estimator in the case of Fig. 4 for different OSNRs and BER levels. Although valid in most cases, a sequence of 4096 symbols is on the margins of acceptability. As a rule of thumb, we believe a sequence of 8192 symbols can safely ensure a reasonable BER estimation, which is still much time saving compared with MC simulations.

## 4. Conclusion

In conclusion, we propose a stochastic BER estimation approach via GEF based PDF fitting for coherent detection QPSK transmission systems with digital carrier phase recovery. Comparing with MC simulation, the computational intensity of GEF-ISI is reduced significantly. GEF-ISI fitting provides reasonable performance estimation for both linear and nonlinear transmission without relying on time consuming MC.

## Acknowledgment

This work was supported by National Natural Science Foundation of China (No.60877045 and No. 60932004) and the Scientific Research Foundation for the Returned Overseas Chinese Scholars, State Education Ministry. The project was partially supported by VPI Photonics and National Basic Research Program of China (No. 2010CB328201). Fan Zhang also acknowledges the donation of VPI software suite from Alexander von Humboldt foundation.

## References and links

**1. **C. R. S. Fludger, T. Duthel, D. van den Borne, C. Schulien, E.-D. Schmidt, T. Wuth, J. Geyer, E. De Man, G.-D. Khoe, and H. de Waardt, “Coherent Equalization and POLMUX-RZ-DQPSK for Robust 100-GE Transmission,” J. Lightwave Technol. **26**(1), 64–72 (2008). [CrossRef]

**2. **T. Pfau, S. Hoffmann, O. Adamczyk, R. Peveling, V. Herath, M. Porrmann, and R. Noé, “Coherent optical communication: towards realtime systems at 40 Gbit/s and beyond,” Opt. Express **16**(2), 866–872 (2008), http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-16-2-866. [CrossRef] [PubMed]

**3. **G. Goldfarb and G. Li, “BER estimation of QPSK homodyne detection with carrier phase estimation using digital signal processing,” Opt. Express **14**(18), 8043–8053 (2006), http://www.opticsinfobase.org/oe/abstract.cfm?URI=oe-14-18-8043. [CrossRef] [PubMed]

**4. **K. Kikuchi, “Phase-diversity homodyne detection of multilevel optical modulation with digital carrier phase estimation,” IEEE J. Sel. Top. Quantum Electron. **12**(4), 563–570 (2006). [CrossRef]

**5. **J. G. Proakis, *Digital Communications,* (New York: McGraw-Hill, Ed.4, 2001).

**6. **E. Ciaramella, “Effective Approach to Estimate Optical System Performance from Numerical Simulations,” IEEE Photon. Technol. Lett. **20**(20), 1703–1705 (2008). [CrossRef]

**7. **M. Jeruchim, “Techniques for Estimating the Bit Error Rate in the Simulation of Digital Communication Systems,” IEEE J. Sel. Areas Comm. **2**(1), 153–170 (1984). [CrossRef]