## Abstract

An analytical model considering modulation-dependent nonlinear effects and second-order interactions between signal and optical amplifier noise is presented for Nyquist-spaced wavelength-division-multiplexing optical communication systems. System performance of dual-polarization modulation formats, such as DP-QPSK, DP-16QAM, and DP-64QAM, is investigated using both the analytical model and numerical simulations. A good agreement between analytical and numerical results shows that, in the case of full-field nonlinearity compensation, accounting for second-order interactions becomes essential to predict system performance of both single- and multi-channel systems at optimum launched powers and beyond. This effect is validated via numerical simulations for signal bandwidths up to $\sim 1\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{THz}$.

Published by The Optical Society under the terms of the Creative Commons Attribution 4.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Over the past four decades, data rates in optical communications systems have seen a dramatic increase. The development of a series of new technologies have contributed to this increase. These include techniques such as wavelength-division multiplexing (WDM), advanced optical signal modulation formats, optical amplifiers and novel fibers, coherent detection, and fast signal processing for mitigating linear and nonlinear (NL) optical effects. Suppressing optical nonlinearity, which is mainly due to the optical Kerr effect, has now become the greatest challenge for enhancing the achievable capacity of optical communications [1,2]. A multitude of nonlinearity compensation techniques have been proposed and demonstrated to mitigate Kerr nonlinearity-induced distortions, including single- and multi-channel (MC) digital backpropagation (DBP), optical phase conjugation, eigenvalue communication, NL Fourier transform, etc., as recently reviewed in [2].

Regardless of the specific scheme, when nonlinearity compensation is ideally operated over the entire transmitted bandwidth, complete suppression of all “deterministic” signal–signal (S–S) interactions can be achieved, while “stochastic” NL distortions consisting of NL signal–noise (S–N) interactions are left uncompensated. This leads to an increased optimal signal-to-noise ratio (SNR) and optimal transmitted power compared with conventional receiver schemes, such as electronic dispersion compensation (EDC). In addition, operating beyond the optimum power may enable potential advantages, e.g., in submarine transmission [3] and for improved detection schemes (Section 10, [4]).

In the range of powers, which are of interest for nonlinearity-compensated systems, first-order perturbation analysis is no longer sufficient for accurate characterization of system performance, and second-order nonlinear effects need to be taken into consideration. For instance, in [5] it was pointed out that, beyond the optimum power threshold and in the case of full-field (FF) DBP, the SNR decreases with a rate of 3 SNR [dB]/power [dB], rather than 1 SNR [dB]/power [dB] as conventional first-order analytical models predict [6]. This rapid SNR degradation has been attributed to second-order S–N interactions [5]. This term arises from the additional NL mixing process between signal and residual first-order S–N interactions, originating in the previous uncompensated fiber spans within a “virtual” DBP link, as schematically shown in Fig. 1. This effect was studied in [5] for single-channel optical transmission systems with a dual-polarization (DP) quadrature phase-shift keying (QPSK) signal. The analysis was based on analytical closed-form expressions derived for orthogonal frequency division multiplexing transmission, assuming that NL interference has a Gaussian distribution as well as being independent of input signal modulation format. Hence, the dependence on the number of channels as well as the impact of modulation format on the effective variance of NL distortions have not been analyzed. However, the implementation of a model accounting for number of channels and modulation format dependency in second-order S–N interactions enables an accurate investigation of the system achievable information rate (AIR), which is a natural figure of merit in coded communication systems [7,8].

In this Letter, within the framework of regular perturbation analysis [9], an analytical model for Nyquist-spaced WDM optical communication systems employing nonlinearity compensation is developed. Such a model extends the work in [5] by accounting for the modulation format dependency of second-order S–N interactions in a multi-channel transmission scenario. Different modulation formats, including DP-QPSK, DP 16-ary quadrature amplitude modulation (DP-16QAM), and DP-64QAM, were investigated. The SNR and optimum launched power for FF DBP schemes were analyzed as a function of transmission distance, accounting for first- and second-order S–N contributions. Finally, the relevance of second-order S–N interactions was quantified for different transmission distances and bandwidths. The developed analytical model allows us to predict the system performance at optimum power and beyond, which gives rise to further investigations of AIRs in coded transmission systems.

The performance of a dispersion-unmanaged optical communication system can be evaluated by introducing the so-called effective receiver SNR, which includes the impact of linear amplified spontaneous emission (ASE) noise and NL distortions due to the optical Kerr effect, as follows [6,10]:

Assuming that all the WDM channels have the same modulation format and launched power, the NL distortion coefficient in Eq. (2) is quantified by the following [12]:

where the first term ${\eta}_{0}$ does not depend on the signal modulation format because the signal is assumed to be strongly dispersed and, hence, Gaussian distributed. The second term ${\eta}_{0}^{\prime}$ is modulation format-dependent and includes the corrections needed for non-Gaussian input signal distribution [12]. Following the model developed in [12,13], this modulation-dependent term can be decomposed as follows:Employing FF DBP, closed-form expressions for the optimum launched power, i.e., the power corresponding to the maximum SNR at a given number of fiber spans, for the case of first-order S–N interactions only is ${P}_{\mathrm{opt}}^{(1)}=\sqrt{\frac{{N}_{\mathrm{s}}}{3{\xi}_{1}\eta}}$ and for first- and second-order S–N interactions is derived as

If DBP is applied over a partial bandwidth (dashed–dotted lines in Figs. 2 and 3), the residual S–S interactions ${P}_{\mathrm{s}-\mathrm{s}}$ in Eq. (1) can be obtained by using an approach similar to the one in [8,15]. It is also worth mentioning that, in the case of partial-bandwidth DBP, the nonlinear distortions owing to residual uncompensated S–S interactions are still more prevalent in comparison with both first- and second-order S–N interactions. Hence, the dramatic effect of second-order S–N interactions can be properly distinguished in the case of FF DBP only, i.e., when S–S interactions are completely suppressed [5,6].

Numerical simulations have been performed to assess the accuracy of the analytical model. The investigated scenario was a Nyquist-spaced WDM optical transmission system, using DP-QPSK, DP-16QAM, and DP-64QAM, with parameters shown in Table 2. A factor of 8 is used to oversample the simulation bandwidth, and the SNR is estimated over ${2}^{17}$ symbols based on the received constellation clusters, similar to [8,16,17]. The transmitted symbol sequences in each channel and polarization are independent and randomly generated. The signal propagation in standard single-mode optical fiber was simulated using the split-step Fourier method to solve the Manakov equation, where a logarithmic step-size distribution was adopted for each fiber span [18]. EDFAs were used to compensate for fiber attenuation. At the receiver, the signal was mixed with an ideal free-running local oscillator to ensure ideal coherent detection of the optical signal. EDC was implemented using an ideal frequency domain filter [19], whereas MC DBP was realized using the reverse split-step Fourier solution of the Manakov equation using the same step-size as in the forward propagation [20,21] to ensure an ideal operation of MC DBP. An ideal root-raised-cosine filter with a roll-off factor of 0.1% is applied to select the backpropagated bandwidth for MC DBP, whereas no filtering is applied for the case of FF-DBP. Laser phase noise and polarization-mode dispersion are neglected.

Analytical model and numerical simulations were carried out for a single- and five-channel WDM transmission system over a transmission distance of 2000 km. The results are shown in Figs. 2 and 3 for the single-channel and five-channel cases, respectively. Excellent agreement between the analytical and numerical calculations is observed. It can be seen that the second-order interactions have a significant impact on performance in the case of FF DBP, whereas it is negligible when DBP is applied over a fraction of the transmitted bandwidth. In the case of FF DBP, for values of launched power per channel beyond 7 dBm for a single-channel system (with the optimum power of 9 dBm) and beyond 5 dBm for the five-channel system (where the optimum power is 7 dBm), accounting only for first-order S–N interactions (dashed black lines) yields inaccurate predictions. It also can be seen that the input signal modulation format does not have any substantial influence on the S–N NL distortions, when FF DBP is applied.

Figure 4 shows the variation of the SNR as a function of transmission distance and a fixed launched power in a five-channel Nyquist-spaced system using FF DBP. It can be observed that neglecting second-order S–N interactions leads to analytical prediction whose inaccuracy grows with both transmission distance and launched power. On the other hand, the “gap” between the predictions of optimum launched power given by the analytical models with and without accounting for the second-order interactions remains approximately constant with transmission distance. Such a gap can be quantified in about 1.25 dB.

This effect persists in systems with wider bandwidths. Figure 5 reflects the SNR at FF DBP versus the number of transmitted channels at optimum launched power for fixed 2000 km transmission distance. A perfect agreement between simulations and second-order S–N interaction model was demonstrated for a bandwidth up to 31 channels ($\sim 1\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{THz}$) and across different modulation formats. Because numerical simulations become computationally intractable, the analytical model is highly beneficial for predicting the SNR performance of systems with wider transmission bandwidths (e.g., C-band).

Figure 6 indicates the growth rate of second-order S–N interactions relative to the corresponding contribution of first-order S–N interactions as a function of launched power per channel for different transmission distances calculated for C-band ($\sim 4.8\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{THz}$) system using the analytical model. The two horizontal dashed lines indicate the half- and equal contributions of second-order S–N interactions relative to first-order effects. It was found that, as both distance and launched power are increased, second-order effects become more significant. In particular, the contribution of second-order effects grows faster with the launched power for longer transmission distances.

In summary, the impact of second-order S–N interactions in the presence of modulation-format-dependent NL distortion of a Nyquist-spaced WDM system with nonlinearity compensation has been analyzed, where the accurate assessment of this effect in the NL regime was presented. Analytical and numerical studies have been carried out for both single- and multi-channel WDM transmission systems. It was shown that, in the case of FF DBP, the consideration of second-order S–N interactions becomes substantial at both optimum launched power and beyond. The proposed extended analytical model allowed us to accurately predict the performance of multi-channel transmission systems with full-field nonlinearity compensation. Additionally, the power thresholds, corresponding to half or equal contribution of second-order S–N interactions relative to the first-order S–N effects, have been quantified in the C-band transmission system for different transmission distances, which suggest a range of launched powers when second-order S–N interactions become important.

## Funding

Engineering and Physical Sciences Research Council (EPSRC) (UNLOC, EP/J017582/1); University College London (UCL) (GRS Scholarship).

## REFERENCES

**1. **R.-J. Essiambre, G. Kramer, P. J. Winzer, G. J. Foschini, and B. Goebel, J. Lightwave Technol. **28**, 662 (2010). [CrossRef]

**2. **P. Bayvel, R. Maher, T. Xu, G. Liga, N. A. Shevchenko, D. Lavery, A. Alvarado, and R. I. Killey, Phil. Trans. R. Soc. A **374**, 20140440 (2016). [CrossRef]

**3. **S. Abbott, A. Pilipetskii, D. Foursa, and H. Li, *SubOptic* (2016).

**4. **E. Agrell, M. Karlsson, A. Chraplyvy, D. J. Richardson, P. M. Krummrich, P. Winzer, K. Roberts, J. K. Fischer, S. J. Savory, B. J. Eggleton, M. Secondini, F. R. Kschischang, A. Lord, J. Prat, I. Tomkos, J. E. Bowers, S. Srinivasan, M. Brandt-Pearce, and N. Gisin, J. Opt. **18**, 063002 (2016). [CrossRef]

**5. **M. A. Al-Khateeb, M. McCarthy, C. Sánchez, and A. Ellis, Opt. Lett. **41**, 1849 (2016). [CrossRef]

**6. **D. Rafique and A. D. Ellis, Opt. Express **19**, 3449 (2011). [CrossRef]

**7. **M. Secondini, E. Forestieri, and G. Prati, J. Lightwave Technol. **31**, 3839 (2013). [CrossRef]

**8. **T. Xu, N. A. Shevchenko, D. Lavery, D. Semrau, G. Liga, A. Alvarado, R. I. Killey, and P. Bayvel, Opt. Express **25**, 3311 (2017). [CrossRef]

**9. **A. Vannucci, P. Serena, and A. Bononi, J. Lightwave Technol. **20**, 1102 (2002). [CrossRef]

**10. **P. Poggiolini, J. Lightwave Technol. **30**, 3857 (2012). [CrossRef]

**11. **G. P. Agrawal, *Nonlinear Fiber Optics* (Academic, 2007).

**12. **A. Carena, G. Bosco, V. Curri, Y. Jiang, P. Poggiolini, and F. Forghieri, Opt. Express **22**, 16335 (2014). [CrossRef]

**13. **R. Dar, M. Feder, A. Mecozzi, and M. Shtaif, Opt. Express **21**, 25685 (2013). [CrossRef]

**14. **R. Dar, M. Feder, A. Mecozzi, and M. Shtaif, Opt. Express **22**, 14199 (2014). [CrossRef]

**15. **T. Tanimura, M. Nölle, J. K. Fischer, and C. Schubert, Opt. Express **20**, 28779 (2012). [CrossRef]

**16. **T. Fehenberger, A. Alvarado, G. Böcherer, and N. Hanik, J. Lightwave Technol. **34**, 5063 (2016). [CrossRef]

**17. **R. Dar and P. J. Winzer, J. Lightwave Technol. **35**, 903 (2017).

**18. **G. Bosco, A. Carena, V. Curri, R. Gaudino, P. Poggiolini, and S. Benedetto, IEEE Photon. Technol. Lett. **12**, 489 (2000). [CrossRef]

**19. **T. Xu, G. Jacobsen, S. Popov, J. Li, E. Vanin, K. Wang, A. T. Friberg, and Y. Zhang, Opt. Express **18**, 16243 (2010). [CrossRef]

**20. **E. Ip and J. M. Kahn, J. Lightwave Technol. **26**, 3416 (2008). [CrossRef]

**21. **R. Maher, T. Xu, L. Galdino, M. Sato, A. Alvarado, K. Shi, S. J. Savory, B. C. Thomsen, R. I. Killey, and P. Bayvel, Sci. Rep. **5**, 08214 (2015). [CrossRef]