## Abstract

In this paper, we first describe a 9-symbol non-uniform signaling scheme based on Huffman code, in which different symbols are transmitted with different probabilities. By using the Huffman procedure, prefix code is designed to approach the optimal performance. Then, we introduce an algorithm to determine the optimal signal constellation sets for our proposed non-uniform scheme with the criterion of maximizing constellation figure of merit (CFM). The proposed nonuniform polarization multiplexed signaling 9-QAM scheme has the same spectral efficiency as the conventional 8-QAM. Additionally, we propose a specially designed GF(3^{2}) nonbinary quasi-cyclic LDPC code for the coded modulation system based on the 9-QAM non-uniform scheme. Further, we study the efficiency of our proposed non-uniform 9-QAM, combined with nonbinary LDPC coding, and demonstrate by Monte Carlo simulation that the proposed GF(2^{3}) nonbinary LDPC coded 9-QAM scheme outperforms nonbinary LDPC coded uniform 8-QAM by at least 0.8dB.

© 2016 Optical Society of America

## 1. Introduction

In the recent years, the exponential Internet traffic growth projections place enormous transmission rate demand on the underlying information infrastructure at every level, from the long haul submarine transmission to optical metro networks [1]. To facilitate this trend, the new advanced modulation scheme combined with advanced forward error correction code should be proposed. Different approaches to advanced coded modulation have been investigated in order to achieve higher spectral efficiency and deal with degradation caused by nonlinearities. Different constellation design algorithms have been studied in [2–5]. Meanwhile, high dimensional modulation schemes have also been investigated in [6–8]. However, these research activities are focused on the problem of placing ${2}^{n}$ constellation points to a multidimensional space in order to achieve better performance by maximizing the minimum distance of the constellation sets, which can be regarded as the coding gain. Shaping gain, on the other hand, is a way to improve the energy efficiency by transmitting different symbols with different probabilities. In the conventional data transmission schemes, the probability of each point in a given constellation is transmitted equally likely and the constellation sets sizes are set to${2}^{n}$. However, this scheme does not take into account the different energy cost of various constellation points. The idea of choosing constellation points with a non-uniform probability distribution is explored in previous studies [9–11], but few in optical communication context. Such non-uniform signaling will reduce the entropy as well as the transmitter output. However, if the points with low energy are transmitted more often than the others with large energy, savings of energy may (more than) compensate for the loss in bit rate [9]. Moreover, we can also transmit a larger constellation sets to match the bit rate of the conventional case; for example, our proposed 9-QAM has the same data rate as 8-QAM in this paper. Moreover, the non-uniform signaling scheme may be more suitable for optical communication because the transmitted points with large probabilities, which have small energies, suffer less fiber nonlinear effects.

In this paper, we propose a non-uniform signaling scheme for 9-ary signal constellation sets employing the Huffman code. The Huffman code tree dictates the structure of the corresponding signal constellation. Further, we introduce an overall search algorithm to determine this 9-point signal constellation by optimizing the constellation figure of merit (CFM). Such designed nonuniform signal constellation sets has been called here 9-QAM, and it has the same spectral efficiency as the conventional 8-QAM. One of the main problems in probability shaping of the modulation format is to properly combine the nonuniform modulation scheme with forward error correction (FEC) code. The overall code rate will be the product of source code rate and channel code rate. In order to solve this problem, we propose a GF(3^{2}) nonbinary quasi-cyclic nonbinary LDPC code for this 9-QAM non-uniform signaling scheme. For fair comparison, a GF(2^{3}) nonbinary LDPC code will be used in the 8-QAM coded modulation scheme, which serves as the reference case. The Monte Carlo simulations indicate that the proposed nonbinary GF(9) LDPC-coded 9-QAM non-uniform signaling scheme with Huffman code based constellation shaping outperforms the corresponding nonbinary LDPC-coded 8-QAM by at least 0.8 dB.

The paper is organized as follows. In Section 2, we introduce the Huffman coding based procedure for non-uniform signaling. In Section 3, we present the proposed optimal signal constellation design algorithm to determine the constellation sets with size of 9. In Section 4, we introduce the GF(3^{2}) nonbinary quasi-cyclic LDPC code. The corresponding coded modulation scheme employing the new signal constellation sets obtained by algorithm, introduced in Section 3, is described in Section 5. The results of the Monte Carlo simulation are summarized in Section 6. Concluding remarks can be found in Section 7.

## 2. Constellation shaping with Huffman code

The method of achieving non-uniform signaling schemes for the transmission of binary data is implemented by using the Huffman code [12]. In this paper, we mainly focus on the two-dimensional (2D) 9-point constellation sets because it has same spectral efficiency as 2D 8-QAM. The non-uniform signaling process is illustrated in Fig. 1. In this scheme, each symbol can carry two, three or four bits per symbol and symbols are selected for transmission with different probabilities. It is obvious that the center point carrying 00 bits has the largest probability of 0.25 and the symbols carrying 3 or 4 bits are transmitted with probabilities 0.125 and 0.0625, respectively. We can also calculate that the spectral efficiency of the proposed 9-point constellation sets as follows

where*i*is the number of bits represented by symbols. ${P}_{i}$ and ${N}_{i}$ denote the probability and number of the symbols representing

*i*bits. The spectral efficiency $S$ of 9-QAM is the same as that as star 8-QAM,

*S*= 3 bits/symbol. The mapping rule is determined by employing the Huffman procedure shown as a tree diagram.

There are two rules in this non-uniform signaling scheme [13]. The first one is that the symbols carrying the same number of bits should be arranged at the same layer; that is to say, the symbols representing the same number of bits must have the same symbol energy. Another rule for the non-uniform scheme is that the symbols with higher energies should be transmitted with smaller probabilities. With these two rules in mind, we know that the structure of constellation set can be determined by the Huffman code. The tree structure dictates the number of layers in the constellations and number of points on each layer. In Section 3, we will introduce an overall search algorithm to find the optimal constellation sets design rules just described. In Fig. 1, we have already provided the resulting constellation obtained by using the design algorithm from Section 3, in which the probability shaping is taken into account as well. In incoming section, we will demonstrate that this 3×3 square constellation set (9-QAM) is optimal for proposed non-uniform signaling scheme.

## 3. Constellation design for non-uniform signaling scheme

As already discussed in Section 2, the constellation structure of the non-uniform constellation sets are dictated by the structure of the binary tree of the Huffman code. In order to determine the signal constellation that is suitable for non-uniform signaling scheme, an overall search algorithm maximizing the constellation figure of merit (CFM) is used in signal constellation design. In any data transmission scheme, the goal is to transmit at a higher bit rate, with higher reliability and as low transmitter power as possible [9]. The CFM is defined as

where ${d}_{\mathrm{min}}^{2}$ denotes the minimum distance of the signal constellation, while denotes the average energy of the signal constellation set.The proposed algorithm for non-uniform signal constellation design is an overall search method to determine the radius of each layer and the relative angles for each layer. As already mentioned above, the constellation structure has been dictated by the structure of the Huffman code tree, so that the input of the algorithm is the number of points on each layer and number of layers, denoted by ${N}_{i}$ and$S$, respectively. The algorithm will search over all possible${R}_{i}$’s, where *R _{i}* denotes the radius of the

*i*-th layer, with a predefined step size $\Delta r$. Additionally, the algorithm also searches over the relative angles for each layer with another step size $\Delta \theta $. After increasing the step sizes for radius and angle, the CFM will be calculated. The algorithm will keep searching over a given range of radii and angles and the combination with the maximum CFM will represent the output of the algorithm. With the radius of each layer and the relatively angle for each layer being determined, the signal constellation set can be found with the help of ${N}_{i}$ and $S$. The constellation optimization procedure is illustrated in Fig. 2.

In this algorithm, the range of the radius can be set in the optimization procedure. As shown in Fig. 3, if the range of radius is set to a very large value, we can find many scaled versions of the same constellation having the maximum CFM. The corresponding constellation design results are all 9-QAM but with different scaling values. In order to reduce the complexity of the algorithm, we can set the range of radius to only cover one optimization cycle, which is shown in Fig. 3. The relative angle for each layer can be set as [$0,\frac{\pi}{4}$] and the step size can be chosen as small as possible but with reasonable run time.

As introduced in previous section, the 9 symbols in Huffman code can be represented with binary sequences: 00, 010, 110, 011, 100, 1110, 1111, 1010 and 1011. With this requirement, we find that the structure of the constellation should be ${N}_{0}=1,{N}_{1}=4,{N}_{2}=4$, and $S=3$. We first put one point in the center because the first layer only has one symbol, then we place 4 points on each of other two layers. The next step would be to search over all possible radii of each layer with a step size $\Delta r$ and also over the relative angles for the second and third layers, which is shown in Fig. 2. After searching over a given range is completed, we find that the $3\times 3$ square 9-QAM has the maximum CFM value. So the 9-QAM is the optimal constellation set, in CFM sense, for 9 symbols non-uniform signaling. Meanwhile, the mapping method for star 8-QAM is shown in Fig. 4.

## 4. Nonbinary quasi-cyclic LDPC coding for 9-QAM

To perfectly match the constellation size with coding, we use GF(3^{2}) nonbinary LDPC coding. GF(3^{2}) is not a popular choice of traditional communication system, since it is not binary based extension field, but it is the best choice of 9-ary constellation with non-uniform distribution, which is hard to implement with binary based coding.

The GF(3^{2}) LDPC code is constructed by randomly replacing nonzero elements in binary LDPC code with nonzero elements from GF(3^{2}), which is illustrated in Fig. 5. We employ regular girth-10 LDPC code with length 16935 and rate 0.8 as the mother code [14]. Sum-product algorithm is then used in the decoding process. The variable node processor (sum operator) update rule is the same as in typical nonbinary LDPC decoder, while the check node processor (product operator) update rule is based on BCJR algorithm operating on corresponding finite state machine.

In the BCJR algorithm-based check-node processor, shown in the Fig. 6, the number of states is equal to the finite field size for single parity check (SPC) code and the number of stages is determined by the check node degrees. For additional details about the BCJR algorithm, please refer to [15].

## 5. Advanced coded modulation scheme for non-uniform 9-QAM signaling

The proposed polarization-multiplexed GF(3^{2}) nonbinary LDPC-coded scheme based on 9-QAM non-uniform signaling is depicted in Fig. 7. To facilitate the explanations, only signal polarization state is shown. In this paper, as we introduced before, we mainly focus on 2D-9-QAM constellation combined with GF(3^{2}) nonbinary QC-LDPC at rate $r=0.8$, codeword length $n=16935$, with the number of parity bits $n-k=3385$. In this coded modulation scheme, the key ingredient of our scheme is the structure of block-interleaver for non-uniform signaling. The proposed interleaver scheme suitable for non-uniform signaling is shown in Fig. 8. Some of the locations in the proposed interleaver will be left as blank because of different symbols representing different number of bits. The *interleaving process* can be formulated as follows:

- • The binary information bits will be first written into the last two rows of the interleaver. The interleaver size for non-uniform signaling is $n\times {L}_{M}$, where ${L}_{M}$ is the maximum source codeword length of the corresponding Huffman code. In our case, ${L}_{M}=4$.
- • Read the bits in column-wise fashion from bottom to the top for the last two rows. If these two bits are not 00, then continue writing the information bits to the second row; otherwise leave the position blank, which is denoted as the red B in Fig. 8.
- • Read the bits in column-wise fashion from bottom to the top for the last three rows. If the three bits are not in the set $\{010,100,110,011\}$, then continue writing the information bits to the first row; otherwise, leave the position at blank.
- • Read the interleaver column by column and map it to 9-QAM with the mapping rule illustrated in Fig. 1. These symbols are then used as outputs of the interleaver.

After the interleaver, the binary bits are mapped to non-uniform symbols, which will be then passed to the nonbinary LDPC encoder. Because the probability of the center point in 9-QAM is highest, there will be many sequences with zero energy, which might cause problems in DSP process at the receiver side. In order to solve this problem, we add a power distributer block, which will rearrange the order of symbols in the codeword. The power distributer block will avoid the presentence of long zero energy sequences. After that, the signal will be fed into the MZM (I/Q) modulator to perform the electrical-to-optical conversion and signal will be transmitted over fiber-optics communication system of interest.

On receiver side, the homodyne coherent detection is used and followed with the compensation of linear and nonlinear impairments and carrier phase estimator (CPE). The former process is implemented by digital back-propagation scheme combined sliding-window MAP equalizer. Only a small number of coefficients are used in digital back-propagation scheme [15], just to reduce the channel memory so that the complexity of the MAP equalizer that follows is not too high. With the help of sliding-MAP equalizer and digital back propagation, the corresponding distribution is Gaussian-like, similar to back-to-back scenario. The sliding-MAP equalizer provides soft symbol LLRs, which are sent to the nonbinary LDPC decoder. Once decoding is completed, we can get information bits after deinterleaving. Note that the simulation scheme of 8-QAM and 9-QAM is the same except from the signal constellation, interleaver structure, and nonbinary LDPC coding scheme. So the 9-QAM scheme has no additional complexity compared to 8-QAM.

## 6. Simulation results

The results of Monte Carlo simulations for the proposed non-uniform 9-QAM coded modulation scheme are summarized in Figs. 7–9. The symbol rate is set to *R _{s}* = 31.25 GS/s and GF(3

^{2}) (16935, 3385) nonbinary LDPC code is used as introduced before.

Note that our proposed scheme is compared against star 8-QAM coded with GF(2^{3}) nonbinary LDPC code with same codeword length. Meanwhile, the decoding method and system complexities are comparable. The non-uniform 9-QAM and uniform star 8-QAM, as introduced in Section 2, have the same spectral efficiencies. In Fig. 9, we provide the symbol error rate (SER) vs. signal-to-noise ratio plot. It is evident that our proposed non-uniform signaling scheme outperforms star 8-QAM by around 0.8 dB. According to [16], even 0.51 dB gain in back-to-back configuration can result in significant improvement in the presence of nonlinearities, when digital back propagation is not used.

In Fig. 10, we show the bit error rate (BER) versus SNR curves for 9-QAM and 8-QAM. Notice also that the 9-QAM also performs better that 8-QAM even in the uncoded case. In our proposed scheme, the 8-QAM can actually benefit more from the digital back propagation. The 9-QAM scheme suffers less fiber nonlinearities in this system because the point with zero energy (center point) is transmitted with highest probability and this point is insensitive to nonlinearities. The similar situation occurs at each layer at 9-QAM because the points with smaller energy, which turns out to be less sensitive to nonlinearities, are transmitted with higher probabilities. So the 9-QAM can achieve better nonlinearity tolerance comparing to 8-QAM if the digital back propagation is not used.

## 7. Concluding remarks

In conclusion, we have proposed a polarization multiplexed non-uniform 9-QAM coded modulation scheme based on GF(3^{2}) nonbinary LDPC codes. The non-uniform structure is achieved by using the prefix code. By using the Huffman procedure, prefix code is designed to approach the optimal performance. We first use the Huffman code procedure to decide the structure of the constellation with size 9. Then we use an overall search algorithm to determine 9 points in signal constellation that maximizes the CFM. The result of this algorithm is 3 × 3 9-QAM signal constellation set. We have also described the implementation of the non-uniform scheme and the design of GF(3^{2}) nonbinary LDPC codes. The most important part in the 9-QAM coded modulation scheme is the interleaver configuration proposed. Finally, the Monte Carlo simulations indicate that the proposed non-uniform 9-QAM scheme outperforms star 8-QAM by 0.8 dB for the same spectral efficiency.

## References and links

**1. **P. Winzer, “Beyond 100G Ethernet,” IEEE Commun. Mag. **48**(7), 26–30 (2010). [CrossRef]

**2. **M. Seimetz, *High-Order Modulation for Optical Fiber Transmission* (Springer, 2009).

**3. **T. Liu and I. B. Djordjevic, “On the optimum signal constellation design for high-speed optical transport networks,” Opt. Express **20**(18), 20396–20406 (2012). [CrossRef] [PubMed]

**4. **N. J. A. Sloane, R. H. Hardin, T. D. S. Duff, and J. H. Conway, “Minimal-energy cluster of hard sphere,” Discrete Comput. Geom. **14**(3), 237–259 (1995). [CrossRef]

**5. **C. Hager, A. Graell i Amat, A. Alvarado, and E. Agrell, “Design of APSK constellations for coherent optical channels with nonlinear phase noise,” IEEE Trans. Commun. **61**(8), 3362–3373 (2013). [CrossRef]

**6. **J. E. Porath and T. Aulin, “Design of multidimensional signal constellations,” IEEE Proc. Commun. **150**(5), 317–323 (2003). [CrossRef]

**7. **M. Karlsson and E. Agrell, “Four-dimensional optimized constellations for coherent optical transmission systems,” in *European Conference Optical Communication 2010* (Optical Society of America, 2010), paper. We.8.C.3. [CrossRef]

**8. **G. Takahara, F. Alajaji, N. C. Beaulieu, and H. Kuai, “Constellation Mappings for Two-Dimensional Signaling of Nonuniform Sources,” IEEE Trans. Commun. **51**(3), 400–408 (2003). [CrossRef]

**9. **F. Kschischang and S. Pasupathy, “Optimal non-uniform signaling for gaussian channels,” IEEE Trans. Inf. Theory **39**(3), 913–929 (1993). [CrossRef]

**10. **F. Buchali, G. Bocherer, W. Idler, L. Schmalen, P. Schulte, and F. Steiner, “Experimental demonstration of capacity increase and rate-adaptation by probabilistically shaped 64-QAM,” in *European Conference Optical Communication 2015* (Optical Society of America, 2015), paper PDP.3.4. [CrossRef]

**11. **M. Yanko, D. Zibar, K. J. Larsen, L. P. B. Christensen, and S. Forchammer, “Constellation shaping for fiber-optic channels with QAM and high spectral efficiency,” IEEE Photon. Technol. Lett. **26**(23), 2407–2410 (2014). [CrossRef]

**12. **T. Liu and I. B. Djordjevic, “LDPC-coded BICM-ID based nonuniform signaling for ultra-high-speed optical transport,” in *Optical Fiber Communication 2016* (Optical Society of America, 2016), paper M3A.3.

**13. **A. K. Khandani and P. Kabal, “Shaping multidimensional signal spaces. Part I: optimal shaping shell mapping,” IEEE Trans. Inf. Theory **39**(6), 1799–1808 (1993). [CrossRef]

**14. **M. Arabaci, I. B. Djordjevic, R. Saunders, and R. M. Marcoccia, “Polarization-multiplexed rate-adaptive non-binary-quasi-cyclic-LDPC-coded multilevel modulation with coherent detection for optical transport networks” in *International Conference on Transparent Optical Networks* (2009), paper. Tu.B2.2.

**15. **I. B. Djordjevic, L. L. Minkov, L. Xu, and T. Wang, “Suppression of fiber nonlinearities and PMD in coded-modulation schemes with coherent detection by using turbo equalization,” J. Opt. Commun. **1**(6), 555–564 (2009). [CrossRef]

**16. **T. H. Lotz, X. Liu, S. Chandrasekhar, P. J. Winzer, H. Haunstein, S. Randel, S. Corteselli, B. Zhu, and D. W. Peckham, “Coded PDM-OFDM transmission with shaped 256-iterative-polar-modulation achieving 11.15-b/s/Hz intrachannel spectral efficiency and 800-km reach,” J. Lightwave Technol. **31**(4), 538–545 (2013). [CrossRef]