1. Introduction

Optical image encryption techniques based on double random phase encoding (DRPE) play an important role in the field of information security due to its inherent advantage of high-speed parallel processing of complex multi-dimensional data [1–8]. To improve the performance of the encryption system, some researchers have proposed fully-phase image encryption method to encode pure-phase images [9–13], and the computer simulation results demonstrate that fully phase-based encryption could perform better than traditional DRPE-based encryption under the noise conditions [9,10]. However, the DRPE-based techniques are vulnerable to various attacks because of the linearity and symmetry of these encryption systems. Phase-truncated Fourier transform (PTFT) based encryption systems have been also proposed to further enhance security [14]. Unfortunately, some of the DRPE-based cryptosystems, especially the fully phase encryption methods are experimentally difficult to verify because they usually require a complex interferometer configuration with spatial light modulators (SLMs) for displaying the phase-encoded image and random phase masks. Besides, the decoding process of those DRPE-based cryptosystems usually requires very precise alignment [2,3].

Recently, deep learning technique has been employed to analyze the security of optical image encryption systems. Some conventional encryption systems based on DRPE [15], computer-generated hologram [16], interference-based encryption [17], joint transform correlation encryption [18] and ghost imaging [19] were found vulnerable to neural networks trained by ciphertext-plaintext pairs [20–29]. What should be pointed out is that different methods of attack not only test the security of cryptosystems but also provide new ideas for designing new optical systems for image encryption [30]. In addition, some experimental setups for traditional optical image encryption are cumbersome and sensitive to vibrations. For example, a complex holographic scheme was necessary for recording complex-valued encrypted data in the DRPE system and precise alignment of the phase code was required for decryption. Thus, it is worth considering how to take advantage of the new technique for practical and secure image encryption [31,32].

In this paper, we propose an experimental optical image encryption based on deep learning. In the optical experiment, the pure phase objects are placed behind a random binary mask (RBM) made of polyethylene terephthalate (PET), and the speckle pattern, i.e., the cyphertext, is captured by a Charge-Coupled Device (CCD) after the object beam passing through the RBM. In the decryption processing, a modified end-to-end convolutional neural network (CNN) model based on U-type network is used to recover the original image from the cyphertext. The CNN model trained by using a series of object and encrypted images pairs allows for a high-quality recovery of object images from cyphertexts. In the proposed cryptosystem, the keys used for encryption are different from the keys used for decryption and the decryption process is not the inverse of encryption process. Different from most of previously proposed traditional methods, deep learning technique is adopted to obtain a pre-trained model for achieving a high-quality decryption and a high level of security.

The rest of this paper is organized as follows. The principle of the proposed method is described in Section 2. The experiment results and discussions are given in Section 3. Finally, concluding remarks are given in Section 4.

2. Principle and method

The optical experimental setup we built up for fully-phase encryption and decryption is shown in Fig. 1. The spatial light modulator (DHC, GCI-770402) is illuminated by a collimated and expanded beam from a pumped semiconductor laser (MGL-III-532, wavelength: 532 nm), the sampling interval and resolution of the SLM are $6.3\mu m \times 6.3\mu m$ and $1280 \times 720$ respectively. A pure-phase object image $\textrm{exp} [{j\mathrm{\pi }o({x,y} )} ]$ is loaded to the SLM for modulating the incident beam. The light beam with the information of pure-phase object then propagates through a PET film after propagation of a distance d₁ in free space. A CCD detector is applied on the other side with the distance d₂ to capture the speckle patterns diffracted by the PET film.

 

Fig. 1. Optical experimental arrangement for fully-phase encryption and decryption.

Download Full Size | PDF

In the experiment, the images with the size of $28 \times 28$ pixels in the MNIST and Fashion-mnist databases [33,34] are applied as pure-phase object images since they are readily available and widely used in the research of various deep learning problems. Before being fed into SLM, those images are resized into $512 \times 512$ and zero-padded. At the receiver end, a CCD camera is placed and digitally record the diffraction patterns. We use the central $512 \times 512$ portion of the data captured by CCD.

Information recovery from noisy optical transmission is considered challenging because the collected data are usually ambiguous and strongly ill-conditioned. With the development of computer technology and algorithm, deep neural networks (DNNs) have been shown to be successful in various inverse problems in imaging in the last few years overcoming the problems of stagnation and slow convergence of traditional phase retrieval algorithms [35,36]. In this work, an end-to-end CNN based on U-net is used to recover the primary pure-phase objects from the cyphertexts. The overall architecture is based on the encoder-decoder U-net framework, but with some proper modifications. The modified network architecture used in this work is depicted in Fig. 2(a). We replace the convolutional layers with the dense blocks to strengthen feature propagation. Besides, an additional convolution layer at the output is used to further increase the depth of the network and nonlinearity. In the training process, the cyphertexts and original images are respectively set as the inputs and outputs of the proposed model. The input layer generates feature maps with the size of $64 \times 64 \times 24$ from the input speckle pattern by using convolution operation, of which the kernel size is $3 \times 3$ and the number of filters is 24. Each convolved image is successively decimated in the encoder path, consisting of six dense blocks connected by transition blocks. Each dense block contains four layers and each layer consists of batch normalization (BN), rectified linear unit (ReLu), and a convolutional layer (Conv) with filter size of $3 \times 3$, which is shown in Fig. 2(b). Let $[{{F_{i,0}},{F_{i,1}}, \cdots ,{F_{i,c - 1}}} ]$ denote the channel combination of the feature maps from layer 0 to the output of layer c, i.e. the concatenation into one tensor. The cth layer of dense block can be described as

 

Fig. 2. The structure of CNN. (a) The architecture of the CNN. (b) The 4-layer dense block.

Download Full Size | PDF

The transition block contains BN, ReLu, a convolutional layer with filter size of $3 \times 3$, followed by an average pooling layer with a kernel size of $2 \times 2$, stride size of 2. These successively-decimated signal then goes through decoder path and additional “BN + convolutional” blocks. In the decoder path, there are six dense blocks which are connected by upsampling convolutional layers, increasing the dimension of the input by a factor of 2. Finally, a convolutional layer followed by the last layer produces the network outputs.

Let the input of $i\textrm{th}$ layer in the proposed CNN structure be ${x_{i - 1}}$. The resultant output of this layer can be expressed as

The adaptive moment estimation (Adam) optimization algorithm [37] is applied in the training process. A learning rate constant of 0.001 and fifty epochs are used. The neural network is installed with the TensorFlow framework, and the training process is implemented on a NVIDIA GTX1650 GPU.

3. Results and discussion

Optical experiments were carried out to verify the feasibility of the proposed method for fully-phase encryption and decryption. The propagation distances are ${d_1} = 20\textrm{ cm}$ and ${d_2} = 10\textrm{ cm}$. 2500 pure-phase objects are selected randomly from the MINST handwritten digit database, which are resized into $512 \times 512$ and then zero-padded. We sequentially loaded pure-phase objects into the SLM and captured their corresponding speckle patterns with the CCD detector. The central $512 \times 512$ portion of the captured data is further reduced by a factor of 4. The input and output images of the neutral network are both resized into $128 \times 128$ pixels. In the training process, 2000 object-speckle pattern pairs are used for training and another 500 object–speckle pattern pairs are used as the test data. The number of training epochs is set as 100. The execution time of CNN model training is about 6.0 hours.

To verify the effectiveness of the designed CNN model, test sets are fed into each of the pre-trained model. The results of the prediction are shown in Fig. 3. The diffraction patterns without using the RBM of PET are shown in Fig. 3(b), and the encrypted images resultant from PET film and the reconstructed images are respectively shown in Fig. 3(c) and Fig. 3(d). It is obvious that high quality images can be reconstructed by using the proposed CNN model. We calculated the structural similarity index (SSIM) and peak signal to noise ratio (PSNR) for all test images. The SSIM and PSNR evaluate the structural similarity and average difference between the ground truth and predicted image, respectively. The average values of SSIM and PSNR of the Mnist sets we used are 0.97 and 25.96, and those of the Fashion-mnist are 0.91 and 22.87, respectively.

 

Fig. 3. Input primary images and decrypted results. (a) Plaintext images, (b) the diffraction results without using RBM, (c) the encrypted results, (d) the reconstructed images by using the proposed CNN model, (e) decryption results with wrong CNN model. The decryption images obtained (f) when the eavesdropping rate of w_i is 99.7%, (g) when the eavesdropping rate of b_i is 99.75%.

Download Full Size | PDF

The trained learning model and its parameters, e.g., the size of kernel, the number of kernels, the activation function, and the matrix ${w_i}$ and vector ${b_i}$, can be used as security keys in the proposed scheme. For simplicity and illustration purpose, we only test the performance of ${w_i}$ and ${b_i}$ in security testing. When all of the keys except ${w_i}$ are correct, the decrypted images cannot be identified with the naked eye. Figure 3(f) shows the decrypted images obtained when the eavesdropping rate of ${w_i}$ is lower than 99.80%. When the eavesdropping rate of ${b_i}$ is below 99.85%, the resultant decrypted images are shown in Fig. 3(g). These results imply that the security of the proposed deep learning-based optical cryptosystem can be fully guaranteed.

We further consider the effect of the axial displacement of CCD on the quality of reconstructed image. In general, the axial displacement of the CCD decreases the correlation of the scatter intensity measurements. In traditional optical encryption schemes, deviation in positions of the optical components in encryption may lead to a seriously degraded decryption. For example, in the Fourier domain DRPE system, the encrypted image is recorded in the back focal plane of the second lens, i.e., the imaging plane of the system. A position deviation in imaging at the encryption stage will undoubtedly degrade the quality of decrypted images. Fortunately, DNN can be applied to this problem since it was proved to be able to make high-quality object predictions in various inverse problems in imaging [35,36]. Our experimental results show that high-quality decrypted images can still be obtained by using the proposed CNN model even when the CCD is deviated from the set position by 30 mm.

The experimental principle of measurement is illustrated in Fig. 4(a), where the initial position of the CCD is ${Z_4}$. The CCD successively moved along the Z axis to the left (right) by 1 cm, 2 cm and 3 cm, i.e., moved to the positions of ${Z_3}({\; {Z_5}} )$, ${Z_2}({{Z_6}} )$ and ${Z_1}({{Z_7}} )$. When these measurements are input to the pre-trained neural network model, the outputs are given by Figs. 4(b) and 4(c), which demonstrates remarkable stability and visual quality. The error bars for the average PSNR and SSIM of Mnist and Fashion-mnist data are shown in Fig. 5. As we can see, the mean SSIM and PSNR of the reconstructed images with respect to the two testing datasets are greater than 0.65 and 15.5 respectively, which implies that the proposed method can offer a high position tolerance for the CCD detector. As shown in Fig. 1, the scheme is simple and flexible control. The toleration about the position of the detector further ensures that the system is able to provide an acceptable quality of decrypted images. As mentioned above, precise alignment is required in many traditional systems and the optically decrypted image usually has a poor quality. The combination of the deep leaning technique and random mask encoding provide an alternative for solving the problems of low position tolerance and poor image quality in optical image encryption.

 

Fig. 4. Experimental investigation on the position tolerance of the CCD camera. (a) Schematic diagram of deviation in the CCD position, the testing results at different locations with respect to (b) Mnist data and (c) Fashion-mnist data.

Download Full Size | PDF

 

Fig. 5. Evaluation of the image quality. The mean values and the standard deviations of the (a) SSIM, (b) PSNR of 50 reconstructed images.

Download Full Size | PDF

Sometimes, data may be lost during transmission. We investigate the robustness of the proposed method against data loss by randomly setting the pixel values of the encrypted image to zero. Specifically, some pixels in an encrypted image are randomly replaced with zero-values. Figure 6 shows the images decrypted from encrypted images with different percentages of data loss using pre-trained model. Figures 6(a)–6(e) are the reconstructed images when the encrypted images with percent of data loss of 10%, 20%, 30%, 40% and 50%, respectively. The mean values of SSIM and PSNR between the ground truth and reconstructed image are calculated, respectively, as shown in Fig. 7. It should be noted that all the reconstructed images with high quality are obtained without retraining the CNN model.

 

Fig. 6. Robustness test against data loss. (a)-(e) The images decrypted from the cyphertext with percent of data loss of 10%, 20%, 30%, 40% and 50%, respectively.

Download Full Size | PDF

 

Fig. 7. Evaluation of the quality of the decrypted images using the encrypted images with data loss. The data points and the error bars in (a) and (b) represent the mean values and the standard deviations of the SSIM/PSNR of 50 reconstructed images.

Download Full Size | PDF

4. Conclusion

In conclusion, we have proposed an experimental approach towards a flexible realization of optical encryption by using a PET RBM and deep learning technique. The modified CNN model is built up according to the densely CNN architecture and then trained by a series of primary and encrypted image pairs. The plaintext can be efficiently decrypted from the cyphertext with high quality. The position tolerance and the robustness against data loss have been investigated in detail. The proposed optical scheme can be directly used as a RBM encrypted image processor. The presented considerations might also be helpful for the application of deep learning in the imaging of the pure phase objects.