Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Speckle autocorrelation separation for multi-target scattering imaging

Open Access Open Access

Abstract

Imaging through scattering media remains a big challenge in optics while the single-shot non-invasive speckle autocorrelation technique (SAT) is well-known as a promising way to handle it. However, it usually cannot recover a large-scale target or multiple isolated small ones due to the limited effective range of the optical memory effect (OME). In this paper, we propose a multi-target scattering imaging scheme by combining the traditional SA algorithm with a Deep Learning (DL) strategy. The basic idea is to extract each autocorrelation component of every target from the autocorrelation result of a mixed speckle using a suitable DL method. Once we get all the expected autocorrelation components, a typical phase retrieval algorithm (PRA) could be applied to reveal the shapes of all those corresponding small targets. In our experimental demonstration, up to five isolated targets are successfully recovered.

© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

1. Introduction

The techniques of imaging through scattering media have important applications in biomedical imaging, astronomical imaging, remote sensing, etc. [16]. As we know, most of the photons will deviate from their original paths resulting in a noisy-like speckle pattern due to the interaction between the light beam and the micro-particles buried in a scattering medium, e.g., fog, biological tissue, and ground glass [7]. To reconstruct a hidden target, several kinds of techniques have been invented and developed in the past decades. The most straightforward way should be the holographic- or conjugation-based one, which is separately and independently proposed by Goodman and Leith in the 1960s [8, 9]. Another intuitive way is the transmission-matrix-based one, which is assumed to be capable of making a full description of the relationship between the input and output optical wavefronts [1012]. These early methods are easy to understand but relatively difficult to implement because the involved operations are invasive and usually very complex. In 1988, Freund et al. found a crucial secret of the scattering system, that’s the so-called optical memory effect (OME) [13]. This property reveals that an imaging system with a scattering layer, to some certain extent, could be regarded as a linear shift-invariant (LSI) system. Therefore, an easy-to-implement deconvolution-based scheme became available, though it’s also an invasive way [14, 15]. It also provided the basis for the iterative time-consuming wavefront-shaping-based techniques [16, 17], in which the wavefront shaping process could be also treated as a deconvolution operation. Recently, a milestone work was done by Bertolotti et al. in 2012 and Katz et al. in 2014, unlike the previous ways, they proposed a “non-invasive” technique to see through a scattering medium based on speckle autocorrelation (SA) algorithm [18, 19]. Noted that the SA-based methods also relied on the fact that the target should locate within the OME’s effective range. Beyond this fact, another critical assumption was made, that the autocorrelation of a captured noisy-like speckle pattern should be almost the same as that of the corresponding hidden target itself. This could be mathematically written as (ignore the coordinates for simplicity):

$$I \otimes I = ({O \ast S} )\otimes ({O \ast S} )= ({O \otimes O} )\ast ({S \otimes S} )\approx O \otimes O$$
where the symbols “*” and “${\otimes} $” represent the convolution and correlation operators, respectively, I and O denote the captured speckle and the target hidden behind a scattering layer, S is the point spread function (PSF) of the scattering system and its autocorrelation is assumed to be a delta function. According to the Wiener-Khinchin theorem, a signal’s power spectrum is the Fourier transform of its autocorrelation, therefore, the power spectrum of the target can be obtained from the autocorrelation of the speckle, see Eq. (1). Then, a traditional phase retrieval algorithm (PRA) could be applied to reconstruct the target [20]. During the phase retrieval process, the power spectrum of the target is employed as the constrain in the frequency domain while a priori information that the target is real (nonnegative) can be used as the constraint in the object domain. In this way, the iteration is performed back and forth in the object and frequency domains.

However, due to the limited effective range of OME, the above original method would fail when it faces a large-scale target or multiple small targets located in different OME areas. In this case, the whole scattering system can be regarded as a combination of multiple LSI sub-systems [21], and the autocorrelation of the mixed speckle can be mathematically expressed as:

$$\begin{aligned}{c} I \otimes I & { = }\left( {\sum\limits_{i = 1}^n {\left( {{O_i} * {S_i}} \right)} } \right) \otimes \left( {\sum\limits_{i = 1}^n {\left( {{O_i} * {S_i}} \right)} } \right)\\ & \textrm{ = }\sum\limits_{i = 1}^n {\left[ {\left( {{O_i} \otimes {O_i}} \right) * \left( {{S_i} \otimes {S_i}} \right)} \right]} + \sum\limits_{i,j = 1\atop i \ne j}^n {\left[ {\left( {{O_i} \otimes {O_j}} \right) * \left( {{S_i} \otimes {S_j}} \right)} \right]}\end{aligned}$$
where the cross-correlations (second term on the right side), between every two different PSFs associated with two LSI sub-systems, can be treated as ignorable background noise since the PSFs of different OME ranges are uncorrelated. So, a simplified version of Eq. (2) could be written as:
$$I \otimes I \approx \sum\limits_{i = 1}^n {({{O_i} \otimes {O_i}} )} $$
It follows that the autocorrelation of the speckle only reveals the overlapped information from every single target. Thus, it will result in a failed reconstruction unless the autocorrelations can be individually separated. Figure 1 shows results when only two isolated targets (Letter “S” and Letter “Z”) are involved.

 figure: Fig. 1.

Fig. 1. (a) the ground truth of two targets located at two different OME ranges; (b) the captured speckle and its autocorrelation (c); (d) the reconstructed result (by using the traditional SAT) which shows nothing meaningful

Download Full Size | PDF

.

Therefore, here comes the problem that how to separate the autocorrelation component of each target from the overlapped autocorrelation (OAC). This straight strategy has become a hot issue because it closely relates to one of the drawbacks of the famous PSF- and SAT-based techniques, that’s the extremely small Field of View (FOV). Recently, Wang et al. successfully separate the autocorrelation terms from a dual-target speckle by introducing a Fourier spectrum guessing and iterative energy-constrained compensation [22]. Soon later, they further explored the multi-target retrieval and location from two speckles captured at different imaging distances [23]. Li et al. proposed a method to alternatively separate the speckle components instead of the autocorrelation terms by employing an independent component analysis, but it requires multiple acquisition [24]. Meanwhile, it’s worth pointing out that there are also some contributions have been made to increase the FOV, e.g., splicing the adjacent PSFs of LSI sub-systems [25, 26], acquiring prior information [27, 28] or specializing the scenarios [2830]. However, these methods do not focus on dividing the overlapped speckle or autocorrelation term.

Recently, deep learning (DL), as a powerful tool, has achieved state-of-the-art performance in imaging through scattering medium [3135]. Recently, Lai et al. presented a DL-based method for reconstructing two adjacent targets located at different depths behind the scattering medium [36]. Yet it is only applicable for two targets. In this paper, we successfully separate the mixtures of five autocorrelation components into individual ones by using a DL strategy. We developed a Tree-net to demonstrate our methodology for imaging multiple small targets, where the distances between each target are much beyond the range of OME.

2. Method

Before the description of our employed Tree-net, we’d like to first introduce a much more famous network U-net and show its performance in our scenario. This network has been proven to be very effective while facing imaging processing tasks, especially in the end-to-end scenario. However, it cannot help us to separate the mixed autocorrelation of multiple targets. A simple demonstration experiment is carried out as follows: We arbitrarily choose 10,000 images from MNITS handwritten digit database [37] as the hidden targets, whose autocorrelation results are calculated respectively. To train the U-net shown in Fig. 2, we take the 5,000 OAC images as the input while the corresponding 2 × 5,000 individual autocorrelation (IAC) images as the output labels. Figure 3 shows results anticipated from the U-net. It is obvious that the trained U-net could not acceptably extract the IAC components, even though only two targets are involved in this situation. Unlike traditional “one-to-one” learning tasks, here we are facing a “one-to-many” issue, which therefore falls out of the U-net’s expertise.

 figure: Fig. 2.

Fig. 2. An attempt to separate the overlapped autocorrelation by a U-net architecture.

Download Full Size | PDF

 figure: Fig. 3.

Fig. 3. A test from the trained U-net. (a) an overlapped autocorrelation image of the testing data set; (b) the real autocorrelation images of two different targets; (c) the anticipated autocorrelations from trained U-net.

Download Full Size | PDF

Here, we introduced a “Tree-net”, which is evolved from “Y-net [38], to achieve autocorrelations separation. Just like the U-net, the basic architecture of the Tree-net is also composed of an “encoder” and a “decoder”, as shown in Fig. 4(d), where three branches, corresponding to three IACs to be extracted, are involved for instance. In our work, the input is an image with 128 × 128 pixels, which is first extended into 6 channels (feature maps) by a convolutional layer with 6 convolutional kernels. These feature maps are then processed by 4 convolutional blocks where they are downsampled and convolved by another 12, 24, 48, and 96 kernels subsequently leading to the final 96 feature maps (8 × 8). All the above convolutional blocks in the encoder contain four operation layers, that are the pooling layer, convolutional layer, activation function layer, and batch normalization layer. The main function of the pooling layer is downsampling, which can remove redundant information and compress features to simplify the network complexity and reduce the amount of computation. The convolutional layer is used to extract local features, and then the activation function layer performs a nonlinear mapping on the output results of the convolutional layer, introducing the nonlinear characteristics into the neural network. Batch normalization layer can normalize a batch of data in the middle layer of the neural network, which can accelerate the convergence speed of the network and improve the training accuracy. Finally, the anticipated images are reconstructed from feature maps by the decoder which has three paths (branches) and each one includes 5 convolutional blocks. Similarly, each convolutional block in the decoder also contains four operation layers, that are the convolutional layer, activation function layer, batch normalization layer, and upsampling layer. In this work, to avoid the problems of gradient explosion and gradient disappearance, we adopted the rectified linear unit (ReLU) as the activation function, which is more efficient in gradient descent and backpropagation. Meanwhile, a major difference between the employed Tree-net and the popular U-net is that the former one’s decoder has multiple upsampling paths, which are designed to get different outputs. We believe this is an apparent reason why the Tree-net is suitable for separating a mixed signal into independent ones. While training the Tree-net, the several upsampling paths in the decoder could simultaneously learn to choose their feature maps corresponding to different output labels, so that it is assumed to be able to solve the feature confusion problem. Another point should be mentioned, in the Tree-net, we removed the operation of “skip connection”, which could help to keep the high-frequency information in the U-net. However, it will be helpless or even harmful to our Tree-net since the confusing feature maps extracted by the encoder’s convolutional layers are still strongly related to the OAC.

 figure: Fig. 4.

Fig. 4. Schematic of the Tree-net framework.

Download Full Size | PDF

We selected the Mean absolute error (MAE) as the loss function while training our Tree-net. The MAE is defined as:

$$MAE = \frac{1}{n}\sum\limits_{i = 1}^n {\left( {\frac{1}{N}\sum\limits_{i = 1}^N {|{{x_i} - {y_i}} |} } \right)} $$
where n represents the number of output images of the network, N represents the number of pixels in each image, xi and yi are the pixel values of the input and output images, respectively. Adam optimization algorithm is selected to update the weights and biases of our network to minimize the MAE value. The learning rate of the Adam optimizer is set to 0.0001, and the epoch of iterations is set to 10.

Meanwhile, we’d like to emphasize that the number of upsampling paths in Tree-net architecture is adjustable and could be easily extended to many more, achieving the purpose of one input to multiple outputs, as like many “branches” growing out of one tree trunk. However, the acquisition of the training set will be a problem since the inputs, consisting of different autocorrelations of the targets should be changed with the number of the “branches”, that’s to say, a training data set is only applicable to a specific network. For the Tree-net with “K” branches, a general way to prepare the training data is as follows:

  • (1) “K” small targets are randomly selected, and “K” speckles associated with every single target are collected. Then “K” individual autocorrelations calculated from “K” speckles can be utilized as the labels.
  • (2) A mixed speckle is collected when all the “K” targets are simultaneously involved. An OAC, calculated from the mixed speckle, is used as the input of the training network. A specific training pair is made up of this “input” and the “labels” in Step (1).
  • (3) Repeat Steps (1)-(2) until there are enough training pairs.

It is apparent that the “input” has to be re-prepared along with “K” (number of the targets). The mixed speckles associated with “K” targets are only applicable for training the Tree-net with “K” branches. That is an onerous job to acquire the mixed speckles corresponding to different number of targets. Here, we take a more efficient way to realize the acquisition of the “input” data. As the autocorrelation of the mixed speckle is almost the same as the superposition of all the IACs of the targets (see Eq. (3)), it is no longer necessary to directly collect all the mixed speckles since the “inputs” can be produced by the sum of the corresponding IACs. As shown in Fig. 4, we first randomly select a large number of digits and letters (Fig. 4(a)) as the targets, and the corresponding speckles (see Fig. 4(b)) produced by the scattering system are collected. Figure 4(c) presents the IACs calculated from the speckles. For Tree-net with three branches in Fig. 4(d), three arbitrary IACs are picked to be the labels, and their superposition is used to be the input. In this way, only the IACs is necessary for the training and all the IACs are available no matter how many branches are there in the Tree-net.

3. Experimental demonstrations

3.1. Experimental setup

Figure 5(a) shows the experimental setup. A laser beam with wavelength of 632.8 nm passes through a rotating ground glass and a collimating lens, illuminates on a spatial light modulator (SLM, Holoeye LC2012, 1024 × 768 px with a pixel size of 36 × 36 µm) on where the images of handwritten digits and letters are displayed. Then it further passes through a scattering diffuser (Thorlabs, DG10-220-MD), forming a speckle and captured by a camera (LUCID ATL314S-MT, 6464 × 4852 px with a pixel size of 3.45 × 3.45 µm) placed at the image plane. The distance between the diffuser and the SLM is 440 mm while it is 60 mm between the diffuser and the camera. The effective magnification of this scattering system is M = 60/440 ≈ 0.14. We also measured the in-plane OME range of the scattering system. Here, cross correlations between speckle when a pinhole placed at the central of the object plane and speckles when it laterally moves to different positions were calculated. Figure 5(b) shows the normalized correlation values, the solid red line is a Gaussian fit to the data and the dashed red line shows the correlation value that drops to 1/2 of the peak value (half-width at half-maximum). The results indicate that the in-plane OME range of our setup is about 1 mm in radius.

 figure: Fig. 5.

Fig. 5. (a) Optical schematic of the experimental setup; (b) OME range measurement

Download Full Size | PDF

3.2. Data acquisition

We choose 10,000 images with 128 × 128 pixels (5000 digits and 5000 letters), as targets hidden behind the scattering medium, from the MNIST and NIST datasets. 10,000 speckles are collected as every single image is successively loaded onto the SLM. Then 10,000 IACs associated with the speckles are calculated. It should be noted that although the size of images is 128 × 128 pixels, the effective height/width of the target is about 50 pixels, and most of the pixels’ values are zero, so that the size of its autocorrelation will not exceed 128 × 128 pixels (the height/width of the object autocorrelation is about twice of that of the object itself). Therefore, the height/width of the targets is about 50 × 36 µm = 1.8 mm, which is smaller than the OME range.

For the situation in which “K” targets are involved, “K” IACs are randomly chosen as the labels while their summation result is used as the input of the training network. This operation is repeated 10,000 times for acquiring 10,000 pairs of training data. In the meantime, 1000 mixed speckles are collected when 1000 different groups of “K” targets are simultaneously loaded onto the SLM. The autocorrelations of the mixed speckles are taken as the testing data, half of which are prepared for the test with the same diffuser in the training while the other half is for the test with different diffusers.

3.3. Experimental results

The experimental results shown in Fig. 6 demonstrate the feasibility of our strategy. When multiple targets are simultaneously located at the plane of interest, the OAC calculated from the corresponding speckle will play the role as the input of our trained Tree-net. Consequently, the anticipated results (outputs) are obtained. The output is visually similar to the real individual autocorrelation (R-IAC) of every single target. As the IAC is well-retrieved, the reconstruction can be successfully realized by traditional SAT. Figures 6 (a-c) show three cases when the number of targets increases from 3 to 5. The distance between the adjacent targets is about 8 mm, which is almost 4 times the OME range in our experiments. In Fig. 6(c), the distance between the two targets farthest apart is about 15 mm.

 figure: Fig. 6.

Fig. 6. A group of anticipated results of separating different numbers of individual autocorrelations by our Tree-net. (a)-(c) present three situations when 3 to 5 targets are involved, respectively. Scale bars: 800μm in “Objects” and “Speckle” and 100μm in the others.

Download Full Size | PDF

Both the values of MAE and structural similarity (SSIM) [39] for different cases are averaged and shown in Table 1. As the targets’ number increases, the MAE increases and the SSIM decreases. It can be explained that both the complexity of the OAC and the cross-talk in the recovered IAC will increase along with more hidden targets, resulting in an increment in the deviation between the outputs and the R-IACs.

Tables Icon

Table 1. The MAE and SSIM between the anticipated IACs (outputs) and the real IACs for cases associated with different numbers of targets

Meanwhile, we have considered the case in which the scattering medium is different between the training and the testing process. In this experiment, the illuminated area on the diffuser has been changed when we prepare the testing data. Though the characteristic (or say the point spread function) of the scattering system is different, the speckle autocorrelations keep almost the same. Figure 7 shows the fine reconstruction results, indicating the robustness of our method against changes of the scattering medium.

 figure: Fig. 7.

Fig. 7. Experimental results with different scattering mediums in acquiring testing data. (a)-(c) present three situations when 3 to 5 targets are involved, respectively. Scale bars: 800μm in “Objects” and “Speckle” and 100μm in the others.

Download Full Size | PDF

So far, we can successfully reconstruct multiple targets with our Tree-net. However, what if we only get a mixed speckle (or its autocorrelation) without knowledge about the number of the individual targets (or individual autocorrelations)? In this case, we could not decide which type of Tree-net should be applied. To handle this issue, we further developed a classification convolutional neural network which can classify the number of hidden targets. Six convolutional layers in the neural network are used to extract the feature maps of the overlapped autocorrelation, and then two full connection layers are used to do the classification. Finally, the output of the neural network is the probability of 3 categories corresponding to 3-5 targets respectively. The architecture of the classification convolutional neural network is shown in Fig. 8. We selected 2000 images and 500 images respectively from the overlapped autocorrelations of 3, 4 and 5 targets for training and testing. Consequently, the training data set contains 6000 images, and the testing data set contains 1500 images. The trained classification network could classify the testing datasets with 92% accuracy.

 figure: Fig. 8.

Fig. 8. Schematic of the classification neural network framework.

Download Full Size | PDF

Likewise, other networks for achieving specific functions can also connect with our Tree-net. For example, a network for retrieving target from its autocorrelation can be employed after the accomplishment of autocorrelations separation. It is equivalent to utilizing a network to realize the phase retrieval process [40].

4. Discussions

According to the experimental results shown in Section 3, our method offers three advantages. (1) Our Y-net can achieve up to 5 autocorrelations separation, which is better than previous methods [2224, 37]. A comparison between these representative methods and ours is shown in Table 2; (2) The input of our training set is artificially generated by superposing multiple autocorrelations of different individual targets. This strategy is based on the physics priori of the speckle correlation and greatly reduce the time cost and the difficulty of data set acquisition as well as increase the diversity of the data set; (3) Our method has a reliable generalization capability in imaging through a different scene in which the scattering medium is previously untrained, as long as the medium has the similar statistical characteristic with the trained one. At the same time, since the intensity autocorrelation is position-independent, the mixed autocorrelation will keep the same no matter where the targets are located as long as they are isolated. Therefore, our method is still available when the targets locate at different positions from the training stage.

Tables Icon

Table 2. Comparison among state-of-the-art multiple targets scattering imaging methods

Some limitations of our method are as follows. It is not able to reconstruct an extended target with the size larger than the OME range. The reconstruction of the target is achieved by a phase retrieval algorithm. To ensure the convergence stability, the structure of the target is supposed to be simple and sparse. Besides, with the increase of the number of objects, the quality of the network outputs will gradually decline due to the heavy crosstalk between different autocorrelations. Consequently, some autocorrelations can be restored but some cannot during the separation process.

5. Conclusion

In conclusion, we developed a DL-based method to recover multiple targets through a scattering diffuser beyond the OME’s effective range. By using the proposed Tree-net, all the individual autocorrelation (IAC) components of the hidden targets can be extracted from a mixed speckle. We experimentally demonstrate that up to 5 IACs can be simultaneously separated with high fidelity by our Tree-net, which is also applicable when the test data is generated from different scattering mediums.

Funding

Natural Science Foundation of Guangdong Province (2021A1515011801); National Natural Science Foundation of China (61805152, 61875129, 62061136005); Chinesisch-Deutsche Zentrum für Wissenschaftsförderung (GZ 1391, M-0044).

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. V. Ntziachristos, “Going deeper than microscopy: the optical imaging frontier in biology,” Nat. Methods 7(8), 603–614 (2010). [CrossRef]  

2. O. Katz, E. Small, and Y. Silberberg, “Looking around corners and through thin turbid layers in real time with scattered incoherent light,” Nat. Photonics 6(8), 549–553 (2012). [CrossRef]  

3. R. Horstmeyer, H. Ruan, and C. Yang, “Guidestar-assisted wavefront-shaping methods for focusing light into biological tissue,” Nat. Photonics 9(9), 563–571 (2015). [CrossRef]  

4. M. E. Hanafy, M. C. Roggemann, and D. O. Guney, “Reconstruction of images degraded by aerosol scattering and measurement noise,” Opt. Eng. 54(3), 033101 (2015). [CrossRef]  

5. D. Li, S. K. Sahoo, H. Q. Lam, D. Wang, and C. Dang, “Non-invasive optical focusing inside strongly scattering media with linear fluorescence,” Appl. Phys. Lett. 116(24), 241104 (2020). [CrossRef]  

6. S. Yoon, M. Kim, M. Jang, Y. Choi, W. Choi, S. Kang, and W. Choi, “Deep optical imaging within complex scattering media,” Nat. Rev. Phys. 2(3), 141–158 (2020). [CrossRef]  

7. J. W. Goodman, Speckle Phenomena in Optics: Theory and Applications (2007).

8. E. Leith and J. Upatnieks, “Wavefront reconstruction photography,” Phys. Today 18(8), 26–32 (1965). [CrossRef]  

9. J. W. Goodman, W. H. Huntley, D. W. Jackson, and M. Lehmann, “Wavefront-Reconstruction Imaging through Random Media,” Appl. Phys. Lett. 8(12), 311–313 (1966). [CrossRef]  

10. S. M. Popoff, G. Lerosey, R. Carminati, M. Fink, A. C. Boccara, and S. Gigan, “Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media,” Phys. Rev. Lett. 104(10), 100601 (2010). [CrossRef]  

11. S. M. Popoff, G. Lerosey, M. Fink, A. C. Boccara, and S. Gigan, “Controlling light through optical disordered media: transmission matrix approach,” New J. Phys. 13(12), 123021 (2011). [CrossRef]  

12. M. Kim, W. Choi, Y. Choi, C. Yoon, and W. Choi, “Transmission matrix of a scattering medium and its applications in biophotonics,” Opt. Express 23(10), 12648–12668 (2015). [CrossRef]  

13. I. Freund, M. Rosenbluh, and S. Feng, “Memory effects in propagation of optical waves through disordered media,” Phys. Rev. Lett. 61(20), 2328–2331 (1988). [CrossRef]  

14. E. Edrei and G. Scarcelli, “Memory-effect based deconvolution microscopy for super-resolution imaging through scattering media,” Sci. Rep. 6(1), 33558 (2016). [CrossRef]  

15. X. Xu, X. Xie, H. He, H. Zhuang, J. Zhou, A. Thendiyammal, and A. P. Mosk, “Imaging objects through scattering layers and around corners by retrieval of the scattered point spread function,” Opt. Express 25(26), 32829 (2017). [CrossRef]  

16. A. P. Mosk, A. Lagendijk, G. Lerosey, and M. Fink, “Controlling waves in space and time for imaging and focusing in complex media,” Nat. Photonics 6(5), 283–292 (2012). [CrossRef]  

17. O. Katz, E. Small, Y. Bromberg, and Y. Silberberg, “Focusing and compression of ultrashort pulses through scattering media,” Nat. Photonics 5(6), 372–377 (2011). [CrossRef]  

18. J. Bertolotti, E. G. van Putten, C. Blum, A. Lagendijk, W. L. Vos, and A. P. Mosk, “Non-invasive imaging through opaque scattering layers,” Nature 491(7423), 232–234 (2012). [CrossRef]  

19. O. Katz, P. Heidmann, M. Fink, and S. Gigan, “Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations,” Nat. Photonics 8(10), 784–790 (2014). [CrossRef]  

20. J. R. Fienup, “Phase retrieval algorithms: a personal tour,” Appl. Opt. 52(1), 45–56 (2013). [CrossRef]  

21. Y. Okamoto, R. Horisaki, and J. Tanida, “Noninvasive three-dimensional imaging through scattering media by three-dimensional speckle correlation,” Opt. Lett. 44(10), 2526–2529 (2019). [CrossRef]  

22. X. Wang, X. Jin, J. Li, X. Lian, X. Ji, and Q. Dai, “Prior-information-free single-shot scattering imaging beyond the memory effect,” Opt. Lett. 44(6), 1423–1426 (2019). [CrossRef]  

23. X. Wang, X. Jin, and J. Li, “Blind position detection for large field-of-view scattering imaging,” Photonics Res. 8(6), 920–928 (2020). [CrossRef]  

24. W. Li, J. Liu, S. He, L. Liu, and X. Shao, “Multitarget imaging through scattering media beyond the 3D optical memory effect,” Opt. Lett. 45(10), 2692–2695 (2020). [CrossRef]  

25. D. Tang, S. K. Sahoo, V. Tran, and C. Dang, “Single-shot large field of view imaging with scattering media by spatial demultiplexing,” Appl. Opt. 57(26), 7533–7538 (2018). [CrossRef]  

26. H. Zhuang, H. He, X. Xie, and J. Zhou, “High speed color imaging through scattering media with a large field of view,” Sci. Rep. 6(1), 32696 (2016). [CrossRef]  

27. C. Guo, J. Liu, W. Li, T. Wu, L. Zhu, J. Wang, G. Wang, and X. Shao, “Imaging through scattering layers exceeding memory effect range by exploiting prior information,” Opt. Commun. 434, 203–208 (2019). [CrossRef]  

28. M. Hofer and S. Brasselet, “Manipulating the transmission matrix of scattering media for nonlinear imaging beyond the memory effect,” Opt. Lett. 44(9), 2137–2140 (2019). [CrossRef]  

29. L. Li, Q. Li, S. Sun, H.-Z. Lin, W.-T. Liu, and P.-X. Chen, “Imaging through scattering layers exceeding memory effect range with spatial-correlation-achieved point-spread-function,” Opt. Lett. 43(8), 1670–1673 (2018). [CrossRef]  

30. Y. Wei, D. Lu, M. Liao, W. He, and X. Peng, “Noninvasive imaging of two isolated objects through a thin scattering medium beyond the 3D optical memory effect,” Opt. Express 29(12), 18807–18816 (2021). [CrossRef]  

31. S. Li, M. Deng, J. Lee, A. Sinha, and G. Barbastathis, “Imaging through glass diffusers using densely connected convolutional networks,” Optica 5(7), 803–813 (2018). [CrossRef]  

32. Y. Li, Y. Xue, and L. Tian, “Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media,” Optica 5(10), 1181 (2018). [CrossRef]  

33. M. Lyu, H. Wang, G. Li, S. Zheng, and G. Situ, “Learning-based lensless imaging through optically thick scattering media,” Adv,Photonics 1(3), 036002 (2019). [CrossRef]  

34. Y. Sun, J. Shi, L. Sun, J. Fan, and G. Zeng, “Image reconstruction through dynamic scattering media based on deep learning,” Opt. Express 27(11), 16032–16046 (2019). [CrossRef]  

35. Q. Li, J. Zhao, Y. Zhang, X. Lai, Z. Chen, and J. Pu, “Imaging reconstruction through strongly scattering media by using convolutional neural networks,” Opt. Commun. 477, 126341 (2020). [CrossRef]  

36. X. Lai, Q. Li, Z. Chen, X. Shao, and J. Pu, “Reconstructing images of two adjacent objects passing through scattering medium via deep learning,” Opt. Express 29(26), 43280 (2021). [CrossRef]  

37. D. Li, “The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web],” IEEE Signal Processing Magazine 29(6), 141–142 (2012). [CrossRef]  

38. K. Wang, J. Dou, Q. Kemao, J. Di, and J. Zhao, “Y-Net: a one-to-two deep learning framework for digital holographic reconstruction,” Opt. Lett. 44(19), 4765–4768 (2019). [CrossRef]  

39. Z. Wang and A. C. Bovik, “Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures,” IEEE Signal Processing Magazine 26(1), 98–117 (2009). [CrossRef]  

40. M. Liao, S. Zheng, S. Pan, D. Lu, W. He, G. Situ, and X. Peng, “Deep-learning-based ciphertext-only attack on optical double random phase encryption,” Opto-Electron. Adv. 4(5), 200016 (2021). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (8)

Fig. 1.
Fig. 1. (a) the ground truth of two targets located at two different OME ranges; (b) the captured speckle and its autocorrelation (c); (d) the reconstructed result (by using the traditional SAT) which shows nothing meaningful
Fig. 2.
Fig. 2. An attempt to separate the overlapped autocorrelation by a U-net architecture.
Fig. 3.
Fig. 3. A test from the trained U-net. (a) an overlapped autocorrelation image of the testing data set; (b) the real autocorrelation images of two different targets; (c) the anticipated autocorrelations from trained U-net.
Fig. 4.
Fig. 4. Schematic of the Tree-net framework.
Fig. 5.
Fig. 5. (a) Optical schematic of the experimental setup; (b) OME range measurement
Fig. 6.
Fig. 6. A group of anticipated results of separating different numbers of individual autocorrelations by our Tree-net. (a)-(c) present three situations when 3 to 5 targets are involved, respectively. Scale bars: 800μm in “Objects” and “Speckle” and 100μm in the others.
Fig. 7.
Fig. 7. Experimental results with different scattering mediums in acquiring testing data. (a)-(c) present three situations when 3 to 5 targets are involved, respectively. Scale bars: 800μm in “Objects” and “Speckle” and 100μm in the others.
Fig. 8.
Fig. 8. Schematic of the classification neural network framework.

Tables (2)

Tables Icon

Table 1. The MAE and SSIM between the anticipated IACs (outputs) and the real IACs for cases associated with different numbers of targets

Tables Icon

Table 2. Comparison among state-of-the-art multiple targets scattering imaging methods

Equations (4)

Equations on this page are rendered with MathJax. Learn more.

I I = ( O S ) ( O S ) = ( O O ) ( S S ) O O
c I I = ( i = 1 n ( O i S i ) ) ( i = 1 n ( O i S i ) )  =  i = 1 n [ ( O i O i ) ( S i S i ) ] + i , j = 1 i j n [ ( O i O j ) ( S i S j ) ]
I I i = 1 n ( O i O i )
M A E = 1 n i = 1 n ( 1 N i = 1 N | x i y i | )
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.