Open Access
Add to BibSonomy
Abstract
Based on the laser Doppler coherent detection method, a laser Doppler Non-Line-of Sight imaging technique (LD-NLOS) is proposed to obtain a series of effective information about the detected objects outside the line of sight. According to the analysis of the frequency and light intensity characteristics of the scattered signal, the information of the detected object hidden in the intermediate scattering surface is decoded. Without relying on complicated back-end algorithm processing and expensive experimental detection cost, the proposed LD-NLOS technique can detect the target vibration velocity and stably reconstruct its 2D shape.
© 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
The widespread availability of high-sensitivity optical signal detectors, coupled with advanced image processing algorithms, along with the vigorous development of laser technology, had spawned a new seemingly impossible imaging technology—Non-line-of-sight (NLOS) imaging [1,2]. The ability to image targets that are obscured by an intermediary scattering surface (around corners, after fog) would prove valuable in its wide range of applications. Such as braking prediction in driving, safety industrial monitoring, searching and rescuing operations in extreme environments, and military activities against terrorism and violence have proved the unique capabilities of NLOS imaging [3,4]. Although the information of the detected targets cannot be directly obtained in the NLOS space, many observable surfaces in this space contain such information. How to apply appropriate technical means to decode the information concealed by an intermediary scattering surface has become the core problem of realizing NLOS imaging.
The previous approaches to seeing around corners had involved using short-pulse tunable laser and high time resolution sensors [5–7]. For example, in the ultra-fast transient imaging, detection devices such as a streak tube camera or a TOF camera were used to detect the time-of-flight (TOF) of multiple scattered photos, thereby restoring the information of obscured targets [8–10]. The Photon counting imaging based on single photon avalanche diode (SPAD) and time correlated single photon counting (TCSPC) achieved NLOS imaging by using high-time-resolved sensors and timers solved measurements [10–13]. Facing saturating the single photon detector and suffering the pile-up effect, the Nonlinear Gated Single Photon Detection (NGSPD) system solves the difficult problem of counting the very few information photons buried in the strong background [14]. At the same time, it is accompanied by disadvantages such as small imaging range, high cost, and high system complexity [15–17].
Based on the laser Doppler technique, we demonstrate the coherent imaging non-line-of-sight target detection technique. For the vibrating target (living or vibrating static target) that is tracked or visualized outside the line of sight, its two-dimensional (2D) shape is observed using the Doppler frequency shift information in the target's diffuse reflected light. Adopting the coherent detection method effectively improves the anti-interference ability of the system. Compared with the aforementioned detection schemes, the LD-NLOS technique ensures fast frequency detection and reconstruction quality while reducing the detection cost and complicated backend algorithm [18–21], providing a more cost-effective method for non-line-of-sight imaging problems.
2. Experiment setup
The experimental laser Doppler imaging scheme used for this study is sketched in Fig. 1. The experimental setup can be recognized as a modified Mach-Zehnder optical interferometer. The light source used for the experiments is a Nd:YVO4 laser (Lighthouse Photonics Sprout-G, 5W, 532nm). The main laser beam is split into two channels. In the reference channel, an optical attenuator, and a half-wave plate are used to control the beam power, polarization angle to ensure a flat illumination of the detector. In the signal channel, the collimated laser illuminates a spot that reflects light toward the unknown object. To improve the signal-to-noise ratio, a BOPP film (Biaxially oriented polypropylene film) is pasted at the laser incident point of the wall. The object is located at 12 cm from the wall, the laser incident point is located at 17.7 cm from the center of the object, and the vibrating target is 1.6 m from the detector. In this system, a speaker vibrating in 1100 Hz-3000 Hz is used as the object, and the vibrating object is assumed to have an optically rough surface. During the multiple reflections of the laser beam on the wall, a spot area to be detected will be formed on the wall. The single-point scanning method is applied to the spot area where the naked eye cannot receive any valid information about the detected object, through which the light is collected by L1. The doppler-shifted signal, due to the vibrating target in the signal channel, is combined at a beam splitter with the unmodulated signal from the reference channel. This includes a beam of signal light that undergoes multiple diffuse reflections (wall-object-wall) and carries a Doppler frequency, a beam of unmodulated signal light and ambient background light mixed radiation that is incident on the surface of a PMT (SensL’s silicon photomultipliers modules, PDE 20%, GAIN 2.3×106, Dark count rate (Typical) 44kHz, Bandwidth 20MHz) which is used as a photomixer. The phase conjugation technique is used to compensate for the signal light wavefront distortion to solve the wavefront matching problem between the signal light and the local oscillation light [22–24]. After the mixing and detection process, a “beat” signal at the Doppler frequency is obtained. An analysis of this frequency is then performed by displaying it on a spectrum analyzer.
Fig. 1. (a) The schematic diagram of the laser Doppler non-line-of-sight imaging experimental system consists of three parts: light source and filter, coherent detection, and optical signal reception. The coherent detection is further divided into reference and detection optical paths, which include the detection target and its vibration signal generating module. (b) Top view of the light propagation path.
The photoelectric fields of the reference path and detection path in the Mach-Zehnder interferometric system can be expressed as:
In formula (4), $\alpha = e\eta \textrm{/(}h\nu \textrm{)}$ is the photoelectric conversion coefficient, $h\nu$ is the photon energy, and $\eta$ is the quantum efficiency of the detector. Since the detection rate of the PMT is limited, only the average value of the optical frequency signal can be detected, so the first three terms in formula (4) are the DC component, and the fourth term is the difference frequency term. After DC filtering, the instantaneous current signal output by the detector:
Under the optimized Mach-Zehnder interferometer NLOS imaging conditions, the laser beam is irradiated by the first reflection from the wall to the detected object vibrating at a fixed frequency, and the laser beam scattered by the detected object with Doppler shift is then captured by the detector through the reflection from the wall. The angle between the direction of the scatterer's vibration velocity and the direction of the laser beam after primary reflection from the wall is θ1, and the other between the vibration direction of the detected object and its scattered light direction is θ2. Assuming $V\textrm{(}t\textrm{)}$ is the instantaneous vibration velocity of the measurement target, the simplified instantaneous Doppler shift or frequency:
The detection target with changing vibration rate connected by the signal generator, combined with formula (5)(6), the instantaneous current signal output by PMT:
According to formula (6) and (7), the frequency of the target blocked by the mask can be obtained.
3. Result and discussion
3.1 Restoration of the vibration frequency on the NLOS target
As a first demonstration, a vibrating object is hidden in Fig. 1, so that direct-light paths between the object and the laser or the PMT are blocked. A signal spot on the wall is illuminated by a laser beam, and keep the position of the illuminated point unchanged.
Figure 2 shows Doppler signals produced in such a system as a function of the target Vibration Frequency. The range of the target vibration frequency is 1100 Hz-3000 Hz. The measurement values results were in good agreement with the target Vibration Frequency. The main factor account for profile broadening is the variation in vibration velocity. And intensity fluctuations at different frequencies are most probably due to variations in amplitude of the object. To illustrate the influence of the amplitude on spectral intensity in detail, we take more amplitudes into consideration (see Fig. 3). The results clearly show that the signal intensity increases with the vibrating amplitude of the object. From these measurements we can see that for the vibrating object hidden from view, we can obtain the vibration frequency by measuring the scattered light.
Fig. 2. Corresponding relationship diagram with restoration the vibration frequency of the detected object relying on detecting Doppler-Shifted Signals by Experiment and the frequency generated by the signal generator (a)(c)(e) Frequency range: 1100-2000 Hz (b)(d)(f) Frequency range: 2000-3000 Hz.
Fig. 3. The transformation relationship between sound intensity and amplitude in the frequency range of 300-1300 Hz. (a)3D view, (b)top view.
3.2 Two dimensional shape reconstruction of the NLOS target
To realize the imaging and tracking of occluded objects, in addition to the detection of the frequency information of the target, it is also necessary to reconstruct the two-dimensional shape of the target.
After attaching the two-dimensional shape target shown in Fig. 4(a) to the mask of the vibration target signal generation module in Fig. 1, it can vibrate at a certain frequency. During the detection process, the three reflections of the laser beam on the detection path form an area to be detected on the wall as shown in Fig. 4(b). The intensity of the light amplitude received by the naked eye cannot obtain any valid information about the detection target. The position of the incident laser beam on the wall is changed by a set of galvanometer-actuated mirrors. Single point scanning is applied to the area to be detected formed by the laser beam irradiating on the wall, and the light signal obtained from the scan is subjected to spectrum analysis. In the area to be detected on the wall, the Doppler shift signal will be captured by the detector due to the vibration of the detected target. On the contrary, in the area outside the coverage of the vibrating target, no Doppler shift can be detected, through which the two-dimensional shape of the detected object outside the line of sight can be reconstructed, as shown in Fig. 4(c). By adjusting the position and the receiving angle of the detector, the received scattered light signal from the center of the target is the strongest. When imaging the edge of an object, the light scattered by the object back to the wall has a larger scattering angle, and the intensity of the light signal received by the detector will be weaker than that of the central part. In addition, the scattered light scattered to the wall at the edge part is blocked by the object, which will also cause the signal strength to be weaker than the central part. Considering the aforementioned limitations, the 2D shape of the target can still be restored by the proposed reconstructed detection method.
Fig. 4. (a) The 2D shape of the target is covered by the obstacle, connecting to the signal generator and vibrating at a certain frequency (b) The partial schematic diagram of the experimental detection path (c) The 2D shape reconstruction effect of LD-NLOS technology.
4. Conclusion
In this paper, we proposed a DL-NLOS technique to detect two-dimensional shapes and frequency information of occluded targets in NLOS imaging. Compared with other non-line-of-sight imaging detection techniques, the proposed scheme reduces redundant back-end algorithms and detection costs. The optimized Mach-Zehnder coherent detection system meets the coherent light requirements of heterodyne laser detection, and improves the anti-interference ability of the system compared with the direct detection method. The relationship between the Doppler signal changes under different vibration frequencies is discussed, and the association between the signal strength and the vibration amplitude of the object is further revealed. It is verified that under the NLOS imaging condition, the laser Doppler detection scheme extracts the two-dimensional information of the occluded object contained in the frequency domain. Therefore, this method provides a more cost-effective choice for NLOS imaging fields.
Disclosures
The authors declare no conflicts of interest.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. O. Katz, E. Small, and Y. Silberberg, “Looking around corners and through thin turbid layers in real time with scattered incoherent light,” Nat. Photonics 6(8), 549–553 (2012). [CrossRef]
2. D. Faccio, A. Velten, and G. Wetzstein, “Non-line-of-sight imaging,” Nat. Rev. Phys. 2(6), 318–327 (2020). [CrossRef]
3. O. Katz, P. Heidmann, M. Fink, and S. Gigan, “Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations,” Nat. Photonics 8(10), 784–790 (2014). [CrossRef]
4. J. Klein, C. Peters, J. Martín, M. Laurenzis, and M. B. Hullin, “Tracking objects outside the line of sight using 2D intensity images,” Sci. Rep. 6(1), 32491 (2016). [CrossRef]
5. Y. Fu, M. Guo, and P. B. Phua, “Cross-talk prevention in optical dynamic measurement,” Opt. Laser Eng. 50(4), 547–555 (2012). [CrossRef]
6. H. Liu, E. Jonas, L. Tian, J. Zhong, B. Recht, and L. Waller, “3D imaging in volumetric scattering media using phase-space measurements,” Opt. Express 23(11), 14461–14471 (2015). [CrossRef]
7. X. Liu, S. Bauer, and A. Velten, “Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems,” Nat. Commun. 11(1), 1645 (2020). [CrossRef]
8. J. Boger-Lombard and O. Katz, “Passive optical time-of-flight for non-line-of-sight localization,” Nat. Commun. 10(1), 3343 (2019). [CrossRef]
9. O. Salhov, G. Weinberg, and O. Katz, “Depth-resolved speckle-correlations imaging through scattering layers via coherence gating,” Opt. Lett. 43(22), 5528–5531 (2018). [CrossRef]
10. J. H. Nam, E. Brandt, S. Bauer, X. Liu, M. Renna, A. Tosi, E. Sifakis, and A. Velten, “Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second,” Nat. Commun. 12(1), 6526 (2021). [CrossRef]
11. S. Xin, S. Nousias, K. N. Kutulakos, A. C. Sankaranarayanan, and I. Gkioulekas, “A Theory of Fermat Paths for Non-Line-Of-Sight Shape Reconstruction,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019), pp. 6791–6802.
12. S. Chan, R. E. Warburton, G. Gariepy, J. Leach, and D. Faccio, “Non-line-of-sight tracking of people at long range,” Opt. Express 25(9), 10109–10117 (2017). [CrossRef]
13. Z. Li, X. Liu, J. Wang, Z. Shi, L. Qiu, and X. Fu, “Fast non-line-of-sight imaging based on first photon event stamping,” Opt. Lett. 47(8), 1928–1931 (2022). [CrossRef]
14. S. Zhu, Y. M. Sua, P. Rehain, and Y. Huang, “Single photon imaging and sensing of highly obscured objects around the corner,” Opt. Express 29(25), 40865–40877 (2021). [CrossRef]
15. K. L. Bouman, V. Ye, A. B. Yedidia, F. Durand, and W. T. Freeman, “Turning Corners into Cameras: Principles and Methods,” in 2017 IEEE International Conference on Computer Vision (2017), pp. 2289–2297.
16. Y. Wang, Y. Zhang, M. Huang, Z. Chen, Y. Jia, Y. Weng, L. Xiao, and X. Xiang, “Accurate but fragile passive non-line-of-sight recognition,” Commun. Phys. 4(1), 88 (2021). [CrossRef]
17. T. Sasaki, C. Hashemi, and J. R. Leger, “Passive 3D location estimation of non-line-of-sight objects from a scattered thermal infrared light field,” Opt. Express 29(26), 2021–43661 (2021). [CrossRef]
18. J. Ye, X. Huang, Z. Li, and F. Xu, “Compressed sensing for active non-line-of-sight imaging,” Opt. Express 29(2), 1749–1763 (2021). [CrossRef]
19. T. Li, Y. Luo, Z. Xie, Y. Liu, S. Xia, S. Xu, G. Ren, H. Ma, B. Qi, and L. Cao, “Non-line-of-sight fast tracking in a corridor,” Opt. Express 29(25), 41568–41581 (2021). [CrossRef]
20. S. Zheng, M. Liao, F. Wang, W. He, X. Peng, and G. Situ, “Non-line-of-sight imaging under white-light illumination: a two-step deep learning approach,” Opt. Express 29(24), 40091–40105 (2021). [CrossRef]
21. X. Feng and L. Gao, “Improving non-line-of-sight image reconstruction with weighting factors,” Opt. Lett. 45(14), 3921–3924 (2020). [CrossRef]
22. Haiping Xu, Shungxiang Shi, Liju Chen, and Yanling Sun, “Investigation into self-pumped and mutually pumped phase conjugation coexisting in Cu: KNSBN crystal,” Acta Photonica Sinica 31(5), 588–591 (2002).
23. G. Pang, H. Liu, P. Hou, M. Qiao, and S. Han, “Optical phase conjugation of diffused light with infinite gain by using gated two-color photorefractive crystal LiNbO3:Cu:Ce,” Appl. Opt. 57(10), 2675–2678 (2018). [CrossRef]
24. T. Bach, K. Nawata, M. Jazbinšek, T. Omatsu, and P. Günter, “Optical phase conjugation of picosecond pulses at 1.06 µ in Sn2P2S6:Te for wavefront correction in high-power Nd-doped amplifier systems,” Opt. Express 18(1), 87–95 (2010). [CrossRef]
