1. Introduction

In ghost imaging [1–7], the object is illuminated by sequential patterns and the corresponding echoes are collected with a bucket detector. Then the image of the object can be reconstructed from the correlation between the patterns and the bucket signals. Relationship between the object and the image is constructed via second-order correlation of the illumination field, which is obviously different from traditional imaging that establishes a point-to-point relationship between the object and the detection plane with lenses. Compared with traditional imaging, GI provides several advantages, such as high sensitivity [8–12], robustness against ambient noise [13–16], multi-dimensional information acquisition [17], lensless imaging [18], etc. Therefore, GI is widely studied towards applications in remote sensing [19,20], biomedical imaging [21,22], imaging through scattering media [13,23], and so on. As an idea of information acquisition, GI has also been extended into different domains such as frequency spectrum [24], Fourier domain [25], time domain [26], as well as different wavebands such as microwave [27], X ray [22,25], THz wave [28]. Since the image is reconstructed via second-order correlation, GI enables separation between imaging and spatial-resolving detection [6], which opens novel applications. For instance, imaging can be achieved in a bistatic way, with the transmitter (including the illumination source) and the receiver (including the bucket detector and image reconstruction module) spatially separated. This can also help to approach imaging against turbulence [29]. Furthermore, GI can also promote distributed structures in remote sensing easier [30,31], with multi-transmitters and multi-receivers working in coordination.

Practically, it is required to match each pattern to its corresponding echo one by one to ensure correct reconstruction. This matching requires a certain channel to transmit synchronization information or a certain protocol. With physical synchronization signals [14,32,33], the projected patterns and the echoes can be matched, which increases the complexity of the system. Especially, it can be hard under different scenarios. For example, the refreshing rate of illumination patterns is required to be high for real-time imaging thus the timeliness of the synchronization signal will be strictly required. This will be harder when handling objects at different distances, as well as the case that one or both platforms of the transmitter and the receiver are moving. Through protocols, it is mostly used to achieve matching by agreeing on a starting bit [34], in which the illumination starts with several entirely black or white patterns. However, the signals of the starting bit might be lost or submerged under weak echo or noisy conditions. And there is also the probability of failure due to crosstalk when multiple transmitters work simultaneously.

In this article, we propose a bistatic GI protocol where the illumination is encoded in both temporal and spatial domain, with no need of physical synchronization. The matching is achieved via second-order correlation in temporal domain in a gradual way with the correlation done progressively from coarse to fine, while images are reconstructed based on spatial correlation. Our scheme does not require additional channels for synchronization to achieve imaging, and it can also combat low signal-to-noise ratio (SNR), crosstalk, and time jitter caused by moving or system clock error. Experimentally, we verify our scheme by imaging under the condition that the echo signals are submerged in noise. We also demonstrate overcoming of crosstalk when two transmitters work simultaneously. With our method, it is also possible to achieve faster imaging by cooperating with transmitters of different codings. Therefore, our method can promote broader applications of GI.

2. Idea of encoding and decoding

In typical GI systems, an object $O(x,y)$ is illuminated with different patterns ${{I}_{k}}(x,y)$ of $p\times q$ pixels, with the echoes being collected by a bucket detector which provides integration among the aperture as ${{B}_{k}}=\sum\limits_{x}{\sum\limits_{y}{{{I}_{k}}(x,y)}\cdot O(x,y)}$. Then the image can be reconstructed as

To achieve such matching with no physical synchronization, we propose a robust dual encoding method, as shown in Fig. 1. In our scheme, the spatial encoding is still used for image reconstruction, while the temporal encoding is used for synchronization. The $k^{th}$ illumination is designed as

Here $A_k(t)$ serves as temporal modulation on the total intensity of each illumination pattern, and $p(x,y)$ represents the normalized spatial distribution within the illuminated area. Then the overall profile in time domain appears as $A(t)=[A_1(t_1),A_2(t_2),\ldots,A_L(t_L)]$. Accordingly, the bucket detection results $B(t)=[B_1(t_1),B_2(t_2),\ldots,B_L(t_L)]$ will reflect the modulation in time domain, as well as that from the interaction between the object and the spatial modulation of the illumination, as

To match the $k^{th}$ bucket value $B_k$ to the $k^{th}$ illumination pattern $p_k(x,y)$, with no physical signal, we propose to perform correlation between $B(t)$ and $A(t)$ and locate the echoes in time domain by positioning the peak of such correlation. The temporal correlation shows

 

Fig. 1. The spatio-temporal encoding for GI. The illumination is encoded by spatial coding pattern $p_k(x,y)$ in order, meanwhile different waveform $A_k(t)$ corresponding to $p_k(x,y)$ is used to encode in time domain. And $D$ is the duration of a pattern.

Download Full Size | PDF

However, if the interested object appears moving in the longitudinal direction during the whole sampling duration, the interval between echoes will vary with time and differ from that of illumination patterns. Even if global correlation in time domain can help to roughly locate the echoes, refined locating is still required for precise matching. To achieve this, we propose to calculate correlation with part of the designed temporal profile as

To achieve accurate locating, the maximum value of correlation results is required to occur only at the correct position of echoes, which requires the correlation between the part $A_p(t)$ and other parts selected among its neighboring part is unimodal. That is, the used coding waveform should not appear self-similarity. It is obviously not feasible for periodic waveform, therefore non-periodic waveforms should be used in this scheme.

Although $\sum _{x,y}P(x,y,t)O(x,y)$ in Eq. (4), which will actually also show up when calculating partial correlation with Eq. (5), is not a constant, its influence can be small by properly design the profile of $A(t)$.

Then we will discuss the requirements on encoding to combat possible crosstalk and noise. For a certain time-domain coding, ${{C}_{r}}=\int {{{A}_{p}}(t){{A}_{p}}(t)dt}$ is denoted as the peak value of correlation result with the right position of the entire coding, while ${{C}_{w}}=\max \left [\int {A'_p(t){{A}_{p}}(t)dt}\right ]$ is the maximum of cross correlation when $A'_{p}(t)$ and $A_p(t)$ contain no overlap. Define $S$ to characterize the unimodal property of the correlation result as

3. Results

To verify our method, we carried out experiments, with the setup shown in Fig. 2. The spatial encoding is performed by modulating a laser with a digital micromirror device (DMD). While the temporal encoding is done with an electro-optic modulator (EOM). Whenever the pattern is switched to ${{I}_{k}}(x,y)$, an external trigger signal will be provided by the DMD, to the field programmable gate array (FPGA) that controls the EOM. Then temporal modulation ${{A}_{k}}$ is operated under control of FPGA. Therefore, the actually emitted illumination beam is dually encoded by ${{A}_{k}}$ and ${{I}_{k}}(x,y)$ simultaneously. The encoded light illuminates the object through an optical imaging system consisted of lenses L1 and L2. The bucket detection is performed by a single pixel photomultiplier tube (PMT) following a lens L3. Then, correlation between the data from PMT and the known coding waveform is calculated to extract the bucket detection value and match it with the right pattern. Finally, imaging can be achieved by the receiver with no physical synchronization.

 

Fig. 2. A sketch of the experimental setup. In the transmitter part, an electro-optic modulator (EOM) and a field programmable gate array (FPGA) are employed to encode the laser in time domain, a digital micromirror device (DMD) encodes it further with Hadamard patterns in spatial domain. DMD will send trigger signals to the FPGA to synchronize spatial encoding and temporal encoding. The patterns are projected onto the object through an optical imaging system consisting of L1 and L2. In the receiver part, a photomultiplier tube (PMT) performs bucket detection with lens L3, and the computer processes the detection results to achieve imaging. There is no physical synchronization between the transmitter and receiver.

Download Full Size | PDF

According to the above section, each temporal coding waveform is required to be uncorrelated with others. However, in order to reduce the pressure of generating random waveforms as well as that of storing temporal profile, we propose using a pseudo random binary sequence (PRBS) to arrange only two random waveforms to achieve the task. The PRBS is easily generated by a linear feedback shift register. For one illumination pattern, if the generated value of PRBS is 0, one coding waveform is used to encode the illumination in time-domain. If the value is 1, the other waveform is used for temporal encoding. The sequence of PRBS is set to be non-repetitive, and the corresponding temporal coding will be also unimodal in second-order correlaton.

In the experiments, the bandwidth of PMT we used is 400 MHz, the duration of each pattern is $D=50$ µs, and totally $N=8192$ Hadamard patterns with $p=q=64$ are projected. For each Hadamard pattern, the temporal waveform is encoded into a sequence of uneven distributed 64 pulses. The intervals between neighbouring pulses are set to be an arithmetic sequence in our experiments, with the minimal time unit being 2.5 ns (the size of one time pixel) and the width of pulses being 5 ns. The waveform for one pattern is generated as

 

Fig. 3. A piece of used temporal profile. Two waveforms are employed, with waveform-1 being the reverse of waveform-0. Switching between two waveforms is arranged according to a pseudo random binary sequence (PRBS).

Download Full Size | PDF

To verify the applicability of the designed waveforms, we calculted correlation between different waveforms generated via Eq. (8), with the peak values shown in Table 1. The minimum value of correlation between different waveforms is verified to be 0. The waveform-2 in the table is generated with $s=26$ and $d=3$, and waveform-3 is the reverse of waveform-2. It can be seen that the waveform only shows great correlation value with itself when matched, which reflects that $S$ is great ($S\sim 0.875$). This meets the coding requirements mentioned in Eq. (7).

Table 1. The peak values of correlation between waveforms. The minimum of correlation between different waveforms is 0.

View Table

With the above settings, we verify our scheme for imaging, with an object made of paper, shown in Fig. 4(a). For comparison, we also performed traditional ghost imaging with no physical synchronization, where the sample ratios for both are set to be equal. As shown in Fig. 4(b) , it is impossible for the traditional scheme to image the object with no physical synchronization. Then we carry out with our scheme, using a sequence of waveform randomly switched between waveform-0 and waveform-1 according to the PRBS for temporal encoding. Experiments are firstly done under high signal to noise ratio. Since the echoes are significantly higher than noise, the maximum of correlation significantly surpasses the values when mismatched, even if only a small sequence of $A_k$ is used for calculation. According to the position that achieves the peak correlation, time position of the illumination patterns can be easily obtained. Then image of the object can be reconstructed, as shown in Fig. 4(c) with contrast to noise ratio (CNR) [35] listed below the image. That is, our scheme achieves GI without physical synchronization.

 

Fig. 4. Experimental results. (a) The object ’K’ made of white paper. (b)Result from traditional GI with no physical synchronization. (c)Results of imaging via our scheme with no physical synchronization under different detection signal to noise ratios (DSNRs). The DSNRs and CNRs are shown below the images. The scale bars: 2 mm.

Download Full Size | PDF

Then the capability of our scheme under low detection signal to noise ratio (DSNR) is verified. When the intensity of echoes is not so low, the echoes can still be located and matched using only a small sequence of $A_k$s. However, when the intensity decays to a certain extent, the contrast of calculated correlation decreases. Thus the sequence of $A_k$s for calculation is required to be longer, since a longer waveform provides better performance against noise. Since the noise in the experiment comes from the detector’s electrical noise, and the detector’s parameters remain unchanged during detection, the intensity of the electrical noise remains unchanged. To test our scheme under different DSNRs, we change the intensity of echoes. With the echoes weakened by an attenuation plate (different transmittance corresponds to different attenuation), experiments at different intensities of echoes are performed, and the images under different DSNRs can be reconstructed. We use $R_{D}=\frac {\overline {b}}{\sigma }$ to evaluate DSNR, with $\overline {b}$ representing the mean intensity of all echoes and $\sigma$ is the standard deviation of noise. Under different levels of DSNRs, we performed matching and imaging, with the imaging results shown in Fig. 4(c), with the value of measured $R_{D}$ shown below the reconstructed image. To achieve matching under weak echoes, longer sequence of temporal coding will be more helpful. But, more time it will take. Sequences covering the duration of 50 patterns (as long as $50D$) are used in our experiments. The results show that such length is sufficient under our experimental condition. For each sequence of 50 patterns, the temporal profile is unique, such that each pattern can be located. By continuously selecting the sequences of different 50 patterns, the echoes corresponding to all patterns can be located. To achieve imaging under weak echoes, temporal correlation within the waveform of 64 pulses corresponding to one pattern is used to take place of the bucket detection [36]. It can be seen in Fig. 4(c) that, image is no more reconstructed when $R_{D}$ is as low as 0.416, due to poor signal. Actually, the echoes can still be precisely located via temporal correlation, under this condition. For comparison, we also conducted a test using a physical synchronization signal, with the time delay caused by light propagation included. The image obtained appears of the same quality with that of our scheme. That is, our scheme succeeds in signal matching while does not improve imaging quality. Also, to match the echoes is easier than imaging under low DSNR, with our scheme.

Another thing to be noted is the influence by time jitter from the used devices. Under this situation, the waveform of echoes will not be totally the same as that of the encoded illumination. While, the overall structure should be roughly maintained, so long as the time jitter is not very large or changing fast, compared to the intervals between pulses. Considering this, we firstly use a long waveform for roughly locating the echoes, then perform finer correlation with short waveforms within the range determined by coarse correlation. This helps to enhance the capability of signal matching, with the cost of longer data processing. In the experiments, we use sequences covering the duration of 50 patterns as long waveforms and single $A_k$ as short waveforms for coarse-to-fine correlation. The time jitter caused by the DMD is measured to be about 10 ns for every 50 patterns, which will lead to mismatch repeatedly since the time jitter is longer than the width of pulses. To improve the tolerance of our scheme against time jitter, the pulse width for correlation calculation is numerically extended. By numerically extending each pulse into 40 ns, the echoes are easily matched in the experiments. It should be noted that, the pulses are only numerically extended during data processing for correlation calculation, while the actually emitted laser pulses were kept to 5 ns.

Based on our method, imaging with multi-transmitters and multi-receivers can be achieved, with crosstalk suppressed. In our experiments, we demonstrate imaging with two transmitters and a single receiver, as sketched in Fig. 5. Light from two transmitters is encoded differently, with waveform-0 and 1 used for the first transmitter and waveform-2 and 3 used for the second. Two beams from two sources illuminate the same object, and the reflected light of both is collected by a single receiver. At this time, light from one source appears as disturbance for imaging with the other source.

 

Fig. 5. Experimental setup of ghost imaging with two transmitters and one receiver. There are two encoded light beams illuminating the same object, with echoes both detected by one PMT. Without synchronous signal, image can be reconstructed via correlation with two sources separately.

Download Full Size | PDF

The experimental results are shown in Fig. 6. With two sources illuminating together, image cannot be achieved with traditional GI due to crosstalk between two sources. As a contrast, images can be achieved, by picking out the corresponding echoes for each source with our method. To evaluate the intensity of crosstalk, we compare the average intensity of echoes from two sources. We take $R=\frac {\overline {{{b}_{2}}}}{\overline {{{b}_{1}}}}$ as an indicator, where $\overline {{{b}_{1}}}$ represents the average intensity of echoes from transmitter 1, and $\overline {{{b}_{2}}}$ for transmitter 2. For simultaneous lighting case, the object is illuminated by both sources, therefore signals from both exist in the echoes. Images are reconstructed by referring to either transmitter, with echoes from the other taken as crosstalk. When $R=0.158$, it is the limit case in our experiments where the echoes of transmitter 2 can be matched. When $R=0.139$, correct matching failed, therefore imaging is no longer obtained. To show that the failure is not due to weak echoes, the imaging results with separate lighting are also shown. That is, two transmitters work separately in time. Then no crosstalk occurs. With separate lighting, of which intensity is not changed compared to the above, imaging can be achieved, indicating that the inability to image mentioned above is due to crosstalk. Therefore, our scheme can work with the crosstalk larger than the wanted signal, almost by one order of magnitude. Further, if the receiver take echoes from both transmitters as useful signals, the actual refresh rate is therefore doubled, which will result in a higher speed of imaging.

 

Fig. 6. Imaging results with two transmitters and one receiver. Images from traditional GI and that via our scheme with transmitter 1 and transmitter 2 with our method are shown. $R$ is the ratio of the average intensity of echoes from Transmitter 2 to that of Transmitter 1. The CNRs are shown below the images. The scale bars: 2 mm.

Download Full Size | PDF

It should be noted that, when doing comparison between our scheme and traditional GI, the number of used pulses and the intensities are set as the same. While in practice, our scheme requires aperiodic coding, which infers that the duty cycle can be lower than periodic pulses usually used in traditional GI. For specific systems, optimization considering effective energy, time consumption and robustness can be further studied. Also, correlation takes a little bit more time than directly locating the pulses using physical synchronization. When using physical signals, each pulse can be processed directly. Our method is to process a signal string, but it is not necessary to wait until all signals are collected. So there is not much loss in real-time performance. In practice, by optimizing the coding waveform and algorithm of correlation, the real-time performance can be improved.

4. Conclusion

In this paper, we proposed a way for bistatic ghost imaging based on dual encoding in both time and spatial domain, using aperiodic temporal waveform and progressive correlation for synchronization between the transmitter and receiver. Through our method, imaging can be achieved with no physical synchronization, making bistatic ghost imaging easier. At the same time, our design is verified experimentally to extract signals for imaging under low signal-to-noise ratio, as well as under strong crosstalk. Therefore our method makes GI more robust against harsh environments. The longer waveform used for matching, the stronger the ability of signal extraction. Towards practical applications, we believe our method opens new possibilities of different configurations.