1. Introduction

The water droplet suspension in fog causes random scattering of light, which exponentially degrades the visibility as a function of distance [1–9]. Heavy fog can severely affect driving on the road. The travel speed needs to be reduced so that the safe stopping distance is less than the fog reduced viewing range. Although other waves [1,5] such as the radio waves used in radar imaging that are insensitive to water droplets can be utilized to assist driving, they typically lack the spatial resolution and contrast of optical measurement. A robust optical imaging system that can greatly outperform conventional optical imaging techniques in imaging depth and contrast may lead to better and safer driving in foggy conditions. Moreover, the technology developed for optical imaging may potentially be adopted to further improve the imaging range for other waves.

The refractive index difference between water and air causes optical wavefront distortion. Takes a point source at a far distance as an example, the incoming wavefront is to an approximation a plane wave without the presence of these water droplets. Thus a lens can focus the wave to a spot on the image plane. Each of the water droplets can cause wavefront distortion. The combined results after passing through a countless number of the droplets is a highly distorted wavefront which can be decomposed into many plane waves propagating towards different directions (optical modes). Thus, many other spots appear on the image plane surrounding the original signal spot. As a result, the image contains a scattering induced random background. The scattering background is difficult to remove computationally as the scattering event is by nature highly dynamic. More critically, the optical shot noise associated with a strong random background can easily overwhelm the true signal.

One way to overcome this problem is to utilize gated detection [2,8,10–16] based on both the temporal coherence and the spatial coherence of light, which can help filter out the scattering background. These gating strategies take advantages of the fact that the waves that undergo the least amount of scattering tend to have well-preserved temporal and spatial profile. By confining the temporal detection window and the spatial detection zone, we can remove substantial scattering background and more importantly the associated shot noise. Temporal gating can be achieved by using pulsed laser source and temporally gated detection to select signals arriving within the predefined time window. Alternatively, low temporal coherence interferometry with broadband light source can be used to achieve gating. Spatial gating can be achieved by using spatially incoherent extended light source for illumination and interferometric recording. Alternatively, sequential point scanning and point detection can achieve the same function [2]. These gating strategies have been successfully applied for many microscopic biomedical imaging applications [2,10,13–15], as the biological tissue, similar to fog, is inhomogeneous in refractive index and causes random light scattering.

To adopt such strategies for practical implementation on vehicles, new challenges need to be overcome. Different from microscopy applications, the optical numerical aperture (NA) for imaging objects at distance through fog is very limited. Thus the axial spatial confinement available to confocal detection is very weak. In addition to random scattering, the imaging in fog may also suffer from aberration, the low order wavefront distortion. As a result, the commonly used single mode detection (e.g. in confocal microscopy [14,15], optical coherence tomography [10,13]) can be very fragile. The air turbulence can easily distort the focus and reduce its peak intensity or shift the focus position.

To address these challenges, we developed an off-axis spatiotemporally gated multimode detection method (Fig. 1). The off-axis configuration means that the illumination laser beam and the signal detection beam travel along different paths. Such a strategy comes with two major benefits. One is that it can achieve axial spatial confinement despite of the low NA of each beam as the overall imaging point spread function (PSF) is the product of the illumination PSF and the detection PSF. The crossing of the two paths effectively achieves the needed axial spatial confinement. The other is that the off-axis configuration can help suppress the scattered illumination light from entering the detection path [13–15]. For comparison, the illumination and the signal detection in the widely used common-axis configuration share the same path and thus the scattered illumination light can easily make its way to the detector. For practical implementation on vehicles, we can install the illumination and the detection module at the position of the two side view mirrors. Furthermore, to make the system robust in the presence of optical aberration [17], we implemented a multimode detection, in which we simultaneously detect multiple spatial modes and incoherently combine their power as the total detected signal. Thus the low order wavefront distortion [17] induced energy spreading or jittering of the focus location will not affect the recorded signal strength.

 

Fig. 1. Off-axis confocal scanning with multimode detection and temporal gating.

Download Full Size | PDF

2. Methods

To experimentally evaluate the imaging performance of the proposed method, we set up a compact optical system (Fig. 2) to image targets located inside a fog mimicking liquid suspension. Typically, the temporal gating can be implemented by nanosecond pulsed laser source and nanosecond scale electronic gating, as in the case of LIDAR imaging [18]. As the total light path inside the liquid suspension is less than 1 meter, we need gating time much shorter than nanosecond. Experimentally, we utilized a 592 nm diode laser source of 7 mm coherence length (MPB Communications) and employed hologram recording [2,19] to achieve temporal gating. For spatial gating, we overlapped the focus of the illumination path (the laser beam focused by L1 onto the imaging target in Fig. 2) and the detection path (the scattered light collected by L2 in Fig. 2) and scan the common focus using a voice coil driven two axes scanning mirror (OIM102.3 OpticsInMotion) to form images. For multimode detection, we implemented pupil plane off-axis holography using a large full well charge capacity camera (MV-D1024E-160-CL-12, Photofocus). Essentially, the pupil plane of the signal collection lens (L2 in Fig. 2) was imaged onto the camera and interfered with the reference. As the reference beam is linearly polarized, the interferometric detection also attained the function of polarization gating. The multimode selection is implemented digitally after the Fourier transform of the recorded hologram. In theory, we can also perform multimode recording at the focal plane of L3 in Fig. 2. Different spatial modes will directly appear on different camera pixels. However, much less number of camera pixels will take part in the interferometric measurement for the detection of tens of spatial modes without substantial image zoom and thus the achievable SNR will be limited. In comparison, we can use a compact configuration for pupil plane detection employing 128 × 118 pixels. Regardless of the number of spatial modes we choose to record, their information is evenly supported by the 128 × 118 pixels and the detection noise floor remains a constant.

 

Fig. 2. Experimental implementation of the off-axis spatiotemporally gated multimode detection system. L1-4, optical lenses of focal length, 500, 500, 250, 50 mm, respectively. M, mirror. The expanded laser beam travels through a beam sampler. The transmitted light is focused by L1 onto the imaging target. The scattered light from the target is collected by L2. L3 and L4 form a reduced image of the back focal plane of L2 onto the camera sensor where it interferes with the reference beam (the reflected light from the beam sampler).

Download Full Size | PDF

The data recording involves driving the two-axis scanner and synchronously triggering the camera exposure. The scanner control signal is the same as that of commonly used laser scanning microscopes. The camera is configured for external exposure control mode such that the external trigger signal controls both the exposure start (rising edge of the TTL trigger signal) and the exposure duration (high level TTL signal duration). Using a region of interest containing 128 × 118 pixels, we achieved 4 kHz frame rate with 50% duty cycle. The recorded image frames are continuously transferred to the computer memory during the recording through a camera-link frame grabber (Neon-CLB, Bitflow).

The data processing (Fig. 3) is similar to that of digital off-axis holography. First, we normalized the acquired image intensity to eliminate laser illumination power variation. Second, we subtracted the reference beam profile from the acquired images to account for the non-uniform spatial profile of the reference beam. Third, we performed Fourier transform. In the Fourier domain, we could flexibly select modes and incoherently combine their power to represent the signal strength.

 

Fig. 3. Data processing procedure. From raw data to processed data: Image normalization and removal of reference background. Abs(FFT2)^2: The modulus square of the two-dimensional Fourier transformed image, i.e. signal intensity at the Fourier plane. Binary mask achieves the function of spatial filtering. The intensity summation of the filtered image represents the signal level at the current beam scanning angle (i.e. one pixel of the final image).

Download Full Size | PDF

3. Results

To evaluate the effect of the multimode detection, we placed a flat wood deer shape figurine inside a rectangle glass tank filled with water. The distance between the target and the viewing window of the tank is 0.37 meter. We gradually added milk into the water tank and measured the scattering path length. First, we compare the images acquired with the off-axis laser scanning method and with conventional camera (Fig. 4(a)-4(d)). Due to slight aberration, the Fourier transformed image has slight spreading around the two complex conjugate peaks (Fig. 4(b)). We compare two methods of processing the Fourier transformed images. One is to use the incoherent summation of the power from 49 pixels (spatial modes) to represent the signal intensity (Fig. 4(a)) and the other is to use just one pixel to represent the signal intensity (Fig. 4(c)). It is worth noting that the incoherent summation utilized in the multimode detection method yields smoother images. In comparison, the image based on single mode detection appears to have more speckles. At the imaging depth of 6.4 scattering path lengths, the off-axis scanning multimode detection approach can still yield clear images with excellence contrast (Fig. 4(e)). In contrast, the camera recorded image is overwhelmed by the scattering background (Fig. 4(f)).

 

Fig. 4. Image of flat deer figurine. (a) The image acquired with the off-axis scanning method through clear water. The signal intensity is the incoherent summation of 49 spatial modes. (b) Fourier transformed hologram from one pixel in a. (c) Image using the same data set of a and only the power of one spatial mode to compute pixel intensity. (d) Image acquired by a camera through clear water. (e) The image acquired by the off-axis scanning method through 6.4 scattering path lengths. (f) Camera image recorded through 6.4 scattering path lengths.

Download Full Size | PDF

To quantitatively evaluate the effect of random scattering on imaging quality, we positioned a plastic ruler at a fixed position and gradually increased the milk concentration and measured the scattering path lengths (Fig. 5). We show the images recorded at 0, 3.5, 5.3, 6.1 and 7.0 scattering path lengths. The scattering induced background gradually overwhelmed the spatial information on the camera (Fig. 5(a)-5(e)), even with the help of averaging 10 (Fig. 5(d)) and 20 (Fig. 5(e)) frames. In drastic comparison, the off-axis laser scanning multimode detection can always produce clear high contrast images. Although the image signal level at 7.0 scattering path lengths is weak, an average of 20 images (Fig. 5(j)) still yield similar sharpness and contrast as that obtained through clear water.

 

Fig. 5. (a)-(e) Camera images recorded at 0, 3.5, 5.3, 6.1, 7.0 scattering path lengths, respectively. (f)-(j) The corresponding images recorded by the off-axis scanning multimode detection method. The images shown in d, e, and j are the average of 10, 20 and 20 image frames, respectively.

Download Full Size | PDF

4. Discussion

Even at seven scattering path lengths, we have not yet reached the imaging limit of the off-axis scanning method. Although the signal has been significantly attenuated by the scattering (14 scattering path lengths in round trip), there is no noticeable reduction in image quality. Several approaches can be applied to improve the signal level and thus push for even greater imaging depth. First, the quantum efficiency (QE) of the employed camera is ∼ 30% at 592 nm. Cameras with above 80% QE can greatly increase the signal level. Second, the beam splitter used for the interferometry has 50% coupling efficiency. Beam samplers with 1% reflectivity can be used instead to double the signal level at the camera. Third, the maximum effective laser illumination power was 85 mW in the experiment. A more powerful laser source can also help to yield a greater signal level.

To adapt the system for practical implementation on vehicle, we need to make several adjustments. First, with the long distance (tens of meters) between the viewing plane and vehicle, we can abandon laser interferometry and directly employ nanosecond pulsed light source and electronic gating on the detected signal as in LIDAR imaging [18]. Fast analog switch can easily offer nanosecond scale gating time. Second, we can position the detector at the image plane (the focal plane of L3 in Fig. 2) directly to implement multimode detection and position an iris in front of the detector to control the number of detected modes. Third, we can employ longer wavelength for its inherently greater scattering path length. Fourth, instead of displaying raw measurement images (e.g. Figure 4 and 5) to the user, we can employ model based image processing [20] to further improve the image quality, especially for the case of weak signal level. Fifth, as the illumination module and the detection module will be located on each side of the vehicle, we can use two separated synchronized beam scanners to scan the common focus (Fig. 1).

5. Conclusion

In summary, we present an off-axis spatiotemporally gated multimode detection method towards deep fog imaging for applications on vehicles. We evaluated the imaging performance using fog mimicking liquid droplet suspension. At seven scattering path lengths, the method can still yield sharp high contrast images as if imaging through clear water. Further improvement on signal detection efficiency and illumination power can push for even greater imaging depth. Using pupil plane interferometry, we digitally select the multiple spatial modes of interest for detection. The incoherently combined multimode signal offers great tolerance to optical aberration and potentially optical focus position jitter. Through experiment, we found that an additional benefit of the incoherent multimode detection is the more uniform image profile. In comparison, the single mode detection based images appear specklier. Further engineering adaption such as the electronics based temporal gating and the synchronized dual scanners are expected to enable practical system for applications on vehicles. As imaging through many other types of random scattering media such as smog is similar to the situation of fog, the same imaging approach may also be adopted for deep imaging through these scattering media or underwater imaging as well. Moreover, the pupil plane interferometry based multimode detection may also help biomedical imaging techniques such as confocal reflectance imaging and optical coherence tomography to achieve better imaging performance.