Open Access
Add to BibSonomy
Abstract
We present a short-wave infrared (SWIR) single photon camera based on a single superconducting nanowire single photon detector (SNSPD) and compressive imaging. We show SWIR single photon imaging at a megapixel resolution with a low signal-to-background ratio around 0.6, show SWIR video acquisition at 20 frames per second and 64x64 pixel video resolution, and demonstrate sub-nanosecond resolution time-of-flight imaging. All scenes were sampled by detecting only a small number of photons for each compressive sampling matrix. In principle, our technique can be used for imaging faint objects in the mid-IR regime.
1. Introduction
Single-photon detection offers high timing resolution [1] and detection of the photon flux at the shot-noise limit [2]. The combination of both these features yields a detection system that can measure the photon’s arrival time from an object with very low photon return flux [3]. When spatial resolution is desired, an array of single photon detectors serves as a camera, still maintaining shot-noise limited detection along with good timing resolution. In addition, an array of single photon detectors may serve as part of a sensitive spectrometer operating at the shot-noise limit, able to resolve the time-of-arrival of the single photons simultaneously as well as their spectral information. Ideally, for this application, a single photon detector array would cover a large spectral range to allow for hyper- or multi-spectral imaging. When combining spectral and spatial resolution, such an array can be utilized for 3D-ranging imaging applications where the measurement of the emission/return spectrum may simultaneously be acquired. To probe the emission spectra of various objects, compounds or molecules at a distance, the system should be sensitive in the short-wave and mid-wave infrared regime of the electromagnetic spectrum. However, there is no array of such single-photon sensitive detectors available beyond 1700 nm. Silicon based technologies are available, but only offer a narrow range of accessible wavelengths (400 nm - 1000nm) [4, 5] and InGaAs based technologies are sensitive out to about 1700 nm [6].
Superconducting nanowire single-photon detectors (SNSPDs) are a good candidate for detection of single photons beyond 1700 nm. Recently, a single-photon imager was demonstrated based on this superconducting detector technology [7]. The SNSPDs were also recently implemented in an 8 × 8-pixel array enabling low-resolution, real-time imaging [8]. Boosting the resolution, i.e. number of pixels, of such devices to beyond 1 megapixel will require years of development. However, when utilizing compressive imaging, the scene can be acquired with megapixel-resolution using only one single detector [9–11]. Compared to InGaAs based technologies, SNSPDs can be operated in a free-running mode where no gating is required. In addition, the spectral responsivity for 1550 nm optimized SNSPDs has been shown to reach into the mid-IR regime [12]. When optimizing the SNSPDs for the mid-IR, detection of single photons out to 7 μm was recently achieved [13]. Furthermore, SNSPDs can offer excellent timing resolution [14] and extremely low dark count rates [15]. Also, work is currently underway to develop compact cryocoolers for ease of use and field-deployment [16]. All these features make the use of SNSPDs with compressive imaging an attractive application for single-photon imaging in and beyond the SWIR regime.
Utilizing a single SNSPD and compressive imaging, we present a single-photon camera capable of video acquisition, megapixel and sub-nanosecond timing resolution in the short-wave infrared (SWIR) regime up to 2500 nm.
2. Theory
In the following, we briefly describe the current compressive imaging theory and our algorithm for sparse successive image (video) reconstruction. Compressive imaging utilizes the sparsity of a scene by finding the major components of a small basis set representing the scene [9]. Most commonly, a single pixel detector is used for detection, and a digital micro mirror device (DMD) chooses a subset of a quasi-random orthogonal sampling matrix with known basis set. We can represent the image as a single vector (x) of length N2, where N2 is the resolution of the DMD, e.g., 256 × 256 pixels (N = 256):
where Φ represents an M × N2 matrix of sampling vectors, ϕ, each with length N2, and M is the number of sampling vectors used for the image acquisition. Each vector ϕ is reshaped into an N × N pattern that will be uploaded onto the DMD and intercept the image’s information (intensity) content. The measurement outcomes y represent the projection of the image vector onto the sampling vectors, thus the total reflected intensities. As sampling vectors, we use a quasi-random arrangement of + 1 and −1, based on Sylvester-Hadamard (SM) matrices of size N2 × N2. The use of these matrices allows fast reconstructions due to existing fast Hadamard transforms. Since a raw SM matrix has repetitive patterns, a random permutation of all columns and a subsequent random choice of M rows are required to obtain the quasi-random sampling vectors ϕ [9]. To translate the ± 1-SM matrix representation to the on/off (1/0) setting choices of the DMD, we use a positive and a negative projection of the same sampling vector in sequence. By subtracting the two measurement outcomes we obtain a background-subtracted measurement outcome equivalent to probing the image with the ± 1-SM matrix representation [17].Compressing images can be effectively used with minimal loss of information for sparse scenes. Sparsity (or approximate sparsity) states that a signal can be well represented with few signal components within an appropriate basis. As a result, sparse scenes (images) can be distinguished from noise because of this structure. Instead of raster-scanning the scene and requiring N2 measurements, one can acquire the image with M < N measurements for complicated scenes and M << N for well-structured signals. After performing a still image acquisition, we analyze the measured intensities using the TVAL3 algorithm [18], where we solve the TV/L2 + problem.
Additionally, compressed sensing can be leveraged to obtain images in rapid succession to form video. With sufficient frame rates, the temporal gradient between frames should also be sparse and help with smooth image transitions. We modified the total-variation definition used in the still image reconstructions to be that of a weighted anisotropic total-variation operation defined as
where is a finite gradient operator, T is a transpose operation, ||·||1 is the L1-norm, x and y are the two spatial degrees and t is the temporal degree of freedom, respectively. Note the larger weight given to the temporal sparsity. The factor of 2 was determined empirically and allowed for a continuous flow of the video while reducing the frame-to-frame noise. Drawing from [19], augmented Lagrangian algorithms specifically used with circulant sensing matrices were altered to use fast-Hadamard transforms. Additionally, the gradient operators were designed with periodic boundary conditions such that video reconstructions could be accomplished with only fast-Hadamard transforms, fast-Fourier transforms, gradient calculations, and soft-thresholding.3. Experiments and results
3.1 Still image and real-time video acquisition
We present experimental results demonstrating the performance of our system. Figure 1 shows the experimental setup for imaging a vintage light bulb filament in the SWIR regime. The objects are projected onto a DMD using a 69 mm focal-length imaging lens. The DMD samples the objects’ projection with many quasi-random sampling patterns at the desired resolution. The raw-pixel resolution of our DMD is 1920 × 1080 pixels. However, we are only using a subset of the available DMD pixels, a square pattern of 1024 × 1024 pixels. To reduce the raw-pixel resolution of our DMD, we collate S × S raw DMD pixels into one super pixel. For example, a 64 × 64 image will occupy the 1024 × 1024 area of the DMD with a super pixel size (S × S) of 16 × 16. A two-lens collection setup is used to couple the reflected intensity into a 17 μm core diameter multimode fiber. A series of ND filters serve as attenuators. A 1450 nm long pass filter ensured that only SWIR-photons will enter the multimode fiber. The silica multimode fiber had a cutoff at 2500 nm, resulting in a photon detection region from 1450 nm to 2500 nm. Our SNSPD operates in an optical cryostat at approximately 1 K. The SNSPD was optimized for 1550 nm [20], but was still sensitive out to 2500 nm [12].
Fig. 1 Experimental Setup. An object is projected onto the DMD. Collection optics guides the reflected photon stream into a multimode fiber and towards a superconducting nanowire detector (SNSPD), cooled to 1 K. The data collection is done using a standard single photon counter (SPC) or via a custom digital to analog converter (DAC) and data acquisition hardware (DAQ).
For the experimental demonstrations, we use two different data collection setups. We use a commercially available photon counter to measure the reflected intensity off the DMD when acquiring still images. However, the data transfer rate of the photon counter is limited. Therefore, for SWIR video acquisitions we used a custom digital-to-analog converter, converting the detected photon detection events into an analog voltage. Using a data-acquisition system, sampling at 2 MHz, allowed us to record the analog voltage stream. This quasi real-time acquisition allowed us to flash DMD patterns at a rate of about 10 kHz while counting the detected photons for each pattern. The image processing is performed after all the data was acquired, and we use the TVAL3 algorithm [18] to reconstruct still images while our custom TV-minimization algorithm was used to reconstruct video.
Figure 2(b) shows a 1024 × 1024 image of a vintage light bulb filament (a conventional photograph of the light bulb with its filament is shown in Fig. 2(a)). The image was acquired using 41943 quasi-random sampling matrices, and therefore a 4% sampling ratio compared to raster scanning. Note that a total of 83886 measurements were needed to project into the ± 1-SM matrix representation. The background-subtracted average photon count rate was 470000 s−1, the average background count rate was 730000 s−1 and the integration time per DMD pattern was 100 ms. Most of the relatively large background originated from the black body radiation emitted by the DMD and its surrounding non-active areas. A smaller fraction of the background rate originated from the multimode fiber, about 70000 s−1. However, even though the background is larger than the signal, a good reconstruction of the image is possible. The fact that the ± 1-SM matrix representation requires the subtraction of two signals resulting from subsequent DMD patterns, rejects the background and intensity variation within the pattern flash rate except for the associated shot noise. This demonstration shows that this method can tolerate large-background environments if the signal due to projective measurements is above the shot noise originating from background and signal combined. The total image acquisition time was 2.3 hours, neglecting the time required for data transfer. The long integration time stems from the 83886 measurements at an integration time of 100 ms. Below we show video acquisitions using much shorter integration times, when the DMD was flashed at a rate of about 10 kHz. The reconstruction of the 1024 × 1024 image shown in Fig. 2(b) took about 160 s on a 6-core desktop PC, running at 3.33 GHz. When sampling with lower sampling ratios, the filament can still be discerned, as shown in Figs. 2(c) and 2(d). Both images were taken with 1% and 0.25% sampling ratio, respectively.
Fig. 2 Experimental results of still images. (a) Conventional photograph a vintage light bulb and its filament. (b)-(d) 1024 × 1024 images of the vintage light bulb filament for sampling ratios of 4%, 1% and 0.25%, respectively.
The above results show the capability of acquiring SWIR regime images with a single pixel SNSPD. However, in those cases the total image acquisition time was long and limited to the single photon counter response time. To speed up the image acquisition time and allow for real-time video acquisition at 10 frames per second (fps), we implemented an 8-bit, 1.3 GHz digital counter converting the single photon pulses into an analog output voltage. The resulting voltage ramp was used to determine the photon count for each DMD pattern. However, while this method offers potential for high speed acquisition, the timing resolution is lost. To capture the timing information along with the intensity at high photon rates, fast data-throughput time taggers will be required.
We acquired two 10 s, 10 fps videos of the vintage light bulb filament at a resolution of 32x32 and 64x64 pixels. The 64x64 pixel video data was then further processed to achieve a video rate of 20 fps. Visualization 1, Visualization 2, Visualization 3, and Visualization 4 accompany this paper in the supplemental material. The videos were taken with an average detected photon count rate of 1.4·106 s−1. The sampling ratio, photon counts and background counts per DMD pattern are summarized in Table 1.
Table 1. Summary of sampling ratio, photon and background count per DMD pattern for 10 fps and 20 fps video acquisition at a resolution of 32x32 pixels and 64x64 pixels, respectively.
For both 10 fps videos, 960 patterns were flashed onto the DMD within the 100 ms integration time for each of the 100 video frames. Thus, the DMD patterns were flashed at a rate of 9.6 kHz. Since the full-array DMD switching time is about 56 μs, we only used the remaining 48 μs for integration of the photon count at each DMD pattern. The average photon count per DMD pattern during the bright phase of the light bulb was about 150, and the background count was about 10. For the video acquisitions, we reduced the background count rate (compared to the still image acquisition) to the natural background count rate emitted by the multimode fiber using better spatial filtering and by placing a neutral density filter in front of the DMD. Table 1 also shows the summary of parameters for the video at 20 fps. In this case, we used the data acquired for the 10 fps video, but divided the number of patterns flashed onto the DMD for one video frame by two, therefore doubling the video frame rate in post-processing. Good reconstruction of the light bulb filament is still possible at this frame rate and with an average photon count of 77 photons per DMD pattern. We speculate that at higher resolution, e.g. 128 × 128, the average detected photon number per DMD pattern will further decrease due to the sparsity of the imaged scene. The sparsity of the scene plays a dominant role for the minimum number of detected photons required to enable reconstruction of the scene. If the return photon count signal variation due to random DMD patterns is above the shot noise of the background and signal combined, a reconstruction of the scene should be achievable.
3.2 Time-of-flight measurements
In the preceding section, we demonstrated video and still image acquisition in the SWIR regime, where the spatial degree of freedom was measured for each image or video frame. Since our SNSPDs have exquisite timing resolution, we can perform single photon time-of-flight measurements, in principle capable of acquiring the depth of a scene [10]. To demonstrate the time-of-flight measurement capability of the system, we use a fiber spectrometer [21] and compressive imaging to probe the spatial-spectral distribution of a pulsed laser illuminating a diffuser. The measurement setup is shown in Fig. 3.
Fig. 3 Experimental setup to measure the time-of-flight on sub-nanosecond time scales. Distinct frequency modes of a pulsed laser with 10 MHz repetition rate are converted into distinct time modes by use of a dispersion compensation module. The resulting light is directed to a diffuser and subsequently imaged onto a DMD. The return photon signal is collected into a single mode fiber and detected with an SNSPD. Time tagging electronics allowed sorting the returned photons into individual time bins.
We used a 100 ps-pulse laser diode with distinct frequency modes, about 1 nm apart. The output spectrum, measured with an optical spectrum analyzer, is shown in the inset of Fig. 3. Sending the laser pulses through a dispersion compensation module (DCM) translates the frequency modes into distinct time modes. This setup is similar to the fiber spectrometer used in earlier work to measure the output spectrum of spontaneous parametric down-conversion processes [21–23]. The DCM had a dispersion of about 1 ns/nm, thus transferring the distinct frequency modes into distinct time modes, about 1 ns apart. After the DCM, the light was free-space coupled onto a diffuser, and the speckle pattern was imaged onto the DMD. The reflected light was coupled into a single mode fiber and directed to an SNSPD. The laser output trigger served as a timing reference and trigger for our time-tagging electronics. All return photons were time-stamped and subsequently sorted into time bins. We flashed 19000 patterns onto the DMD while integrating the photon return flux for 100 ms for each DMD pattern. The DMD resolution was 512 × 512 pixels, resulting in a sampling ratio of 3.6%. We then used our custom TV-minimization algorithm to reconstruct an individual image for each time bin. We chose a time bin width of 128 ps, compatible with the temporal jitter of our SNSPDs. A compilation of 275 reconstructed frames (35.2 ns total length) into a video is accompanying this paper in the supplementary material.
Figure 4(a) shows the full-resolution diffuser speckle pattern during illumination with the laser pulse at a wavelength of about 1543 nm, corresponding to the setup detection time of about 15 ns. The speckle pattern is clearly visible and changes in the speckle pattern can be observed as a function of wavelength in the accompanying video. Figures 4(c)-4(f) show a zoom-in into the center region of the diffusor. The images were reconstructed at sampling ratios of 3.6%, 1.8%, 1.2% and 0.9%, respectively. We lowered the sampling ratio on the same data set using less measurements for the individual reconstructions. The full-resolution speckle pattern at the peak intensity shown in Fig. 4(a) was reconstructed with a total photon detection count of 107, thus an average detected photon count of about 526 photons per DMD pattern. The total photon count for all 275 frames was about 3·108, resulting in an average detected photon count of about 57 per frame and DMD pattern. Figures 5(a)-5(d) show the acquired time histograms for the first four DMD patterns. A large variation in the time-dependent signal can be observed. This means that at the resolution of the DMD the spatial-spectral distribution varies significantly, which is not surprising since we are observing a laser speckle pattern, where the spatial distribution strongly depends on the wavelength. Figures 5(e)-5(h) show a zoom-in into the central region of the diffusor at a system detection time of 5.2 ns and sampling ratios of 3.6%, 1.8%, 1.2% and 0.9%, respectively. At the system detection time of 5.2 ns, the average number of detected photons per DMD pattern is 59. Analyzing the photon statistics for all DMD patterns, we find that the standard deviation in the number of detected photons is about 39, a factor of 5 larger than the shot noise of the average detected photon count.
Fig. 4 (a) 512x512 pixels image reconstructed at the brightest frame system detection time of 15 ns, corresponding to a wavelength of 1543 nm. The speckle pattern from the diffuser can clearly be observed. (b) Pulsed laser spectrum (in time) after the DCM. A video accompanies this paper and shows the image reconstructions at all time bins. The time bin width was 128 ps. The red lines correspond to a system detection time of 5.2 ns and 15ns, respectively. (c)-(f) Zoom into the central region of the diffusor at the brightest frame system detection time of 15 ns at sampling ratios of 3.6%, 1.8%, 1.2% and 0.9%, respectively.
Fig. 5 (a)-(d) Acquired time histograms for the first four DMD patterns. (e)-(h) Zoom into the central region of the diffusor at a system detection time of 5.2 ns and sampling ratios of 3.6%, 1.8%, 1.2% and 0.9%, respectively
4. Conclusion
In conclusion, we demonstrated sub-nanosecond single-photon imaging in the SWIR regime using superconducting nanowire single-photon detectors and compressive imaging. We also showed single-photon megapixel image acquisition with large background, video acquisition and time-of-flight imaging with detected mean photon numbers of less than 100 photons per DMD frame. In addition, due to the responsivity of the SNSPDs in the mid-IR regime, this technology can be used for faint light image acquisition in the mid-IR regime where large background signals may be present.
Funding
NIST Quantum Information Science Initiative (QISI).
References and links
1. E. A. Dauler, A. J. Kerman, B. S. Robinson, J. K. W. Yang, B. Voronov, G. Goltsman, S. A. Hamilton, and K. K. Berggren, “Photon-number-resolution with sub-30-ps timing using multi-element superconducting nanowire single photon detectors,” J. Mod. Opt. 56(2-3), 364–373 (2009). [CrossRef]
2. R. H. Hadfield, “Single-photon detectors for optical quantum information applications,” Nat. Photonics 3(12), 696–705 (2009). [CrossRef]
3. A. McCarthy, N. J. Krichel, N. R. Gemmell, X. Ren, M. G. Tanner, S. N. Dorenbos, V. Zwiller, R. H. Hadfield, and G. S. Buller, “Kilometer-range, high resolution depth imaging via 1560 nm wavelength single-photon detection,” Opt. Express 21(7), 8904–8915 (2013). [CrossRef] [PubMed]
4. C. Niclass, C. Favi, T. Kluter, M. Gersbach, and E. Charbon, “A 128 x128 Single-Photon Imager with on-Chip Column-Level 10b Time-to-Digital Converter Array Capable of 97ps Resolution,” in 2008 IEEE International Solid-State Circuits Conference - Digest of Technical Papers, 2008), pp. 44–594. [CrossRef]
5. E. R. Fossum, J. Ma, S. Masoodian, L. Anzagira, and R. Zizza, “The Quanta Image Sensor: Every Photon Counts,” Sensors (Basel) 16(8), 1260 (2016). [CrossRef] [PubMed]
6. M. A. Itzler, M. Entwistle, U. Krishnamachari, M. Owens, X. Jiang, K. Slomkowski, and S. Rangwala, “SWIR Geiger-mode APD detectors and cameras for 3D imaging,” in SPIE Sensing Technology + Applications (SPIE, 2014), paper 91140F.
7. Q.-Y. Zhao, D. Zhu, N. Calandri, A. E. Dane, A. N. McCaughan, F. Bellei, H.-Z. Wang, D. F. Santavicca, and K. K. Berggren, “Single-photon imager based on a superconducting nanowire delay line,” Nat. Photonics 11(4), 247–251 (2017). [CrossRef]
8. M. S. Allman, V. B. Verma, M. Stevens, T. Gerrits, R. D. Horansky, A. E. Lita, F. Marsili, A. Beyer, M. D. Shaw, D. Kumor, R. Mirin, and S. W. Nam, “A near-infrared 64-pixel superconducting nanowire single photon detector array with integrated multiplexed readout,” Appl. Phys. Lett. 106(19), 192601 (2015). [CrossRef]
9. E. J. Candes, J. Romberg, and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory 52(2), 489–509 (2006). [CrossRef]
10. G. A. Howland, P. B. Dixon, and J. C. Howell, “Photon-counting compressive sensing laser radar for 3D imaging,” Appl. Opt. 50(31), 5917–5920 (2011). [CrossRef] [PubMed]
11. R. G. Baraniuk, “Single-pixel imaging via compressive sampling,” IEEE Signal Process. Mag. 25, 83 (2008).
12. F. Marsili, V. Verma, M. J. Stevens, J. A. Stern, M. D. Shaw, A. Miller, D. Schwarzer, A. Wodtke, R. P. Mirin, and S. W. Nam, “Mid-Infrared Single-Photon Detection with Tungsten Silicide Superconducting Nanowires,” in CLEO:2013, OSA Technical Digest (online) (Optical Society of America, 2013), paper CTu1H.1.
13. L. Chen, D. Schwarzer, V. B. Verma, M. J. Stevens, F. Marsili, R. P. Mirin, S. W. Nam, and A. M. Wodtke, “Mid-infrared Laser-Induced Fluorescence with Nanosecond Time Resolution Using a Superconducting Nanowire Single-Photon Detector: New Technology for Molecular Science,” Acc. Chem. Res. 50(6), 1400–1409 (2017). [CrossRef] [PubMed]
14. I. E. Zadeh, J. W. N. Los, R. B. M. Gourgues, V. Steinmetz, G. Bulgarini, S. M. Dobrovolskiy, V. Zwiller, and S. N. Dorenbos, “Single-photon detectors combining high efficiency, high detection rates, and ultra-high timing resolution,” APL Photonics 2(11), 111301 (2017). [CrossRef]
15. E. E. Wollman, V. B. Verma, A. D. Beyer, R. M. Briggs, B. Korzh, J. P. Allmaras, F. Marsili, A. E. Lita, R. P. Mirin, S. W. Nam, and M. D. Shaw, “UV superconducting nanowire single-photon detectors with high efficiency, low noise, and 4 K operating temperature,” Opt. Express 25(22), 26792–26801 (2017). [CrossRef] [PubMed]
16. V. Kotsubo, R. Radebaugh, P. Hendershott, M. Bonczyski, B. Wilson, S. W. Nam, and J. N. Ullom, “Compact 2.2 K Cooling System for Superconducting Nanowire Single Photon Detectors,” IEEE Trans. Appl. Supercond. 27(4), 1–5 (2017). [CrossRef]
17. W.-K. Yu, X.-F. Liu, X.-R. Yao, C. Wang, Y. Zhai, and G.-J. Zhai, “Complementary compressive imaging for the telescopic system,” Sci. Rep. 4(1), 5834 (2015). [CrossRef] [PubMed]
18. C. Li, W. Yin, and Y. Zhang, 18. C. Li, W. Yin, and Y. Zhang, “User’s guide for TVAL3: TV minimization by augmented lagrangian and alternating direction algorithms,” CAAM Report (2009).
19. W. Yin, S. Morgan, J. Yang, and Y. Zhang, “Practical compressive sensing with Toeplitz and circulant matrices,” in Visual Communications and Image Processing 2010 (SPIE, 2010), 10.
20. F. Marsili, V. B. Verma, J. A. Stern, S. Harrington, A. E. Lita, T. Gerrits, I. Vayshenker, B. Baek, M. D. Shaw, R. P. Mirin, and S. W. Nam, “Detecting single infrared photons with 93% system efficiency,” Nat. Photonics 7(3), 210–214 (2013). [CrossRef]
21. T. Gerrits, F. Marsili, V. B. Verma, L. K. Shalm, M. Shaw, R. P. Mirin, and S. W. Nam, “Spectral correlation measurements at the Hong-Ou-Mandel interference dip,” Phys. Rev. A 91(1), 013830 (2015). [CrossRef]
22. R.-B. Jin, T. Gerrits, M. Fujiwara, R. Wakabayashi, T. Yamashita, S. Miki, H. Terai, R. Shimizu, M. Takeoka, and M. Sasaki, “Spectrally resolved Hong-Ou-Mandel interference between independent photon sources,” Opt. Express 23(22), 28836–28848 (2015). [CrossRef] [PubMed]
23. P. S. Kuo, T. Gerrits, V. B. Verma, and S. W. Nam, “Spectral correlation and interference in non-degenerate photon pairs at telecom wavelengths,” Opt. Lett. 41(21), 5074–5077 (2016). [CrossRef] [PubMed]
