We introduce a new method of hyperspectral imaging which encodes a varying temporal intensity modulation onto the excitation (or illumination) power spectrum. Functionally, we have moved the spectrometer from the back end of an experiment to the front, where intensity modulations uniquely label wavelengths within the excitation power spectrum at different frequencies, thereby creating a temporal light label which we can identify after subsequent light–matter interactions. To demonstrate this method, we acquire two-dimensional micrographs of background-free absorption spectra by capturing the intensity modulations transferred from the excitation spectrum into the emitted fluorescent intensity. Both the temporal light labeling method and the demonstrated excitation-labeled fluorescence application are readily adaptable to hyperspectral acquisition rates far beyond the frame rates of high-speed cameras.
© 2015 Optical Society of America
Hyperspectral imaging (HI) generates a three-dimensional data set, a so-called hypercube, composed of two-dimensional (2D) spatially resolved spectroscopic data. Owing to the richness of information contained in the hypercube, HI has found widespread use in a diversity of fields from biomedical applications such as clinical cancer pathology [1,2] to remote sensing . Within biomedical applications, HI is a noninvasive diagnostic technique  using tissue reflectance, absorption, scattering, and fluorescence as indicators for disease progression . Taken together, the different optical characteristics provide a more comprehensive assessment of the specimen such as quantifying changes in cellular morphology (e.g., nuclear content and epithelial thickening) and metabolism .
As HI is a means of adding chromatic information to a spatial imaging system, powerful hybrid systems are possible by combining hyperspectral data acquisition with advanced imaging techniques. For example, spatial frequency-domain imaging  in conjunction with HI yields depth-sectioned, high-spatiotemporal resolutions for quantifiable imaging of neurophysiology and metabolism .
Currently, the dominant HI methods are (much like traditional microscopy) point, line, and area scanning . The first two scanning methods rely on an array detector to capture spatially dispersed spectra from subsections of the total field of view, whereas area scanning captures the whole field of view while filtering and scanning the illumination spectrum.
In this Letter, we introduce a new method of performing HI and demonstrate it within a particularly novel application: background-free absorption spectra imaging. Our HI method encodes a wavelength-dependent temporal modulation onto the excitation (or illumination) power spectrum. The temporal modulations shift the spectroscopic wavelength axis into a time-domain measurement, eliminating the need to spatially disperse the light to acquire spectral information. This is formally equivalent to down-sampling the optical carrier frequency bandwidth into a much slower laboratory frequency, , well within the bandwidth of square-law detectors: where is an offset frequency, , is angular optical frequency, is the central angular frequency of the excitation pulse, and is the down-sampling coefficient. The temporal light labeling (LiLa) method, then, is a means of performing spectroscopy on a single element detector at any wavelength.
In wavelength regions lacking dense array detectors, such as the mid-infrared and terahertz, LiLa offers an elegant solution to building a spectrometer. The number of spectral points, , for a LiLa spectrometer is fully decoupled from an array detector, easily enabling systems of . Particularly in the mid-infrared, this should be a huge improvement over the current state of power spectral measurements, where the cost of adding pixels to an array detector is exceedingly expensive.
While the spectral data in this Letter are acquired at low frequencies (several Hz), this is purely a limitation of the high-speed camera frame rate used to demonstrate LiLa. In the future, the LiLa encoding rate can be boosted by with the current hardware and over by moving to resonant galvo mirrors or a polygonal mirror ( will scale inversely to the encoding rate). These faster encoding rates would enable the development of hyperspectral, confocal laser-scanning microscopes.
LiLa and area scanning alone among HI techniques are adaptable to hybrid systems when combined with nonarray detector-based, high-speed imaging systems such as spatial frequency modulated imaging (SPIFI) [8–10] or fluorescent imaging using radiofrequency-tagged emission (FIRE) . Both SPIFI and FIRE perform a similar dimensional encoding as LiLa, writing spatial information into the frequency domain, which greatly reduces the imaging system’s sensitivity to scattering in turbid media . With FIRE, 2D frame rates as high as 4.4 kHz have been reported using a single high-gain, high-bandwidth photomultiplier tube to capture the fluorescence from a galvo-scanned line focus .
Extending LiLa to single-pixel imaging, via SPIFI or FIRE type space-to-time encoding techniques, increases the information content in the electronic frequency domain by exploiting the large electronic detector bandwidth. The LiLa spectrum in the modulation frequency domain can be multiplexed into modulation frequencies encoding spatial location so that the information can be simultaneously encoded in the detector signal.
The primary difference between LiLa and area scanning is the integrated illumination time per spectral point, because LiLa is a multiplexed form of area scanning. For a fixed total scan time, a LiLa style measurement will increase the integrated illumination over area scanning by a factor of . The LiLa signal-to-noise ratio improvement over area scanning then, in the white-noise limit, scales as , which equates to large improvements for systems where is .
LiLa is also a means of multiplexing nonlinear imaging modalities such as stimulated Raman scattering. Recent work by Liao et al. captured stimulated Raman loss across a broad spectrum pump field with a photodiode array and a custom-designed, 16-channel tuned amplifier array . Using LiLa to modulate the broad pump field and detecting stimulated Raman gain in a narrowband Stokes field could dramatically increase the resolution in the captured Raman spectrum, again because is not coupled to the number of detectors.
We demonstrate our HI technique in a wide-field, epifluorescence microscope, where the hypercube contains a 2D micrograph of background-free absorption spectra by virtue of the broad spectrum fluorescence containing the temporal intensity modulation encoded into the excitation spectrum. We dub this method HI via excitation-labeled fluorescence (HI-ELF). Figure 1 is a concept figure, showing how the unique intensity modulation frequency at each wavelength of the excitation pulse is transferred to absorbers and maintained through fluorescent emission. The net result of HI-ELF is a time-domain encoding of the absorbed spectral energy, detectable through temporal modulations in the emission intensity integrated across the entirety of the fluorescence spectrum. By detecting the absorption spectrum through temporal modulations in the fluorescence spectrum, we drastically lower the noise floor of an absorption measurement, as it can be shot-noise limited for small changes to the average of the small fluorescent intensity, , as opposed to small changes on top of the large background excitation intensity, . Using HI-ELF, we are able to reliably differentiate between absorbed spectra whose centroids differ by only 3 nm.
The temporal LiLa technique relies on encoding a wavelength-dependent temporal intensity modulation onto the excitation power spectrum prior to the microscope. We accomplish this by passing the excitation pulse through a folded Martinez stretcher with a frequency-modulated reticle  at the focal plane; see Fig. 2(a) for a reference schematic. The spinning-disk reticle [Fig. 2()] modulates the spectral intensity of wavelengths spatially dispersed across the line focus at distinct frequencies, creating a unique mapping between intensity modulation frequency and wavelength.
The spectral resolution, , contained in the temporal modulation frequencies is dependent on the focal spot size of the monochromatic wavelets comprising the spectral line focus and the smallest reticle feature size, where the larger of the two sets the resolution. The focal spot size of the monochromatic wavelets is estimated using the paraxial approximation, where the intensity distribution of the spectral line focus impinging on the reticle at the focal plane of the Martinez stretcher is given by , where is the angular frequency bandwidth of the pulse and is the radius of the electric field of the first-order diffracted beam, leaving the grating with an associated Rayleigh length of . Monochromatic wavelets at propagate parallel to the optical axis of the wavelet at , but are displaced spatially by , where , is the speed of light, is the grating groove density, is the first-order diffraction angle of the wavelet at , and is the focal length of the achromatic lens in the Martinez stretcher.
The reticle feature resolution is determined by its spatial modulation pattern, which is a radially dependent cosine mask expressed as , where is the number of cycles per unit radius (i.e., the chirp parameter), is the rotation frequency of the reticle, and enforces binary amplitude modulations. Making the reticle symmetric about the line with polar coordinates [the vertical symmetry plane in Fig. 2() with oriented downward], has the effect of making the smallest feature on the mask equal to . In our mask, , making the smallest feature size 200 μm. Using and , the monochromatic wavelet focal spots are full-width at half-maximum, well below the smallest feature size of the mask. The spectral resolution in this implementation is therefore set by the spatial resolution of the reticle, , and is given by . Using our system properties of , a grating incidence angle of 17.5°, and , the LiLa spectral resolution is .
The number of spectral points comprising the LiLa spectrum is given by , where is the total spectral bandwidth of the excitation source (i.e., the width encompassing of the power spectrum) and is the modulation frequency resolution and is equal to . The modulation frequency bandwidth encoded across the spectral bandwidth is , where is the width of the spectral line focus incident on the reticle. Using , the number of spectral points in our setup is .
To represent the LiLa spectrum in the modulation frequency domain, we need to physically locate the spectral line focus on the reticle. The radial position of the central frequency wavelet on the reticle sets , the central modulation frequency. This parameter is adjustable by translating the reticle transverse to the spectral line focus. The spatial location of focused wavelets, , becomes , where we have oriented the Martinez stretcher and reticle such that the spectral line focus occurs along the dimension. In the limit where the monochromatic wavelet focal spot sizes are much smaller than the smallest feature on the reticle, the mask modulation is transferred directly into the frequency domain, , as the spatial variations in the mask correspond to spatially isolated (focused) monochromatic wavelets. The excitation intensity after the Martinez stretcher, where the power spectrum has been spatially recompressed, is . Capturing the full spatial and spectral extent of the excitation intensity on a photodetector yields a voltage signal , where accounts for the quantum efficiency of the detector, collection efficiency, and other factors. The first sideband of the Fourier transform of , , is the excitation power spectrum in the modulation frequency domain: the LiLa spectrum.
In using the paraxial approximation to express the spectral intensity through the Martinez stretcher, there is the implicit assumption that monochromatic wavelets at all the field angles subtended by the diffracted power spectrum are focused with minimal aberrations, especially Petzval field curvature . This approximation quickly breaks down for a system with large field angles (e.g., for the achromatic lens in our setup), as the field curvature becomes commensurate with the Rayleigh length of focused wavelets with the highest field angles within the power spectrum. Modeling our system in Zemax, with maximum field angles of , we find that the field curvature across the spectral line focus is less than the respective wavelet Rayleigh lengths everywhere.
To calibrate the wavelength to intensity modulation frequency mapping, we step a razor blade through the spectral line focus and colocate the edge in the transmitted power spectrum between a wavelength-domain measurement (a traditional spectrometer and a stationary reticle) and a modulation frequency-domain measurement (a single-element photodetector and rotating reticle, i.e., LiLa). Figure 2(b) shows resultant position to wavelength and modulation-frequency calibration curves. The theoretical spatial locations of the wavelengths, , are also shown in Fig. 2(b) as red circles. The root-mean-squared error between measured wavelengths and the LiLa calibration is 0.2 nm. Figure 2(c) shows the excitation pulse power spectrum captured by temporal LiLa as compared to the power spectrum captured by a spectrometer with a static reticle set to an angle with high modulation [ from the orientation shown in Fig. 2()]. As a redundant check, we can recompute the number of spectral points using as determined from in Fig. 2(c): at yields .
The Martinez stretcher and reticle constitute a pulse shaper with a rapidly varying amplitude mask, a system that is very similar to pulse-shape modulation systems used for nonlinear signal optimization  and Fourier-transform spectroscopy. Our setup also generates pulse pairs based on the time-varying amplitude modulation density at the Fourier plane of the Martinez stretcher, but we are leveraging a wholly different and new capability than pump-probe or Fourier-transform spectroscopy. We are utilizing the down-sampling of the optical carrier frequency bandwidth by (at ) to directly capture it with a single-element square-law detector. Faster systems such as a reticle motor spin rate of or a polygonal mirror would shift to , respectively, increasing the possible image acquisition rate.
For HI-ELF, the captured signal on a camera from a wide-field microscope becomes , where is the magnitude of the down-telescope between the input lens and the objective (Zeiss numerical aperture), is the image magnification onto the camera between the tube lens and the objective, and . is the spatial distribution of fluorophores, where represents different fluorophores and includes cross-sections, emission spectra transmission through filters, and other such factors. The captured fluorescent emission intensity retains the encoded excitation frequency modulations, enabling fluorescent species identification based on the absorbed spectra recovered from the LiLa signal.
We dyed two separate pieces of lens tissue with Alexa Fluor 514 and 546 and situated them in close proximity to one another underneath a cover slip. As an excitation source, we frequency-doubled 25 fs pulses at 1075 nm with 10 nJ of pulse energy  in a 250 μm potassium–dihydrogen–phosphate crystal. The two dyes’ absorption spectra cut off and on in roughly the middle of the second-harmonic generation spectrum. HI-ELF images were acquired using the setup in Fig. 2(a) with a camera (Andor Neo) operating at a frame rate of 735 Hz over a pixel area of and , which keeps the highest frequency in the excitation LiLa spectrum below the Nyquist limit of the camera frame rate. The signal levels on the camera were kept beneath saturation, leaving 16 bits of dynamic range. Image-processing flowchart and resultant HI-ELF are shown in Fig. 3.
To generate the absorbed HI hypercube, , we Fourier-transform the temporal signal at each pixel, , with a noise-reducing gate function. As our excitation spectra did not fully enclose the absorption spectra of the two fluorophores, we use the unabsorbed spectra as the contrast mechanism. The difference between an appropriately scaled reference spectrum and the measured absorbed spectrum at each pixel constitutes an unabsorbed spectrum. A surface reflection from a wedge captured on a photodetector generates the reference excitation LiLa spectrum. The reference spectrum is scaled for each pixel such that, within modulation-frequency windows [black-dashed boxes in Fig. 3(c)], it circumscribes the absorbed spectrum. Reference unabsorbed spectra for the two dyes are shown in Fig. 3(d). Fitting the unabsorbed spectra of each pixel with the reference unabsorbed spectra scaled by coefficients generates a species map. The fit parameters are found using a least-mean-squared-error search. The integrated absorbed power spectrum of each pixel, Fig. 3(e), is then scaled by , creating two fluorescent species labeled micrographs. A combination image is shown in Fig. 3(f), with the Alexa Flour 514 and 546 coefficients taking blue and red color schemes, respectively.
We have introduced a new method of HI using temporal LiLa. The potency of this new technique is demonstrated by performing background-free absorption imaging via ELF. Merging the LiLa method for acquiring spectra with SPIFI or FIRE will enable a vast increase in frame rate, well beyond the limits of array detectors.
Colorado State University (CSU).
1. D. G. Ferris, R. A. Lawhead, E. D. Dickman, N. Holtzapple, J. A. Miller, S. Grogan, S. Bambot, A. Agrawal, and M. L. Faupel, J. Low. Genit. Tract Dis. 5, 65 (2001).
2. Y. Zhang, Y. Chen, Y. Yu, X. Xue, V. V. Tuchin, and D. Zhu, J. Biomed. Opt. 18, 077003 (2013). [CrossRef]
3. Z. Lee, K. L. Carder, C. D. Mobley, R. G. Steward, and J. S. Patch, Appl. Opt. 38, 3831 (1999). [CrossRef]
4. T. Vo-Dinh, Biomedical Photonics Handbook: Biomedical Diagnostics (CRC Press, 2014), Vol. 2.
5. J. R. Weber, D. J. Cuccia, W. R. Johnson, G. H. Bearman, A. J. Durkin, M. Hsu, A. Lin, D. K. Binder, D. Wilson, and B. J. Tromberg, J. Biomed. Opt. 16, 011015 (2011). [CrossRef]
6. D. Abookasis, C. C. Lay, M. S. Mathews, M. E. Linskey, R. D. Frostig, and B. J. Tromberg, J. Biomed. Opt. 14, 024033 (2009). [CrossRef]
7. G. Lu and B. Fei, J. Biomed. Opt. 19, 10901 (2014). [CrossRef]
8. G. Futia, P. Schlup, D. G. Winters, and R. A. Bartels, Opt. Express 19, 1626 (2011). [CrossRef]
9. D. G. Winters and R. A. Bartels, Opt. Lett. 40, 2774 (2015). [CrossRef]
10. D. J. Higley, D. G. Winters, G. L. Futia, and R. A. Bartels, J. Opt. Soc. Am. A 29, 2579 (2012). [CrossRef]
11. E. D. Diebold, B. W. Buckley, D. R. Gossett, and B. Jalali, Nat. Photonics 7, 806 (2013). [CrossRef]
12. C.-S. Liao, M. N. Slipchenko, P. Wang, J. Li, S.-Y. Lee, R. A. Oglesbee, and J.-X. Cheng, Light Sci. Appl. 4, e265 (2015). [CrossRef]
13. J. S. Sanders, R. G. Driggers, C. E. Halford, and S. T. Griffin, Opt. Eng. 30, 1720 (1991). [CrossRef]
14. S. R. Domingue and R. A. Bartels, Opt. Lett. 40, 253 (2015). [CrossRef]
15. E. Frumker, E. Tal, Y. Silberberg, and D. Majer, Opt. Lett. 30, 2796 (2005). [CrossRef]