Open Access
Add to BibSonomyAbstract
Spectral imaging is a powerful tool for providing in situ material classification across a spatial scene. Typically, spectral imaging analyses are interested in classification, though often the classification is performed only after reconstruction of the spectral datacube. We present a computational spectral imaging system, the Adaptive Feature-Specific Spectral Imaging Classifier (AFSSI-C), which yields direct classification across the spatial scene without reconstruction of the source datacube. With a dual disperser architecture and a programmable spatial light modulator, the AFSSI-C measures specific projections of the spectral datacube which are generated by an adaptive Bayesian classification and feature design framework. We experimentally demonstrate multiple order-of-magnitude improvement of classification accuracy in low signal-to-noise (SNR) environments when compared to legacy spectral imaging systems.
© 2016 Optical Society of America
1. Introduction
Spectral imaging is a powerful analytical tool, combining two important scientific techniques: spectroscopy and imaging [1,2]. It allows for improved discrimination of objects in a scene, as well as the ability to make in situ material classification [3–5]. In this article we introduce the Adaptive Feature-Specific Spectral Imaging Classifier—AFSSI-C—a computational imaging system which yields direct classification at each spatial location in a scene.
Spectral imagers take measurements of the spectral datacube, a three dimensional representation of the scene consisting of two spatial dimensions and a spectral dimension. Typically the measured datacube can have upwards of 109 signal elements. For example, the NASA AVIRIS system, a workhorse for Earth remote sensing, sweeps 677 linear spatial locations at 224 spectral channels [6, 7], producing 109 signal elements in under 10 minutes. Measuring each of these signal elements directly is costly in both time and data capacity due to the sheer number of signal elements and the associated transmission and storage requirements. The latter is exacerbated in the AVIRIS airborne imager because it scans continuously as it flies over an area of interest. Even the most advanced systems strive to reconstruct the spectral datacube [8–15], though many rely on compressive and computational techniques to greatly reduce capture time and data load.
We argue that the goal of the majority of spectral imaging analysis is material classification [3, 16,17]. Indeed, once the spectral datacube is reconstructed, the spectrum at each spatial location is compared to known spectra to make a classification. By arriving directly at a classification decision without direct measurement of every element in the spectral datacube, the AFSSI-C has the potential to greatly improve classfication accuracy as well as reducing transmission and storage requirements.
A challenge to all spectral imaging systems is the measurement of the three dimensional spectral datacube using at most a two dimensional detector array [1]. Traditional systems rely on scanning to overcome this restriction. Spotlight or whiskbroom systems [18] gather the spectra for a single spatial location, sweeping through both spatial dimensions. Pushbroom systems [18,19] gather the spectral information for an entire row, scanning through the remaining spatial dimension. Tuned filter systems gather the information at a single spectral channel across the entire image, scanning through the spectral dimension [20]. Each of these approaches require the system to discard a significant portion of the available light to improve resolution, either spatially, spectrally, or both. The AFSSI-C, on the other hand, benefits from an open aperture and instead relies on encoding the spectral datacube with adaptive spectral filters to achieve spatial and spectral resolution, thus utilizing significantly more light than the traditional systems mentioned above.
The Adaptive Feature-Specific Spectrometer (AFSS) was previously introduced by our group [21]. By utilizing adaptive spectral features and a Bayesian probability framework for feature design and classification, the AFSS is able to provide significantly improved spectroscopic classification accuracy over classical spectrometers without reconstruction of the source spectrum. One might naively propose extending this idea to spectral imaging by simply forming an array of AFSS systems to achieve classification across a spatial scene, though this approach would be very costly due to massive component redundancy. The AFSSI-C system provides a similar feature-based measurement approach and Bayesian framework, but via a more compact architecture. Instead of using multiple spectrometers, it relies on multiplexing and encoding of the spatial and spectral information to achieve spectral classification across a spatial scene. Not only does this approach share resources across multiple spatial locations, it avoids the expense of pixel-specific components, while still leveraging the multiplex advantage gained from the AFSS design.
With its adaptive features, multiplex advantage, and open aperture, the AFSSI-C system is a spectral imaging-classifier which greatly improves classification accuracy in low signal-to-noise (SNR) environments. This additional feature will be highlighted in both the reported simulation and experimental results. This system has the potential to benefit the myriad industries that utilize spectral imaging for material classification, such as medical diagnostics, geological exploration, art conservation, and security applications [22–26].
2. System approach
2.1. Hardware architecture and system model
The design of the AFSSI-C is, shown in Fig. 1, essentially two 4f open-aperture spectrographs arranged back-to-back. A spatial light modulator—a digital micromirror device (DMD)—divides the two halves. An objective lens (not shown) relays the object to the input plane of the instrument. The first lens of the system collimates the rays. At this point in the system we can imagine the source spectral datacube as depicted in Fig. 2(a), where x and y are the spatial axes, and λ the spectral axis. The collimated light then falls on the first grating, which spectrally disperses the light from the incident spectral datacube. This can be thought of as a shearing action on the spectral datacube. As shown in Fig. 2(b), the spectral dimension of the cube is now at an angle—in the direction of the dispersion—to the surface of the spatial plane. This sheared spectral datacube is focused to an intermediate image plane at the DMD.
Fig. 1 Schematic of the AFSSI-C. Light from the source enters from the left, where the entrance optic develops the intermediate image plane prior to lens 1. The light is dispersed by the first diffractive grating, and imaged onto the DMD by the second lens. Shown here are three spectral channels, illustrating the different positions each channel has on the DMD, all from the same spatial location (experimental system has 38 spectral channels). The DMD directs light to the dump or lens 3 on a mirror-by-mirror basis. Collimated light from lens 3 is sent through a second grating, and finally imaged onto the detector by the 4th lens.
We implement mirror patterns on the dense array of micromirrors at the DMD which either allow the light to continue to the rest of the system, or block it by sending it to a beam dump, on a mirror-by-mirror basis. This pattern encodes the datacube, portrayed in Fig. 2(c) as holes punched through the sheared cube.
Fig. 2 Visualization of datacube progression through the AFSSI-C system. (a) The input cube is (b) sheared by the first grating, (c) encoded at the DMD, and finally (d) spatially re-registered by the second grating.
The now-encoded signal is once again collimated before hitting the second grating, which is identical to the first, but with the dispersion direction flipped so as to undo the applied dispersion. This removes the shear in the encoded spectral datacube, as depicted in Fig. 2(d). The final lens in the system focuses that result onto the detector array, which integrates the cube through the spectral dimension. The result at each pixel is the inner product of the source spectrum at that location in the scene with a spectral filter—the implemented feature vector—imposed on that location by the mirror pattern on the DMD.
The choice of a dual-disperser architecture rather than single-disperser architectures such as [9] or [27] is because the dual disperser architecture implements a spatially-varying spectral encoding rather than a fully spatial-spectral encoding. The spatially-varying spectral encoding is more appropriate for parallel direct classification, as the measurements can be mapped one-to-one with the spatial locations in the scene, allowing for parallel operation of the Bayesian inference approach, rather than having to solve a single larger joint inference problem.
The mirror pattern at the DMD gives rise to the spectral filters that act at each spatial location in the scene. We turn to a system model to understand this relationship. The system model, and the relationship between the pixelation of the detector and the pixel size at the DMD, are similar to the treatment in [11]. We start with this model, and extend it to provide the restriction imposed on the feature design.
If we consider a source spectral density of S0 (x, y; λ) and assume unit magnification throughout, with pixelization of the DMD and detector array of size Δ, we can write the measurements at location (n, l) on the detector array as
Here T is an array that describes the pixelated mirror pattern on the DMD, α is the linear dispersion of the gratings, and λc is the center wavelength of the system. As will be shown later, the system relies on calibration which accommodates deviations from these assumptions.
If we analyze two constrained cases, we can gain an understanding of how the spectral filter influences the measured intensity. We first consider a monochromatic source S0 (x, y, λ) = I0 (x, y) δ(λ − λc), where I0 is the intensity distribution of the monochromatic scene. In this case, Eq. (1) simplifies to
where Inl is a spatially pixelated version of the monochromatic source with intensity distribution I0 (x, y)Now we consider a case with the monochromatic source is shifted by a spectral channel Δλ = Δ/α. Then,
where we find that a shift by a spectral channel results in a shift in the mirror pattern.For a natural scene with content spanning the spectral range, we can consider spectral channel index κ out of C total spectral channels. We can also define a discretized source spectral datacube Snlκ, and then the detector signal Γnl as a result of mirror pattern T acting on the pixelated source is
which shows the measurement at each pixel being the inner product of the source spectrum and the feature vector which results from the mirror pattern. If we look at an adjacent pixel on the detector, Γn(l+1) we find thatThe spatial location (n, l) sees the effect of the pattern in mirror locations Tnl to Tn(l+C−1), while the neighboring spatial location at n, (l + 1) is encoded by the mirror pattern from Tn(l+1) to Tn(l+C). What we notice is that the mirror pattern for neighboring pixels overlap, and have the mirror pattern from Tn(l+1) to Tn(l+C−1) in common. Essentially what each source location sees is a spectral filter formed by a length C segment of a mirror pattern which spans the entire row, with the neighboring location seeing a length C segment of that underlying pattern offset by 1. The overlap of the spatial and spectral information at the DMD disallows independent feature vectors for each spatial location. The spectral filters imposed by the DMD pattern are unique for every spatial location, but cannot be implemented independently, requiring the features to be designed jointly for spatial locations along a row.
With a system model in place, our attention turns to the feature implementation, and how to determine classification.
2.2. Feature design and decision framework
The AFSSI-C uses adaptive features designed from information in previous measurements to make a classification decision from hypothesis spectra in a spectral library. This library is common to all spatial locations, and it is helpful to first look at the process of feature design and classification decision for a single spatial location. We can then expand the analysis to include the entire scene, building from the single-location tool set.
The measurement at each detector pixel is compared to the spectral library, subject to the spectral filter implemented at that spatial location. A probability is assigned to each hypothesis in the library using Bayes’ theorem. The conditional probability of hypothesis hi given the measurement history up to the kth measurement {m}k, can be written as:
The probabilities are assigned and updated using a Gaussian noise model and log-likelihood ratios, as implemented by the AFSS in [21]. The probability assignment is on a per-pixel basis and is independent of the probability assignment of adjacent spatial locations in the AFSSI-C.
Features are designed to yield measurements which discriminate between the hypotheses with the greatest probability. To this end we use a modification of principal component analysis (PCA) that employs the updated probability estimates of each hypothesis as a weighting factor. We consider this an intuitively reasonable approach, keeping in mind that the first principal component (PC) is an eigenvector of the scatter matrix, along the direction of greatest variance in the library spectra [28]. By taking the inner product of the spectra at each location with this first eigenvector, we significantly separate the possible measurement outcomes, facilitating discrimination. Using standard PCA does not allow for informing subsequent measurements with previous results, so we modify PCA as follows.
At each individual pixel we create a scatter matrix with R spectral hypotheses, with individual spectrum sr, weighted in relation to the prior probability associated with each hypothesis given the measurement history. The individual scatter matrices Q(k) at each spatial location with spectral library element hypothesis hr take the form
Here X(k) is the matrix of weighted spectral elements, where the row index i is from 1 to the number of spectral channels C, with columns j from 1 to R as follows,
and s̄ is the probabilistically-weighted sum of the spectral library:We can then compute the first eigenvector of Q(k), analogous to PCA. We call this probabilistic-PCA or pPCA, and in the case of the AFSS this eigenvector becomes the feature for the next measurement. The scatter matrix Q(k) is then updated with every measurement.
The effect this has on the first PC is illustrated in the two-dimensional (i.e. a spectral library with two spectral channels) example in Fig. 3. In Fig. 3 (a) each hypothesis is equiprobable, whereas in Fig. 3(b) the hypotheses are probabilistically-weighted after the first measurement, showing a new direction for the first PC vector. In the case of the AFSS [21], the features derived from this PC vector allow the system to further discriminate between the most likely library spectra when making a classification decision.
Fig. 3 Depiction of the pPCA (simple 2D example). (a) First principal component before a measurement has been made. All of the hypotheses are equiprobable, depicted here as all points having the same grayscale value. (b) After a measurement has been made: the darker points are more probable hypotheses, with the less probable hypotheses taking on lighter shades of gray. The first principal component has now been shifted to the direction of greatest variation in the weighted data.
As discussed above, the AFSSI-C system architecture mixes spatial and spectral information at the coding plane in a way that requires joint design of the mirror pattern on each row. Therefore, pPCA is further modified by instead using the matrix X̃uv (k), constructed from the single-location probabilistically weighted hypothesis matrices Xij (k) from each spatial location (l′) in a row (n′). The individual Xij (k) matrices become block elements in this much larger matrix, with each block offset in v by the appropriate value of l′. The Xij (k) matrices for neighboring spatial locations share C − 1 columns in this larger matrix, which represents the overlap of spatial and spectral information at the DMD as illustrated by the system model discussion. Figure 4(b) illustrates the stacking of the elements of the mean-centered, weighted library spectra from Fig. 4(a) into a larger array X̃(k). We arrive at a joint scatter matrix similar in form to Equation (7),
where is the mean of each column in X̃, and the sum is over the library elements r̃ from all of the row locations. The first PC of Q̃(k) is a vector pointing in the direction of greatest variation in the probability-weighted data for the entire row. We repeat this process for all of the rows in the datacube to arrive at the features to be implemented for the subsequent measurement. We refer to this feature design method as joint-pPCA.Fig. 4 Joint-pPCA schematic. (a) The pictorial illustration of the formation of X(k) from Equation(7). The hypothesis spectra (1) are centered around the mean (2), and given a weight based on their probabilities (3). (b) The joint-pPCA calculation involves the stacking of the 1, . . . , C − 1 elements of Xi,j (k) for each location (n′, l′) along a row into a larger matrix X̃uv (k). The final joint-pPCA scatter matrix is formed from X̃(k) multiplied by its mean-centered transpose, to arrive at scatter matrix Q̃(k). The first principal component of this much larger scatter matrix is then in the direction of greatest variation in the joint-position data.
With the system able to take advantage of prior information to adapt features for the subsequent measurement, the AFSSI-C is able to accurately and rapidly determine pixel-level classification. For this work, the classification decision after each measurement is given by the maximum a-posteriori probability (MAP) estimate (the mode of the probability mass function) [29] of the hypothesis spectra at each location. We now look at the components and design of the experimental system.
3. Experimental system design
3.1. Hardware
Certain considerations were made before fabrication and assembly of the experimental AFSSI-C system. Here we outline some of the design decisions and consequences of those decisions.
Lenses are off-the-shelf achromatic doublets to avoid the need for custom fabrication. This also limited the degrees of freedom in the optical design, requiring the design focal lengths to match available lenses. A compact, fixed focal length, lens (Edmund Optics Stock No. 58-001, f = 12mm) is used as an objective lens, relaying the object onto an intermediate image plane. A fast entrance optic is needed to meet the layout and magnification requirements. Referring to Fig. 1, lenses 1 and 4 are 75 mm focal length achromatic doublets with a 50 mm diameter (Edmund Optics Stock No. 49-291), and lenses 2 and 3 are 150 mm focal length achromatic doublets with 25 mm diameter (Edmund Optics Stock No. 47-643).
For a repeatable, updatable synthetic datacube source, we use an LED display (Dell P2311H LED monitor with 248μm pixel pitch). Though this limits the spectra we are able to generate, it still allows for programmable spectra at every spatial location. The center 1080 × 1080 pixels are used to generate the source spectral datacubes. Spectra consist of combinations of the RGB monitor colors.
The design spectral range is 425–625nm; filters were placed in the collimated space to attenuate light outside of this range. The system is designed for 38 spectral channels, resulting in roughly 5nm/spectral channel. The system magnification to the DMD, which then dictates the lateral spread allowed per spectral channel, requires custom 0.10 lines/μm holographic blaze gratings as the dispersive elements, fabricated by Wasatch Photonics. The gratings are designed to minimize diffraction in all the orders except the first order.
The DMD is a Texas Instruments DLP Discovery 4100 DMD development kit, with the DLP9500 0.95” 1080p DMD chipset. The DMD has 1080 × 1920 10.8μm mirrors on the array, and the mirrors pivot ±12° on a diagonal axis. The DMD is oriented such that the bottom of the array is parallel to the optical table, which forces the second arm of the AFSSI-C to rise off the optical table at an angle of 17.5°; this orientation was chosen so that the mirrors would be oriented as squares rather than diamonds. While the periodic structure of the DMD can produce strong diffraction for certain types of illumination, no diffraction effects are observed in the AFSSI-C as the light striking the DMD is highly incoherent.
An SBIG ST-10XME detector with a 2184 × 1472, 6.8μm pitch CCD array is used as the detector array. Because of the angle of the optical axis at the detector, a goniometer was constructed using a rapid prototyping 3D printer to facilitate easy alignment of the CCD array.
Because we are interested in a quantitative system analysis, we compare measurements on the discretized detector to what was projected by our discretized source. The layout dimensions, component pixel size, and lens design were tuned to yield pixel groups—referred to here as system pixels—at each component that are an integer number of the component pixel, or in the case of the DMD, integer number of mirrors. We therefore have the following system pixel dimensions: 16 × 16 pixels on the source, 8 × 8 DMD mirrors, and 12 × 12 pixels at the detector.
Once the optical layout was finalized, the physical system was designed in SolidWorks. The lens mounts, detector mount, DMD mount, beam dump, and locating plates were then fabricated on a Eden 350 rapid prototyping 3D printer.
Light baffling is utilized to shield the system from ambient light as well as stray light within the system. A baffling structure was designed and printed to surround the detector, with smaller baffling elements to attenuate stray light around the optical paths.
3.2. Calibration
As previously mentioned in [30], calibration is fundamental for the functioning of a computational imaging system as the AFSSI-C. Here we will elaborate on the derived challenges.
A measurement at the detector must be compared to the source to assess system performance, but the DMD geometry creates a tilted object plane for the second arm of the AFSSI-C. The DMD plane is normal to the optical axis of the first arm, but it is at an angle (equal to twice the tilt of the mirrors on the diagonal) to the second arm. We can think of the DMD plane as the object for the second arm of the system. This object is extended along the optical axis of the second arm, which causes keystone distortion at the detector as explained by the Scheimpflug condition [31]. The distortion is rectified by a mapping routine which creates fiducial markers on the source. The points are registered to the corresponding image on the detector, and a lookup table can then be used to map detector pixels to the source locations.
The optical system has aberrations, and there are intensity variations across the measured ROI of the source. The system is therefore calibrated for spectral response. This is done any time physical adjustments to the system have been made. System calibration involves activating each of the available colors on the source, and then sweeping a single column of ‘on’ system pixels (i.e., angled to send light to the detector) across the DMD. A datacube consisting of the spectral response at each spatial location is compiled from the measurements. Knowledge of the spectral response of the system is used to accommodate aberrations and imperfect recombination of the spectral signal at the second grating.
Finally, a map of the source brightness is determined at the beginning of each experiment to account for spatial variations in the spectral throughput. This map is created by turning the entire DMD to ‘on’ and measuring the intensity at each spatial location from a white source. Subsequent measurements rely on this flat field measurement to accommodate for the spatially varying illumination response of the system.
4. Experiment motivation and technique
The goal of the AFSSI-C effort was to design and build a system that would yield direct classification and outperform traditional spectral imaging systems in classification accuracy. This comparison is to be performed via simulation, so we must validate our ability to simulate the real world performance of such a system. Thus, we first compare the experimental results of the AFSSI-C system (Fig. 5) with simulation. This allows for confident comparison to the conventional approaches.
Fig. 5 Photograph of the experimental system. The detector is high on the left; the DMD is in the background. The entrance optic is on a linear stage in the foreground
To be convinced in our experimental corroboration of the simulation results, we need to be able to claim agreement for different levels of classification difficulty. To this end, we corrupt the experimental measurements in software with white Gaussian noise, and hence we manipulate the difficulty of classification. As in [32], the experimental results reported here are for a 4-class problem, with an input spectral datacube of 64 × 64 spatial pixels and 38 spectral channels.
Figure 6 is an illustration of the spectral source used, with the associated 4-class spectral library. While this spectral datacube is small compared to those used in remote sensing, the implemented size was driven by practical considerations regarding the source, DMD and detector on hand. The dual-disperser architecture, in general, places no significant limitation on possible datacube sizes. Similarly, the processing involved in the Bayesian inference and feature design are computationally lightweight and do not create a computational limit on datacube size.
Fig. 6 Left: The source used in the classification experiments and simulations; Right: The 4-class spectral library.
Classification difficulty is quantified as the task Signal-to-Noise ratio—TSNR—which considers the noise in the system relative to the separation distance between hypotheses spectra. TSNR is defined as the ratio of the minimum separation between the spectra in the reference library to the standard deviation of the system noise in decibels. If we assume white Gaussian noise with standard deviation σn, and the minimum Euclidean distance between the hypotheses in the spectral library dmin, the TSNR is then given by the expression:
When using this definition for TSNR, a value of 0 dB TSNR is the point where the noise is equal to the minimum distance between the library elements.
In simulation we can add the noise derived from the desired TSNR; in experiment we have intrinsic system nonidealities and model mismatch that lead to a noise floor which must be accounted for before any noise is added to adjust TSNR. We assume this model mismatch to be inherent system noise which can be measured through Monte Carlo pixel-level spectral prediction experiments. These experiments generate random displays of spectra with random features at the DMD, and the deviation between the measured and predicted results is considered the system noise. Once we know the inherent system noise, different levels of TSNR above this baseline can be realized by adding additive white Gaussian noise (AWGN) to the acquired measurements.
The features must be modified because of the fact that the DMD implements binary valued mirror patterns, while the first eigenvector of Q̃(k) has grayscale values, containing both positive and negative elements. This first PC vector is modified by applying a threshold such that positive values become a 1 and negative values become −1. A bipolar measurement scheme is used: the first measurement uses the 1 values as mirror ‘on’ (pointing to the rest of the system) and the −1 values as ‘off’ (to the beam dump), and the second reverses the mirror pattern. The final result is the difference between the two detector measurements.
The traditional system simulations are based on the scanning action of each of the three systems considered, assuming every element of the spectral datacube has been measured for an equivalent measurement duration. At any given measurement duration the signal intensity for each element in the spectral datacube is reduced by the equivalent scanning mechanism. This simulation architecture is designed to consider the input of the traditional system to be identical to the AFSSI-C, with the same detector characteristics. Classification is made by finding the classification hypothesis with the least Euclidean distance to the measurement; recall that the output from a traditional system is the reconstruction of the spectrum at each spatial location, so classification involves comparing spectral vectors.
The synthetic source datacube used for the experiment is generated by first posterizing an image to the number of classes. This posterization reduces the number of tone levels in an image to match the number of classes in the experiment. Finally the appropriate spectrum at each location is generated by assigning intensities to the RGB monitor LEDs according to the posterization levels. The result is a repeatable source spectral datacube that can be compared to the measurements for system analytics.
5. System simulation and experimental results
Figure 7 presents a snapshot of a video (see Visualization 1) comparing system performance at different TSNR levels. As the classification difficulty increases, the system takes longer to correctly classify. However, the AFSSI-C feature-design process intelligently adjusts the codes to focus attention on uncertain spatial locations
Fig. 7 Frame from a movie (see Visualization 1) showing experimental results for three different levels of TSNR, over the course of 30 measurements. The left column is a depiction of the DMD code, center is the output from the detector, and the right is the classification decision at the current measurement.
Figure 8 shows the comparison between the AFSSI-C system experiment results and the simulated values. The simulated and experimental data are the average of multiple runs, i.e., 30 simulations and 10 experiments, for each of the four TSNR values representing octaves of classification difficulty (0, −3, −6, and −9dB TSNR). Figure 8 shows good agreement between the simulated and measured classification accuracy. This agreement allows us to compare the performance of the AFSSI-C with traditional systems via simulation. We can then assess the gain in performance over these traditional scanning systems.
Fig. 8 Comparison of the AFSSI-C experimental system results to the simulation results for multiple TSNR levels by plotting the classification error versus measurement. Shown are repeated experiments of a 64× 64× 38 spectral datacube and a 4-class library.
Figure 9 compares the AFSSI-C to traditional scanning spectral imaging systems. It also compares different feature design modalities. To test feature designs, the adaptive, joint-pPCA designed features are compared to static, pseudo-random features. The traditional systems that were investigated as simulations are the pushbroom, whiskbroom, and tunable filter systems. The performance of the simulated traditional systems follows intuition—a whiskbroom system has to sweep over all 4096 spatial locations, while the pushbroom and tunable filter system only make 64 and 38 scanning steps, respectively. This explains the greater SNR and hence greater classification accuracy of the tunable filter system, with the whiskbroom system being least accurate of the three. Note that the sequential nature of the measurements in the AFSSI-C limits its applicability to those scenes which are slowly varying with respect to the time needed to achieve a desired classification accuracy.
Fig. 9 Simulation comparing the classification performance for different measurements at TSNR = 0 for different systems: the AFSSI-C with designed features (joint pPCA), the AFSSI-C with random features, the traditional pushbroom imager, the traditional tunable filter imager, and the traditional whiskbroom imager. The input is a 64× 64× 38 spectral datacube with a 4-class library.
In this manuscript, we are explicitly considering the case where the spectral library is completely known—there are no ‘nuisance parameters’ such as amplitude variations of the spectra—and we are investigating the performance of the AFSSI-C in this baseline scenario. However, there do exist methods for addressing nuisance parameters in Bayesian inference schemes [33,34], which could be incorporated into the design and inference framework of the AFSSI-C. Alternatively, a single uncoded measurement would capture the total signal energy at each spatial location and the library could be locally normalized to the appropriate value prior to proceeding with the algorithm described here.
There are a number of phenomena which give the AFSSI-C an advantage over traditional systems. When comparing the AFSSI-C system with joint-pPCA designed features to traditional systems at the 5th measurement in Fig. 9, we see that the classification accuracy improves by 250 ×. This improvement in performance is attributed to a combination of factors such as the open aperture architecture of the AFSSI-C design, lack of scanning, and adaptivity. Figure 9 also demonstrates the benefit of using joint-pPCA to design spectral features that discriminate between the possible hypothesis spectra. Looking again at the 5th measurement, we see a 100 × performance gain relative to static, pseudorandom coding. The performance curve for the random coding case is generated by directly classifying from the acquired measurements using an identical Bayesian inference framework. The information processing inequality of information theory [35] guarantees us that the alternative approach of classifying after datacube reconstructions can do no better than direct classification. Thus, the separation of these two curves represents the true performance improvement arising from adaptive measurement, while the separation between the static-coded curve and the traditional systems represents the performance improvement arising from the increased open aperture. Moreover, we expect further improvement in classification accuracy for larger systems as the traditional systems become even more starved for light, having to sweep through even larger spatial arrays. In all, these simulated results indicate that there is great potential for the AFSSI-C system to outperform traditional systems where classification is the desired result of the analysis.
6. Conclusion
In this paper we introduced the AFSSI-C, a spectral imager classifier that utilizes adaptive spectral features in a resource-lean configuration. We are able to show multiple order-of-magnitude improvement in classification accuracy compared to traditional spectral imaging systems when the noise in the system is equal to the minimum separation between the library spectra, by employing a robust system simulation corroborated with experimental results. By taking advantage of its adaptiveness, the AFSSI-C performance with designed features also achieves multiple order-of-magnitude improvement over a random feature implementation. These adaptive features are designed via Bayesian inference and a novel joint-probabilistic PCA approach, which drives the measurement decision evolution to boost the discrimination ability between spectra candidates at every spatial location.
Direct classification is a useful modality for many of the applications of spectral imaging. By making measurements of an encoded datacube instead of explicit measurement of every element, huge performance gains are realized. It is reasonable to imagine the AFSSI-C system being of great benefit to a number of industries that rely on in situ material classification. Further work is underway to test the system with larger spectral libraries, as well as a second generation system based on a transmissive spatial light modulator (SLM). Using a transmissive SLM will reduce keystone distortion, and simplify the optical path.
Funding
Defense Advanced Research Projects Agency (DARPA) (N66001-10-1-4079).
References and links
1. Y. Garini, I. T. Young, and G. McNamara, “Spectral imaging: Principles and applications,” Cytom. Part A 69A, 735–747 (2006). [CrossRef]
2. M. T. Eismann, Hyperspectral Remote Sensing (SPIE, 2012). [CrossRef]
3. C.-I. Chang, Hyperspectral Imaging: Techniques for Spectral Detection and Classification (SpringerUS, 2003). [CrossRef]
4. A. Ibrahim, S. Tominaga, and T. Horiuchi, “Spectral imaging method for material classification and inspection of printed circuit boards,” Opt. Eng. 49, 057201 (2010). [CrossRef]
5. H.-H. K. Burke and Gary A. Shaw, “Spectral imaging for remote sensing,” Lincoln Lab. J. 14, 3–28 (2003).
6. NASA/JPL-Caltech, “Aviris:airborne visible / infrared imaging spectrometer,” (2014).
7. S. A. Macenka and M. P. Chrisp, “Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) Spectrometer Design and performance,” Proc. SPIE 0834, 32–43 (1987). [CrossRef]
8. N. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52, 090901 (2013). [CrossRef]
9. A. A. Wagadarikar, N. P. Pitsianis, X. Sun, and D. J. Brady, “Video rate spectral imaging using a coded aperture snapshot spectral imager,” Opt. Express 17, 6368–6388 (2009). [CrossRef] [PubMed]
10. C. P. Warren, D. Even, W. Pfister, K. Nakanishi, A. Velasco, D. Breitwieser, S. Yee, and J. Naungayan, “Miniaturized visible near-infrared hyperspectral imager for remote-sensing applications,” Opt. Eng. 51, 111720 (2012). [CrossRef]
11. M. E. Gehm, R. John, D. J. Brady, R. M. Willett, and T. J. Schulz, “Single-shot compressive spectral imaging with a dual-disperser architecture,” Opt. Express 15, 14013–14027 (2007). [CrossRef] [PubMed]
12. X. Lin, G. Wetzstein, Y. Liu, and Q. Dai, “Dual-coded compressive hyperspectral imaging,” Opt. Lett. 39, 2044–2047 (2014). [CrossRef] [PubMed]
13. D. Kittle, K. Choi, A. Wagadarikar, and D. J. Brady, “Multiframe image estimation for coded aperture snapshot spectral imagers,” Appl. Opt. 49, 6824–6833 (2010). [CrossRef] [PubMed]
14. H. Rueda, H. Arguello, and G. R. Arce, “DMD-based implementation of patterned optical filter arrays for compressive spectral imaging,” J. Opt. Soc. Am. A 32, 80–89 (2015). [CrossRef]
15. G. Arce, D. Brady, L. Carin, H. Arguello, and D. Kittle, “An introduction to compressive coded aperture spectral imaging,” IEEE Sig. Process. Mag. 31, 105–115 (2014). [CrossRef]
16. E. DuPont, D. Chambers, J. Alexander, and K. Alley, “A spatial-spectral classification approach of multispectral data for ground perspective materials,” in “IEEE International Conference on Systems, Man, and Cybernetics (SMC),” (2011), pp. 3125–3129.
17. C. Liu and J. Gu, “Discriminative Illumination: Per-Pixel Classification of Raw Materials Based on Optimal Projections of Spectral BRDF,” IEEE Trans. Pattern Anal. Mach. Intell. 36, 86–98 (2014). [CrossRef]
18. W. L. Wolfe, Introduction to imaging spectrometers, vol. v. TT 25. (SPIE, Bellingham, Wa, 1997). [CrossRef]
19. C. Yang, J. H. Everitt, M. R. Davis, and C. Mao, “A CCD camera-based hyperspectral imaging system for stationary and airborne applications,” Geocarto International 18, 71–80 (2003). [CrossRef]
20. N. Gat, “Imaging spectroscopy using tunable filters: a review,” Proc. SPIE 4056, 50–64 (2000),. [CrossRef]
21. D. V. Dinakarababu, D. R. Golish, and M. E. Gehm, “Adaptive feature specific spectroscopy for rapid chemical identification,” Opt. Express 19, 4595–4610 (2011). [CrossRef] [PubMed]
22. G. Lu and B. Fei, “Medical hyperspectral imaging: a review,” J. Biomed. Opt. 19, 010901 (2014). [CrossRef]
23. F. D. Van der Meer, H. M. Van der Werff, F. J. van Ruitenbeek, C. A. Hecker, W. H. Bakker, M. F. Noomen, M. van der Meijde, E. J. M. Carranza, J. B. de Smeth, and T. Woldai,“Multi- and hyperspectral geologic remote sensing: A review,” Internat. J. Appl. Earth Observ. Geoinform. 14, 112–128 (2012). [CrossRef]
24. E. Marengo, M. Manfredi, O. Zerbinati, E. Robotti, E. Mazzucco, F. Gosetti, G. Bearman, F. France, and P. Shor, “Development of a technique based on multi-spectral imaging for monitoring the conservation of cultural heritage objects,” Analytica Chimica Acta 706, 229–237 (2011). [CrossRef] [PubMed]
25. M. Hinnrichs, J. O. Jensen, and G. McAnally, “Handheld hyperspectral imager for standoff detection of chemical and biological aerosols,” Proc. SPIE 5268, 67–78(2004). [CrossRef]
26. T. Arnold, M. De Biasio, and R. Leitner, “Near-infrared imaging spectroscopy for counterfeit drug detection,” Proc. SPIE 8032, 80320Y (2011),. [CrossRef]
27. X. Lin, Y. Liu, J. Wu, and Q. Dai, “Spatial-spectral encoded compressive hyperspectral imaging,” ACM Trans. Graphics (TOG) 33, 233 (2014). [CrossRef]
28. H. Abdi and L. J. Williams, “Principal component analysis,” Wiley Interdisciplinary Reviews: Computational Statistics 2, 433–459 (2010). [CrossRef]
29. H. V. Poor, An Sntroduction to Signal Detection and Estimation (Springer Science & Business Media, 2013).
30. M. Dunlop, P. K. Poon, D. R. Golish, E. Vera, and M. Gehm, “Calibration challenges and initial experimental demonstration of an adaptive, feature-specific spectral imaging classifier,” in “Imaging and Applied Optics,” (Optical Society of America, 2013), p. CW2C.3. [CrossRef]
31. W. Smith, Modern Optical Engineering, 4th ed. (McGraw-Hill Education, 2007).
32. M. Dunlop, P. Poon, E. Vera, and M. E. Gehm, “Experimental validation of the adaptive feature-specific spectral imaging classifier,” in “Frontiers in Optics 2014,” (Optical Society of America, 2014), p. FTh2B.5.
33. D. Hernández-Lobato, V. Sharmanska, K. Kersting, C. H. Lampert, and N. Quadrianto, “Mind the Nuisance: Gaussian Process Classification using Privileged noise,” in “Advances in Neural Information Processing Systems,” (2014), pp. 837–845.
34. L. Hong and K. Ho, “Classification of BPSK and QPSK signals with unknown signal level using the Bayes technique,” in “Proceedings of the 2003 International Symposium on Circuits and Systems, ISCAS’03,” (IEEE2003), vol. 4, pp. IV–1.
35. T. M. Cover and J. A. Thomas, Elements of Information Theory (John Wiley & Sons, 2012).
