Open Access
Add to BibSonomy
Abstract
This paper presents the design and fabrication of a fiber-based snapshot imaging spectrometer working in both visible (490 nm-732 nm) and short-wave infrared (1090 nm - 1310 nm) ranges. To maximize the light collection efficiency, a custom relay system with 0.25 NA and 20 mm field of view (FOV) was designed and integrated. The bench setup showed that the custom relay system could fully resolve 10 µm fiber cores over the entire FOV among visible and short-wave infrared ranges. The numerical aperture (NA) match provided a 2.07X fold throughout improvement in the visible range and about 10X fold in the SWIR range compared to the previous generations, enabling imaging with a fast frame rate and under low illumination conditions. The presented imaging spectrometer generated spectral datacubes with 35000 spatial samplings and 23 spectral channels. Spectral urban imaging results obtained by the spectrometer in both visible and SWIR ranges are presented. Finally, we collected spectral images of apple bruising to show potential applications in the food quality industry.
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement
1. Introduction
Hyperspectral imaging (HSI) systems allow collection of spatial and spectral information of the object and generate 3D datacubes $(x,y,\lambda )$ [1–3]. It has been utilized for applications in agriculture [4–6], astronomy [7–9], biotechnology [3,10] and environment [11]. The spectral imaging systems can be divided into three main categories: spatial scanning [12], spectral scanning [13,14], and non-scanning/snapshot [1,10,15–17]. This last group can be further divided into direct/field integral imaging [1,2], aperture split [2,18], image duplication through polarization/filter components or coded computational methods like computed tomographic imaging spectrometer (CTIS) or coded aperture snapshot spectral imaging (CASSI) [2,18]. The most significant advantages of snapshot spectral imaging systems are instantaneous capture of spectral and spatial information [1,2]. This allows monitoring of fast processes (avoiding motion artifacts), enables efficient image conditioning (overlapping scenes), and is more light efficient (longer integration possible at same imaging rates as scanning methods).
Fiber bundles are one of the options for building field integral snapshot imaging spectrometers. The approach allows a high degree of freedom to reformat the input or output configurations arbitrarily [19–23]. In addition, the overall optical layout is conceptually simple. The fiber-based snapshot imaging spectrometer includes fore-optics, a fiber bundle, a reimaging system with a dispersed component, and an image sensor. On the other hand, custom fiber bundle implementations are limited as they are built with individual fiber or ribbons, making the process time-consuming and costly. Available fibers used in these bundles result in large output areas and high NA (0.2-0.6) making reimaging challenging in terms of imaging performance over a large field and overall collection efficiency. Specific previous implementations allowing examples of the largest fiber-based spectrometer implementations, in terms of spatial spectral sampling, allowed approximately 3% or 14% light throughput. Some other off-the-shelf optics allow higher light collection [24] but are not well-matched with the pixel size of available cameras and thus result in smaller data cubes.
One of the first applications of short-wave infrared (SWIR) imaging was the determination of moisture and fat in meat [25]. For example, by capturing the reflecting spectra of samples, SWIR imaging can determine the moisture rate fast and preciously. Hyperspectral imaging expands the applications of SWIR imaging by improving its spatial dimensions and spectral resolutions, which results in increased sensitivity of SWIR imaging [26]. The improved sensitivity of hyperspectral SWIR imaging enables detection of quality, contaminants, and adulterants in the food industry [27–29] and pharmaceutical industry [30,31]. It can detect if melamine is added to milk powder to increase the protein measurement results. It also observed the homogeneity or degradation of medicine tablets. The imaging spectrometers with SWIR range were also reported for remote sensing applications. The Hyperion Imaging spectrometer was one of the imaging spectrometers to routinely acquire data on a global scale [32,33]. These imaging spectrometers allow quantitative spectral monitoring of earth surfaces.
This paper presents a snapshot fiber-based imaging spectrometer with high spatial sampling and low integration time in both visible and SWIR wave ranges. This is the first snapshot imaging spectrometer in both ranges we presented using our custom fiber bundle. By replacing the dispersion unit (a band-pass filter and a prism), the spectrometer can switch between visible mode and SWIR mode very quickly. High-resolution spectral imaging in visible range provides a color signature of different objects, which can be used for segmentation. Combined with the ability of SWIR spectral imaging to detect the chemical signature of many materials, the imaging spectrometer can capture real-time and measurable contrast of the monitored objects. However, no off-the-shelf objectives can efficiently collect the light from the fiber bundle in our system due to their limited NA and FOV in both wave ranges. Even only in the visible range, the commercial objectives lost more than 50% of light from the fiber bundle [16,34] due to small NA and vignetting. A custom-designed relay system with high NA and large FOV is critical for the imaging spectrometer to maintain high SNR in both wave ranges. The design, fabrication, and characterization of the relay system are reported in detail. The improved light collection efficiency combined with the snapshot modality allows the capture of spectral images of fast-moving objects under low illumination conditions. The final spectral imaging results of urban buildings are shown to demonstrate the spatial resolution and spectral responses. Additionally, the SWIR spectral imaging of a bruised apple is presented for its potential application in food quality detection.
2. Principle of light-guide snapshot imaging spectrometer
The optical layout of a light-guide/fiber-based snapshot imaging spectrometer is illustrated in Fig. 1(a). The object is first captured by a photographic objective lens and then imaged onto a custom fiber bundle. The fiber bundle array is fabricated by stacking many layers of fiber ribbons of $6 \times 6$ blocks (simliar to [16]), which are densely stacked at the input end. However, at the output, the fiber bundle layers are rearranged into rows with blank spaces between each row. A photomask with specifically designed slits is attached to the output to select fibers core to avoid overlap after dispersion. Afterward, those selected fiber cores are imaged by a relay system to the sensor. The prism and filters are positioned at the center of the relay system to introduce dispersion fully covering the blank spaces generated by the fiber bundle array. This design will map all the spatial and spectral information onto the sensor without any spatial or spectral scanning. The datacube size is limited by the total number of pixels that can be utilized. By adjusting the void space between fiber rows at the output, we can balance spectral sampling with spatial sampling. A calibration process is necessary to determine the look-up table mapping from the raw image to the 3D datacube. The calibration includes two main steps: spatial calibration and spectral calibration [15,16]. In the spatial calibration, a DLP projector is put in front of the objective lens to project six $\frac {\pi }{3}$ step phase shift sinusoidal patterns in x and y directions, respectively. A phase-shift algorithm [16] is implemented here to calculate the mapping of the spatial positions from the sensor to the object. In the spectral calibration process, three laser-line filters are used to determine the spectral positions of these three wavelengths for each spatial position, while the spectral positions for the other wavelengths are interpolated based on these position data. Eventually, a 3D spectral datacube can be reconstructed.
3. Design of high throughput VIS/SWIR imaging spectrometer
The goal of this work is to design a spectral imaging system to capture spectral images of objects at a fast frame rate in both visible and SWIR wave ranges. As described in the principle, the imaging spectrometer consists of two critical components: a custom fiber bundle and a reimaging system, whose qualities will significantly affect the final imaging performance. All the components used in the setup are listed in Table 1. A Kowa LM25HC-SW F/1.4 f25mm photographic lens was used as the objective lens to image the scene onto the fiber bundle. The fiber bundle was fabricated by stacking fiber ribbons manufactured by Schott ($6 \times 6$ blocks, 10 $\mu m$ diameter). The fiber ribbons were stacked in a custom mold to allow dense assembly at the input and separated with uniform spacing at the output. The fiber bundle was then glued together with the mold using optical epoxy. Finally, the fiber bundle was cut by a diamond saw to the designed dimension and polished by diamond pads to allow optical surface quality. The fabrication process was explained in detail in our previous papers [16,34]. To fully leverage the advantage of simultaneous datacube acquisition, the overall light-throughput of the system is critical. A radiometric model was built to estimate the output signal level [24]. The reimaging system should, optimally, match the fiber core’s NA to avoid coupling losses from the fiber bundle. However, many off-the-shelf objectives or their combinations were evaluated, while none of them satisfied the requirement of large FOV, high NA, telecentricity, and minimal vignetting (fiber NA = 0.28, fiber bundle output is $12 mm \times 16 mm$). The previous systems used either a pair of identical achromatic doublets (Edmund 33918, 75mm Diameter, 100mm EFL) [15] or the Olympus 0.63X stereo microscope objective (Olympus MVPLAPO 0.63X) and tube lens [34] as the relay systems. When the two identical achromatic doublets were used, the aperture stop in the system had to be closed to meet the resolution requirement. The highest NA the system can work with while meeting satisfactory resolution is 0.05. Considering the NA of the fibers is 0.28, more than 97 percent of light gets lost due to the low efficiency of the reimaging system. The radiometric design model [16] provides a setup with a stereo microscope objective and tube lens from the Olympus MVX microscope as covering the entire FOV while operating at 0.0995 collection NA. Even though a better combination was found by using the stereo microscope objective and tube lens, the new reimaging system only provides about 14% throughput and causes a redundant 0.8X magnification. The major light loss in the fiber-based spectrometer is due to the relay system, which limits fast frame rate imaging and the ability to image under low illumination conditions. Moreover, vignetting is another significant issue due to the limited size of the commercial lens. As a result, developing a custom optical system to collect the light emitted from the fiber bundle at high NA while maintaining suitable FOV is necessary. The final system spectrometer implemented with the custom reimaging system is shown in Table 2.
Table 1. Fiber-based snapshot imaging spectrometer component list.
Table 2. Fiber-based snapshot imaging spectrometer system Spec.
4. Design and fabrication of the high throughput relay system
4.1 Optical design
The reimaging system is a critical component in the imaging spectrometer because it determines the light collection efficiency and also spatial resolution. Designing a custom relay system solves the bottleneck of the fiber-based imaging spectrometer, which could expand the applications requiring high frame rate and high signal-to-noise ratio. The reimaging system consists of a relay system, filters, and a prism. The first requirement for the relay system is to cover the entire fiber bundle. The target FOV is set to be $\pm$ 10 $mm$. This is slightly larger than the fiber bundle size. The single fiber core size is 10 $\mu m$. To fully resolve the fiber bundle, the RMS spot size has to be around 10 microns over the entire FOV. In our previous paper [15,16,34], the collection NA is limited to 0.05 and 0.0995, which causes significant light loss. Matching the fiber bundle NA is the most important and challenging part of designing the custom relay system. Another difficult part is that the light must be collimated in the middle because the prim and the band pass filter only perform well at a limited incident angle. Incident angles larger than 10$^{\circ }$ will cause aberrations on the prism and loss of filtration on the filters. At last, the chromatic aberrations need to be controlled over the visible and SWIR range required by our applications.
The design parameters are shown in Table 3. The MVCam camera sensor size is similar to the dimension of custom fiber bundle output. Considering the sensor pixel size is also close to the fiber mold field diameter of the custom fiber bundle, the optimized magnification is 1X. The custom relay system was designed in OpticStudio (ZEMAX, USA). The system was optimized to achieve the best modulous transfer function (MTF) at 100 lp/mm. The schematic of the design is shown in Fig. 2. Optical prescriptions for the final objective design are given in Table 4. The relay system consists of six achromatic doublets made of materials from the Schott preferred glass category. There are many advantages to forcing the relay system to be symmetric. First, two identical sets of lenses can be used together to perform a 1X relay, which significantly reduces the cost and lead time of the lenses and the assembly mounts. Coma and spherical aberrations are known as primarily pupil-dependent fourth-order aberrations. Adopting a symmetric design eliminates coma at the beginning of the design and enables more freedom to correct the spherical aberrations due to the high NA. The distance between the image and the last lens backside is 19.98 $mm$. Theoretically, the reimaging system should be placed close to the image plane to reduce the field-related aberrations much more efficiently. However, considering the distance from the camera front to sensor position and the thickness of a fused silica protection window, around 20 mm should be kept for the mounts and the range for adjusting the focusing distance. To force the light at the center to be collimated during optimization, a bunch of rays ($H=0, \rho = 0, 0.3, 0.5, 0.77, 1$) are selected at the optimization merit function editor, whose incident angles at the stop plane (the center of the relay system) were set to be zero.
Table 3. Spec sheet for the objective design.
Table 4. Lens prescription for the relay lens.
4.2 Tolerance and performance optimization
The manufacturability and opto-mechanics were also assessed in OpticStudio. The large size of the lenses results in the system being sensitive to tolerance parameters, such as radius, refractive index, and Abbe number. The summary of the tolerance parameters is shown in Table 5. The average MTF at 100 lp/mm is the criterion during the process of tolerance. To ensure the good resolution of the fiber cores, the MTF value at 100 lp/mm needs to be higher than 0.4 over the entire FOV. The paraxial distance and the distance at the center are allowed to be the compensators during tolerance optimization as these parameters can be adjusted during alignment without affecting the design of the lens mounts. The designed MTF value at 100 lp/mm is approximately 0.5 as shown in Fig. 3. According to 10,000 Monte Carlo simulation runs, the possibility of success is 60% to satisfy the MTF requirement. Further tightening the tolerance parameter would improve the success rate, but it would also significantly increase the cost of the system. To increase the probability of success, high opto-mechanical tolerances can be obtained after lens fabrication by design to accommodate specifically manufactured lenses. The RMS spot size shown in the figure is close to the fiber core size. In addition, only 0.06% of vignetting is detected in the design, which won’t affect the collection efficiency over FOV and result in uniform SNR even under poor illumination conditions.
Table 5. Spec sheet for tolerance parameters
4.3 Lens fabrication and opto-mechanics design
All the lenses were fabricated by Optimax Systems Inc., New York. The tolerances of the lenses were within the OPTIMAX tolerance chart’s precision catalog with a minor variance in Abbe number. To allow relatively high transmission in the visible and SWIR wave ranges at the same time, no anti-reflection coatings were applied to the lens surfaces for this setup. A custom designed visible-infrared anti-reflection coating would help improve the transmission further later on. The opto-mechanics was designed in SolidWorks 2020 (Dassault Systemes, France) as demonstrated in Fig. 4(a). The assembly mounts are divided into two pieces to allow room for prisms and filters with an automatic filter switcher. On each side, two spacers are employed to control the spacing between lenses. The thickness and bevel angles of these spacers have been carefully regulated to enable for simple drop-in assembly without additional alignment. The assembly mounts were fabricated by Rice University’s in-house machine shop with a tolerance of $-0.0000/+0.0125 mm$ for the inner diameter of the lens tube, which met the element decenter requirement and diameters of fabricated doublets. The final assembly is shown in Fig. 4(b).
Fig. 4. Mechanical schematic of the relay system. (a) Design of the assembly in SOLIDWORKS. (b) Picture of the assembly on the optical bench.
5. Custom relay performance assessment
5.1 Imaging performance
The imaging performance of the relay systems was evaluated for spatial resolution, FOV, and vignetting. The spatial resolution of the relay system was evaluated by imaging a standard 1951 United States Air Force (USAF) resolution target. The optical setup used to assess the resolution of the relay system is shown in Fig. 5(a). A halogen lamp with collecting lenses (Thorlabs, NJ) was used as the light source. A 405 nm long-pass filter (BLP01-405R-25, Semrock, NY) and a 694 nm short pass filter (FF02-694/SP-25) were placed after the light source to narrow the illumination waveband to the design visible waveband. A diffuser was used to ensure that the illumination would fill up the entire designed stop aperture of the relay system. After that, the USAF resolution target was imaged by the relay system onto a scientific CMOS camera (DMK 38U542, The Imaging Source, Germany). The image is shown in Fig. 5(b). Based on the target image result, the relay lens was able to clearly resolve the features up to group 6, element 6 (114 $lp/mm$) in the proposed visible range. The resolution target position was moved to the corner of the FOV to make sure the relay system provides satisfactory resolution over the entire FOV. Vignetting test was done by comparing the pixel values from the edge of the FOV to the ones at the center when no object was present. A $10 \times 10$ pixel region was placed at the corner and then at the center of the raw image. The pixel values from these two regions show a $2.7\%$ difference, which led to the conclusion that the relay system has vignetting within tolerance. The resolution of the relay system was mainly evaluated in the visible range because commercial cameras with small pixels in visible range are easy to access. There are limited commercial SWIR cameras available on the market. The SWIR MVCam (12 $\mu m$ camera pixel size) in the system can not provide enough sampling. Instead, an image of the fiber bundle with a photo mask at the output, shown in Fig. 5(c), was taken when a 1200 nm filter was inserted to limit dispersion. As mentioned in Table 1, the photo mask has 7 $\mu m$ 45$^{\circ }$ tilt slits. The expected resolution from the ZEMAX simulation is 17.5 $\mu m$. Because the axial chromatic aberration is neglectable, the resolution image at center wavelength can represent the relay system performance over the entire designed SWIR range. The zoom-in images at three different field positions in Fig. 5(d) demonstrate the images of the entire photo mask occupies only two pixels at the sensor, which shows the resolution is smaller than 24 $\mu m$. SWIR Cameras with smaller pixels can specify the actual resolution, but this experimental test clarifies the relay system already provides satisfying performance for our application, which is further proven by the high-resolution reconstructed images in Section 6.2 and 6.3.
Fig. 5. Resolution assessment. (a) Optical setup for resolution assessment. (b) Image of the standard USAF resolution target. (c) Raw image of the fiber bundle output with photo mask. (d) Zoom-in images of three filed positions.
5.2 Throughput evaluation
The setup for the throughput measurement, shown in Fig. 6(a), is similar to the resolution assessment setup. The USAF resolution target was replaced by a custom fiber bundle with a square aperture at the output. The custom relay system (1X magnification) was compared to the olympus stereo microscope relay system (0.8X magnification). Because of the different magnifications, we could not directly compare the signal level of these two systems. To compensate for the magnification difference, a $4 \times 4 mm$ square aperture was put after the output of the fiber bundle to create an object shown in Fig. 6(b), which can be fully captured by the two imaging setups. The throughput was evaluated by adding all of the pixel values with subtraction of the background signal. According to the result, the custom relay system presented in this study collected 2.07X fold more light from the fiber bundle, which is about 79.96% of all the light emitting from the fiber bundle. The significant improvement of light throughput would reduce the required integration time. As stated in Section 4.3, the light collection efficiency can be optimized by custom VIS/SWIR anti-reflection coatings. Our previous paper [24] accesses the light collection efficiency when coupling light from the custom fiber bundle using different lens setups. Adding the result from this custom relay system, the collection efficiency of these lenses is shown in Fig. 7.
6. Spectral imaging results
6.1 System setup
The experiment bench setup of the VIS/SWIR imaging spectrometer was shown in Fig. 1(b). Visible imaging mode and SWIR imaging mode used different sets of filters and prisms to leverage the void spaces created by the fiber bundle. An automatic filter wheel and an automatic transnational stage can be used to switch filter sets and camera positions for the two imaging conditions. A Kowa LM25HC-SW F/1.4 f25mm photographic lens was utilized to image the scene onto the input of the fiber bundle. The objective lens is designed for imaging in SWIR wavelength range. The infrared coating allows more than 90% of transmission from 1000 nm to 1350 nm. It also passes through more than 40% of visible light. Therefore, to limit the change of components, we used the same objective for the two imaging modes. Note that the objective lens can be easily changed to accommodate specific imaging conditions or application requirements. The custom fiber bundle and custom relay system mentioned in Section 4 were implemented. The MVCam, a megapixel InGaAs camera from Princeton Infrared Technologies, can run at over 90 fps with $1280 \times 1024$ resolution. It has more than 75% quantum efficiency from 1 $\mu m$ to 1.6 $\mu m$ and more than $30\%$ quantum efficiency in the visible range. The spatial resolution of reconstructed images was determined by the number of fiber cores captured, whereas the spectral sampling was determined by the void spaces at the output of the fiber bundle. The calibration procedure is required to reconstruct the 3D spectral datacubes. It directly affects spatial resolution and spectral accuracy. The calibration procedure in the visible range is described in [16,34]. The procedure for the SWIR wavelength range is very similar except that different filter sets are used. Therefore, two look-up tables were generated and stored specifically. The final spectral datacubes have 35,000 spatial samplings and 23 spectral samplings. The spatial resolution when using the Kowa objective is 3.93 arcsecs in the horizontal direction and 0.99 arcsecs in the vertical direction. The average spectral resolution is 22.0 nm in the visible range and 20.0 nm in the short-wave infrared range.
6.2 Assessment of spectrum measurement
In order to validate the spectrum measurement, an experiment was conducted where the results obtained from the fiber-based spectrometer system were compared with those obtained from two commercial spectrometers (Ocean Optics USB4000 and Ocean Insight Flame-NIR+). The experiment employed the use of three narrow-band filters (633 nm, 1100 nm, and 1200 nm) that were placed after a halogen lamp. Two systems are applied to capture the filtered light and plot the spectra, as shown in Fig. 8. The TuLIPSS data was evaluated using Gaussian fitting to determine if the system could accurately locate the center wavelength of the filters. The results showed that the peak positions of the Gaussian fitting were within the spectral resolution of both visible and SWIR ranges, with measured center wavelengths of 631.5 nm, 1097.0 nm, and 1200.1 nm, respectively. The fiber-based spectrometer was capable of precisely capturing the center wavelengths of the filters, with the Ocean Insight spectrometers exhibiting a narrower FWHM, especially in the visible range, due to their high samplings. However, when the commercial spectrometer’s samplings were closer to the TuLIPSS in the SWIR range, the two systems showed similar measured data.
Fig. 8. Comparison between the spectrum obtained from the fiber-based interferometer and an Ocean Insight spectrometer. (a), (b), and (c) are the measurement of 633nm, 1100nm, and 1200 nm narrow-band filters, respectively. The results of the measurements are presented in the form of TuLIPSS measurement, Gaussian fitting corresponding to the TuLIPSS measurement, and Ocean Insight reference measurement on each figure.
To further validate the spectrum measurement, we performed a leaf spectrum measurement in a controlled laboratory experiment as shown in Fig. 9. A leaf illuminated by a halogen lamp was set up as the object for both systems. As shown, the spectra from the two systems are closely matched. As a reference measurement, an Ocean Insight Flame-NIR+ spectrometer was used to validate the TuLIPSS measurements. However, comparing these two measurements directly can be difficult as the Ocean Optics spectrometer only provides a spectrum measurement from an optical fiber and does not reconstruct an image. To overcome this, six points that match where the Ocean Insight spectrometer fiber was located were selected to plot the average spectrum of the leaf from the fiber-based imaging spectrometer. As shown in Fig. 9(b), the spectra from these two systems are closely matched.
Fig. 9. Comparison of spectrum measurement from the fiber-based interferometer and Ocean Optics spectrometers. (a) A leaf image from one spectral channel of TuLIPSS marked with six selected fiber positions. (b) The average spectrum of the six selected points on (a) and the reference measurement from Ocean Insight spectrometer.
6.3 Visible imaging
Urban imaging from a laboratory window was performed when the spectrometer was running in visible mode. The results in the visible range from this setup are shown in Fig. 10 with pseudo color. The visible imaging experiment focuses on demonstrating the improvement of image quality and the improvement of signal level by the custom relay lens because the concept of visible fiber-based imaging spectrometer was already introduced in [16,34]. The raw data were taken through the windows of our lab on a sunny day while outdoor illumination is around 100,000 lux. The integration time when the Olympus relay (NA = 0.0995) was implemented in the same configuration was 60 milliseconds. This setup with the custom relay lens, on the other hand, enabled 30 milliseconds of exposure time for a similar intensity level on the sensor. The relay system did collect more light from the fiber bundle due to better NA matching between the fiber bundle and the relay system. The different features show varied responses over the 23 spectral channels in Fig. 10(a). The pseudo-color composite image from the 23 spectral channels is shown in Fig. 10(c). Comparing the composite image with the high-resolution photo of the scene in Fig. 10(b), we can see the images from the spectrometer has enough spatial resolution for feature detection and segmentation. Many detailed features are well resolved, such as the windows on the building on the left-top side and the dark shadow on the right side. Two spots (trees and buildings) are selected from the image and their spectra are extracted and plotted in Fig. 10(d). These features show considerable differences in the spectrum domain as expected, especially the corresponding wavelengths at the intensity peaks. Objects in the scene can be selected based on their spectral signature from the spectrometer.
Fig. 10. Imaging results from the imaging spectrometer working in visible range through a laboratory window of Houston Medical Center. (a) All 23 spectral channels of a scene from the windows of Rice BioScience Research Collaborative are shown. (b) A photo taken by iPhone camera of the same scene. (c) The pseudo-color composite image of all spectral channels. (d) Spectra of two selected points (tree and building).
6.4 SWIR imaging
The SWIR imaging mode was also evaluated by capturing similar urban images. The system was assembled outside the lab building because the coatings on our laboratory windows block more than 95% of SWIR light. Hence, the images of the same scene were taken from a different view angle compared to the previous visible imaging experiment. When the spectrometer runs at SWIR mode, it still has the same number of spatial samplings and spectral channels but shifted wavelength range and spectral resolution. The SWIR image was taken on the Rice University campus on a sunny day (100,000 lux). The integration time was 30 ms. Compared to 300 $ms$ for the same intensity level when the Olympus Microscope MVX system was utilized, the custom relay showed more throughput improvement in the SWIR wavelength range. The extra light throughput increase is because the coatings on the Olympus lenses are optimized for visible range and the materials chosen for the custom relay system have better transmission at SWIR wavelength range. The system effectively increased the frame rate from 3.3 fps to 33 fps, which demonstrates the importance of the custom relay when video imaging is required. The images from all 23 spectral channels (Fig. 11(a)) and the spectrum comparison between TuLIPSS and Ocean Insight Flame-NIR+ spectrometer on grass (Fig. 11(b)) and tree (Fig. 11(c)) illustrate that the system can provide enough spatial sampling and spectral resolution for object detection and spectral feature extraction in SWIR range.
Fig. 11. Urban imaging results from the imaging spectrometer in SWIR mode. (a) All 23 spectral channels from 1090 nm to 1310 nm. (b) Comparison of the tree spectrum between TuLIPSS and Ocean Insight Flame-NIR+. (c) Comparison of the grass spectrum between TuLIPSS and Ocean Insight Flame-NIR+.
SWIR spectral imaging can simultaneously detect both chemical and physical properties of food materials. There are some reported studies applying SWIR spectral imaging for non-destructively detecting bruises on apples [35]. However, the development of a real-time SWIR spectral imaging system is still a challenge. Considering our imaging spectrometer’s parallel acquisition modality and high light collection efficiency, we performed experiments to image bruising apples in the lab, as shown in Fig. 12. The apple was put in front of the imaging spectrometer and illuminated by a halogen lamp. By adjusting the focal lens of the first objective lens, the system focused on the surface of the apple. The integration time was set to 20 milliseconds. Figure 12 shows the spectrometer still maintains a considerable signal-to-noise ratio for the bruise apple imaging application when the fiber-based spectrometer worked at 50 fps. A photo of the apple is shown in Fig. 12(a), where the bruising region is not visible. However, the bruising region shows a strong signal in SWIR range as demonstrated in Fig. 12(b). The spectra from the bruising region (blue) and normal region (red) are plotted in Fig. 12(c), which are consistent with the results shown in [36]. The significant discrepancy between these two spectra indicates high sensitivity of the bruising detection using the imaging spectrometer. The experiment further illustrates the capability of our imaging spectrometer to capture the spectrum changes with a short integration time.
Fig. 12. Imaging results of a bruising apple in SWIR mode. (a) A bruising apple image taken by iPhone camera. (b) Selected channels of the bruising apple. (c) The spectra of a bruising region and a normal region.
7. Conclusion and discussion
In this study, a snapshot fiber-based spectral imaging system working in both visible and SWIR wavelength ranges was presented for the first time. The spectrometer generates datacubes with 35000 spatial samplings and 23 spectral channels. The spatial resolution and spectral resolution of the system are suitable for object detection and spectral extraction. A high NA and large FOV achromatic relay system was designed, fabricated, and evaluated. When implemented in the imaging spectrometer, it increased the light throughput about 2 times in the visible range and almost 10 times in the SWIR range compared to the commercial Olympus MVX microscope. This improvement overcomes the biggest challenge with light-guide fiber-based spectrometers, which allow for fast video imaging or imaging under insufficient illumination. The ability to capture visible and SWIR spectral images in a snapshot mode allows potential applications in many fields, such as water vapor evaporation and fruit bruise detection.
Even though we have achieved the spectral imaging capability over VIS/SWIR range and improved the light collection efficiency, several improvements still remain. As mentioned in Section 3, the achromatic doublets in the custom relay system are not coated, which still causes considerable light loss. A custom-designed visible and SWIR anti-reflection coatings can be applied on the lens surfaces to significantly improve the light throughput if necessary. Additionally, the fiber bundle can be further improved by optimizing fabrication techniques. Two-photon polymerization can be potentially used to directly print the fiber bundle structures. If so, we can achieve more regular structures at the input and output to eliminate spatial calibration and improve spatial sampling. The 3D-printed fiber bundle could also lead to a more compact spectrometer, which will broaden the applications of this type of imaging spectrometer.
Funding
National Aeronautics and Space Administration (NNX17AD30G).
Acknowledgments
We would like to acknowledge all the members in Tkaczyk lab for the helpful discussions and assistance.
Disclosures
Dr. Tomasz Tkaczyk has financial interests in Attoris LLC.
Data availability
Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
References
1. L. Gao and L. V. Wang, “A review of snapshot multidimensional optical imaging: measuring photon tags in parallel,” Phys. Rep. 616, 1–37 (2016). [CrossRef]
2. N. A. Hagen and M. W. Kudenov, “Review of snapshot spectral imaging technologies,” Opt. Eng. 52(9), 090901 (2013). [CrossRef]
3. G. Lu and B. Fei, “Medical hyperspectral imaging: a review,” J. Biomed. Opt. 19(1), 010901 (2014). [CrossRef]
4. T. Ad ao, J. Hruška, L. Pádua, J. Bessa, E. Peres, R. Morais, and J. J. Sousa, “Hyperspectral imaging: A review on uav-based sensors, data processing and applications for agriculture and forestry,” Remote Sens. 9(11), 1110 (2017). [CrossRef]
5. B. Park and R. Lu, Hyperspectral imaging technology in food and agriculture (Springer, 2015).
6. B. Lu, P. D. Dao, J. Liu, Y. He, and J. Shang, “Recent advances of hyperspectral imaging technology and applications in agriculture,” Remote Sens. 12(16), 2659 (2020). [CrossRef]
7. E. K. Hege, D. O’Connell, W. Johnson, S. Basty, and E. L. Dereniak, “Hyperspectral imaging for astronomy and space surveillance,” Proc. SPIE 5159, 380–391 (2004). [CrossRef]
8. B. Rafert, R. G. Sellar, E. Holbert, J. H. Blatt, D. W. Tyler, S. E. Durham, and H. D. Newby, “Hyperspectral imaging fourier transform spectrometers for astronomical and remote sensing observations,” Proc. SPIE 2198, 338–349 (1994). [CrossRef]
9. A. A. Gowen, C. P. O’Donnell, P. J. Cullen, G. Downey, and J. M. Frias, “Hyperspectral imaging–an emerging process analytical tool for food quality and safety control,” Trends in food science & technology 18(12), 590–598 (2007). [CrossRef]
10. L. Gao, R. T. Kester, N. Hagen, and T. S. Tkaczyk, “Snapshot image mapping spectrometer (ims) with high sampling density for hyperspectral microscopy,” Opt. Express 18(14), 14330–14344 (2010). [CrossRef]
11. W. L. Wolfe, Introduction to imaging spectrometers, vol. 25 (SPIE Press, 1997).
12. S. C. Wang and R. C. Flagan, “Scanning electrical mobility spectrometer,” Aerosol Sci. Technol. 13(2), 230–240 (1990). [CrossRef]
13. M. Sun and S. Zigman, “An improved spectrophotometric assay for superoxide dismutase based on epinephrine autoxidation,” Anal. Biochem. 90(1), 81–89 (1978). [CrossRef]
14. J. Kerr, “New methodology for deriving total ozone and other atmospheric variables from brewer spectrophotometer direct sun spectra,” J. Geophys. Res.: Atmos. 107(D23), ACH 22-1 (2002). [CrossRef]
15. Y. Wang, M. E. Pawlowski, and T. S. Tkaczyk, “High spatial sampling light-guide snapshot spectrometer,” Opt. Eng. 56(8), 081803 (2017). [CrossRef]
16. Y. Wang, M. E. Pawlowski, S. Cheng, J. G. Dwight, R. I. Stoian, J. Lu, D. Alexander, and T. S. Tkaczyk, “Light-guide snapshot imaging spectrometer for remote sensing applications,” Opt. Express 27(11), 15701–15725 (2019). [CrossRef]
17. J. Lu, X. W. Ng, D. Piston, and T. S. Tkaczyk, “Snapshot image mapping spectrometer with 3d printed multifaceted mapping mirror for biomedical applications,” Proc. SPIE 12216, 122160B (2022). [CrossRef]
18. N. Hagen and E. L. Dereniak, “Analysis of computed tomographic imaging spectrometers. i. spatial and spectral resolution,” Appl. Opt. 47(28), F85–F95 (2008). [CrossRef]
19. D. W. Fletcher-Holmes and A. R. Harvey, “Real-time imaging with a hyperspectral fovea,” J. Opt. A: Pure Appl. Opt. 7(6), S298–S302 (2005). [CrossRef]
20. J. Kriesel, G. Scriven, N. Gat, S. Nagaraj, P. Willson, and V. Swaminathan, “Snapshot hyperspectral fovea vision system (hypervideo),” Proc. SPIE 8390, 83900T (2012). [CrossRef]
21. P. S. Hsu, D. Lauriola, N. Jiang, J. D. Miller, J. R. Gord, and S. Roy, “Fiber-coupled, uv–swir hyperspectral imaging sensor for combustion diagnostics,” Appl. Opt. 56(21), 6029–6034 (2017). [CrossRef]
22. B. Khoobehi, A. Khoobehi, and P. Fournier, “Snapshot hyperspectral imaging to measure oxygen saturation in the retina using fiber bundle and multi-slit spectrometer,” Proc. SPIE 8229, 82291E (2012). [CrossRef]
23. N. Bedard and T. S. Tkaczyk, “Snapshot spectrally encoded fluorescence imaging through a fiber bundle,” J. Biomed. Opt. 17(8), 080508 (2012). [CrossRef]
24. D. Zheng, C. Flynn, R. I. Stoian, J. Lu, H. Cao, D. Alexander, and T. S. Tkaczyk, “Radiometric and design model for the tunable light-guide image processing snapshot spectrometer (tulipss),” Opt. Express 29(19), 30174–30197 (2021). [CrossRef]
25. I. Ben-Gera and K. H. Norris, “Direct spectrophotometric determination of fat and moisture in meat products,” Journal of Food Science 33(1), 64–67 (1968). [CrossRef]
26. S. Lohumi, S. Lee, H. Lee, and B.-K. Cho, “A review of vibrational spectroscopic techniques for the detection of food authenticity and adulteration,” Trends in Food Science & Technology 46(1), 85–98 (2015). [CrossRef]
27. M. Kamruzzaman, D.-W. Sun, G. ElMasry, and P. Allen, “Fast detection and visualization of minced lamb meat adulteration using nir hyperspectral imaging and multivariate image analysis,” Talanta 103, 130–136 (2013). [CrossRef]
28. M. Kamruzzaman, Y. Makino, and S. Oshita, “Hyperspectral imaging in tandem with multivariate analysis and image processing for non-invasive detection and visualization of pork adulteration in minced beef,” Anal. Methods 7(18), 7496–7502 (2015). [CrossRef]
29. P. Mishra, C. B. Cordella, D. N. Rutledge, P. Barreiro, J. M. Roger, and B. Diezma, “Application of independent components analysis with the jade algorithm and nir hyperspectral imaging for revealing food adulteration,” J. Food Eng. 168, 7–15 (2016). [CrossRef]
30. J. Cruz and M. Blanco, “Content uniformity studies in tablets by nir-ci,” J. Pharm. Biomed. Anal. 56(2), 408–412 (2011). [CrossRef]
31. L. de Moura França, M. F. Pimentel, S. da Silva Sim oes, S. Grangeiro Jr, J. M. Prats-Montalban, and A. Ferrer, “Nir hyperspectral imaging to evaluate degradation in captopril commercial tablets,” Eur. J. Pharm. Biopharm. 104, 180–188 (2016). [CrossRef]
32. J. Pearlman, S. Carman, C. Segal, P. Jarecke, P. Clancy, and W. Browne, “Overview of the hyperion imaging spectrometer for the nasa eo-1 mission,” in IGARSS 2001. Scanning the Present and Resolving the Future. Proceedings. IEEE 2001 International Geoscience and Remote Sensing Symposium (Cat. No. 01CH37217), vol. 7 (IEEE, 2001), pp. 3036–3038.
33. J. S. Pearlman, P. S. Barry, C. C. Segal, J. Shepanski, D. Beiso, and S. L. Carman, “Hyperion, a space-based imaging spectrometer,” IEEE Transactions on Geoscience and Remote Sensing 41(6), 1160–1173 (2003). [CrossRef]
34. C. Flynn, R. I. Stoian, B. D. Weers, J. E. Mullet, J. A. Thomasson, D. Alexander, and T. S. Tkaczyk, “Ruggedized, field-ready snapshot light-guide-based imaging spectrometer for environmental and remote sensing applications,” Opt. Express 30(7), 10614–10632 (2022). [CrossRef]
35. I. Baek, C. Mo, C. Eggleton, S. A. Gadsden, B.-K. Cho, J. Qin, D. E. Chan, and M. S. Kim, “Determination of spectral resolutions for multispectral detection of apple bruises using visible/near-infrared hyperspectral reflectance imaging,” Frontiers in plant science 13, 963591 (2022). [CrossRef]
36. P. Baranowski, W. Mazurek, J. Wozniak, and U. Majewska, “Detection of early bruises in apples using hyperspectral data and thermal imaging,” J. Food Eng. 110(3), 345–355 (2012). [CrossRef]
