Abstract

A new filter array and a demosaicking method for snapshot multispectral polarization imaging are proposed in this paper. The proposed filter array is a thin-film wavy multilayer structure regarded as a photonic crystal that can be fabricated using the autocloning method. The multispectral polarization filter array is developed by altering the wave structure of the photonic crystal at each pixel. In addition, we propose a demosaicking method for multispectral polarization images by considering snapshot imaging as a linear model. In the experiments, we evaluated the recovered spectrum error in some color charts and showed various demosaicked images such as multispectral polarization images, specific-band degree of linear polarization images, polarized RGB images, and non-polarized RGB images.

© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

The Bayer color filter array (CFA) has been widely used in conventional RGB digital cameras. These cameras that are based on a filter array mechanism can recover an RGB image using the demosaicking method after capturing one color channel at each pixel. Since this mechanism can achieve snapshot imaging in a simple manner, multispectral filter arrays (MSFAs) have been studied for multispectral imaging [1–4]. An MSFA has different color filter at each pixel and can capture a mosaicked image similar to a CFA. Reconstructing the original image from the mosaicked image is referred to as demosaicking.

The filter array mechanism can also be applied to polarizers. Some polarized filter arrays have been proposed in the literature [5–14]. V. Gruev [5] proposed a simple filter array consisting of 0°-, 45°-, 90°, and 135°-polarimeters at each pixel to obtain linear polarizations and developed a CCD device, PolarCam [8], which is commercially available. G. Myhre proposed a division-of-focal-plane polarimeter based on a dichroic dye and liquid crystal polymer guest-host system [12]. X. Tu designed an achromatic elliptical polarizer for a broadband division-of-focal-plane full-Stokes imaging polarimeter for the visible wavelength band using a combination of multiple birefringent waveplates [14]. Since these polarization imaging methods capture a monochrome image, they require multiple measurement shots for spectral analysis while changing the wavelength of the light (or the bandpass filter).

Multispectral polarization imaging enables the simultaneous acquisition of reflection, scattering, and transmission features that are beneficial for the detection and identification of an object; some multishot imaging techniques have been explored in the literature [15–17]. Y. Zhao [15] demonstrated a multispectral polarization imaging system for image segmentation. This system first splits an incident light into two parts using non-polarization beam splitters; multispectral images are obtained by rotating a spectral filter wheel with six narrow band filters, and polarization images are obtained by rotating a polarimetric filter wheel with four polarizers, whose azimuth angles are set at 0°, 45°, 90°, and 135°, respectively. C. Fu [17] proposed a compressive spectral polarization imaging system that consists of an objective lens, a colored detector aligned with a micropolarizer array, and a prism. This system requires multiple shots while rotating the prism.

The filter array mechanism is expected to contribute toward the further development of snapshot multispectral polarization imaging. S. Junger [18] demonstrated the fabrication of wire grid polarizers and hole arrays for color sensing using a complementary metal-oxide semiconductor (CMOS) process. M. Kulkarni [19] designed a multispectral polarization filter array whose each element comprises spectrally sensitive vertically stacked photodetectors integrated with a nanowire linear polarizer. X. Tu [20] proposed an optimization method for designing multispectral and polarization filters separately and introduced an example of three-band multispectral polarization filter arrays that can recover four Stokes parameters in simulation. T. York [21] and M. Garcia [22] both proposed a bio-inspired imager which captures co-registered color and polarization information in real time with high resolution by monolithically integrating nanowire polarizers with vertically-stacked photodetectors. These photodetectors capture three different spectral bands per pixel by exploiting wavelength-dependent depth absorption of photons.

However, the conventional snapshot multispectral polarization imaging approaches use a linear polarizer (i.e., polarizer having a strong extinction ratio) for polarization measurement, with the angle of the linear polarizer changed at each pixel. This polarization filter is required to transmit wide spectrum in one angle as developed in [23], and by contrast, it is required to cut wide spectrum incident at a right angle. Meanwhile, the spectral filter requires to transmit all polarization angles evenly regardless of its transparent wavelength. These constraints may cause an increase in manufacturing cost or a limiting the wavelength band.

A new filter array based on photonic crystals and a demosaicking method for snapshot multispectral polarization imaging are proposed in this paper. Figure 1 shows the conceptual example of a photonic crystal, and Fig. 2 shows the outline of the proposed multispectral polarization filter array imaging. As shown in Fig. 1, a two-dimensional multilayer wavy structure is fabricated as a photonic crystal. As the transparent spectral sensitivity and polarization property can be controlled by changing the lattice pitch and direction of the wavy structure, a multispectral polarization filter array can be obtained by changing the lattice pitch and direction at each pixel separately as shown in Fig. 2. A snapshot image can be obtained by attaching the proposed filter array to a monochrome camera sensor; however, each pixel value of a captured grayscale image has a different meaning depending on filter sensitivity. Therefore, we propose not only a filter array but also a demosaicking method for recovering multispectral polarization image. The proposed demosaicking method assumes filter array imaging as a linear model and then solves the l2 minimization problem of the estimation error between the original incident light and demosaicked image. In the experiments, we evaluate the spectral and polarization properties of the prototype filter array, and show various types of images such as non-polarized multispectral images, polarized RGB images, and degree of linear polarization (DOLP) images that can be demosaicked from a snapshot grayscale image.

 figure: Fig. 1

Fig. 1 Conceptual chart of photonic crystal.

Download Full Size | PPT Slide | PDF

 figure: Fig. 2

Fig. 2 Outline of the proposed multispectral polarization filter array imaging.

Download Full Size | PPT Slide | PDF

This paper is organized as follows: Section 2 discusses the proposed multispectral polarization filter array based on photonic crystals. In Section 3, we formulate the proposed imaging system as a linear model and propose a demosaicking method. An experiment to evaluate the method as well as the corresponding results are presented in Section 4. Section 5 concludes the study.

2. Multispectral polarization filter array based on photonic crystal

In this section, we briefly review the feature of 2D wave structure photonic crystals and show the design of the proposed multispectral polarization filter array.

A photonic crystal is an optical nanostructure with a period comparable to the wavelength of light. The transmitted spectrum is changed depending on the period. In particular, Y. Ohtera [24,25] proposed a photonic crystal filter (PhCF), which is a thin-film wavy multilayer structure that is fabricated by the autocloning method based on a radio frequency (RF) bias sputtering process. Autocloning is a method to fabricate a multilayered-type photonic crystal based on lithography and sputtering [26,27]. First, a lattice pattern is prepared on a substrate by lithography and dry etching. Next, by stacking multiple dielectric films on the substrate using the RF bias sputtering process, a structure that has refractive index modulation in both horizontal and vertical directions can be obtained. According to this method, by changing the period of lattice (in-plane lattice constant) on the initial substrate from one position to another, it is possible to fabricate multiple photonic crystal regions with different horizontal lattice structures and common vertical index profiles using a single sputtering process.

The PhCF has a different spectral sensitivity in transverse electric (TE) and transverse magnetic (TM) polarization modes due to form birefringence, as shown in Fig. 1. Here, an electric field parallel to the grooves and perpendicular to the plane of incidence is referred to as a TE mode, while a polarization orthogonal to it is referred to as a TM mode. Y. Ohtera [24,25] used only one of these modes to fabricate edge filters by attaching an additional linear polarization filter. In contrast, we focused on the transmittance difference in TE and TM modes and utilized both modes for multispectral polarization imaging.

We designed a multispectral polarization filter array as shown in Fig. 3. Figure 3(a) is a lattice pattern on a substrate in the lithography process. Each pixel size is 4, 650 to the pixel size of a monochrome imager (described in Section 4). The proposed filter array has 265, 280, 290, and 305nm pitches as filters 1, 2, 3, and 4, with each of them having four lattice directions, 0, 45, 90, and 135°. The total of filter types is 4 × 4 = 16 pixels. The filter array is arranged periodically in horizontal and vertical directions until 100 × 100 pixels. The lithography process, sputtering process, material, thickness, and other parameters of multilayer structures can be referred to in [24]. First, the photoresist for electron beam (EB) lithography (ZEP-520A, ZEON Co. Ltd.) was spin coated on a quartz substrate. The line and space patterns of the period according to Fig. 3(a) are written on the substrate by an EB lithography system (ELS-G125S, Elionix Inc.). Then, the substrate was dry etched through the photoresist by reactive ion etching. Finally, an alternating multilayer of 20 periods (40 layers in total) consisting of Nb2O5 and SiO2 was deposited using the RF bias sputtering. Figure 3(b) shows the scanning electron microscope (SEM) image of the surface of the fabricated filter array. From the experiments, it is clear that there is almost no disturbance of the pattern.

 figure: Fig. 3

Fig. 3 Designed multispectral polarization filter array.

Download Full Size | PPT Slide | PDF

Figure 4 shows the measured spectral transmittance of filters. Figure 4(a) shows the transmittance when the incident light is non-polarized, whereas (b) and (c) show TE and TM mode transmittance. Regions with the same pitch were assumed to represent the same transmission property regardless of their lattice angles. The transmittance in Fig. 4 can be derived from the average of the four lattice angles. As shown in Figs. 4(b) and 4(c), the fabricated filter has different spectral transmittances in TE and TM modes because of optical anisotropy. Moreover, because of changing lattice pitches, the spectrum peak moves from shorter to longer wavelength as shown from filters 1 to 4. For visual comparison, we reproduce sRGB components of the transmittance of filter array in three cases: the incident light is non-polarized, 0° polarized, and 90° polarized all over the filer array area, as shown in Fig. 5. The sRGB components are calculated from the transmittance of Fig. 4 and lattice angle of Fig. 3(a) using D65 standard illuminant, XYZ tristimulus function, and sRGB color conversion from XYZ. Each 2 × 2 pixel has the same color in Fig. 5(a) because the non-polarized transmittance does not depend on lattice angle. Meanwhile, Figs. 4(b) and 4(c) has a different color pattern because the lattice angle is changed at each pixel. From the results, it is clear that the fabricated multispectral polarization filter array has a different spectral and polarization transmittance property at each pixel.

 figure: Fig. 4

Fig. 4 Transmittance of filter array.

Download Full Size | PPT Slide | PDF

 figure: Fig. 5

Fig. 5 RGB reproduction of filter array (4 × 4pixel).

Download Full Size | PPT Slide | PDF

The proposed filter array based on PhCF has two advantages. First, we can control both spectral and polarization properties in one device (multilayer structure); also, there is no need to attach any bandpass filter or polarization filter. Second, we can change the filter array pattern and transmittance easily by changing the lattice pattern in the lithography process. Therefore, it is expected to suppress the development and manufacturing cost of multispectral polarization imaging using the proposed filter array mechanism.

However, the captured pixel value of the proposed filter array does not mean a single spectral band or a single polarization parameter because the transmittance is not a narrow bandpass. In addition, when comparing Fig. 4(b) and (c), the extinction ratio is changed depending on the wavelength (e.g., transmittance is 0.23 in the TE mode, whereas it is 0.40 in the TM mode at 600 nm in filter 1). From these (undesired) features, PhCF was used in a limited wavelength having a high extinction ratio [25]. We attempted to solve this problem by proposing a demosaicking algorithm that is suitable for the proposed filter array in Section 3.

3. Capturing and demosaicking model

We can produce snapshot multispectral polarization imaging by attaching the fabricated multi-spectral polarization filter array to a monochrome imager. However, it requires the demosaicking process to recover a multispectral polarization image from the captured grayscale image because various spectral and polarization components are mixed in one pixel.

S. Gao [28] and J. Zhang [29] proposed a gradient-based interpolation method that switches the bicubic convolution direction by using the spatial gradient. A. Ahmed [30] proposed a residual interpolation, where the residual is the difference between an observed and a tentatively estimated pixel value. Most of the interpolation methods in division of the focal plane polarization sensor are designed assuming that they will be applied to linear polarizers of 0, 45, 90, and 135° within a 2 × 2-pixel window. Therefore, they can obtain a local gradient (or local tentative estimated pixel) from a small area. However, in the proposed filter array, the observed pixels in the same channel (i.e. same spectral transmittance and polarization angle) are down-sampled at every 4 × 4 pixels. Moreover, the measured pixel value does not mean a single spectral band and a single polarization angle. Since we must first deconvolve the multispectral polarization information from the captured pixel value, it is a little difficult to apply conventional interpolation methods to the proposed filter array directly.

Therefore, a simple and fast demosaicking method of multispectral polarization images is proposed. The capturing process is considered as a linear model and various images are recovered by solving the inverse problem of the linear model. First, we define the first three Stokes parameters of the incident light at a pixel position (x, y) and a wavelength λ as sx,y,λ=[s0s1s2]x,y,λT. The relationship between the Stokes parameters and polarization intensity Ix,y,λ=[I0I45I90]x,y,λT (i.e., the intensity of multispectral polarization image) is

sx,y,λ=[101101121]Ix,y,λ=PIx,y,λ.
In the proposed imaging architecture using filter array and monochrome imager, the measured intensity signal of an imager gx,y is
gx,y=λTλaRθx,yMx,y,λRθx,ysx,y,λ,
where Tλ is the sensitivity at λ of an imager, a = [1 0 0] is the extraction vector of s0 by an imager, Rθx,y ∈ ℝ3×3 is the rotation matrix depending on the lattice angle θ at (x, y), Mx,y,λ ∈ ℝ3×3 is the matrix having left-up 3 × 3 components of the Mueller matrix of the filter at (x, y, λ). The right side of Eq. (2) is expressed as follows:
gx,y=λTλ[[m]x,y,λ0,0[m]x,y,λ0,1cos2θx,y[m]x,y,λ0,2sin2θx,y[m]x,y,λ0,1sin2θx,y+[m]x,y,λ0,2cos2θx,y]Tsx,y,λ,
where [m]x,y,λ1,j is the i-th row and j-th column element of M at (x, y, λ). Since the proposed filter array corresponds to a linear diattenuator [31] and has a different extinction ratio at each wavelength, each element of the Mueller matrix of the filter is
[m]x,y,λ0,0=[U]x,y,λTE+[U]x,y,λTM2,
[m]x,y,λ0,1=[U]x,y,λTE[U]x,y,λTM2,
[m]x,y,λ0,2=0,
where [U]x,y,λmode is a TE or TM mode transmittance of the filter at (x, y, λ) and corresponds to the transmittance of Figs. 4(b) and 4(c). Thus, Eq. (3) can be expressed as follows:
gx,y=12λTλ[[U]x,y,λTE+[U]x,y,λTM([U]x,y,λTE[U]x,y,λTM)cos2θx,y([U]x,y,λTE[U]x,y,λTM)sin2θx,y]Tsx,y,λ=12λTλMx,y,λsx,y,λ.
Therefore, the captured pixel value gx,y is determined by the sensitivity of the imager, TE and TM mode transmittance of the filter, lattice angle of each filter, and Stokes parameters of the incident light. This linear model has been mentioned in a focal plane array literature [32], but their model does not consider the spatial and spectral correlations for demosaicking. Therefore, we assume that this measurement model is independent at each pixel and can express this three-dimensional elements (x, y, λ) to one-dimensional data. We define the pixel size as (X, Y), the number of measurement spectral band as L, g=[g0,0g0,1gX1,Y1]TXY, s=[s0,0,0Ts0,0,1TsX1,Y1,L1T]T3XYL, and I=[I0,0,0TI0,0,1TIX1,Y1,L1T]T3XYL as a column vector. g can be expressed as follows:
g=TMs=TMPI=HI,
where, T ∈ ℝXY×XYL, M′ ∈ ℝXYL×3XYL, and P′ ∈ ℝ3XYL×3XYL are
T=12EXY[Tλ0Tλ1TλL1],
M=diag(M0,0,λ0,M0,0,λ1,M0,0,λL1,M0,1,λ0,,MX1,Y1,λL1),
P=EXYLP,
where En ∈ ℝn×n is the identify matrix, ⊗ is the Kronecker product, diag(X) is the diagonal matrix whose diagonal element is X.

Since g and H given in Eq. (8) are known values after snapshot imaging, demosaicking corresponds to the inverse problem of Eq. (8) as

I^=Wg.
In this paper, Wiener estimation [33] is applied for demosaicking that minimizes the l2-norm of the error between the original and estimated multispectral polarization images as
argminWII^2s.t.I^=Wg.
The norm term of Eq. (13) is expressed as follows:
II^2=Tr[RI2RIHTWT+WHRIHTWT],
where Tr [·] denotes the trace of a matrix and RI ∈ ℝ3XYL×3XYL is the autocorrelation matrix of I. Therefore, the matrix W is
W=RIHT(HRIHT)1.
We calculated RI as the first-order Markov model [34,35] in this paper.

Various types of images can be obtained from the demosaicked multispectral polarization image Î. In addition, since a simple inverse model is used for demosaicking, various images can be obtained from a snapshot grayscale image by multiplying only one constant matrix. For example, non-polarized multispectral image Îmsi ∈ ℝXYL (corresponds to the first Stokes parameter) is

I^msi=Wmsig,
Wmsi=(EXYL[101])W.
Similarly, non-polarized RGB image Îrgb ∈ ℝ3XY is
I^rgb=Wrgbg,
Wrgb=(EXY(CTL))Wmsi,
where C ∈ ℝ3×3 is the conversion matrix from XYZ to RGB, T ∈ ℝL is the XYZ tristimulus function, and L ∈ ℝL×L is the diagonal matrix of the standard illuminant.

There are two advantages of expressing the proposed imaging and demosaicking flow as a linear model. First, the computational complexity and time can be suppressed for snapshot imaging. This model can recover various images by only one matrix multiplication; in addition, it does not need to recover all the components of multispectral polarization image if a lower-dimensional image, such as MSI or RGB, is demosaicked. As shown in Eqs. (16) and (18), the dimension of Wmsi is XYL × XY, and the dimension of Wrgb is 3XY × XY. From these calculations, it is clear that they do not need to recover the multispectral polarization image vector I ∈ ℝ3XYL. Fast demosaicking may be advantageous for various snapshot imaging applications.

Second, various types of spectral transmittance are acceptable for filter array. The fabricated filter array is not an ideal linear polarizer and has various extinction ratios along the optical wavelength as shown in Fig. 4. The conventional study of PhCF [24] that did not apply the demosaicking method used only a limited wavelength area because the area having a strong extinction ratio was limited. The proposed filter array has various transmittances in both TE and TM modes; moreover, the proposed demosaicking model considers the diversity of transmittance. Although the recovered image quality depends on the filter array design, the proposed imaging and demosaicking architecture can cover a wide wavelength and various extinction ratios.

4. Experimental results

In the experiment, we attached the fabricated filter array to a monochrome imager (ICX205AL, SONY) by ultraviolet curing. For alignment, we used a manipulator while viewing a captured monochrome image in real time. In the UV curing process, the filter array was moved on the surface of the imager using a manipulator. We exposed UV light at a position which maximizes the contrast of the captured mosaic pattern.

We used an 8-bit USB camera casing (ARTCAM-150P5-BW-WOM, ARTCAM) for snapshot imaging as shown in Fig. 6. The exposure time of the camera was set to 80 ms. The recovering target wavelength was in the range of 420–720 nm at 10 nm interval (L = 31 bands); the D65 standard illuminant was used for RGB reproduction.

The spectral transmittance of 4 × 4 pixels after casing in the monochrome camera is shown in Fig. 7. Here, 0, 45, 90, and 135° indicate the lattice angle of the filter array. Almost all pixels have the same spectral transmittance, as shown in Fig. 4, therefore, there are no great problems in the UV curing and casing process. The transmittances of 90° of Figs. 7(f) and 7(g) both have a sharp dip in the transmittance peak. This feature may be caused by a crosstalk, but its effect on a captured pixel value is clearly small.

 figure: Fig. 6

Fig. 6 Implementation of the proposed snapshot camera with filter array.

Download Full Size | PPT Slide | PDF

 figure: Fig. 7

Fig. 7 Spectral transmittance in 16 pixels.

Download Full Size | PPT Slide | PDF

In the following experiments, first, the spectrum reflectance was compared with an original spectrum in color chart (ColorChecker Classic, x-rite); then, the quality of the recovered multispectral polarization image was evaluated by comparing various types of images.

The results of non-polarized spectral reflectance demosaicked from a captured grayscale image are shown in Fig. 8. Here, we compare the original measured spectrum using a spectrometer (USB2000+, OceanOptics) and the demosaicked spectrum where the patch index is 7, 8, 14, and 15. From the comparison, the form of the demosaicked spectrum approximately matches the original spectrum. However, major difference can be seen in wavelengths shorter than 500 nm of patch 14 and wavelengths longer than 650 nm of all patches. For visual comparison, we show the reproduced RGB image from the original spectrum in Fig. 9 and demosaicked non-polarized RGB image in Fig. 10. The visual colors of demosaicked RGB are close to the original spectrum; in particular, there is almost no difference in patches 7 and 8. The color of patches 14 and 15 is a little low in the demosaicked results. Table 1 shows the root mean square error (RMSE) of the spectrum and the color difference in CIE76 delta E. RMSEs are almost the same in all the patches, but the delta E of patches 14 and 15 is larger; this result matches with the visual comparison of Fig. 10.

 figure: Fig. 8

Fig. 8 Spectral reflectance comparison of color chart.

Download Full Size | PPT Slide | PDF

 figure: Fig. 9

Fig. 9 RGB reproduction of color chart from original spectrum.

Download Full Size | PPT Slide | PDF

 figure: Fig. 10

Fig. 10 Demosaicked RGB image of color chart.

Download Full Size | PPT Slide | PDF

Tables Icon

Table 1. RMSE of spectral reflectance and CIE76 delta E.

It is possible to reduce the demosaicked error by increasing the number of filter types. We designed only four types of transmittance (i.e., four types of lattice pitch) in Fig. 3; however, each filter has a similar transmittance curve in few wavelengths (particularly over 650 nm), as shown in Fig. 4(a). This similarity may weaken the linear independence with each filter and may lead to degradation of demosaicking quality [4]. It is easy to increase the number of filters in our method as mentioned in Section 2; therefore, a larger number of different lattice pitches should be used to improve the demosaicking quality in the future.

Next, we measured the DOLP error when capturing a wire grid polarizer (Edmund Optics, 34319) under a white light. This wire grid polarizer has a flat transmittance and over 200:1 extinction ratio for visible wavelengths. Therefore, we assumed that the ideal DOLP value is 1 for all wavelengths and calculated the RMSE between the ideal and demosaicked DOLP. Here, the DOLP value Idolp is calculated as follows:

[Idolp]x,y,λ=[s12+s22s0]x,y,λ.
Figure 11 shows the RMSE graph of DOLP for four angles of wire grid polarizer. The RMSE is small at the shorter wavelength, but it becomes large at the longer wavelength. This error may be improved by changing the filter array pattern as mentioned in the previous paragraph. On the other hand, the RMSE is not 1 for all wavelengths. Although the DOLP values are a little different from the ideal ones, it is clear that we can detect polarized properties at all wavelengths from a snapshot image.

We present captured grayscale and demosaicked images of a polarized object. The overview of captured objects is shown in Fig. 12. We set a smartphone (Nexus 5, Google) in front of the test chart (ISO 12233); the rectangle area is captured using the proposed snapshot camera. The liquid crystal display of this smartphone works as a 0° linear polarizer; therefore, it is expected that 90° linear polarization light of a demosaicked multispectral polarization image is darker than 0° in the display area. Note that Fig. 12 does not serve as a guidance of color correctness because this picture is captured by a non-calibrated RGB camera.

 figure: Fig. 12

Fig. 12 Captured scene. This image is captured by an RGB camera, and the color is not calibrated to sRGB correctly.

Download Full Size | PPT Slide | PDF

Figure 13 shows the captured and demosaicked images. Here, Fig. 13(a) shows the snapshot grayscale image, and all the other images are demosaicked from Fig. 13(a). Note that the pixel value is scaled by a factor of two in Fig. 13(b), (c), (e), and (f) because their intensity is half of the non-polarized intensity. The cyan texture appears in the display area of Fig. 13(b) when compared with Fig. 13(c). The RGB images of Fig. 13(d) to (f) can be recovered with almost the same color appearance and spatial structure of Fig. 12. In particular, it can be seen that the display of Fig. 13(f) is darker than that of (e) as expected. This result means that the display cuts the transmitted light of 90° linear polarization and works as a 0° linear polarizer. For more detailed analysis, the DOLP images at 500 nm and 600 nm are shown in Figs. 13(g) and 13(h), respectively.

 figure: Fig. 13

Fig. 13 Snapshot and demosaicked images.

Download Full Size | PPT Slide | PDF

As shown in Fig. 13(g), the DOLP value is large in the cyan area of the display, whereas the DOLP value is small in the magenta area. In Fig. 13(h), the DOLP values are almost the same in the display area, but not zero as compared to the test chart of the background. These results show the advantage of the proposed multispectral polarization imaging: even if an object has a polarized property and various spatial textures and colors, the proposed method can detect and show the polarization property from a snapshot image.

In contrast, the entire display of Fig. 13(f) appears as a weak magenta area and not as a complete dark area. One of the major reasons is that the captured pixel value of the display area is small, as shown in Fig. 13(a). Since the 90° polarization light was cut in the display area, the total intensity of the display area of the snapshot image is necessarily small. If the captured pixel value is small, it is difficult to recover the spectrum correctly. The bit depth of the camera needs to be increased from 8 to 16 bits to improve the demosaicked image quality.

Finally, we measured the computational time of the demosaicking algorithm. For this, the demosaicking algorithm was simulated from a random mosaicked image to a non-polarized RGB image in 1280 × 1024 pixels using Visual C++ 2015, Intel Core i7-5600U CPU (2.6 GHz), and 8 GB RAM. The computational time was 0.112 s in an average of 100 frames, which indicates that the proposed imaging system can work about 10 frames/second even if the system is a prototype and is not optimized for fast computing.

5. Conclusion

This paper proposed a photonic crystal-based filter array and a demosaicking method for snapshot multispectral polarization imaging. It was verified that by changing the lattice pattern, the fabricated filter array had a different spectral and polarization transmittance at each pixel. The demosaicked image quality of spectral reflectance of color charts and the polarized liquid crystal display were also shown. From the results, the proposed imaging system achieved a simple device (using only a multilayered structure film with a monochrome camera), simple demosaicking (using only one matrix multiplication), and wide spectrum recovery including visible wavelength (420–720 nm at 10 nm, 31 bands).

In future work, we will try to modify the lattice pattern of the filter array to increase the number of types of spectral transmittance. This modification may lead to improvement in demosaicking image quality at low cost as we are only modifying the pattern in EB lithography. On the other hand, there is definitely a trade-off between the number of filter types and the spatial resolution, as mentioned in our previous work [35]. We should not only increase the number of filter types but also explore the best balance between them in future. A new demosaicking method that considers a spatial-spectral-polarization gradient is also required to improve the quality of demosaicked images. Additionally, we have not investigated the crosstalk in detail, but it is thought that the crosstalk will occur in the filter array. In particular, since the proposed filter array has a different lattice structure at each pixel, the crosstalk effect may not be a uniform in the spatial-spectral-polarization dimension. Future investigations of crosstalk effects are required.

Funding

JSPS KAKENHI (18K11368).

Acknowledgments

The authors would like to thank Photonic Lattice and Prof. Masahiro Yamaguchi at the Tokyo Institute of Technology for their valuable suggestions. This work was supported by the Konica Minolta Imaging Science Encouragement Award.

References and links

1. J. Brauers and T. Aach, “A color filter array based multispectral camera,” Proc. of Workshop Farbbildverarbeitung (2006).

2. F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. on Image Process. 19(9), 2241–2253 (2010). [CrossRef]  

3. Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015). [CrossRef]   [PubMed]  

4. J. Jia, K. J. Barnard, and K. Hirakawa, “Fourier spectral filter array for optimal multispectral imaging,” IEEE Trans. on Image Process. 25(4), 1530–1543 (2016). [CrossRef]  

5. V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010). [CrossRef]   [PubMed]  

6. X. Zhao, A. Bermak, F. Boussaid, and V. G. Chigrinov, “Liquid-crystal micropolarimeter array for full Stokes polarization imaging in visible spectrum,” Opt. Express 18(17), 17776–17787 (2000). [CrossRef]  

7. D. S. Sabatke, M. R. Descour, E. L. Dereniak, W. C. Sweatt, S. A. Kemme, and G. S. Phipps, “Optimization of retardance for a complete Stokes polarimeter,” Opt. Lett. 25(11), 802–804 (2000). [CrossRef]  

8. 4D Technology, “Polarcam polarization camera,” http://www.4dtechnology.com.

9. T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009). [CrossRef]   [PubMed]  

10. Z. Xiaojin, F. Boussaid, A. Bermak, and V. G. Chigrinov, “Thin photo-patterned micropolarizer array for CMOS image sensors,” IEEE Photon. Technol. Lett. 21(12), 805–807 (2009). [CrossRef]  

11. V. Gruev, J. V. d. Spiegel, and N. Engheta, “Dual-tier thin film polymer polarization imaging sensor,” Opt. Express 18(18), 19292–19303 (2010). [CrossRef]  

12. G. Myhre, W. L. Hsu, A. Peinado, C. LaCasse, N. Brock, R. A. Chipman, and S. Pau, “Liquid crystal polymer full-stokes division of focal plane polarimeter,” Opt. Express 20(25), 27393–27409 (2012). [CrossRef]   [PubMed]  

13. T. Ohfuchi, M. Sakakura, Y. Yamada, N. Fukuda, T. Takiya, Y. Shimotsuma, and K. Miura, “Polarization imaging camera with a waveplate array fabricated with a femtosecond laser inside silica glass,” Opt. Express 25(20), 23738–23754 (2017). [CrossRef]   [PubMed]  

14. X. Tu, L. Jiang, M. I. Elhaj, and S. Pau, “Design, fabrication and testing of achromatic elliptical polarizer,” Opt. Express 25(9), 10355–10367 (2017). [CrossRef]   [PubMed]  

15. Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009). [CrossRef]  

16. Y. Zhao, L. Zhang, and Q. Pan, “Spectropolarimetric imaging for pathological analysis of skin,” Appl. Opt. 48(10), 236–246 (2009). [CrossRef]  

17. C. Fu, H. Arguello, B. M. Sadler, and G. R. Arce, “Compressive spectral polarization imaging by a pixelized polarizer and colored patterned detector,” J. Opt. Soc. Am. A 32(11), 2178–2188 (2015). [CrossRef]  

18. S. Junger, W. Tschekalinskij, N. Verwaal, and N. Weber, “Polarization and spectral filter arrays based on sub-wavelength structures in CMOS,” SENSOR+TEST Conferences, 161–165 (2011).

19. M. Kulkarni and V. Gruev, “Integrated spectral-polarization imaging sensor with aluminum nanowire polarization filters,” Opt. Express 20(21), 22997–23012 (2012). [CrossRef]   [PubMed]  

20. X. Tu and S. Pau, “Optimized design of N optical filters for color and polarization imaging,” Opt. Express 24(3), 3011–3024 (2016). [CrossRef]   [PubMed]  

21. T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014). [CrossRef]  

22. M. Garcia, C. Edmiston, R. Marinov, A. Vail, and V. Gruev, “Bio-inspired color-polarization imager for real-time in situ imaging,” Optica 4(10), 1263–1271 (2017). [CrossRef]  

23. J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007). [CrossRef]  

24. Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007). [CrossRef]  

25. Y. Ohtera, D. Kurniatan, and H. Yamada, “Design and fabrication of multichannel Si/SiO2 autocloned photonic crystal edge filters,” Appl. Opt. 50(9), 50–54 (2011). [CrossRef]  

26. S. Kawakami, “Fabrication of submicrometre 3D periodic structures composed of Si/SiO2,” Electron. Lett. 33(14), 1260–1261 (1997). [CrossRef]  

27. T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

28. S. Gao and V. Gruev, “Gradient-based interpolation method for division-of-focal-plane polarimeters,” Opt. Express 21(1), 1137–1151 (2013). [CrossRef]   [PubMed]  

29. J. Zhang, H. Luo, B. Hui, and Z. Chang, “Image interpolation for division of focal plane polarimeters with intensity correlation,” Opt. Express 24(18), 20799–20807 (2016). [CrossRef]   [PubMed]  

30. A. Ahmed, X. Zhao, V. Gruev, J. Zhang, and A. Bermak, “Residual interpolation for division of focal plane polarization image sensors,” Opt. Express 25(9), 10651–10662 (2017). [CrossRef]   [PubMed]  

31. R. A. Chipman, “Polarimetry,” in Handbook of Optics, 2nd ed. (McGraw-Hill, 1995), Chap 22 pp. 1–37.

32. S. B. Powell and V. Gruev, “Calibration methods for division-of-focal-plane polarimeters,” Opt. Express 21(18), 21039–21055 (2013). [CrossRef]   [PubMed]  

33. W. K. Pratt, Digital Image Processing (Wiley, 1978), pp. 356–367.

34. W. K. Pratt and D. E. Mancill, “Spectral estimation techniques for the spectral calibration of a color image scanner,” Appl. Opt. 15(1), 73–75 (1976). [CrossRef]   [PubMed]  

35. K. Shinoda, Y. Yanagi, Y. Hayasaki, and M. Hasegawa, “Multispectral filter array design without training images,” Optical Review 24(4), 554–571 (2017). [CrossRef]  

References

  • View by:
  • |
  • |
  • |

  1. J. Brauers and T. Aach, “A color filter array based multispectral camera,” Proc. of Workshop Farbbildverarbeitung (2006).
  2. F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. on Image Process. 19(9), 2241–2253 (2010).
    [Crossref]
  3. Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015).
    [Crossref] [PubMed]
  4. J. Jia, K. J. Barnard, and K. Hirakawa, “Fourier spectral filter array for optimal multispectral imaging,” IEEE Trans. on Image Process. 25(4), 1530–1543 (2016).
    [Crossref]
  5. V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010).
    [Crossref] [PubMed]
  6. X. Zhao, A. Bermak, F. Boussaid, and V. G. Chigrinov, “Liquid-crystal micropolarimeter array for full Stokes polarization imaging in visible spectrum,” Opt. Express 18(17), 17776–17787 (2000).
    [Crossref]
  7. D. S. Sabatke, M. R. Descour, E. L. Dereniak, W. C. Sweatt, S. A. Kemme, and G. S. Phipps, “Optimization of retardance for a complete Stokes polarimeter,” Opt. Lett. 25(11), 802–804 (2000).
    [Crossref]
  8. 4D Technology, “Polarcam polarization camera,” http://www.4dtechnology.com .
  9. T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009).
    [Crossref] [PubMed]
  10. Z. Xiaojin, F. Boussaid, A. Bermak, and V. G. Chigrinov, “Thin photo-patterned micropolarizer array for CMOS image sensors,” IEEE Photon. Technol. Lett. 21(12), 805–807 (2009).
    [Crossref]
  11. V. Gruev, J. V. d. Spiegel, and N. Engheta, “Dual-tier thin film polymer polarization imaging sensor,” Opt. Express 18(18), 19292–19303 (2010).
    [Crossref]
  12. G. Myhre, W. L. Hsu, A. Peinado, C. LaCasse, N. Brock, R. A. Chipman, and S. Pau, “Liquid crystal polymer full-stokes division of focal plane polarimeter,” Opt. Express 20(25), 27393–27409 (2012).
    [Crossref] [PubMed]
  13. T. Ohfuchi, M. Sakakura, Y. Yamada, N. Fukuda, T. Takiya, Y. Shimotsuma, and K. Miura, “Polarization imaging camera with a waveplate array fabricated with a femtosecond laser inside silica glass,” Opt. Express 25(20), 23738–23754 (2017).
    [Crossref] [PubMed]
  14. X. Tu, L. Jiang, M. I. Elhaj, and S. Pau, “Design, fabrication and testing of achromatic elliptical polarizer,” Opt. Express 25(9), 10355–10367 (2017).
    [Crossref] [PubMed]
  15. Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009).
    [Crossref]
  16. Y. Zhao, L. Zhang, and Q. Pan, “Spectropolarimetric imaging for pathological analysis of skin,” Appl. Opt. 48(10), 236–246 (2009).
    [Crossref]
  17. C. Fu, H. Arguello, B. M. Sadler, and G. R. Arce, “Compressive spectral polarization imaging by a pixelized polarizer and colored patterned detector,” J. Opt. Soc. Am. A 32(11), 2178–2188 (2015).
    [Crossref]
  18. S. Junger, W. Tschekalinskij, N. Verwaal, and N. Weber, “Polarization and spectral filter arrays based on sub-wavelength structures in CMOS,” SENSOR+TEST Conferences, 161–165 (2011).
  19. M. Kulkarni and V. Gruev, “Integrated spectral-polarization imaging sensor with aluminum nanowire polarization filters,” Opt. Express 20(21), 22997–23012 (2012).
    [Crossref] [PubMed]
  20. X. Tu and S. Pau, “Optimized design of N optical filters for color and polarization imaging,” Opt. Express 24(3), 3011–3024 (2016).
    [Crossref] [PubMed]
  21. T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
    [Crossref]
  22. M. Garcia, C. Edmiston, R. Marinov, A. Vail, and V. Gruev, “Bio-inspired color-polarization imager for real-time in situ imaging,” Optica 4(10), 1263–1271 (2017).
    [Crossref]
  23. J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
    [Crossref]
  24. Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007).
    [Crossref]
  25. Y. Ohtera, D. Kurniatan, and H. Yamada, “Design and fabrication of multichannel Si/SiO2 autocloned photonic crystal edge filters,” Appl. Opt. 50(9), 50–54 (2011).
    [Crossref]
  26. S. Kawakami, “Fabrication of submicrometre 3D periodic structures composed of Si/SiO2,” Electron. Lett. 33(14), 1260–1261 (1997).
    [Crossref]
  27. T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).
  28. S. Gao and V. Gruev, “Gradient-based interpolation method for division-of-focal-plane polarimeters,” Opt. Express 21(1), 1137–1151 (2013).
    [Crossref] [PubMed]
  29. J. Zhang, H. Luo, B. Hui, and Z. Chang, “Image interpolation for division of focal plane polarimeters with intensity correlation,” Opt. Express 24(18), 20799–20807 (2016).
    [Crossref] [PubMed]
  30. A. Ahmed, X. Zhao, V. Gruev, J. Zhang, and A. Bermak, “Residual interpolation for division of focal plane polarization image sensors,” Opt. Express 25(9), 10651–10662 (2017).
    [Crossref] [PubMed]
  31. R. A. Chipman, “Polarimetry,” in Handbook of Optics, 2nd ed. (McGraw-Hill, 1995), Chap 22 pp. 1–37.
  32. S. B. Powell and V. Gruev, “Calibration methods for division-of-focal-plane polarimeters,” Opt. Express 21(18), 21039–21055 (2013).
    [Crossref] [PubMed]
  33. W. K. Pratt, Digital Image Processing (Wiley, 1978), pp. 356–367.
  34. W. K. Pratt and D. E. Mancill, “Spectral estimation techniques for the spectral calibration of a color image scanner,” Appl. Opt. 15(1), 73–75 (1976).
    [Crossref] [PubMed]
  35. K. Shinoda, Y. Yanagi, Y. Hayasaki, and M. Hasegawa, “Multispectral filter array design without training images,” Optical Review 24(4), 554–571 (2017).
    [Crossref]

2017 (5)

2016 (3)

2015 (2)

Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015).
[Crossref] [PubMed]

C. Fu, H. Arguello, B. M. Sadler, and G. R. Arce, “Compressive spectral polarization imaging by a pixelized polarizer and colored patterned detector,” J. Opt. Soc. Am. A 32(11), 2178–2188 (2015).
[Crossref]

2014 (1)

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

2013 (2)

2012 (2)

2011 (1)

2010 (3)

2009 (4)

T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009).
[Crossref] [PubMed]

Z. Xiaojin, F. Boussaid, A. Bermak, and V. G. Chigrinov, “Thin photo-patterned micropolarizer array for CMOS image sensors,” IEEE Photon. Technol. Lett. 21(12), 805–807 (2009).
[Crossref]

Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009).
[Crossref]

Y. Zhao, L. Zhang, and Q. Pan, “Spectropolarimetric imaging for pathological analysis of skin,” Appl. Opt. 48(10), 236–246 (2009).
[Crossref]

2007 (2)

J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
[Crossref]

Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007).
[Crossref]

2004 (1)

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

2000 (2)

1997 (1)

S. Kawakami, “Fabrication of submicrometre 3D periodic structures composed of Si/SiO2,” Electron. Lett. 33(14), 1260–1261 (1997).
[Crossref]

1976 (1)

Aach, T.

J. Brauers and T. Aach, “A color filter array based multispectral camera,” Proc. of Workshop Farbbildverarbeitung (2006).

Achilefu, S.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Ahmed, A.

Aoyama, T.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Arce, G. R.

Arguello, H.

Baba, A.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Barnard, K. J.

J. Jia, K. J. Barnard, and K. Hirakawa, “Fourier spectral filter array for optimal multispectral imaging,” IEEE Trans. on Image Process. 25(4), 1530–1543 (2016).
[Crossref]

Bermak, A.

Boussaid, F.

Z. Xiaojin, F. Boussaid, A. Bermak, and V. G. Chigrinov, “Thin photo-patterned micropolarizer array for CMOS image sensors,” IEEE Photon. Technol. Lett. 21(12), 805–807 (2009).
[Crossref]

X. Zhao, A. Bermak, F. Boussaid, and V. G. Chigrinov, “Liquid-crystal micropolarimeter array for full Stokes polarization imaging in visible spectrum,” Opt. Express 18(17), 17776–17787 (2000).
[Crossref]

Brauers, J.

J. Brauers and T. Aach, “A color filter array based multispectral camera,” Proc. of Workshop Farbbildverarbeitung (2006).

Brock, N.

Chang, Z.

Charanya, T.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Chigrinov, V. G.

Z. Xiaojin, F. Boussaid, A. Bermak, and V. G. Chigrinov, “Thin photo-patterned micropolarizer array for CMOS image sensors,” IEEE Photon. Technol. Lett. 21(12), 805–807 (2009).
[Crossref]

X. Zhao, A. Bermak, F. Boussaid, and V. G. Chigrinov, “Liquid-crystal micropolarimeter array for full Stokes polarization imaging in visible spectrum,” Opt. Express 18(17), 17776–17787 (2000).
[Crossref]

Chipman, R. A.

Cronin, T. W.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Deng, X.

J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
[Crossref]

Dereniak, E. L.

Descour, M. R.

Edmiston, C.

Elhaj, M. I.

Engheta, N.

Fu, C.

Fukuda, N.

Gao, S.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

S. Gao and V. Gruev, “Gradient-based interpolation method for division-of-focal-plane polarimeters,” Opt. Express 21(1), 1137–1151 (2013).
[Crossref] [PubMed]

Garcia, M.

Gruev, V.

M. Garcia, C. Edmiston, R. Marinov, A. Vail, and V. Gruev, “Bio-inspired color-polarization imager for real-time in situ imaging,” Optica 4(10), 1263–1271 (2017).
[Crossref]

A. Ahmed, X. Zhao, V. Gruev, J. Zhang, and A. Bermak, “Residual interpolation for division of focal plane polarization image sensors,” Opt. Express 25(9), 10651–10662 (2017).
[Crossref] [PubMed]

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

S. B. Powell and V. Gruev, “Calibration methods for division-of-focal-plane polarimeters,” Opt. Express 21(18), 21039–21055 (2013).
[Crossref] [PubMed]

S. Gao and V. Gruev, “Gradient-based interpolation method for division-of-focal-plane polarimeters,” Opt. Express 21(1), 1137–1151 (2013).
[Crossref] [PubMed]

M. Kulkarni and V. Gruev, “Integrated spectral-polarization imaging sensor with aluminum nanowire polarization filters,” Opt. Express 20(21), 22997–23012 (2012).
[Crossref] [PubMed]

V. Gruev, J. V. d. Spiegel, and N. Engheta, “Dual-tier thin film polymer polarization imaging sensor,” Opt. Express 18(18), 19292–19303 (2010).
[Crossref]

V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010).
[Crossref] [PubMed]

Hasegawa, M.

K. Shinoda, Y. Yanagi, Y. Hayasaki, and M. Hasegawa, “Multispectral filter array design without training images,” Optical Review 24(4), 554–571 (2017).
[Crossref]

Hashimoto, N.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Hayasaki, Y.

K. Shinoda, Y. Yanagi, Y. Hayasaki, and M. Hasegawa, “Multispectral filter array design without training images,” Optical Review 24(4), 554–571 (2017).
[Crossref]

Hirakawa, K.

J. Jia, K. J. Barnard, and K. Hirakawa, “Fourier spectral filter array for optimal multispectral imaging,” IEEE Trans. on Image Process. 25(4), 1530–1543 (2016).
[Crossref]

Hsu, W. L.

Hui, B.

Inoue, Y.

Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007).
[Crossref]

Ishikawa, W.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Iso, D.

F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. on Image Process. 19(9), 2241–2253 (2010).
[Crossref]

Jia, J.

J. Jia, K. J. Barnard, and K. Hirakawa, “Fourier spectral filter array for optimal multispectral imaging,” IEEE Trans. on Image Process. 25(4), 1530–1543 (2016).
[Crossref]

Jiang, L.

Junger, S.

S. Junger, W. Tschekalinskij, N. Verwaal, and N. Weber, “Polarization and spectral filter arrays based on sub-wavelength structures in CMOS,” SENSOR+TEST Conferences, 161–165 (2011).

Kahan, L.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Kawakami, S.

Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007).
[Crossref]

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

S. Kawakami, “Fabrication of submicrometre 3D periodic structures composed of Si/SiO2,” Electron. Lett. 33(14), 1260–1261 (1997).
[Crossref]

Kawashima, T.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Kemme, S. A.

Kikuchi, S.

Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015).
[Crossref] [PubMed]

Kulkarni, M.

Kurniatan, D.

LaCasse, C.

Lake, S. P.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Liu, X.

J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
[Crossref]

Luo, H.

Mancill, D. E.

Marinov, R.

Marshall, J.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Mitsunaga, T.

F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. on Image Process. 19(9), 2241–2253 (2010).
[Crossref]

Miura, K.

T. Ohfuchi, M. Sakakura, Y. Yamada, N. Fukuda, T. Takiya, Y. Shimotsuma, and K. Miura, “Polarization imaging camera with a waveplate array fabricated with a femtosecond laser inside silica glass,” Opt. Express 25(20), 23738–23754 (2017).
[Crossref] [PubMed]

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Monno, Y.

Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015).
[Crossref] [PubMed]

Myhre, G.

Nayar, S. K.

F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. on Image Process. 19(9), 2241–2253 (2010).
[Crossref]

Ohfuchi, T.

Ohkubo, H.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Ohta, J.

T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009).
[Crossref] [PubMed]

Ohtera, Y.

Y. Ohtera, D. Kurniatan, and H. Yamada, “Design and fabrication of multichannel Si/SiO2 autocloned photonic crystal edge filters,” Appl. Opt. 50(9), 50–54 (2011).
[Crossref]

Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007).
[Crossref]

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Okutomi, M.

Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015).
[Crossref] [PubMed]

Onuki, T.

Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007).
[Crossref]

Pan, Q.

Y. Zhao, L. Zhang, and Q. Pan, “Spectropolarimetric imaging for pathological analysis of skin,” Appl. Opt. 48(10), 236–246 (2009).
[Crossref]

Pana, Q.

Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009).
[Crossref]

Pau, S.

Peinado, A.

Perkins, R.

Phipps, G. S.

Powell, S. B.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

S. B. Powell and V. Gruev, “Calibration methods for division-of-focal-plane polarimeters,” Opt. Express 21(18), 21039–21055 (2013).
[Crossref] [PubMed]

Pratt, W. K.

Raman, B.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Roberts, N. W.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Sabatke, D. S.

Sadler, B. M.

Saha, D.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Sakakura, M.

Sasagawa, K.

T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009).
[Crossref] [PubMed]

Sasaki, Y.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Sato, T.

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

Sciortino, P.

J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
[Crossref]

Shimotsuma, Y.

Shinoda, K.

K. Shinoda, Y. Yanagi, Y. Hayasaki, and M. Hasegawa, “Multispectral filter array design without training images,” Optical Review 24(4), 554–571 (2017).
[Crossref]

Spiegel, J. V. d.

Sweatt, W. C.

Takiya, T.

Tanaka, M.

Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015).
[Crossref] [PubMed]

Tokuda, T.

T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009).
[Crossref] [PubMed]

Tschekalinskij, W.

S. Junger, W. Tschekalinskij, N. Verwaal, and N. Weber, “Polarization and spectral filter arrays based on sub-wavelength structures in CMOS,” SENSOR+TEST Conferences, 161–165 (2011).

Tu, X.

Vail, A.

Verwaal, N.

S. Junger, W. Tschekalinskij, N. Verwaal, and N. Weber, “Polarization and spectral filter arrays based on sub-wavelength structures in CMOS,” SENSOR+TEST Conferences, 161–165 (2011).

Walters, F.

J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
[Crossref]

Wang, J. J.

J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
[Crossref]

Weber, N.

S. Junger, W. Tschekalinskij, N. Verwaal, and N. Weber, “Polarization and spectral filter arrays based on sub-wavelength structures in CMOS,” SENSOR+TEST Conferences, 161–165 (2011).

Xiaojin, Z.

Z. Xiaojin, F. Boussaid, A. Bermak, and V. G. Chigrinov, “Thin photo-patterned micropolarizer array for CMOS image sensors,” IEEE Photon. Technol. Lett. 21(12), 805–807 (2009).
[Crossref]

Yamada, H.

Y. Ohtera, D. Kurniatan, and H. Yamada, “Design and fabrication of multichannel Si/SiO2 autocloned photonic crystal edge filters,” Appl. Opt. 50(9), 50–54 (2011).
[Crossref]

T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009).
[Crossref] [PubMed]

Yamada, Y.

Yanagi, Y.

K. Shinoda, Y. Yanagi, Y. Hayasaki, and M. Hasegawa, “Multispectral filter array design without training images,” Optical Review 24(4), 554–571 (2017).
[Crossref]

Yasuma, F.

F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. on Image Process. 19(9), 2241–2253 (2010).
[Crossref]

York, T.

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010).
[Crossref] [PubMed]

Zhang, D.

Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009).
[Crossref]

Zhang, J.

Zhang, L.

Y. Zhao, L. Zhang, and Q. Pan, “Spectropolarimetric imaging for pathological analysis of skin,” Appl. Opt. 48(10), 236–246 (2009).
[Crossref]

Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009).
[Crossref]

Zhao, X.

Zhao, Y.

Y. Zhao, L. Zhang, and Q. Pan, “Spectropolarimetric imaging for pathological analysis of skin,” Appl. Opt. 48(10), 236–246 (2009).
[Crossref]

Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009).
[Crossref]

Appl. Opt. (3)

Appl. Phys. Lett. (1)

J. J. Wang, F. Walters, X. Liu, P. Sciortino, and X. Deng, “High-performance, large area, deep ultraviolet to infrared polarizers based on 40 nm line/78 nm space nanowire grids,” Appl. Phys. Lett. 90, 061104 (2007).
[Crossref]

Computer Vision and Image Understanding (1)

Y. Zhao, L. Zhang, D. Zhang, and Q. Pana, “Object separation by polarimetric and spectral imagery fusion,” Computer Vision and Image Understanding 113(8), 855–866 (2009).
[Crossref]

Electron. Lett. (1)

S. Kawakami, “Fabrication of submicrometre 3D periodic structures composed of Si/SiO2,” Electron. Lett. 33(14), 1260–1261 (1997).
[Crossref]

IEEE J. Lightwave Technol. (1)

Y. Ohtera, T. Onuki, Y. Inoue, and S. Kawakami, “Multichannel Photonic Crystal Wavelength Filter Array for Near-Infrared Wavelengths,” IEEE J. Lightwave Technol. 25(2), 499–503 (2007).
[Crossref]

IEEE Photon. Technol. Lett. (1)

Z. Xiaojin, F. Boussaid, A. Bermak, and V. G. Chigrinov, “Thin photo-patterned micropolarizer array for CMOS image sensors,” IEEE Photon. Technol. Lett. 21(12), 805–807 (2009).
[Crossref]

IEEE Trans. Biomed. Circuits Syst. (1)

T. Tokuda, H. Yamada, K. Sasagawa, and J. Ohta, “Polarization-analyzing CMOS image sensor with monolithically embedded polarizer for microchemistry systems,” IEEE Trans. Biomed. Circuits Syst. 3(5), 259–266 (2009).
[Crossref] [PubMed]

IEEE Trans. Image Process. (1)

Y. Monno, S. Kikuchi, M. Tanaka, and M. Okutomi, “A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor,” IEEE Trans. Image Process. 24(10), 3048–3059 (2015).
[Crossref] [PubMed]

IEEE Trans. on Image Process. (2)

J. Jia, K. J. Barnard, and K. Hirakawa, “Fourier spectral filter array for optimal multispectral imaging,” IEEE Trans. on Image Process. 25(4), 1530–1543 (2016).
[Crossref]

F. Yasuma, T. Mitsunaga, D. Iso, and S. K. Nayar, “Generalized assorted pixel camera: Postcapture control of resolution, dynamic range, and spectrum,” IEEE Trans. on Image Process. 19(9), 2241–2253 (2010).
[Crossref]

IEICE Trans. Electron. (1)

T. Kawashima, Y. Sasaki, K. Miura, N. Hashimoto, A. Baba, H. Ohkubo, Y. Ohtera, T. Sato, W. Ishikawa, T. Aoyama, and S. Kawakami, “Development of autocloned photonic crystal devices,” IEICE Trans. Electron. E87-C(3), 283–290 (2004).

J. Opt. Soc. Am. A (1)

Opt. Express (12)

M. Kulkarni and V. Gruev, “Integrated spectral-polarization imaging sensor with aluminum nanowire polarization filters,” Opt. Express 20(21), 22997–23012 (2012).
[Crossref] [PubMed]

X. Tu and S. Pau, “Optimized design of N optical filters for color and polarization imaging,” Opt. Express 24(3), 3011–3024 (2016).
[Crossref] [PubMed]

V. Gruev, R. Perkins, and T. York, “CCD polarization imaging sensor with aluminum nanowire optical filters,” Opt. Express 18(18), 19087–19094 (2010).
[Crossref] [PubMed]

X. Zhao, A. Bermak, F. Boussaid, and V. G. Chigrinov, “Liquid-crystal micropolarimeter array for full Stokes polarization imaging in visible spectrum,” Opt. Express 18(17), 17776–17787 (2000).
[Crossref]

V. Gruev, J. V. d. Spiegel, and N. Engheta, “Dual-tier thin film polymer polarization imaging sensor,” Opt. Express 18(18), 19292–19303 (2010).
[Crossref]

G. Myhre, W. L. Hsu, A. Peinado, C. LaCasse, N. Brock, R. A. Chipman, and S. Pau, “Liquid crystal polymer full-stokes division of focal plane polarimeter,” Opt. Express 20(25), 27393–27409 (2012).
[Crossref] [PubMed]

T. Ohfuchi, M. Sakakura, Y. Yamada, N. Fukuda, T. Takiya, Y. Shimotsuma, and K. Miura, “Polarization imaging camera with a waveplate array fabricated with a femtosecond laser inside silica glass,” Opt. Express 25(20), 23738–23754 (2017).
[Crossref] [PubMed]

X. Tu, L. Jiang, M. I. Elhaj, and S. Pau, “Design, fabrication and testing of achromatic elliptical polarizer,” Opt. Express 25(9), 10355–10367 (2017).
[Crossref] [PubMed]

S. Gao and V. Gruev, “Gradient-based interpolation method for division-of-focal-plane polarimeters,” Opt. Express 21(1), 1137–1151 (2013).
[Crossref] [PubMed]

J. Zhang, H. Luo, B. Hui, and Z. Chang, “Image interpolation for division of focal plane polarimeters with intensity correlation,” Opt. Express 24(18), 20799–20807 (2016).
[Crossref] [PubMed]

A. Ahmed, X. Zhao, V. Gruev, J. Zhang, and A. Bermak, “Residual interpolation for division of focal plane polarization image sensors,” Opt. Express 25(9), 10651–10662 (2017).
[Crossref] [PubMed]

S. B. Powell and V. Gruev, “Calibration methods for division-of-focal-plane polarimeters,” Opt. Express 21(18), 21039–21055 (2013).
[Crossref] [PubMed]

Opt. Lett. (1)

Optica (1)

Optical Review (1)

K. Shinoda, Y. Yanagi, Y. Hayasaki, and M. Hasegawa, “Multispectral filter array design without training images,” Optical Review 24(4), 554–571 (2017).
[Crossref]

Proceedings of the IEEE (1)

T. York, S. B. Powell, S. Gao, L. Kahan, T. Charanya, D. Saha, N. W. Roberts, T. W. Cronin, J. Marshall, S. Achilefu, S. P. Lake, B. Raman, and V. Gruev, “Bioinspired polarization imaging sensors: from circuits and optics to signal processing algorithms and biomedical applications,” Proceedings of the IEEE 102, 1450–1469 (2014).
[Crossref]

Other (5)

S. Junger, W. Tschekalinskij, N. Verwaal, and N. Weber, “Polarization and spectral filter arrays based on sub-wavelength structures in CMOS,” SENSOR+TEST Conferences, 161–165 (2011).

J. Brauers and T. Aach, “A color filter array based multispectral camera,” Proc. of Workshop Farbbildverarbeitung (2006).

4D Technology, “Polarcam polarization camera,” http://www.4dtechnology.com .

W. K. Pratt, Digital Image Processing (Wiley, 1978), pp. 356–367.

R. A. Chipman, “Polarimetry,” in Handbook of Optics, 2nd ed. (McGraw-Hill, 1995), Chap 22 pp. 1–37.

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (13)

Fig. 1
Fig. 1 Conceptual chart of photonic crystal.
Fig. 2
Fig. 2 Outline of the proposed multispectral polarization filter array imaging.
Fig. 3
Fig. 3 Designed multispectral polarization filter array.
Fig. 4
Fig. 4 Transmittance of filter array.
Fig. 5
Fig. 5 RGB reproduction of filter array (4 × 4pixel).
Fig. 6
Fig. 6 Implementation of the proposed snapshot camera with filter array.
Fig. 7
Fig. 7 Spectral transmittance in 16 pixels.
Fig. 8
Fig. 8 Spectral reflectance comparison of color chart.
Fig. 9
Fig. 9 RGB reproduction of color chart from original spectrum.
Fig. 10
Fig. 10 Demosaicked RGB image of color chart.
Fig. 11
Fig. 11 RMSE of DOLP.
Fig. 12
Fig. 12 Captured scene. This image is captured by an RGB camera, and the color is not calibrated to sRGB correctly.
Fig. 13
Fig. 13 Snapshot and demosaicked images.

Tables (1)

Tables Icon

Table 1 RMSE of spectral reflectance and CIE76 delta E.

Equations (20)

Equations on this page are rendered with MathJax. Learn more.

s x , y , λ = [ 1 0 1 1 0 1 1 2 1 ] I x , y , λ = PI x , y , λ .
g x , y = λ T λ aR θ x , y M x , y , λ R θ x , y s x , y , λ ,
g x , y = λ T λ [ [ m ] x , y , λ 0 , 0 [ m ] x , y , λ 0 , 1 cos 2 θ x , y [ m ] x , y , λ 0 , 2 sin 2 θ x , y [ m ] x , y , λ 0 , 1 sin 2 θ x , y + [ m ] x , y , λ 0 , 2 cos 2 θ x , y ] T s x , y , λ ,
[ m ] x , y , λ 0 , 0 = [ U ] x , y , λ T E + [ U ] x , y , λ T M 2 ,
[ m ] x , y , λ 0 , 1 = [ U ] x , y , λ T E [ U ] x , y , λ T M 2 ,
[ m ] x , y , λ 0 , 2 = 0 ,
g x , y = 1 2 λ T λ [ [ U ] x , y , λ T E + [ U ] x , y , λ T M ( [ U ] x , y , λ T E [ U ] x , y , λ T M ) cos 2 θ x , y ( [ U ] x , y , λ T E [ U ] x , y , λ T M ) sin 2 θ x , y ] T s x , y , λ = 1 2 λ T λ M x , y , λ s x , y , λ .
g = TM s = TM P I = HI ,
T = 1 2 E X Y [ T λ 0 T λ 1 T λ L 1 ] ,
M = diag ( M 0 , 0 , λ 0 , M 0 , 0 , λ 1 , M 0 , 0 , λ L 1 , M 0 , 1 , λ 0 , , M X 1 , Y 1 , λ L 1 ) ,
P = E X Y L P ,
I ^ = Wg .
arg min W I I ^ 2 s . t . I ^ = Wg .
I I ^ 2 = Tr [ R I 2 R I H T W T + WH R I H T W T ] ,
W = R I H T ( H R I H T ) 1 .
I ^ msi = W msi g ,
W msi = ( E X Y L [ 1 0 1 ] ) W .
I ^ rgb = W rgb g ,
W rgb = ( E X Y ( CTL ) ) W msi ,
[ I dolp ] x , y , λ = [ s 1 2 + s 2 2 s 0 ] x , y , λ .

Metrics