1. Introduction

Reflectance spectra can be used in color characterization, object recognition, material identification and many other fields. Over the past few decades, multispectral imaging systems have been developed to provide a better solution to spectral reflectance measurement than traditional point-based spectrophotometer, and these systems has been successfully applied in many fields [1–4]. Multispectral imaging system transforms reflected optical radiation into multi-channel response values via optical lens and large-area sensor of camera, thus the spectral reflectance of imaging objects and scenes can be reconstructed from corresponding multi-channel response values. Conventional multispectral imaging systems use monochrome camera, combined with narrowband optical filters [5] or programmable LED light sources [6]. Due to the dependence on specific equipment, technical complexity, slow imaging speed and low economy, applications of these techniques are limited at the laboratory level. Alternatively, consumer-level trichromatic camera associated with broadband filters [7] or optimal combinations of light source [8] is a more feasible scheme. Although there exist some compromises on accuracy, it is a more practical and promising solution, especially where advantages of easily obtained camera equipment, loose imaging environment, and three channels per shooting as well as high-resolution pictures are concerned.

However, because of the inherent nonlinearity of RGB camera imaging [9], multi-channel response values yielded by this technique should be carefully post-processed by reasonable spectra reconstruction method to estimate original spectral reflectance, thus the reconstruction method of high-precision reflectance spectra based on trichromatic camera multispectral systems have attracted more and more attentions in recent years.

Generally, dimensions of response obtained by these systems are usually much less than dimensions of reconstructed reflectance spectra, so reconstruction of high-dimensional spectral reflectance from low-dimensional response signal is an information recovery process from compression, which is mathematically called ill-posed inverse problem [10]. Traditionally, Wiener method based on linear imaging model could be used for spectral reflectance reconstruction in [11,12]. However, considering the dependence on system priori knowledge of Wiener method, more generalized methods based on statistical leaning framework [13] are preferred. In this framework, a parametric or nonparametric model is constructed based on training samples and used to give best estimation on testing samples. Ordinary least square (OLS) regression method [14] and modifications [15] are commonly suggested in the field of learn-based spectral reflectance reconstruction. Agahian et al. [16] first introduced a method that used color difference between training and test samples as a weighted matrix for optimal selection of training samples. Based on this idea, Amiri et al. [17] proposed a method that made weighting least square (WLS) regression on polynomial extension of the sample response matrix. And Heikkinen et al. proposed kernel ridge regression (KRR) method [18] for spectral reflectance. Essentially, KRR is a method that nonlinearly transforms low-dimensional camera response into high-dimensional feature space, and conducts regularized least square regression of reflectance data in the feature space. In contrast to least square methods above, Shen et al. reported that partial least squares regression (PLS) method [19] also could be adopted in constructing a regression model based on the correlation between response value and spectral reflectance. All these studies claimed to have achieved good results on different metrics.

Since the photoelectricity transfer function [9] is non-linear, the relationship between response values of the camera and spectral reflectance of imaging objects is non-linear, so nonlinear reflectance reconstruction model is necessary. Vapnik [13] proposed a method to extend the linear model to the nonlinear model, named kernel function, or kernel trick. By using kernel function, input data of the samples can be mapped into a (nonlinear) feature space of high dimension (even infinite dimension), and suitable regression parameters can be found in the high-dimensional feature space. Therefore, it is a promising method that using kernel function to increase dimensions of the original response as well as characterizing the nonlinearity of spectral reflectance estimation of multispectral imaging system.

As we know, there is a potential multidimensional collinearity in either the original response space, or the polynomial extended response space, or the high-dimensional space transformed by kernel function. Multidimensional collinearity leads to large variance and covariance when regression coefficient is calculated by least square method, therefore these models will have poor generalization ability on the new data. In contrast to least square regression, PLS method has three advantages: (1) PLS method can make good use of the existing correlation between input variables and corresponding outputs variables to establish latent orthogonal components on the premise of maintaining most of the information in input variables, which will lead to a low variance estimation. (2) PLS method can fully utilize the potential variable space of high dimensions of output variables, and inherently suits for multi-input to multi-output regression. (3) Selecting potential variables with appropriate dimensions to replace the original response variables for regression can effectively filter out noise information.

Based on these ideas, in this paper, we present a novel spectral reflectance reconstruction method using more general nonlinear kernel partial least squares (KPLS) regression, which is not previously proposed in this field. In our method, we firstly select the appropriate kernel function in reproducing kernel Hilbert space (RKHS) [13] to nonlinearly map the original response values into a feature space, and subsequently construct a partial least square regression model in the feature space. By an explicit expressing regression coefficient in the feature space, the nonlinear KPLS model with good generalization properties is obtained. Moreover, the nonlinear optimization involved in this method can be avoided by using the kernel function corresponding to the regular dot product in RKHS. Based on the properties of the RKHS kernel function, in fact, only the linear algebra like simple linear PLS is needed for calculation.

We conducted sufficient experiments to validate our method, and compared it with existing spectral reflectance reconstruction methods on benchmark metrics. In our experiment, the reflectance datasets of standard color charts as well as a set of printing materials are used. Synthetic response data, and real data acquired from digital RGB camera image under three different illuminant environment were employed for training and testing. Results achieved by our method were evaluated from two aspects of spectral and colorimetric accuracy. The overall performance of our method is superior or at least equivalent to its counterparts and enough satisfactory for color management purpose. Furthermore, the influence of kernel function selection, kernel parameter optimizing and number of potential variables, as well as number of response channels were comprehensively analyzed.

In what follows, a column vector is denoted by a boldface lowercase letter, such as $\mathbf {c}=(c_1,\;c_2,\;c_3)^T \in \mathbb {R}$, a matrix is denoted by a boldface capital letter, e.g. $\mathbf {C}=[\mathbf {c}_1,\mathbf {c}_2,\ldots ,\mathbf {c}_l] \in \mathbb {R}^{3\times l}$, and scalars are represented by plain letters.

2. Imaging model

Incoming photon reflected by the surface of the objects interacts with the lens and sensing system of camera, in which the signal of the reflect light is converted to electric signal in photosensitive sensor chip (e.g. CCD) and quantized to digital response. Let $\Omega$ denotes the spectral wavelength range of the imaging system, which we mainly focus on approximate 400–700 nm in this study, representing the sensitive wavelength range of human. Under the condition of a fixed capture geometry and proper exposure time, a generalized multispectral system with trichromatic camera under an arbitrary illuminant can be mathematically modeled as the following integral process

3. Methods

3.1 Linear estimation methods

In the real applications, the spectral characteristics are usually sampled uniformly at $n$ ($n$=31) intervals from 400–700 nm. For simplicity, we combine the relative SPD of the illuminant and sensor spectral responsivity function into a set of system responsivity function defined as $w_i$, $w_i = l(\lambda )s_i(\lambda )$. Let $\mathbf {c} \in \mathbb {R}^m$, $\mathbf {r} \in \mathbb {R}^n$ be the camera response vector and reflectance vector, respectively. System responsivity function matrix $\mathbf {W} \in \mathbb {R}^{m \times n}$ is occupied by the $i$th row of the vector $\mathbf {w}_i$, and $\mathbf {e} \in \mathbb {R}^n$ denotes the noise vector. Equation (1) can thus be expressed in vector-matrix notation as

Different from above methods based on a priori knowledge of the system responsivity matrix and the noise distribution, the learning-based methods obtain the transform matrix $\mathbf {M}$ from the camera response values and the spectral reflectance of the training sample, and then apply $\mathbf {M}$ on the known camera responses to predict the reflectance of the test samples. Given a pair of training samples set as $S=\{(\mathbf {c}_i, \mathbf {r}_i) \in \mathbb {R}^m\!\times \mathbb {R}^n\}_{i=1}^{l}$, OLS (also known as pseudo inverse) method is given as

3.2 Ordinary least square regression method based on polynomial extension

In order to characterizing the nonlinearity of multispectral imaging system, the nonlinear technique based on polynomial extension of response $\mathbf {c}_i$ was employed to reconstruct the reflectance according to [15,17,19,23]. $\mathbf {c}_i \in \mathbf {R}^m$ is mapped into an $d$-ordered polynomial space, and the components of polynomial vector are constituted by product of different order element of $\mathbf {c}_i$, constrained by not exceeding $d$ order in each components. A 3-order polynomial extension of three-channel camera response vector $\tilde {\mathbf {c}}$ can be demonstrated as

A weighting coefficient matrix for optimal training samples to improve the reconstruction accuracy were suggested by Aaghian et al. [16], Amiri et al. [17] and Liang et al. [23]. The weights are arranged in a diagonal matrix $\mathbf {W}$, as shown by Eq. (10)

3.3 Partial least square method based on polynomial extension

In contrast to OLS, partial least square (PLS) regression is a method based on the projection of input variables to the uncorrelated latent variables, which can model a linear relationship between input and output variables while maintaining most of the information in the input variables. The OLS based on the extension of polynomial responses inevitably causes overfitting and collinearity, in this regard, PLS method based on polynomial extension for spectral reconstruction was proposed [19].

For the mean-centered response matrix $\tilde {\mathbf {C}} \in \mathbb {R}^{M \times l}$ and reflectance matrix $\mathbf {R}\in \mathbb {R}^{n \times l}$ of training samples, $\mathbf {T}\in \mathbb {R}^{l \times p}$, $\mathbf {U} \in \mathbb {R}^{l\times p}$ are defined as score matrices, as well as a loading matrix $\mathbf {P} \in \mathbb {R}^{l \times p}$, associated with two weighting matrices $\mathbf {W}\in \mathbb {R}^{M \times p}$, $\mathbf {V} \in \mathbb {R}^{n \times p}$, where $p$ is the number of PLS components. $\tilde {\mathbf {C}}$ and $\mathbf {R}$ can be expressed as

3.4 Our method

Motived by all above, here we propose a novel kernel PLS (KPLS) regression method for spectral reflectance reconstruction, which combines the advantages of both kernel function and PLS algorithm. Assuming we have a centered training set $S=\{(\mathbf {c}_i, \mathbf {r}_i) \in \mathbb {R}^m\!\times \mathbb {R}^n\}^{l}_{i=1}$, spectral reflectance reconstruction can be extended from linear to the nonlinear case as

In practice, the explicit mapping $\phi (.)$ is unkown or the computation may be numerically intractable. In RKHS framework, the computation of the uncertain mapping $\phi (.)$ can be avoided by applying the kernel function. We chose a positive definition kernel function $k(.,.)$, which corresponds to an inner product in the feature space

By a simpler calculation of eigenvector-eigenvalue problem, and using the $\mathbf {\Phi }$ matrix of mapped input data [24], we can modify the PLS algorithm into its kernel form:

Step 1: calculating eigenvector $\mathbf {t}$ and $\mathbf {u}$ from problem: $\mathbf {\Phi }^T\mathbf {\Phi }\mathbf {R}^T\mathbf {R}\mathbf {t}=\mathbf {t}\alpha ,\ \mathbf {R}^T\mathbf {R}\mathbf {\Phi }^T\mathbf {\Phi }\mathbf {u}=\mathbf {u}\beta$

Step 2: rescaling $\mathbf {t},\ \mathbf {u}$ to unit length: $\mathbf {t}\leftarrow \mathbf {t}/\left \| \mathbf {t}\right \|,\ \mathbf {u}\leftarrow \mathbf {u}/\left \| \mathbf {u}\right \|$

Step 3: deflate $\mathbf {\Phi }^T\mathbf {\Phi },\ \mathbf {R}$ matrices: $\mathbf {\Phi }^T\mathbf {\Phi } \leftarrow (\mathbf {\Phi }-\mathbf {tt\Phi })^T(\mathbf {\Phi }-\mathbf {tt\Phi }),\ \mathbf {R}^T \leftarrow \mathbf {R}^T-\mathbf {ttR}^T$

Applying the kernel function, i.e. the fact that $\phi (\mathbf {c}_i)^T\phi (\mathbf {c}_j)=k(\mathbf {c}_i,\mathbf {c}_j)$, we can see that $\mathbf {\Phi }^T\mathbf {\Phi }$ represents the kernel Gram matrix $\mathbf {K}$ of the inner products between all mapped training response values $[\phi (\mathbf {c}_i)]_{i=1}^l$. Thus, instead of an explicit nonlinear mapping, the kernel function can be used. The deflation of the $\mathbf {K}$ matrix after extraction of the $\mathbf {t}$, $\mathbf {u}$ component is now given by

It is worth noting that in [18,25], the kernel ridge regression (KRR) method was also applied into spectral reflectance reconstruction, and it has the form as

4. Experiments

Our main interest in this study is to evaluate spectral reflectance reconstruction accuracy in the field of spectral printing and color management of printing devices based on the proposed method. For this purpose, we designed a set of experiments to demonstrate the performance of KPLS method in terms of spectral and colorimetric error compared with previous methods. Experiments were carried out based on both simulated and real data. In this section, the datasets and imaging system are introduced and evaluation metrics are illustrated.

4.1 Spectral datasets

In simulated experiment, the Munsell Matte color (MMC) spectra data set [26] was used, which was measured by Perkin-Elmer lambda 9 UV/VIS/NIR spectrophotometer from 1269 chips of the Munsell Book of Color Matte Finish Collection. The MMC dataset was originally sampled at 1nm interval from 380 - 800 nm, but we selected the data at 10nm interval in the 400 - 700nm, for consistency with the real experiment. In real experiment, ANSI IT8.7/3 color chart (928 samples) was measured with the Xrite i1iSis spectrum scanner at 10nm interval in the 400 - 700nm range. IT8.7 spectral color chart was printed out on a Canon imagePROGRAF iPF5100 printer using Epson matte photo paper (260 $g/m^2$). Chromaticity coordinates of two spectra datasets in CIE L*a*b* chromaticity diagram are shown in Fig. 1, calculated by CIE 1964 $10^\circ$ standard observer system and CIE D65 standard illumination.

 

Fig. 1. CIE L*a*b* coordinates for (a) MMC dataset, and (b) IT8.7/3 spectra dataset

Download Full Size | PDF

4.2 Simulated and real camera response

The camera responses for MMC datasets are simulated based on imaging model in Eq. (2) yet ignoring the nonlinear factor $\Gamma$ of a real camera system. A Sony DXC-930 3CCD camera was used for simulated imaging device, of which the spectral sensitivity function was measured by Barnard et al. [27]. In consideration of the potential applications in the printing or related color industries, light sources of simulated system were assumed to be the CIE Standard Illuminant D65, D50 and A. The spectral responsivity function of the camera sensors and the relative SPD of three illuminants can be represented by Fig. 2. Therefore, there are totally nine system responsivity functions, which can be obtained under the three lighting conditions, and 3-channel response value corresponds to D65, 6-channel to D65 and A, 9-channel to D65, D50 and A.

 

Fig. 2. (a) Spectral responsivity of the simulated camera, and (b) relative SPD of the simulated illuminants

Download Full Size | PDF

Additive normally distributed noise is added into the camera response. The noise is with zero mean and $\sigma ^2$ variance on each channel. The signal-to-noise ratio (SNR) for noisy data can be calculated as

In the real experiment, we built a 9-band multispectral imaging system with a Nikon D7200 trichromatic camera and three illuminants modules in X-Rite Spectralight III, which could acquires at most 9-band multispectral image of an object in three shots. In our experiment, the first shot used D65 light source, then A light source and finally D50 light source. The camera was fixed in the front of the object surface, and the imaging plane of the camera was parallel to the sample placement plane and the distance between them was appropriately 1 meter, while the light from light sources was at approximately $45^\circ$ to the object surface. Multispectral images of IT8.7/3 color chart were acquired under different illuminant module in the sequential exposure, and the exposure time, $f$-number and ISO were carefully set to avoid any pixel saturation at each exposure. 3-channel image was obtained by the first shot, and 6-channel and 9-channel image were acquired by combining the two and three images from the first two and all three shots, respectively. Camera response values for each color patch were used by averaging the central $15\times 15$ pixels, approximately equivalent to the aperture size of the spectrophotometric measurement. Moreover, in order to use the same parameter space, the real camera responses are also pre-processed with max-value normalization.

4.3 Kernels and parameter optimization

The kernel-based method (KRR and proposed KPLS) allows us to choose an appropriate kernel function based on the given data. In this study, we evaluated two positive defined kernels commonly used in this field, which are including:

A. polynomial kernel function

B. Gaussian kernel function

For reconstruction method with free parameters (RR, PLS, KRR and KPLS), parameter search space was used to find the optimal parameter. In the case of RR and PLS method, the optimal regularization parameter $\lambda$ and component number $p$ was selected from a 1-dimensional parameter search space, while in the case of KRR and KPLS method, 2-dimensional parameter search space was used to optimize the two free parameters. After some tests, we found the optimal $\lambda$ in the range [0, 1] for RR method and $p$ in the range [20, 100] for PLS method in 1-dimensional search space. In addition, the optimal parameter range for KRR was [0, 1] for the regularization parameter $\lambda$ and [1,5] for polynomial order $d$ based on polynomial kernel function, [$10^{-1}$, 10] for Gaussian kernel parameter $\sigma$ based on Gaussian kernel function, respectively. In the case of KPLS, we used the same optimal parameter range for polynomial order $d$ and Gaussian kernel parameter $\sigma$, but component number $p$ was in range [20, 150]. Moreover, we sampled the parameter using the uniform interval over the corresponding range, and the results were calculated using 20 samples for $\lambda$, 5 samples of $d$, and 20 samples for $\sigma$, respectively. For component number $p$, the sampling interval is set as 1.

4.4 Evaluating metrics

We evaluated recovery performance in terms of spectral and color different metrics. The root mean square error (RMSE) was used as spectral reconstruction metrics, which can be expressed as

Color difference formula can be used to quantify the perceived difference between two colors stimuli in the same environment, which can used to evaluate the practical effect of spectral reconstruction in colorimetric field. In this study, the latest recommendation of CIE DE2000 [28] formula was employed to calculate the color difference under the condition of CIE 1964 $10^\circ$ standard observer system and CIE D65 standard illumination between the original and reconstructed spectrum.

5. Results & discussions

In this section, the selection of optimal parameter are demonstrated, and comparative results are summarized, futhermore the influence on results related to the noise level and number of response channels are analyzed.

5.1 Model evaluation and optimal parameters selection

Unlike the conventional 10-fold cross-validation adopted in [18] which may result in a biased estimation of model performance [29], a $5\times 2$ nested cross-validation was performed in this paper, to select the optimal parameter and simultaneously evaluate the performance of the resulting model. This means that the cross-validation was divided into two loops, where 5-fold cross-validation of outer loop was used to assess the performance of the model that wins in the inner fold, and 2-fold cross-validation of inner loop was used to conduct a parameter grid search separately. The optimal parameters were based on minimum average RMSE over the two folds in each inner loop, and the model performance were evaluated by average RMSE over the five folds of outer loop cross-validation.

Obviously, the type of data set, number of response channels and noise level influence the selection of the optimal parameters for the proposed KPLS method. Figure 3 shows one typical example of 3-dimensional RMSE map over the parameter gird. Figure 3(a) is the mean RMSE over 2 folds of parameter optimization using Gaussian kernel function with MMC dataset, 9-channel simulated response under middle noise level condition, and Fig. 3(b) corresponds to IT8.7/3 color chart, 9-channel real camera response.

 

Fig. 3. (a) Mean RMSE map of the simulated experiment, and (b) mean RMSE map of the real experiment of an 2-fold parameter optimization instance over the different component number $p$ and Gaussian kernel parameter $\sigma$ based on KPLS method with Gaussian kernel function. The optimal parameter is illustrated by a red square.

Download Full Size | PDF

From Fig. 3(a), we can find that the two parameters of KPLS method interdepend on each other. As the increment of the $p$ and $\sigma$ parameter, RMSE initially significantly decreases and converges to global minimal, and then the bigger $p$ and $\sigma$ lead to the worse result. The $p$ and $\sigma$ parameter of the real data show a similar relationship with the simulated case in Fig. 3(b). After some tests with different noise level and different number of response channels, it is also found that the shape of the RMSE surface is similar with Fig. 3, but the values of optimal parameters are variable. Further, 4-order polynomial extension was employed for RR and PLS methods, and the same scheme was used to select the optimal parameters of all other methods mentioned in this paper, using the parameter range described above.

5.2 Reconstruction performance with simulated responses from 9-channel system

The simulated camera responses are composed by 9-channel signal under 80, 40 and 30 SNR level, and the results are obtained by 10 times $5\times 2$ nested cross-validation calculation. Spectral reconstruction results based the proposed KPLS method are compared with methods introduced in previous section (Wiener, RR, PLS, WLS and KRR) in regarding of the spectral and colorimetric performance. The average of results are summarized in Table 1.

Table 1. Spectral reconstruction results for simulated MMC dataset, 9-channel imaging system under different noise level, and Wiener, RR, PLS, WRLS, KRR and KPLS method

View Table | View all tables in this article

It can be observed in Table 1 that our method achieves better or at least equivalent results in terms of colorimetric and spectral accuracy under different noise condition in comparison with other methods, and KRR method that also uses kernel method is slightly worse than our method, but better than other methods. Therefore, the results validate the availability and superiority of kernel framework in the field of spectral reconstruction. In addition, by comparing the results based on two different kernel functions, it seems that Gaussian kernel outperforms polynomial kernel at higher noise level, and it shows the same trend in both KPLS and KRR method.

Meanwhile, similar colorimetric and spectral error of PLS and RR method based on polynomial extension can be seen from Table 1, but PLS performs slightly better at the higher noise case, which is further confirmed that PLS method has the ability of denoising in spectral reconstruction. Moreover, some dismatches in the best resuts between spectral accuracy and colorimetric accuracy can be observed in Table 1, that means that the good performance of the spectral metric does not necessarily result in the good performance of the color difference metric, which is also confirmed in [17,18,25].

In Fig. 4, samples of measured and reconstructed spectral reflectances from the MMC dataset based on KPLS method with Gaussian kernel are illustrated, and the results obtained from the other methods are illustrated in comparison. Figure 4 (left column) corresponds to the sample with the smallest RMSE for KPLS method under 80 (first row), 40 (second row) and 30 (third row) SNR level. It can be seen that the estimated reflectance by KPLS is slightly better than other methods (RMSE = 0.0009, 0.0023 and 0.0026 correspond to 80, 40 and 30 SNR), and in case of Wiener method the result is the worst (RMSE = 0.0050, 0.0109, and 0.0144 correspond to 80, 40 and 30 SNR). In Fig. 4 (middle column), the samples correspond to the largest RMSE for KPLS method under 80 (first row), 40 (second row) and 30 (third row) SNR level are illustrated. It can be observed here that almost all methods behave similarly, but consistently deviate from measured reflectance in higher wavelength. We can identify this sample as an outlier, whose camera responses and reflectance data do not correspond. The samples corresponding to 95th-percentile RMSE for KPLS method (RMSE = 0.0109, 0.0312 and 0.0368 correspond to 80, 40 and 30 SNR) are illustrated in Fig. 4 (left column) in order of 80 (first row), 40 (second row) and 30 (third row) SNR level. For these samples, it is found that the performance of spectral reconstructed by KPLS method based on Gaussian kernel improves compared with other method especially in wavelength range [650, 700].

 

Fig. 4. Sample reflectance and reconstruction results from all methods in simulated experiment: Left and middle column corresponds to the samples with the lowest and highest RMSE from KPLS method with G-kernel, and right column illustrates the samples corresponding to the 95th percentile RMSE for KPLS with G-kernel. The noise levels are arranged in row in order of 80, 40 and 30 SNR

Download Full Size | PDF

5.3 Reconstruction performance with real responses from 9-channel system

The simulation experiment ignored the nonlinearity of the imaging process, thus the results in Section 5.2 are not enough to verify the proposed method in the real world. In this section, the real experiment based on 9-channel imaging system and IT8.7/3 color chart was carried out to further validate our method, and results from reconstruction methods were calculated using the similar scheme as described in Section 5.2. The sensitivity of camera and SPD of the illuminant in the real experiment were not measured, so on this account Wiener method is not participated in comparison here. The numerical results of the comparison are summarized in Table 2.

Table 2. Spectral reconstruction results for IT8.7/3 dataset, 9-channel imaging system, and RR, PLS, WRLS, KRR and KPLS method

View Table | View all tables in this article

The statistical results in Table 2 show that the proposed KPLS method obviously surpass the comparative methods in the real experiment in terms of colorimetric and spectral accuracy, except for standard deviation of RMSE. In view of kernel function, it is found that Gaussian kernel outperforms polynomial kernel on two metrics, which potentially indicates the adaptability of Gaussian kernel to nonlinear imaging or inevitable noise in the practical imaging environment.

Similar to simulated experiment described in Section 5.2, we illustrate three samples of measured and reconstructed spectral reflectances from the IT8.7/3 dataset based on KPLS method with Gaussian kernel in Fig. 5. The reconstruction results indicated by Fig. 5 exhibits consistent characteristics with Fig. 4, which also confirms the good performance of the proposed method on the individual sample.

 

Fig. 5. Sample reflectance and reconstruction results from all methods in real experiment: Left and middle corresponds to the samples with the lowest and highest RMSE from KPLS method with G-kernel, and right illustrates the sample corresponding to the 95th percentile RMSE for KPLS with G-kernel.

Download Full Size | PDF

5.4 Influence of the number of responses channels

The influence of the number of responses channels on reconstruction accuracy was investigated in this study. We obtained 3-channel responses captured under the first light source in real experiment, 6-channel response under the first and second light source, as well as 9-channel responses used in Section 5.3. The reconstruction results from different methods based on different number of responses channels are illustrated in Table 3, of which the 9-channel results have been presented in Table 2 but still unified in Table 3 for intuitive comparison.

Table 3. Spectral reconstruction results for IT8.7/3 dataset, 3-, 6- and 9-channel imaging system, and RR, PLS, WRLS, KRR and KPLS method

View Table | View all tables in this article

Results in Table 3 suggest that the reconstruction method with the best performance changes with the decrease of the number of response channels. The dominant position of the proposed method is replaced by WLS (6-chanel) and KRR (3-channel) under a smaller response channels number, respectively. When considering the partial least square method need to exploit useful information from the input variables to build the potential relationship between the input variables and output variables, it is not surprising that our method do not give the best performance when the response channels change from 9 to 6 or 3. This inference could also be confirmed by the reconstruction performance based on RR and PLS method from 9-, 6- and 3-channel responses, where PLS method outperforms RR in 9-channel, but performs worse in 6- and 3-chanel responses.

When considering color management purpose, one unit of CIE DE2000 error is the threshold value of the perceptual difference for human vision. The result in Table 3 indicates that the average CIE DE2000 error given by our proposed method is smaller than one unit. From this point of view, our proposed method is available in color management applications.

6. Conclusion

In this paper, a new approach based on the KPLS method for spectral reflectance reconstruction with commercial trichromatic camera under multi-illuminant environment was proposed. Sufficient simulated experiments based on MMC dataset with different noise levels and practical experiments based on IT8.7/3 color chart with different number of responses channels were implemented, in order to compare the performance of our proposed method with the existing methods. The results indicated that the proposed method showed a better performance than, or was at least equivalent to, the previously developed methods on the used datasets. The outstanding estimation ability of this spectral reconstruction method is partly attributed to incorporate partial least square algorithm and kernel trick into our method, which can make full of the use of training data and simultaneously extend the dimensions of the input data.

For the proposed method, it can be categorized into the framework of kernel methods. Therefore, the kernel selection plays an important role on the recovery performance. A nested cross validation have been adopted to select the optimal parameter, and Gaussian kernel and polynomial kernel have been both tested in simulated and real experiments. The results suggested that Gaussian kernel seemed to be more robust for noise than polynomial kernel. In addition, the influence of the number of response channels have been researched, which showed that the proposed method performed slightly worse than the best method with low-dimensional responses. Searching for better kernels and methods that are more adaptable to low-dimensional responses will be further investigated in our future research.