Abstract

Partial least squares (PLS) regression is widely used to predict chemical analytes from spectroscopic data, thus reducing the need for expensive and time-consuming wet chemical reference analysis in industrial process monitoring. However, predictions via PLS by definition carry sample-specific errors, and estimation of these errors is essential for correct interpretation of results. To increase trust in PLS regression-based predictions, reliable prediction error estimates must be reported. This can be achieved by determining realistic sample-specific prediction errors using an unbiased mean squared prediction error estimate. This work provides a guide for estimating sample-specific prediction errors, showing the importance of choosing an appropriate error estimator prior to deploying PLS models for industrial applications. We reviewed recent and established methods for estimating the sample-specific prediction error and test them through simulation studies. The methods were subsequently applied for estimating prediction errors in two real-life datasets from the food ingredients industry, where near-infrared spectroscopy was used to quantify i) urea in process water and ii) individual protein concentrations in ultrafiltration retentates from a protein fractionation process. Both the simulations and real data examples showed that the mean squared error of calibration is always a downward biased estimator. Although leave-one-out-cross-validation performed surprisingly well in the data analysed in this work, this paper demonstrated that the appropriate choice of error estimator requires the user to make an informed, data-centered decision.

© 2023 The Author(s)

Error analysis of the spectral shift for partial least squares models in Raman spectroscopy

Haiyi Bian and Jing Gao
Opt. Express 26(7) 8016-8027 (2018)

Partial least squares regression calculation for quantitative analysis of metals submerged in water measured using laser-induced breakdown spectroscopy

Tomoko Takahashi, Blair Thornton, Takumi Sato, Toshihiko Ohki, Koichi Ohki, and Tetsuo Sakka
Appl. Opt. 57(20) 5872-5883 (2018)

Acidity measurement of iron ore powders using laser-induced breakdown spectroscopy with partial least squares regression

Z.Q. Hao, C.M. Li, M. Shen, X.Y. Yang, K.H. Li, L.B. Guo, X.Y. Li, Y.F. Lu, and X.Y. Zeng
Opt. Express 23(6) 7795-7801 (2015)

Fast estimation of optical properties of pear using a single snapshot technique combined with a least-squares support vector regression model based on spatial frequency domain imaging

Xueming He, Tingwei Li, Xiaping Fu, Xu Jiang, Yingwang Gao, and Xiuqin Rao
Appl. Opt. 58(15) 4075-4084 (2019)

Multiple kinds of pesticide residue detection using fluorescence spectroscopy combined with partial least-squares models

Rendong Ji, Shicai Ma, Hua Yao, Yue Han, Xiao Yang, Ruiqiang Chen, Yinshang Yu, Xiaoyan Wang, Dongyang Zhang, TieZhu Zhu, and Haiyi Bian
Appl. Opt. 59(6) 1524-1528 (2020)