Abstract

Biometric signatures of remote photoplethysmography (rPPG), including the pulse-induced characteristic color absorptions and pulse frequency range, have been used to design robust algorithms for extracting the pulse-signal from a video. In this paper, we look into a new biometric signature, i.e., the relative pulsatile amplitude, and use it to design a very effective yet computationally low-cost filtering method for rPPG, namely “amplitude-selective filtering” (ASF). Based on the observation that the human relative pulsatile amplitude varies in a specific lower range as a function of RGB channels, our basic idea is using the spectral amplitude of, e.g., the R-channel, to select the RGB frequency components inside the assumed pulsatile amplitude-range for pulse extraction. Similar to band-pass filtering (BPF), the proposed ASF can be applied to a broad range of rPPG algorithms to pre-process the RGB-signals before extracting the pulse. The benchmark in challenging fitness use-cases shows that applying ASF (ASF+BPF) as a pre-processing step brings significant and consistent improvements to all multi-channel pulse extraction methods. It improves different (multi-wavelength) rPPG algorithms to the extent where quality differences between the individual approaches almost disappear. The novelty of the proposed method is its simplicity and effectiveness in providing a solution for the extremely challenging application of rPPG to a fitness setting. The proposed method is easy to understand, simple to implement, and low-cost in running. It is the first time that the physiological property of pulsatile amplitude is used as a biometric signature for generic signal filtering.

© 2017 Optical Society of America

Full Article  |  PDF Article
OSA Recommended Articles
Skin inhomogeneity as a source of error in remote PPG-imaging

Andreia Vieira Moço, Sander Stuijk, and Gerard de Haan
Biomed. Opt. Express 7(11) 4718-4733 (2016)

Full video pulse extraction

Wenjin Wang, Albertus C. den Brinker, and Gerard de Haan
Biomed. Opt. Express 9(8) 3898-3914 (2018)

Motion robust PPG-imaging through color channel mapping

Andreia V. Moço, Sander Stuijk, and Gerard de Haan
Biomed. Opt. Express 7(5) 1737-1754 (2016)

References

  • View by:
  • |
  • |
  • |

  1. W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmographic imaging using ambient light,” Opt. Express 16(26), 21434–21445 (2008).
    [Crossref] [PubMed]
  2. M. Lewandowska, J. Rumiński, T. Kocejko, and J. Nowak, “Measuring pulse rate with a webcam - a non-contact method for evaluating cardiac activity,” in Proceedings of Federated Conference on Computer Science and Information Systems (IEEE, 2011), pp. 405–410.
  3. M. Z. Poh, D. J. McDuff, and R. W. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng. 58(1), 7–11 (2011).
    [Crossref]
  4. W. Wang, S. Stuijk, and G. de Haan, “A novel algorithm for remote photoplethysmography: Spatial subspace rotation,” IEEE Trans. Biomed. Eng. 63(9), 1974–1984 (2016).
    [Crossref]
  5. G. de Haan and V. Jeanne, “Robust pulse rate from chrominance-based rPPG,” IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013).
    [Crossref] [PubMed]
  6. G. de Haan and A. van Leest, “Improved motion robustness of remote-PPG by using the blood volume pulse signature,” Physiol. Meas. 35(9), 1913–1922 (2014).
    [Crossref] [PubMed]
  7. W. Wang, A. C. den Brinker, S. Stuijk, and G. de Haan, “Algorithmic principles of remote-PPG,” IEEE Trans. Biomed. Eng.PP(99), 1 (2016). (posted 13 September 2016, in press).
  8. W. Verkruysse, M. Bartula, E. Bresch, M. Rocque, M. Meftah, and I. Kirenko, “Calibration of contactless pulse oximetry,” Anesth. Analg. 124(1), 136–145 (2017).
    [Crossref]
  9. W. Wang, B. Balmaekers, and G. de Haan, “Quality metric for camera-based pulse rate monitoring in fitness exercise,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2016), pp. 2430–2434.
  10. C. Julien, “The enigma of Mayer waves: Facts and models,” Cardiovasc. Res. 70(1), 12–21 (2006).
    [Crossref]
  11. A. A. Kamshilin, I. S. Sidorov, L. Babayan, M. A. Volynsky, R. Giniatullin, and O. V. Mamontov, “Accurate measurement of the pulse wave delay with imaging photoplethysmography,” Biomed. Opt. Express 7(12), 5138–5147 (2016).
    [Crossref] [PubMed]
  12. F. Bousefsaf, C. Maaoui, and A. Pruski, “Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate,” Biomed. Sig. Proc. Control 8(6), 568–574 (2013).
    [Crossref]
  13. X. Li, J. Chen, G. Zhao, and M. Pietikäinen, “Remote heart rate measurement from face videos under realistic situations,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2014), pp. 4264–4271.
  14. M. Hülsbusch, An Image-based Functional Method for Opto-electronic Detection of Skin Perfusion (Ph.D. dissertation, 2008).
  15. G. R. Tsouri and Z. Li, “On the benefits of alternative color spaces for noncontact heart rate measurements using standard red-green-blue cameras,” J. Biomed. Opt. 20(4), 048002 (2015).
    [Crossref] [PubMed]

2017 (1)

W. Verkruysse, M. Bartula, E. Bresch, M. Rocque, M. Meftah, and I. Kirenko, “Calibration of contactless pulse oximetry,” Anesth. Analg. 124(1), 136–145 (2017).
[Crossref]

2016 (2)

A. A. Kamshilin, I. S. Sidorov, L. Babayan, M. A. Volynsky, R. Giniatullin, and O. V. Mamontov, “Accurate measurement of the pulse wave delay with imaging photoplethysmography,” Biomed. Opt. Express 7(12), 5138–5147 (2016).
[Crossref] [PubMed]

W. Wang, S. Stuijk, and G. de Haan, “A novel algorithm for remote photoplethysmography: Spatial subspace rotation,” IEEE Trans. Biomed. Eng. 63(9), 1974–1984 (2016).
[Crossref]

2015 (1)

G. R. Tsouri and Z. Li, “On the benefits of alternative color spaces for noncontact heart rate measurements using standard red-green-blue cameras,” J. Biomed. Opt. 20(4), 048002 (2015).
[Crossref] [PubMed]

2014 (1)

G. de Haan and A. van Leest, “Improved motion robustness of remote-PPG by using the blood volume pulse signature,” Physiol. Meas. 35(9), 1913–1922 (2014).
[Crossref] [PubMed]

2013 (2)

G. de Haan and V. Jeanne, “Robust pulse rate from chrominance-based rPPG,” IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013).
[Crossref] [PubMed]

F. Bousefsaf, C. Maaoui, and A. Pruski, “Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate,” Biomed. Sig. Proc. Control 8(6), 568–574 (2013).
[Crossref]

2011 (1)

M. Z. Poh, D. J. McDuff, and R. W. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng. 58(1), 7–11 (2011).
[Crossref]

2008 (1)

2006 (1)

C. Julien, “The enigma of Mayer waves: Facts and models,” Cardiovasc. Res. 70(1), 12–21 (2006).
[Crossref]

Babayan, L.

Balmaekers, B.

W. Wang, B. Balmaekers, and G. de Haan, “Quality metric for camera-based pulse rate monitoring in fitness exercise,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2016), pp. 2430–2434.

Bartula, M.

W. Verkruysse, M. Bartula, E. Bresch, M. Rocque, M. Meftah, and I. Kirenko, “Calibration of contactless pulse oximetry,” Anesth. Analg. 124(1), 136–145 (2017).
[Crossref]

Bousefsaf, F.

F. Bousefsaf, C. Maaoui, and A. Pruski, “Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate,” Biomed. Sig. Proc. Control 8(6), 568–574 (2013).
[Crossref]

Bresch, E.

W. Verkruysse, M. Bartula, E. Bresch, M. Rocque, M. Meftah, and I. Kirenko, “Calibration of contactless pulse oximetry,” Anesth. Analg. 124(1), 136–145 (2017).
[Crossref]

Chen, J.

X. Li, J. Chen, G. Zhao, and M. Pietikäinen, “Remote heart rate measurement from face videos under realistic situations,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2014), pp. 4264–4271.

de Haan, G.

W. Wang, S. Stuijk, and G. de Haan, “A novel algorithm for remote photoplethysmography: Spatial subspace rotation,” IEEE Trans. Biomed. Eng. 63(9), 1974–1984 (2016).
[Crossref]

G. de Haan and A. van Leest, “Improved motion robustness of remote-PPG by using the blood volume pulse signature,” Physiol. Meas. 35(9), 1913–1922 (2014).
[Crossref] [PubMed]

G. de Haan and V. Jeanne, “Robust pulse rate from chrominance-based rPPG,” IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013).
[Crossref] [PubMed]

W. Wang, A. C. den Brinker, S. Stuijk, and G. de Haan, “Algorithmic principles of remote-PPG,” IEEE Trans. Biomed. Eng.PP(99), 1 (2016). (posted 13 September 2016, in press).

W. Wang, B. Balmaekers, and G. de Haan, “Quality metric for camera-based pulse rate monitoring in fitness exercise,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2016), pp. 2430–2434.

den Brinker, A. C.

W. Wang, A. C. den Brinker, S. Stuijk, and G. de Haan, “Algorithmic principles of remote-PPG,” IEEE Trans. Biomed. Eng.PP(99), 1 (2016). (posted 13 September 2016, in press).

Giniatullin, R.

Hülsbusch, M.

M. Hülsbusch, An Image-based Functional Method for Opto-electronic Detection of Skin Perfusion (Ph.D. dissertation, 2008).

Jeanne, V.

G. de Haan and V. Jeanne, “Robust pulse rate from chrominance-based rPPG,” IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013).
[Crossref] [PubMed]

Julien, C.

C. Julien, “The enigma of Mayer waves: Facts and models,” Cardiovasc. Res. 70(1), 12–21 (2006).
[Crossref]

Kamshilin, A. A.

Kirenko, I.

W. Verkruysse, M. Bartula, E. Bresch, M. Rocque, M. Meftah, and I. Kirenko, “Calibration of contactless pulse oximetry,” Anesth. Analg. 124(1), 136–145 (2017).
[Crossref]

Kocejko, T.

M. Lewandowska, J. Rumiński, T. Kocejko, and J. Nowak, “Measuring pulse rate with a webcam - a non-contact method for evaluating cardiac activity,” in Proceedings of Federated Conference on Computer Science and Information Systems (IEEE, 2011), pp. 405–410.

Lewandowska, M.

M. Lewandowska, J. Rumiński, T. Kocejko, and J. Nowak, “Measuring pulse rate with a webcam - a non-contact method for evaluating cardiac activity,” in Proceedings of Federated Conference on Computer Science and Information Systems (IEEE, 2011), pp. 405–410.

Li, X.

X. Li, J. Chen, G. Zhao, and M. Pietikäinen, “Remote heart rate measurement from face videos under realistic situations,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2014), pp. 4264–4271.

Li, Z.

G. R. Tsouri and Z. Li, “On the benefits of alternative color spaces for noncontact heart rate measurements using standard red-green-blue cameras,” J. Biomed. Opt. 20(4), 048002 (2015).
[Crossref] [PubMed]

Maaoui, C.

F. Bousefsaf, C. Maaoui, and A. Pruski, “Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate,” Biomed. Sig. Proc. Control 8(6), 568–574 (2013).
[Crossref]

Mamontov, O. V.

McDuff, D. J.

M. Z. Poh, D. J. McDuff, and R. W. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng. 58(1), 7–11 (2011).
[Crossref]

Meftah, M.

W. Verkruysse, M. Bartula, E. Bresch, M. Rocque, M. Meftah, and I. Kirenko, “Calibration of contactless pulse oximetry,” Anesth. Analg. 124(1), 136–145 (2017).
[Crossref]

Nelson, J. S.

Nowak, J.

M. Lewandowska, J. Rumiński, T. Kocejko, and J. Nowak, “Measuring pulse rate with a webcam - a non-contact method for evaluating cardiac activity,” in Proceedings of Federated Conference on Computer Science and Information Systems (IEEE, 2011), pp. 405–410.

Picard, R. W.

M. Z. Poh, D. J. McDuff, and R. W. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng. 58(1), 7–11 (2011).
[Crossref]

Pietikäinen, M.

X. Li, J. Chen, G. Zhao, and M. Pietikäinen, “Remote heart rate measurement from face videos under realistic situations,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2014), pp. 4264–4271.

Poh, M. Z.

M. Z. Poh, D. J. McDuff, and R. W. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng. 58(1), 7–11 (2011).
[Crossref]

Pruski, A.

F. Bousefsaf, C. Maaoui, and A. Pruski, “Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate,” Biomed. Sig. Proc. Control 8(6), 568–574 (2013).
[Crossref]

Rocque, M.

W. Verkruysse, M. Bartula, E. Bresch, M. Rocque, M. Meftah, and I. Kirenko, “Calibration of contactless pulse oximetry,” Anesth. Analg. 124(1), 136–145 (2017).
[Crossref]

Ruminski, J.

M. Lewandowska, J. Rumiński, T. Kocejko, and J. Nowak, “Measuring pulse rate with a webcam - a non-contact method for evaluating cardiac activity,” in Proceedings of Federated Conference on Computer Science and Information Systems (IEEE, 2011), pp. 405–410.

Sidorov, I. S.

Stuijk, S.

W. Wang, S. Stuijk, and G. de Haan, “A novel algorithm for remote photoplethysmography: Spatial subspace rotation,” IEEE Trans. Biomed. Eng. 63(9), 1974–1984 (2016).
[Crossref]

W. Wang, A. C. den Brinker, S. Stuijk, and G. de Haan, “Algorithmic principles of remote-PPG,” IEEE Trans. Biomed. Eng.PP(99), 1 (2016). (posted 13 September 2016, in press).

Svaasand, L. O.

Tsouri, G. R.

G. R. Tsouri and Z. Li, “On the benefits of alternative color spaces for noncontact heart rate measurements using standard red-green-blue cameras,” J. Biomed. Opt. 20(4), 048002 (2015).
[Crossref] [PubMed]

van Leest, A.

G. de Haan and A. van Leest, “Improved motion robustness of remote-PPG by using the blood volume pulse signature,” Physiol. Meas. 35(9), 1913–1922 (2014).
[Crossref] [PubMed]

Verkruysse, W.

W. Verkruysse, M. Bartula, E. Bresch, M. Rocque, M. Meftah, and I. Kirenko, “Calibration of contactless pulse oximetry,” Anesth. Analg. 124(1), 136–145 (2017).
[Crossref]

W. Verkruysse, L. O. Svaasand, and J. S. Nelson, “Remote plethysmographic imaging using ambient light,” Opt. Express 16(26), 21434–21445 (2008).
[Crossref] [PubMed]

Volynsky, M. A.

Wang, W.

W. Wang, S. Stuijk, and G. de Haan, “A novel algorithm for remote photoplethysmography: Spatial subspace rotation,” IEEE Trans. Biomed. Eng. 63(9), 1974–1984 (2016).
[Crossref]

W. Wang, A. C. den Brinker, S. Stuijk, and G. de Haan, “Algorithmic principles of remote-PPG,” IEEE Trans. Biomed. Eng.PP(99), 1 (2016). (posted 13 September 2016, in press).

W. Wang, B. Balmaekers, and G. de Haan, “Quality metric for camera-based pulse rate monitoring in fitness exercise,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2016), pp. 2430–2434.

Zhao, G.

X. Li, J. Chen, G. Zhao, and M. Pietikäinen, “Remote heart rate measurement from face videos under realistic situations,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2014), pp. 4264–4271.

Anesth. Analg. (1)

W. Verkruysse, M. Bartula, E. Bresch, M. Rocque, M. Meftah, and I. Kirenko, “Calibration of contactless pulse oximetry,” Anesth. Analg. 124(1), 136–145 (2017).
[Crossref]

Biomed. Opt. Express (1)

Biomed. Sig. Proc. Control (1)

F. Bousefsaf, C. Maaoui, and A. Pruski, “Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate,” Biomed. Sig. Proc. Control 8(6), 568–574 (2013).
[Crossref]

Cardiovasc. Res. (1)

C. Julien, “The enigma of Mayer waves: Facts and models,” Cardiovasc. Res. 70(1), 12–21 (2006).
[Crossref]

IEEE Trans. Biomed. Eng. (3)

M. Z. Poh, D. J. McDuff, and R. W. Picard, “Advancements in noncontact, multiparameter physiological measurements using a webcam,” IEEE Trans. Biomed. Eng. 58(1), 7–11 (2011).
[Crossref]

W. Wang, S. Stuijk, and G. de Haan, “A novel algorithm for remote photoplethysmography: Spatial subspace rotation,” IEEE Trans. Biomed. Eng. 63(9), 1974–1984 (2016).
[Crossref]

G. de Haan and V. Jeanne, “Robust pulse rate from chrominance-based rPPG,” IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013).
[Crossref] [PubMed]

J. Biomed. Opt. (1)

G. R. Tsouri and Z. Li, “On the benefits of alternative color spaces for noncontact heart rate measurements using standard red-green-blue cameras,” J. Biomed. Opt. 20(4), 048002 (2015).
[Crossref] [PubMed]

Opt. Express (1)

Physiol. Meas. (1)

G. de Haan and A. van Leest, “Improved motion robustness of remote-PPG by using the blood volume pulse signature,” Physiol. Meas. 35(9), 1913–1922 (2014).
[Crossref] [PubMed]

Other (5)

W. Wang, A. C. den Brinker, S. Stuijk, and G. de Haan, “Algorithmic principles of remote-PPG,” IEEE Trans. Biomed. Eng.PP(99), 1 (2016). (posted 13 September 2016, in press).

M. Lewandowska, J. Rumiński, T. Kocejko, and J. Nowak, “Measuring pulse rate with a webcam - a non-contact method for evaluating cardiac activity,” in Proceedings of Federated Conference on Computer Science and Information Systems (IEEE, 2011), pp. 405–410.

W. Wang, B. Balmaekers, and G. de Haan, “Quality metric for camera-based pulse rate monitoring in fitness exercise,” in Proceedings of IEEE International Conference on Image Processing (IEEE, 2016), pp. 2430–2434.

X. Li, J. Chen, G. Zhao, and M. Pietikäinen, “Remote heart rate measurement from face videos under realistic situations,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2014), pp. 4264–4271.

M. Hülsbusch, An Image-based Functional Method for Opto-electronic Detection of Skin Perfusion (Ph.D. dissertation, 2008).

Cited By

OSA participates in Crossref's Cited-By Linking service. Citing articles from OSA journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (9)

Fig. 1
Fig. 1 Illustration of the amplitude-selective filtering. The essence of this filter is in the step of selecting the RGB frequency components within the pulsatile amplitude-range for pulse extraction, thus removing large components due to motion.
Fig. 2
Fig. 2 (a) The statistically measured relative pulsatile amplitude as a function of RGB channels. (b) The continuous relative pulsatile traces exemplified in extreme lighting conditions and the optical filter responses of the used RGB camera.
Fig. 3
Fig. 3 Experimental setup and video snapshots.
Fig. 4
Fig. 4 Illustration of the two quality metrics (SNR and success-rate) used for evaluating the rPPG performance. In the SNR metric, the frequency components of pulse (green) and noise (red) are defined by the ECG-reference. In the success-rate metric, the inlier estimates (green) and outlier estimates (red) are defined by a tolerance (dashed black line) w.r.t. the ECG-reference.
Fig. 5
Fig. 5 The SNR comparison between eight rPPG algorithms as a function of pre-processing. Different panels show the results obtained by using different filters (e.g., None, BPF, ASF and ASF+BPF) in the pre-processing, where the median values are indicated by red bars, the quartile range by blue boxes, the full range by whiskers, disregarding the outliers (red crosses).
Fig. 6
Fig. 6 The success-rate curves (and corresponding AUC) obtained by eight rPPG algorithms over 23 benchmark videos by using different filters in the pre-processing. Each panel shows the contribution of three filters (i.e., BPF, ASF and ASF+BPF) to a particular rPPG algorithm, where different colors denote the AUC for different filters and the percentage numbers exemplify their success-rate at T = 3, i.e., allowing 3 bpm difference with the ECG-reference.
Fig. 7
Fig. 7 Spectrograms obtained by eight rPPG algorithms on a fitness video by using different filters in the pre-processing. From top to bottom: baseline without pre-processing (None), Band-Pass Filter (BPF), Amplitude-Selective Filter (ASF), and the combination of ASF and BPF (ASF+BPF).
Fig. 8
Fig. 8 Spectrograms obtained by eight rPPG algorithms on a fitness video by using ASF in the pre-processing (Pre-ASF) or the post-processing (Post-ASF). From top to bottom: baseline without pre-/post- processing (None), Pre-ASF, and Post-ASF.
Fig. 9
Fig. 9 The 3D mesh shows different SNR (first row) and AUC of success-rate (second row) when varying the parameters of ASF for eight rPPG algorithms. The two varied parameters are: the sliding window length L (first column) and the maximum amplitude threshold amax (second column). When changing the investigated parameter, the other one remains constant. The red/blue color represents the high/low values for SNR and AUC of success-rate.

Tables (3)

Tables Icon

Algorithm 1: Amplitude-Selective Filtering

Tables Icon

Table 1 The globally averaged SNR (dB) over 23 benchmark videos of each rPPG algorithm by using different filters in the pre-processing. The bold entry denotes the best result obtained by each rPPG algorithm using the corresponding filter.

Tables Icon

Table 2 The AUC of success-rate obtained each rPPG algorithm over 23 benchmark videos by using different filters in the pre-processing. The bold entry denotes the best result obtained by each rPPG algorithm using the corresponding filter.

Equations (8)

Equations on this page are rendered with MathJax. Learn more.

C i = S i , 1 + S i , 2 + + S i , M = n = 1 M S i , n ,
C i = F i , 1 + F i , 2 + + F i , N = n = 1 N F i , n ,
C ^ i = n = 1 N w i , n F i , n with w i , n = { 1 , if n [ b min , b max ] , 0 , elsewhere ,
C ˜ i = C i μ ( C i ) 1 ,
F i = FFT ( C ˜ i ) L ,
W n = { 1 , if abs ( F 1 , n ) < a max , Δ abs ( F 1 , n ) , elsewhere ,
F ^ i = W F i ,
C ^ i = μ ( C i ) ( IFFT ( F ^ i ) + 1 ) ,

Metrics