Home

Awesome

iPhys: An Open Non-Contact Imaging-Based Physiological Measurement Toolbox

In the past few years a lot of attention has been given to methods for remotely measuring physiological signals using low-cost cameras. Imaging PPG (iPPG) focuses on the measurement of volumetric changes in blood flow at distance from the body using imaging devices to capture changes in transmitted or reflected light. Imaging ballistocardiography (iBCG) typically leverages optical flow estimation to track the vertical motion of the head or body from a video sequence. Both iPPG and iBCG methods can be used to recover human vital signals.

This toolbox contains MATLAB implementations of a number of algorithms for non-contact physiological measurement. This will enable researchers to present results on their datasets using standard public implementations of the baseline methods with all parameters known. The toolbox includes implementations of many of the most commonly used baseline methods for imaging photplethysmography (iPPG) and image ballistocardiography (iBCG).

Citation:

If you find this toolbox helpful and make use of it in your work please cite:

iPhys: An Open Non-Contact Imaging-Based Physiological Measurement Toolbox

Daniel McDuff

ArXiv

Link: http://arxiv.org/abs/1901.04366 Cite as: arXiv:1901.04366 [cs.CV]

We welcome suggestions or contact regarding the toolbox. Please contact: damcduff@microsoft.com

Background:

Imaging photoplethysmography (iPPG) has developed as a method for capturing the BVP signal remotely using digital cameras and ambient light. Almost any digital camera (i.e., webcam or cellphone camera) is sufficiently sensitive to capture the pulse signal when the subject is close to the device. Poh et al. (2010) proposed the use of Independent Component Analysis (ICA) to recover the pulse signal and enable a fully automated iPPG framework.

Alt text

Methods inspired by optical models of the skin have helped advance the state-of-the art. The CHROM (2013) method uses a linear combination of the chrominance signals and makes the assumption of a standardized skin color profile to white-balance the video frames. The plane orthogonal to the skin (POS [2017]) algorithm assumes the presence of a pulsatile color space signal and posits that this will be orthogonal to the skin color space.

Licensing:

The code is licensed with the MIT License (https://opensource.org/licenses/MIT) and the RAIL AI Licenses (https://www.licenses.ai/) that allows use of the software including rights to use, copy or modify it, with the exception of harmful applications as specified in: https://www.licenses.ai/open-source-license.

Methods:

The toolbox contains implementations of the several of the most commonly used methods for video-based physiological measurement. Below we give citations and links to the publications on which these implementations are based:

GREEN CHANNEL - Verkruysse, Svaasand, Nelson (2008)

https://www.osapublishing.org/oe/viewmedia.cfm?uri=oe-16-26-21434&seq=0

Citation: Verkruysse, W., Svaasand, L. O., & Nelson, J. S. (2008). Remote plethysmographic imaging using ambient light. Optics express, 16(26), 21434-21445. DOI: 10.1364/OE.16.021434

ICA - Poh, McDuff, Picard (2010)

https://www.osapublishing.org/oe/viewmedia.cfm?uri=oe-18-10-10762&seq=0

Citation: Poh, M. Z., McDuff, D. J., & Picard, R. W. (2010). Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics express, 18(10), 10762-10774. DOI: 10.1364/OE.18.010762

CHROM - De Haan & Jeanne (2013)

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.726.6643&rep=rep1&type=pdf

Citation: De Haan, G., & Jeanne, V. (2013). Robust pulse rate from chrominance-based rPPG. IEEE Transactions on Biomedical Engineering, 60(10), 2878-2886. DOI: 10.1109/TBME.2013.2266196

POS - Wang, den Brinker, Stuijk & de Haan (2017)

https://pure.tue.nl/ws/files/31563684/TBME_00467_2016_R1_preprint.pdf

Citation: Wang, W., den Brinker, A. C., Stuijk, S., & de Haan, G. (2017). Algorithmic principles of remote PPG. IEEE Transactions on Biomedical Engineering, 64(7), 1479-1491. DOI: 10.1109/TBME.2016.2609282

BCG - Balakrishnan, Durand & Guttag (2013)

http://openaccess.thecvf.com/content_cvpr_2013/papers/Balakrishnan_Detecting_Pulse_from_2013_CVPR_paper.pdf

Citation: Balakrishnan, G., Durand, F., & Guttag, J. (2013). Detecting pulse from head motions in video. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on (pp. 3430-3437). IEEE. DOI: 10.1109/CVPR.2013.440

We would happily include additional implementations of iPPG or iBCG algorithms in this toolbox. If you would like to contribute an implementations of a new method please submit a pull request or email: damcduff@microsoft.com

Usage:

The scripts have been created to allow easy processing of any video dataset. We have also included example data in the "test_data" folder.

Each method takes the following inputs:

And produces the following output:

Example:

DataDirectory = [cd '\test_data'];

VideoFile = [DataDirectory 'video_example.mp4'];

FS = 30;

StartTime = 0;

Duration = 60;

ECGFile = [DataDirectory 'ECGData.mat'];

PPGFile = [DataDirectory 'PPGData.mat'];

PlotTF = false;

[BVP, PR, HR_ECG, PR_PPG, SNR] = ICA_POH(VideoFile, FS, StartTime, Duration, ECGFile, PPGFile, PlotTF)

Data

Due to the size of the video example, please download it here and add it to the "test_data" folder.

The ECGData.mat file has three fields: 1) The waveform - ECGData.data, 2) The ECG sampling rate - ECGData.fs, 3) The ECG peak locations (in samples) - ECGData.peaks.

The PPGData.mat file has three fields: 1) The waveform - PPGData.data, 2) The PPG sampling rate - PPGData.fs, 3) The PPG peak locations (in samples) - PPGData.peaks.