Advertisement
Research Article| Volume 8, ISSUE 5, P420-428, October 2022

Download started.

Ok

Performance evaluation of a wrist-worn reflectance pulse oximeter during sleep

Open AccessPublished:July 08, 2022DOI:https://doi.org/10.1016/j.sleh.2022.04.003

      Abstract

      Objectives

      To characterize and evaluate the estimation of oxygen saturation measured by a wrist-worn reflectance pulse oximeter during sleep.

      Methods

      Ninety-seven adults with sleep disturbances were enrolled. Oxygen saturation was simultaneously measured using a reflectance pulse oximeter (Galaxy Watch 4 [GW4], Samsung, South Korea) and a transmittance pulse oximeter (polysomnography) as a reference. The performance of the device was evaluated using the root mean squared error (RMSE) and coverage rate. Additionally, GW4-derived oxygen desaturation index (ODI) was compared with the apnea-hypopnea index (AHI) derived from polysomnography.

      Results

      The GW4 had an overall RMSE of 2.3% and negligible bias of -0.2%. A Bland-Altman density plot showed good agreement between the GW4 and the reference pulse oximeter. RMSEs were 1.65 ± 0.57%, 1.76 ± 0.65%, 1.93 ± 0.54%, and 2.93 ± 1.71% for normal (n = 18), mild (n = 21), moderate (n = 23), and severe obstructive sleep apnea (n = 35), respectively. The data rejection rate was 26.5%, which was caused by fluctuations in contact pressure and the discarding of data less than 70% of saturation. A GW4-ODI ≥5/h had the highest ability to predict AHI ≥15/h with sensitivity, specificity, accuracy, and area under the curve of 89.7%, 64.1%, 79.4%, and 0.908, respectively.

      Conclusions

      This study evaluated the estimation of oxygen saturation by the GW4 during sleep. This device complies with both Food and Drug Administration and International Organization for Standardization standards. Further improvements in the algorithms of wearable devices are required to obtain more accurate and reliable information about oxygen saturation measurements.

      Keywords

      Introduction

      Pulse oximetry significantly contributes to clinical care by permitting noninvasive, convenient, and continuous measurement of oxygen saturation in peripheral blood (SpO2). Photoplethysmography (PPG) is an optical measurement method that enables wearable devices such as pulse oximeters to detect blood volume changes in the microvascular tissue bed; it has various clinical applications, including measuring the heart rate, blood pressure, and SpO2.
      • Allen J
      Photoplethysmography and its application in clinical physiological measurement.
      This technique transmits light to the skin at 2 different wavelengths, red (660 nm) and infrared (940 nm), and measures reflected or transmitted light to calculate absorbance based on the Lambert-Beer law.
      • Tamura T
      • Maeda Y
      • Sekine M
      • Yoshida M
      Wearable photoplethysmographic sensors—past and present.
      Then, it calculates the absorbance ratio at different wavelengths, which reflects the ratio of oxygenated and deoxygenated hemoglobin levels. Several studies have attempted to measure SpO2 at various body locations using various types of wearable devices.
      • Tamura T
      • Maeda Y
      • Sekine M
      • Yoshida M
      Wearable photoplethysmographic sensors—past and present.
      • Longmore SK
      • Lui GY
      • Naik G
      • Breen PP
      • Jalaludin B
      • Gargiulo GD
      A comparison of reflective photoplethysmography for detection of heart rate, blood oxygen saturation, and respiration rate at various anatomical locations.
      • Fine J
      • Branan KL
      • Rodriguez AJ
      • et al.
      Sources of inaccuracy in photoplethysmography for continuous cardiovascular monitoring.
      According to previous results, transmittance PPG on the fingertip was verified as one of the most accurate and stable methods for measuring SpO2. However, the fingertip-attached device can be obtrusive and cumbersome to wear continuously because it prevents users from moving their hands freely; therefore, attempts have been made to develop and validate wrist-worn devices for measuring SpO2.
      • Braun F
      • Theurillat P
      • Proenca M
      • et al.
      Pulse oximetry at the wrist during sleep: performance, challenges and perspectives.
      • Guber A
      • Shochet GE
      • Kohn S
      • Shitrit D
      Wrist-sensor pulse oximeter enables prolonged patient monitoring in chronic lung diseases.
      • Kramer M
      • Lobbestael A
      • Barten E
      • Eian J
      • Rausch G
      Wearable pulse oximetry measurements on the torso, arms, and legs: a proof of concept.
      Obstructive sleep apnea (OSA) is a common sleep disorder, with a 9%–38% prevalence rate among the overall adult population.
      • Senaratna CV
      • Perret JL
      • Lodge CJ
      • et al.
      Prevalence of obstructive sleep apnea in the general population: a systematic review.
      It is characterized by repetitive upper airway obstruction during sleep, thereby resulting in oxygen desaturation, frequent arousals, and increased sympathetic activity. In-laboratory polysomnography (PSG) is the gold standard for measuring the apnea-hypopnea index (AHI) and diagnosing sleep-related breathing disorders; however, it has certain limitations, such as high costs, lengthy time requirements, and the need for an in-hospital setting. Furthermore, PSG has limitations that affect its functionality, including the first-night effect and inherent night-to-night variability.
      • Rashid NH
      • Zaghi S
      • Scapuccin M
      • Camacho M
      • Certal V
      • Capasso R
      The value of oxygen desaturation index for diagnosing obstructive sleep apnea: a systematic review.
      Therefore, accessible and relatively low-cost options are being developed to overcome the limitations of PSG so that daily sleep patterns can be monitored.
      Reflectance PPG sensors are embedded in most commercially available wearable devices for collecting bio-signals. Nevertheless, several challenges associated with wrist-worn reflectance PPG sensors have been encountered because these devices are known to have a relatively low signal-to-noise ratio and are prone to artifacts; such drawbacks have prevented their widespread use in clinical practice.
      Previous studies have validated the accuracy and reliability of wrist-worn reflectance PPG sensors by detecting SpO2 at different hypoxic levels during awake states in laboratory environments.
      • Hermand E
      • Coll C
      • Richalet JP
      • Lhuissier FJ
      Accuracy and reliability of pulse O2 saturation measured by a wrist-worn oximeter.
      • Kirszenblat R
      • Edouard P
      Validation of the Withings ScanWatch as a wrist-worn reflective pulse oximeter: prospective interventional clinical study.
      • Lauterbach CJ
      • Romano PA
      • Greisler LA
      • Brindle RA
      • Ford KR
      • Kuennen MR
      Accuracy and reliability of commercial wrist-worn pulse oximeter during normobaric hypoxia exposure under resting conditions.
      However, detecting wrist-worn reflectance PPG signals during sleep is different because the individuals are unconscious and cannot adjust the watch position. If the measurements during sleep are not properly performed for a long duration, then the test cannot be interrupted to correct for artifacts, resulting in substantially more errors compared to when measurements are obtained during awake states. Only one published study has evaluated the performance of the wrist-worn reflectance PPG device during sleep.
      • Braun F
      • Theurillat P
      • Proenca M
      • et al.
      Pulse oximetry at the wrist during sleep: performance, challenges and perspectives.
      This study aimed to evaluate the SpO2 during sleep as obtained by Galaxy Watch 4, which permits continuous measurement every second. In addition, we attempted to compare the oxygen desaturation index (ODI) obtained using wrist-worn reflectance pulse oximetry with the AHI during sleep.

      Participants and methods

      Participants

      A total of 97 adults with sleep disturbances (age 44.4 ± 13.0 years; 74 men and 23 women) who visited the sleep laboratory at Samsung Medical Center, Seoul, South Korea, to undergo PSG were enrolled in this study. All participants were Asian. The following individuals were excluded: those with parasomnia because of aggressive limb movements during sleep; those with hypoxemia (SpO2 < 90% during waking hours) caused by an underlying pulmonary disease; those with neurological or cardiovascular diseases, including peripheral artery disease; and those with mental illness who were unable to comply with study procedures. All participants provided written informed consent, and the study protocol was approved by the institutional review board of Samsung Medical Center (IRB no. 2021-04-166).

      Tested device

      Our target device was a GW4 series (model SM-R890N or SM-R860N, Samsung Electronics Co., Seoul, South Korea), which is a watch-type of wearable device that includes a reflectance pulse oximeter module on its underside that is worn against the skin (Fig. 1). This module comprises a series of closely located light-emitting diodes for each wavelength (λIR = 940 nm, λRed = 660 nm) at the center. The 8 photodiodes that sense reflected light are located radially, with an average distance of 4.5 mm from the center. This device captured the PPG signal at a sampling frequency of 25 Hz for each wavelength, and it calculated the SpO2 every second. Output SpO2 data from the GW4 (WristO2) were presented as integers, and this device covered 70%–100% range of saturation because of its lack of accuracy for saturation levels less than 70%.
      • Chan ED
      • Chan MM
      • Chan MM
      Pulse oximetry: understanding its basic principles facilitates appreciation of its limitations.
      The GW4 continuously monitored the similarity of repetitive pulse waveforms to filter out inconclusive conditions.
      • Kirszenblat R
      • Edouard P
      Validation of the Withings ScanWatch as a wrist-worn reflective pulse oximeter: prospective interventional clinical study.
      In that case, the PPG signal quality was considered low, and the output value was not analyzed.
      Fig. 1
      Fig. 1Photos of the Galaxy Watch 4 (GW4) used in this study. (A) Underside view indicating the LED-PD module (yellow rectangular area), and (B) Magnified view of the LED-PD module. (C) A participant wearing the GW4 on the wrist. The white and red arrows indicate infrared and red LED, respectively. The green arrow indicates one of the 8 photodiodes located radially. LED, light-emitting diodes; PD, photodetectors.

      Performance evaluation of oxygen saturation obtained using GW4

      All participants underwent level 1 PSG (Embla N7000 PSG System, Medcare Flaga, Reykjavik, Iceland) while wearing the GW4. To compare the accuracy of its reflectance pulse oximeter, we simultaneously measured SpO2 using both the GW4 and a transmittance pulse oximetry system placed on the fingertip as a reference (SpO2Ref). Participants wore the GW4 on the left wrist and were instructed to tighten the device against the skin to obtain a high-quality PPG signal. Participants were classified into 4 groups based on the PSG results as follows: normal (AHI <5/h), mild (5≤ AHI <15/h), moderate (15≤ AHI <30/h), and severe (AHI ≥30/h) OSA. Accordingly, we employed a scoring metric following the guidelines published by the American Academy of Sleep Medicine (AASM).
      • Berry RB
      • Brooks R
      • Gamaldo C
      • et al.

      SpO2 calculation method

      A well-defined alternating component (AC)/direct component (DC) method was used to calculate SpO2. Briefly, noise in the PPG waveform was eliminated using a low-pass filter (fc = 12.5 Hz). Then, the AC and DC were measured based on the recent PPG waveform within a window. The perfusion index (PI) and R-value were calculated as PIλ = ACλ/DCλ and R-value = PIλRed ⁄ PIλIR, respectively. The SpO2 level was estimated based on the R-value using the predefined calibration information. During a previous in-laboratory test following the International Organization for Standardization (ISO) 80601-2-61;2017 standard, we confirmed that the performance of the GW4 conforms to the United States Food and Drug Administration (FDA) guidelines and the ISO standard requirements of root mean squared errors (RMSEs) less than 3.5% and 4%, respectively. To exclude falsely calculated values resulting from motion or venous pulsation, SpO2 values were screened using the morphology of PPG waveforms and stability of the output data. The GW4 automatically rejected the periods showing distorted waveforms distant from the arterial waveform and movements detected from accelerometers.

      Performance evaluation of WristO2 and the association between AHI and GW4-ODI

      The performance of the target device was evaluated using the RMSE and coverage rate. The RMSE was calculated using the squared error (SpO2Ref-WristO2) of all valid sample points when both the SpO2Ref and WristO2 data were available. The coverage rate was defined as the percentage of valid sample points out of the total time asleep. The data rejection rate was calculated as (1—coverage rate). To evaluate the performance, the RMSE and coverage rate were calculated using the pooled SpO2 errors. The RMSE and coverage rate were also calculated for each participant to assess the dependency on the OSA severity. The GW4 oxygen desaturation index (GW4-ODI) was derived from the WristO2, which was defined as the number of desaturation episodes divided by the total sleep time. The number of desaturation events was calculated as the number of drops with a difference of a certain criteria (2%, 3%, or 4%) or more between the maximum SpO2 and minimum SpO2 of a consecutive slope. The total sleep time was estimated using the algorithm of the watch, and the rejected data points were not included.

      Statistical analysis

      Demographics and PSG parameters were analyzed using descriptive statistical analysis to calculate the means and standard deviations of individual variables. MATLAB was used to analyze and visualize the RMSE and coverage rate of the WristO2. Bland-Altman density plots were used to show agreement between the WristO2 and SpO2Ref. Moreover, these plots were used along with correlation plots for correlation and agreement analyses.
      An analysis of the receiver-operating characteristic (ROC) curve was performed to compare the diagnostic performance of different ODI thresholds. All calculated P values were 2-tailed, and statistical significance was defined as P<0.05, using SPSS version 18.0 (SPSS Inc., Chicago, IL, USA).

      Results

      The demographics and polysomnographic findings of the participants are summarized in Table 1.
      Table 1Demographics and polysomnography parameters of the participants
      VariablesAllNormal (n = 18)Mild OSA (n = 21)Moderate OSA (n = 23)Severe OSA (n = 35)P value
      Demographics
      Age, years44.4 ± 13.039.1 ± 15.047.3 ± 12.343.4 ± 13.746.0 ± 11.5.30
      Male, n (%)74 (76.3)10 (55.6)12 (57.1)18 (78.3)34 (97.1)<.001
      BMI, kg/m226.3 ± 4.423.4 ± 2.424.3 ± 3.126.1 ± 5.029.2 ± 3.9<.001
      PSG parameters
      Total sleep time, min342.4 ± 61.3364.9 ± 56.0347.4 ± 52.7351.1 ± 53.4322.2 ± 69.4.22
      Sleep latency, min11.1 ± 12.914.3 ± 14.710.5 ± 8.611.9 ± 18.69.3 ± 9.1.63
      Sleep efficiency, %82.9 ± 11.282.0 ± 11.384.5 ± 9.484.6 ± 9.681.3 ± 13.1.73
      WASO, %14.9 ± 10.815.4 ± 10.613.3 ± 9.312.9 ± 9.416.8 ± 12.5.53
      Arousal index, /h29.3 ± 20.117.1 ± 6.518.8 ± 7.523.1 ± 6.345.9 ± 24.5<.001
      AHI, /h28.9 ± 27.42.8 ± 1.49.2 ± 3.221.2 ± 3.759.1 ± 22.8.73
      Lowest SpO2, %84 ± 8.592.0 ± 2.887.0 ± 5.286.1 ± 3.376.7 ± 8.9<.001
      ODI, /h24.3 ± 26.62.0 ± 1.87.0 ± 3.615.2 ± 6.352.1 ± 25.7<.001
      BMI, body mass index; OSA, obstructive sleep apnea; PSG, polysomnography; WASO, wakefulness after sleep onset; AHI, apnea-hypopnea index; ODI, oxygen desaturation index.

      Oxygen saturation

      Fig. 2 shows a representative image of the WristO2 with its reference during sleep. The WristO2 of a normal participant (AHI, 2.3/h) was distributed in the normal range of 90%–100% (Fig. 2A). Transient false recordings and baseline drift were observed intermittently. The erroneous readings (Fig. 2A) were assumed to be related to positional changes
      • Hickey M
      • Phillips JP
      • Kyriacou P
      Venous pooling and drainage affects photoplethysmographic signals at different vertical hand positions.
      or fluctuations in contact pressure between the sensor and the skin,
      • Fine I
      • Kaminsky A
      Possible error in reflection pulse oximeter readings as a result of applied pressure.
      both of which have been reported previously.
      • Braun F
      • Theurillat P
      • Proenca M
      • et al.
      Pulse oximetry at the wrist during sleep: performance, challenges and perspectives.
      Multiple true desaturation events assosciated with the WristO2 were observed in one patient with severe OSA (AHI, 91/h) (Fig. 2B).
      Fig. 2
      Fig. 2Illustrations of overnight WristO2 traces of normal participants (A) and patients with severe OSA (B). Transient false recordings were observed intermittently (blue arrows). The black trace indicates the WristO2, and the gray-shaded area indicates the SpO2Ref with a ±4% error. The gray arrows indicate examples of motion-related transient drop in the WristO2, and the black arrow points to the baseline shift and slow recovery to the baseline. Magnified views of red rectangular areas in (B) are shown in (C) and (D); these represent periodic desaturation events caused by apnea or hypopnea and relatively small changes in oxygen saturation (C) and large changes with the lowest saturation of 71% (D). WristO2, oxygen saturation derived from a wrist-worn reflectance pulse oximeter; OSA, obstructive sleep apnea; SpO2Ref, oxygen saturation according to the finger-attached transmittance pulse oximeter used as a reference.

      Performance evaluation

      To evaluate the WristO2, we included valid data points for which the WristO2 and SpO2Ref were both available, and these points were used used for analysis. Of 672.3 hours of recorded sleep time, 3.4 hours were excluded due to inconclusive data from SpO2Ref, 171.7 hours from WristO2, and 6.4 hours from both WristO2 and SpO2Ref; thus, a considerable amount of data were missing from the WristO2 calculations. After excluding these missing data, 490.8 hours of recorded sleep time were used for the analysis.
      The coverage rate of the GW4 was 73.5%, and its data rejection rate was 26.5%. The RMSE was 2.3%, which was calculated during the entire valid period; therefore, it met the requirements of the FDA and ISO standards (Table 2). The Bland-Altman density plot showed good agreement between the 2 measurements, with a mean bias of -0.16% (Fig. 3).
      Table 2GW4 pulse oximeter performance
      SpecificationValues
      RMSE2.28%
      Bias-0.16%
      95% lower limit of bias-4.63%
      95% upper limit of bias4.31%
      Total data duration672.3 hours
      Valid data duration490.8 hours
      Data rejection rate26.5%
      GW4, Galaxy Watch 4; RMSE, root mean square error.
      Fig. 3
      Fig. 3Bland-Altman density plot showing good agreement between the WristO2 and SpO2Ref (A). The bar indicates the number of samples. The black dashed lines indicate the upper and lower 95% limits of agreement calculated from the estimated error, ranging from -4.63% to 4.31%. When the saturation value is low, a large estimation error is more likely. Probability density functions of the estimation error (B) and averaged oxygen saturation (C) are shown. The bias of the WristO2 is -0.16%, with a standard deviation of 2.28%. WristO2, oxygen saturation derived from a wrist-worn reflectance pulse oximeter; SpO2Ref, oxygen saturation according to the finger-attached transmittance pulse oximeter used as a reference.

      Performance based on the severity of OSA

      To characterize the performance of WristO2 among participants with different OSA severity, the RMSE and coverage rate of each participant were compared according to AHI and OSA severity groups. Almost all participants had an AHI ≤ 60, and and the performance of the device met FDA and ISO standards (Fig. 4A). However, when participants had a AHI > 60, the estimation error increased and the WristO2 did not meet the FDA and ISO standards.
      Fig. 4
      Fig. 4The obtained WristO2 according to different AHI values (A, B) and OSA severity (C, D) on scatter plots and bar plots. Each dot represents one participant. This image shows a non-linear correlation between the RMSE and the AHI (A). There was no correlation found between the coverage rate and the AHI (B). WristO2, oxygen saturation derived from a wrist-worn reflectance pulse oximeter; AHI, apnea-hypopnea index; OSA, obstructive sleep apnea; RMSE, root mean square error.
      The RMSEs were 1.65 ± 0.57%, 1.76 ± 0.65%, 1.93 ± 0.54%, and 2.93 ± 1.71% for normal, mild, moderate, and severe OSA, respectively. There was no significant difference in the RMSEs of the different groups; however, the severe OSA group had a trend of poor performance of WristO2 (Fig. 4C). Conversely, the coverage rates were 74.30 ± 18.16%, 79.26 ± 14.13%, 77.94 ± 15.48%, and 64.29 ± 19.08% for normal, mild, moderate, and severe OSA, respectively; these were variable regardless of the AHI (Fig. 4B, D).

      Association between AHI ≥15/h and GW4-ODI

      ROC analysis to predict an AHI ≥15/h was performed to compare the GW4-ODI based on different thresholds, specifically 2%, 3%, and 4% (Fig. 5). The areas under the ROC curve (AUCs) were 0.890 (95% confidence interval [CI], 0.829-0.952), 0.908 (95% CI, 0.852-0.963), and 0.893 (95% CI, 0.832-0.953) for the thresholds of 2%, 3%, and 4%, respectively. Therefore, the GW4-ODI was defined as a 3% decrease in the WristO2 from baseline per hour.
      Fig. 5
      Fig. 5Receiver-operating characteristic curves of the ODI with different thresholds for predicting an AHI ≥15/h. ODI: oxygen desaturation index; AUC, area under the curve; AHI: apnea-hypopnea index.
      Strong positive correlations were observed between the AHI and PSG-ODI (Pearson correlation, r = 0.951) and between the AHI and GW4-ODI (r = 0.918) (Fig. 6A, B). Bland-Altman plots were used to show agreement between the AHI and ODI obtained from PSG and GW4 (Fig. 6C, D). The biases were similar under both conditions, and the AHI was higher than both the PSG-ODI (mean, 9.28; SD, 10.26) and GW4-ODI (mean, 9.32; SD, 11.46). Participants with a difference of 10 or more between the AHI and GW4-ODI had a higher total hypopnea index (26.18 ± 12.54 vs. 8.82±7.53, P < .001) and lower saturation (81.84 ± 6.52 vs. 85.36 ± 9.35, P = .03) than those with a difference less than 10.
      Fig. 6
      Fig. 6Correlation plots and Bland-Altman plots of the AHI and ODI derived from PSG (A, C) and the GW4 (B, D). Strong positive correlations are observed between the AHI and ODI derived from PSG (A, r =0.951) and the GW4 (B, r =0.918). Different color codes representing the OSA severity groups are displayed in Bland-Altman plots. The middle line is the mean difference between the 2 measurements. The upper and lower lines represent limits of agreement with a standard deviation ±1.96. Both the PSG-ODI and GW4-ODI showed increasing bias and greater dispersions, concomitant with the AHI increase. AHI, apnea-hypopnea index; ODIEst, estimated oxygen desaturation index; PSG, polysomnography; GW4, Galaxy Watch 4.
      Parameters for predicting AHI ≥15/h were estimated according to the different cutoff values associated with the GW4-ODI (Table 3). The highest sensitivity was observed with a cutoff value of GW4-ODI ≥5/h. A GW4-ODI ≥15/h and ≥20/h provided 100% specificity.
      Table 3Comparison of GW4-ODI performance with different cutoff values to predict the apnea-hypopnea index ≥15
      ODI cutoff valueSensitivity, %Specificity, %PPV, %NPV, %Accuracy, %
      ≥5/h89.764.178.880.679.4
      ≥10/h74.189.791.570.080.4
      ≥15/h56.910010060.974.2
      ≥20/h48.310010056.569.1
      GW4, Galaxy Watch 4; ODI, oxygen desaturation index; PPV, positive predictive value; NPV, negative predictive value.

      Discussion

      Wearable devices are increasingly being used in healthcare to facilitate continuous real-life monitoring of users' data.
      • Kim C
      How will the digital tools change healthcare?.
      ,
      • Kim J
      • Cho JW
      Future sleep medicine: mobile health and big data.
      In this study, we evaluated the estimated SpO2 obtained from a commercially available watch-type wearable device. Previous studies have shown that reflectance pulse oximetry can relatively accurately detect desaturation, except when measurements are performed at high altitudes.
      • Hermand E
      • Coll C
      • Richalet JP
      • Lhuissier FJ
      Accuracy and reliability of pulse O2 saturation measured by a wrist-worn oximeter.
      • Kirszenblat R
      • Edouard P
      Validation of the Withings ScanWatch as a wrist-worn reflective pulse oximeter: prospective interventional clinical study.
      • Lauterbach CJ
      • Romano PA
      • Greisler LA
      • Brindle RA
      • Ford KR
      • Kuennen MR
      Accuracy and reliability of commercial wrist-worn pulse oximeter during normobaric hypoxia exposure under resting conditions.
      However, wrist-worn reflectance pulse oximetry has more limitations than transmittance pulse oximetry.
      • Lee H
      • Ko H
      • Lee J
      Reflectance pulse oximetry: practical issues and limitations.
      Because they are susceptible to motion and noise artifacts, commercially available wearable smartwatches have their own algorithms to enhance PPG measurements.
      • Lee H
      • Ko H
      • Lee J
      Reflectance pulse oximetry: practical issues and limitations.

      Vandecasteele K, Làzaro J, Cleeren E, et al. Artifact detection of wrist photoplethysmograph signals. In Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies. Volume 3: BIOSIGNALS, ISBN 978-989-758-279-0, pages 182-189. DOI: 10.5220/0006594301820189

      Phillips C, Liaqat D, Gabel M, de Lara E. WristO2: reliable peripheral oxygen saturation readings from wrist-worn pulse oximeters. 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops) 2021:623-629. doi: 10.1109/PerComWorkshops51409.2021.9430986.

      Accuracy of SpO2 measurements during sleep

      Reflectance pulse oximetry of the GW4 showed an overall RMSE of 2.28% and a negligible bias of -0.16% during sleep. This performance is comparable to that of reported by previous studies of wrist-worn wearable devices monitoring SpO2 that included an in-house device, the Withings ScanWatch and Garmin-branded devices.
      • Braun F
      • Theurillat P
      • Proenca M
      • et al.
      Pulse oximetry at the wrist during sleep: performance, challenges and perspectives.
      ,
      • Hermand E
      • Coll C
      • Richalet JP
      • Lhuissier FJ
      Accuracy and reliability of pulse O2 saturation measured by a wrist-worn oximeter.
      • Kirszenblat R
      • Edouard P
      Validation of the Withings ScanWatch as a wrist-worn reflective pulse oximeter: prospective interventional clinical study.
      • Lauterbach CJ
      • Romano PA
      • Greisler LA
      • Brindle RA
      • Ford KR
      • Kuennen MR
      Accuracy and reliability of commercial wrist-worn pulse oximeter during normobaric hypoxia exposure under resting conditions.
      Individuals move and change their position unconsciously throughout the sleep cycle, which prevents proper measurement of SpO2 because of a poor sensor-skin interface. Furthermore, in the clinical setting, such as a sleep laboratory, sleep apnea induces oxygen desaturation and subsequent arousals; which are usually followed by body or limb movements. False recordings of SpO2 frequently occur because of venous pulsations,
      • Shelley KH
      • Tamai D
      • Jablonka D
      • Gesquiere M
      • Stout RG
      • Silverman DG
      The effect of venous pulsation on the forehead pulse oximeter wave form as a possible source of error in SpO2 calculation.
      position changes,
      • Hickey M
      • Phillips JP
      • Kyriacou P
      Venous pooling and drainage affects photoplethysmographic signals at different vertical hand positions.
      respirations,
      • Phillips JP
      • Belhaj A
      • Langford RM
      • Kyriacou PA
      Effect of respiratory-induced intensity variations on finger SpO2 measurements in volunteers.
      or issues with the sensor-skin interface.
      • Mannheimer PD
      The light-tissue interaction of pulse oximetry.
      Empirically, we found that recorded SpO2 measurements were much more accurate when participants adhered to the manufacturer's instructions to wear the device as tightly as possible above the wrist bone.
      Although most sampled data from previous studies had an SpO2 range of 90%–100%, this study included participants presenting a wide range of SpO2, with the lowest SpO2Ref of 54%. The Bland-Altman density plot showed good agreement between the WristO2 and SpO2ref, with an acceptable error in the range of 70%–100% of the average saturation (Fig. 3A). Poor performance of WristO2 with profound desaturation may be caused by arousal from sleep due to apnea or hypopnea. It is possible that some motion-derived artifacts in PPG waveforms coincide with the changes in the accelerometric parameters. However, we found that the coverage rate was variable regardless of the OSA severity; that is, low saturation itself is related to increases in error. This result is consistent with that of previous reportings showing a decrease in accuracy with a lower saturation level.
      • Thrush D
      • Hodges MR
      Accuracy of pulse oximetry during hypoxemia.
      ,
      • Severinghaus J
      • Naifeh K
      • Koh S
      Errors in 14 pulse oximeters during profound hypoxia.
      Although the GW4 showed a relatively imprecise estimation of SpO2 for participants with very severe OSA, it demonstrated generally good performance for the other 3 groups.

      Association between AHI ≥ 15/h and GW4-ODI

      The AASM Sleep Apnea Definitions Task Force recommends the use of ≥3% (instead of ≥4%) as the oxygen desaturation criterion to define hypopnea.
      • Berry RB
      • Budhiraja R
      • Gottlieb DJ
      • et al.
      Rules for scoring respiratory events in sleep: update of the 2007 AASM Manual for the Scoring of Sleep and Associated Events. Deliberations of the Sleep Apnea Definitions Task Force of the American Academy of Sleep Medicine.
      However, these guidelines do not specify a definition of the ODI. Using the ODI for screening OSA could be an alternative technique for measuring the AHI, but it is difficult to directly compare the diagnostic performance of the ODI across studies because different definitions and software have been used.
      • Rashid NH
      • Zaghi S
      • Scapuccin M
      • Camacho M
      • Certal V
      • Capasso R
      The value of oxygen desaturation index for diagnosing obstructive sleep apnea: a systematic review.
      ,
      • Ng Y
      • Joosten SA
      • Edwards BA
      • et al.
      Oxygen desaturation index differs significantly between types of sleep software.
      To the best of our knowledge, no previous study has attempted to calculate the ODI using wrist-worn reflectance pulse oximetry during sleep. Instead, wrist-worn wearable devices with transmittance pulse oximetry attached to the finger, such as WatchPAT or Pulsox-300i, have been verified with outstanding discrimination.
      • Ma JR
      • Huang JJ
      • Chen Q
      • Wu HT
      • Xiao KL
      • Zhang YT
      Value of pulse oximetry watch for diagnosing pediatric obstructive sleep apnea/hypopnea syndrome.
      • Lin HC
      • Su CL
      • Ong JH
      • et al.
      Pulse oximetry monitor feasible for early screening of obstructive sleep apnea (OSA).
      • Waseem R
      • Chan MTV
      • Wang CY
      • Seet E
      • Chung F
      Predictive performance of oximetry in detecting sleep apnea in surgical patients with cardiovascular risk factors.
      The ROC analysis in our study found that a 3% decrease in the WristO2 from baseline showed outstanding discrimination, with an AUC of 0.908 to predict an AHI ≥ 15/h.
      • Lemeshow H
      • Hosmer DW
      Assessing the Fit of the Model. Applied Logistic Regression.
      The GW4-ODI ≥5/h was chosen as a predictor for AHI ≥15/h because of its high sensitivity. However, it should be noted that GW4 is not a screening device for OSA due to its high rejection rate.
      It is worth noting that the discrimination between the GW4-ODI and AHI of individuals requires different criteria. The GW4-ODI underestimated the AHI by 9.32 ± 11.46 in our study. Furthermore, a similar degree of bias between the AHI and ODI, defined as desaturation of 4% or more, was reported in a previous study.
      • Overland B
      • Skatvedt O
      • Kvaerner KJ
      • Akre H
      Pulse oximetry: sufficient to diagnose severe sleep apnea.
      Although normal participants showed good agreement between the AHI and GW4-ODI, more severe OSA was related to increased bias and greater dispersions in the Bland-Altman plot (Figure 6D). The bias and variability drastically increased in the severe OSA group. When participants were divided into 2 groups according to the difference between the AHI and GW4-ODI, the group with a difference of ≥10/h had a higher total hypopnea index. This suggests that the GW4-ODI underestimates hypopnea because the ODI is unable to detect hypopneas without desaturation. This group with a difference of ≥10/h also included participants with a lower SpO2 nadir, indicating that patients with more severe OSA may be more susceptible to prediction error.
      In the previous studies that predict OSA using a finger-attached pulse oximeter, the sensitivity and specificity varied with various ODI definitions and target severities.
      • Ma JR
      • Huang JJ
      • Chen Q
      • Wu HT
      • Xiao KL
      • Zhang YT
      Value of pulse oximetry watch for diagnosing pediatric obstructive sleep apnea/hypopnea syndrome.
      • Lin HC
      • Su CL
      • Ong JH
      • et al.
      Pulse oximetry monitor feasible for early screening of obstructive sleep apnea (OSA).
      • Waseem R
      • Chan MTV
      • Wang CY
      • Seet E
      • Chung F
      Predictive performance of oximetry in detecting sleep apnea in surgical patients with cardiovascular risk factors.
      ,
      • Romem A
      • Romem A
      • Koldobskiy D
      • Scharf SM
      Diagnosis of obstructive sleep apnea using pulse oximeter derived photoplethysmographic signals.
      • Chung F
      • Liao P
      • Elsaid H
      • Islam S
      • Shapiro CM
      • Sun Y
      Oxygen desaturation index from nocturnal oximetry: a sensitive and specific tool to detect sleep-disordered breathing in surgical patients.
      • Dawson A
      • Loving RT
      • Gordon RM
      • et al.
      Type III home sleep testing versus pulse oximetry: is the respiratory disturbance index better than the oxygen desaturation index to predict the apnoea-hypopnoea index measured during laboratory polysomnography?.
      • Hang LW
      • Wang HL
      • Chen JH
      • et al.
      Validation of overnight oximetry to diagnose patients with moderate to severe obstructive sleep apnea.
      • Li Y
      • Gao H
      • Ma Y
      Evaluation of pulse oximeter derived photoplethysmographic signals for obstructive sleep apnea diagnosis.
      The present study showed lower accuracy when predicting an AHI ≥ 15/h compared to prior studies using an ODI of 3% and reported sensitivity ranging from 86.1% to 96% and specificity from 89% to 94%.
      • Lin HC
      • Su CL
      • Ong JH
      • et al.
      Pulse oximetry monitor feasible for early screening of obstructive sleep apnea (OSA).
      ,
      • Hang LW
      • Wang HL
      • Chen JH
      • et al.
      Validation of overnight oximetry to diagnose patients with moderate to severe obstructive sleep apnea.
      ,
      • Li Y
      • Gao H
      • Ma Y
      Evaluation of pulse oximeter derived photoplethysmographic signals for obstructive sleep apnea diagnosis.
      The relatively low specificity of the GW4-ODI could be attributed to the vulnerability to artifacts compared to the ODI derived from a firmly attached transmittance pulse oximeter.

      Limitations

      This study had several limitations that require particular attention. First, the error in the WristO2 was larger than the true SpO2 level. During this investigation, transmittance pulse oximetry was used as a reference instead of co-oximetry, and we considered an RMSE of ±4% as acceptable. This resulted in a maximum acceptable range with an RMSE of ±8% with co-oximetry as a reference. Although the FDA guidance requires the use of co-oximetry as a reference,

      Pulse Oximeters - Premarket Notification Submissions [510(k)s]: Guidance for Industry and Food and Drug Administration Staff. 2013.

      we were not able to easily perform invasive co-oximetry measurements during sleep in this study. Second, the WristO2 is inherently vulnerable to artifacts. An average data loss of 26.5% during one night resulted from fluctuations in contact pressure caused by non-supine body positions, movements, and the discarding of data less than 70% of saturation (Supplementary Fig. 1). Therefore, it is important to minimize such artifacts by following strict guidelines or an error-correction algorithm. Improvement of the algorithm will enable the correction of the afore-mentioned errors, such as a sudden and transient drop or baseline shift with the slow recovery of the WristO2, thus allowing improvement of the accuracy or coverage rate. Third, the weakness of wrist-worn reflectance pulse oximetry is its declining accuracy at low saturation levels. Although the accuracy of the device with a low SpO2 range was relatively well-preserved in a previous study, the difference from our study was that participants were in the semi-supine position and remained still during measurements.
      • Kirszenblat R
      • Edouard P
      Validation of the Withings ScanWatch as a wrist-worn reflective pulse oximeter: prospective interventional clinical study.
      When SpO2 decreased during sleep, the WristO2 tended to have a lower value than the reference, thus underestimating the true SpO2.

      Conclusions and future work

      This study characterized the performance of the GW4 when measuring the estimated SpO2 during sleep. The accuracy of the GW4 complies with the FDA and ISO standards. The major strength of this study is that it evaluated the performance of wrist-worn reflectance pulse oximetry when measuring the SpO2 of patients with sleep apnea and of some patients with profound desaturation.
      In the field of sleep medicine, the development of precise and convenient SpO2 measurements should be associated with the screening and monitoring of OSA. With improvements in the accuracy and error correction algorithms of wrist-worn wearable devices, it is expected that future devices will provide more accurate and reliable information to patients and clinicians alike.

      Authors’ contribution

      HJ, DK, and EYJ conceived and designed the study. WL, HS, and JS participated in data curation. HJ and DK conducted data analyses and drafted the manuscript. EYJ and JC reviewed and edited the manuscript. All authors approved the final version of the manuscript.

      Declaration of conflict of interest

      This study was funded by Samsung Electronics. Some authors (HJ, WL, HS, JS, JC, and EYJ) are affiliated with Samsung Electronics. Open access to the data was provided and the research was monitored independently by the 2 institutions.

      Funding

      This study was supported by a grant from Samsung Medical Center (grant OTC1190671).

      Appendix. Supplementary materials

      References

        • Allen J
        Photoplethysmography and its application in clinical physiological measurement.
        Physiol Meas. 2007; 28: R1https://doi.org/10.1088/0967-3334/28/3/R01
        • Tamura T
        • Maeda Y
        • Sekine M
        • Yoshida M
        Wearable photoplethysmographic sensors—past and present.
        Electronics. 2014; 3: 282-302https://doi.org/10.3390/electronics3020282
        • Longmore SK
        • Lui GY
        • Naik G
        • Breen PP
        • Jalaludin B
        • Gargiulo GD
        A comparison of reflective photoplethysmography for detection of heart rate, blood oxygen saturation, and respiration rate at various anatomical locations.
        Sensors (Basel). 2019; 19: 1874https://doi.org/10.3390/s19081874
        • Fine J
        • Branan KL
        • Rodriguez AJ
        • et al.
        Sources of inaccuracy in photoplethysmography for continuous cardiovascular monitoring.
        Biosensors (Basel). 2021; 11: 126https://doi.org/10.3390/bios11040126
        • Braun F
        • Theurillat P
        • Proenca M
        • et al.
        Pulse oximetry at the wrist during sleep: performance, challenges and perspectives.
        Annu Int Conf IEEE Eng Med Biol Soc. 2020; : 5115-5118https://doi.org/10.1109/embc44109.2020.9176081
        • Guber A
        • Shochet GE
        • Kohn S
        • Shitrit D
        Wrist-sensor pulse oximeter enables prolonged patient monitoring in chronic lung diseases.
        J Med Syst. 2019; 43: 230https://doi.org/10.1007/s10916-019-1317-2
        • Kramer M
        • Lobbestael A
        • Barten E
        • Eian J
        • Rausch G
        Wearable pulse oximetry measurements on the torso, arms, and legs: a proof of concept.
        Mil Med. 2017; 182: 92-98https://doi.org/10.7205/MILMED-D-16-00129
        • Senaratna CV
        • Perret JL
        • Lodge CJ
        • et al.
        Prevalence of obstructive sleep apnea in the general population: a systematic review.
        Sleep Med Rev. 2017; 34: 70-81https://doi.org/10.1016/j.smrv.2016.07.002
        • Rashid NH
        • Zaghi S
        • Scapuccin M
        • Camacho M
        • Certal V
        • Capasso R
        The value of oxygen desaturation index for diagnosing obstructive sleep apnea: a systematic review.
        Laryngoscope. 2021; 131: 440-447https://doi.org/10.1002/lary.28663
        • Hermand E
        • Coll C
        • Richalet JP
        • Lhuissier FJ
        Accuracy and reliability of pulse O2 saturation measured by a wrist-worn oximeter.
        Int J Sports Med. 2021; 42: 1268-1273https://doi.org/10.1055/a-1337-2790
        • Kirszenblat R
        • Edouard P
        Validation of the Withings ScanWatch as a wrist-worn reflective pulse oximeter: prospective interventional clinical study.
        J Med Internet Res. 2021; 23: e27503https://doi.org/10.2196/27503
        • Lauterbach CJ
        • Romano PA
        • Greisler LA
        • Brindle RA
        • Ford KR
        • Kuennen MR
        Accuracy and reliability of commercial wrist-worn pulse oximeter during normobaric hypoxia exposure under resting conditions.
        Res Q Exerc Sport. 2021; 92: 549-558https://doi.org/10.1080/02701367.2020.1759768
        • Chan ED
        • Chan MM
        • Chan MM
        Pulse oximetry: understanding its basic principles facilitates appreciation of its limitations.
        Respir Med. 2013; 107: 789-799https://doi.org/10.1016/j.rmed.2013.02.004
        • Berry RB
        • Brooks R
        • Gamaldo C
        • et al.
        The AASM manual for the scoring of sleep and associated events: rules, terminology and technical specifications, version 2.4. American Academy of Sleep Medicine, Darien, IL2017
        • Hickey M
        • Phillips JP
        • Kyriacou P
        Venous pooling and drainage affects photoplethysmographic signals at different vertical hand positions.
        SPIE. 2015; 9332 (Optical Diagnostics and Sensing XV: Toward Point-of-Care Diagnostics): 9332
        • Fine I
        • Kaminsky A
        Possible error in reflection pulse oximeter readings as a result of applied pressure.
        J Healthc Eng. 2019; 7293813https://doi.org/10.1155/2019/7293813
        • Kim C
        How will the digital tools change healthcare?.
        J Sleep Med. 2019; 16: 71-74https://doi.org/10.13078/jsm.190040
        • Kim J
        • Cho JW
        Future sleep medicine: mobile health and big data.
        J Sleep Med. 2019; 16: 1-10https://doi.org/10.13078/jsm.18016
        • Lee H
        • Ko H
        • Lee J
        Reflectance pulse oximetry: practical issues and limitations.
        ICT Express. 2016; 2: 195-198https://doi.org/10.1016/j.icte.2016.10.004
      1. Vandecasteele K, Làzaro J, Cleeren E, et al. Artifact detection of wrist photoplethysmograph signals. In Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies. Volume 3: BIOSIGNALS, ISBN 978-989-758-279-0, pages 182-189. DOI: 10.5220/0006594301820189

      2. Phillips C, Liaqat D, Gabel M, de Lara E. WristO2: reliable peripheral oxygen saturation readings from wrist-worn pulse oximeters. 2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops) 2021:623-629. doi: 10.1109/PerComWorkshops51409.2021.9430986.

        • Shelley KH
        • Tamai D
        • Jablonka D
        • Gesquiere M
        • Stout RG
        • Silverman DG
        The effect of venous pulsation on the forehead pulse oximeter wave form as a possible source of error in SpO2 calculation.
        Anesth Analg. 2005; 100: 743-747https://doi.org/10.1213/01.Ane.0000145063.01043.4b
        • Phillips JP
        • Belhaj A
        • Langford RM
        • Kyriacou PA
        Effect of respiratory-induced intensity variations on finger SpO2 measurements in volunteers.
        Annu Int Conf IEEE Eng Med Biol Soc. 2013; 2013: 3937-3940https://doi.org/10.1109/embc.2013.6610406
        • Mannheimer PD
        The light-tissue interaction of pulse oximetry.
        Anesth Analg. 2007; 105: S10-S17https://doi.org/10.1213/01.ane.0000269522.84942.54
        • Thrush D
        • Hodges MR
        Accuracy of pulse oximetry during hypoxemia.
        South Med J. 1994; 87: 518-521https://doi.org/10.1097/00007611-199404000-00019
        • Severinghaus J
        • Naifeh K
        • Koh S
        Errors in 14 pulse oximeters during profound hypoxia.
        J Clin Monit. 1989; 5: 72-81https://doi.org/10.1007/BF01617877
        • Berry RB
        • Budhiraja R
        • Gottlieb DJ
        • et al.
        Rules for scoring respiratory events in sleep: update of the 2007 AASM Manual for the Scoring of Sleep and Associated Events. Deliberations of the Sleep Apnea Definitions Task Force of the American Academy of Sleep Medicine.
        J Clin Sleep Med. 2012; 8: 597-619https://doi.org/10.5664/jcsm.2172
        • Ng Y
        • Joosten SA
        • Edwards BA
        • et al.
        Oxygen desaturation index differs significantly between types of sleep software.
        J Clin Sleep Med. 2017; 13: 599-605https://doi.org/10.5664/jcsm.6552
        • Ma JR
        • Huang JJ
        • Chen Q
        • Wu HT
        • Xiao KL
        • Zhang YT
        Value of pulse oximetry watch for diagnosing pediatric obstructive sleep apnea/hypopnea syndrome.
        Acta Otolaryngol. 2018; 138: 175-179https://doi.org/10.1080/00016489.2017.1384569
        • Lin HC
        • Su CL
        • Ong JH
        • et al.
        Pulse oximetry monitor feasible for early screening of obstructive sleep apnea (OSA).
        J Med Biol Eng. 2019; 40: 62-70https://doi.org/10.1007/s40846-019-00479-6
        • Waseem R
        • Chan MTV
        • Wang CY
        • Seet E
        • Chung F
        Predictive performance of oximetry in detecting sleep apnea in surgical patients with cardiovascular risk factors.
        PLoS One. 2021; 16e0250777https://doi.org/10.1371/journal.pone.0250777
        • Lemeshow H
        • Hosmer DW
        Assessing the Fit of the Model. Applied Logistic Regression.
        3rd ed. John Wiley & Sons Inc., New Jersey, NJ2013: 177
        • Overland B
        • Skatvedt O
        • Kvaerner KJ
        • Akre H
        Pulse oximetry: sufficient to diagnose severe sleep apnea.
        Sleep Med. 2002; 3: 133-138https://doi.org/10.1016/s1389-9457(01)00122-8
        • Romem A
        • Romem A
        • Koldobskiy D
        • Scharf SM
        Diagnosis of obstructive sleep apnea using pulse oximeter derived photoplethysmographic signals.
        J Clin Sleep Med. 2014; 10: 285-290https://doi.org/10.5664/jcsm.3530
        • Chung F
        • Liao P
        • Elsaid H
        • Islam S
        • Shapiro CM
        • Sun Y
        Oxygen desaturation index from nocturnal oximetry: a sensitive and specific tool to detect sleep-disordered breathing in surgical patients.
        Anesth Analg. 2012; 114: 993-1000https://doi.org/10.1213/ANE.0b013e318248f4f5
        • Dawson A
        • Loving RT
        • Gordon RM
        • et al.
        Type III home sleep testing versus pulse oximetry: is the respiratory disturbance index better than the oxygen desaturation index to predict the apnoea-hypopnoea index measured during laboratory polysomnography?.
        BMJ Open. 2015; 5e007956https://doi.org/10.1136/bmjopen-2015-007956
        • Hang LW
        • Wang HL
        • Chen JH
        • et al.
        Validation of overnight oximetry to diagnose patients with moderate to severe obstructive sleep apnea.
        BMC Pulm Med. 2015; 15: 24https://doi.org/10.1186/s12890-015-0017-z
        • Li Y
        • Gao H
        • Ma Y
        Evaluation of pulse oximeter derived photoplethysmographic signals for obstructive sleep apnea diagnosis.
        Medicine (Baltimore). 2017; 96: e6755https://doi.org/10.1097/MD.0000000000006755
      3. Pulse Oximeters - Premarket Notification Submissions [510(k)s]: Guidance for Industry and Food and Drug Administration Staff. 2013.