Reducing False Positive Atrial Fibrillation detections with the PulseAI neural network
by Dr Alan Kennedy, PhD
Atrial fibrillation (AF) is the most common form of arrhythmia. AF episodes can last from a few seconds to several days, and they may produce symptoms such as heart palpitations, shortness of breath, and tiredness. Although AF is not typically deadly, it can cause significant repercussions such as stroke, therefore it's critical to detect and treat the condition as soon as feasible. False positive AF detections are also an issue, in some cases resulting in needless drug usage, such as anticoagulants, which pose a number of risks to the patients when prescribed unnecessarily.
An electrocardiogram (ECG) is the most popular method for detecting AF. A 12-lead ECG is a clinical test that uses sensors placed on the skin to examine the heart's electrical activity. The majority of commercial ECG devices interpret heart rhythm using software algorithms [1, 2], after which a doctor confirms or corrects the findings to arrive at diagnosis. We recently encountered a case of a patient who was experiencing palpitations when they visited their GP. They had an ECG recorded at the GP's office, which is shown below.
The patient’s ECG in the GP office.
The automated ECG algorithm on the device interpreted the ECG as AF, which prompted anticoagulation by the GP and referral to secondary care (Cardiology). However, it is clear that the 12-lead ECG is in fact not AF, it is sinus rhythm with premature atrial contractions. Many studies over the years have described premature atrial contractions as a significant source of false positives for AF detection from traditional computerised ECG algorithms, and this is a good example.
We wanted to assess how our AI technology would have performed in this scenario. We also wondered if the PulseAI neural network would have helped the patient avoid unnecessary medication and also saved them a referral for secondary care. We extracted the rhythm strip waveform from the PDF using a proprietary internal algorithm. We then passed the digital samples to our ECG analysis platform, which provided an automated interpretation using the PulseAI neural network. The PulseAI neural network was trained on a continually growing proprietary internal database of more than 7.8 million ECG rhythm strips.
Printout of the analysis performed using the PulseAI Rhythm Strip API.
With no prior knowledge of the case, our AI algorithm correctly identified the ECG as normal sinus rhythm with premature atrial contractions.
Testing on MIT-BIH Arrhythmia Database
To validate our AI algorithm, we also tested it on the MIT-BIH Physionet arrhythmia database [4, 5] against a previously published state-of-the-art AF detection algorithm [6]. The state-of-the-art algorithm is similar to the algorithms used in the majority of today's clinical ECG devices. The database (not used for training the PulseAI neural network) is a publicly available database annotated by cardiologists for ECG algorithm benchmarking; it consists of 48 30-minute ECG recordings extracted from 24-hour ambulatory ECG recordings. Half of the recordings contain clinically significant arrhythmias. We divided the database into 30 second rhythm strips (n=2880) for analysis.
The performance of the PulseAI neural network algorithm in comparison to the state-of-the-art heuristic approach.
The results demonstrated that the PulseAI neural network had comparable sensitivity for AF detection but reduced the false positive rate significantly (McNemars Test, p<0.001) improving the specificity by 21% and the PPV by a staggering 64%.
Towards the future
This AF false positive problem is significant and getting worse. In 2017, Dr John Mandrola famously published an article titled: False Atrial Fibrillation: a growing patient safety issue [7]. He described badly performing ECG algorithms as the reason for not only incorrect interpretations but also a deterioration in the physicians’ ability to do ECG interpretation.
“My partners and I regularly see patients who have been referred for AF, or tested for AF, or worse, treated for AF, when they did not have AF. The computer misread the ECG and the doctor did not recognize the mistake.” Dr John Mandrola.
The importance of accurate AF detection from the ECG cannot be understated. In this clinical case study, we saw how an automated ECG algorithm misinterpreted a minor abnormality as AF, resulting in the GP prescribing anticoagulation medication and referring the patient to secondary care unnecessarily. If the ECG had been correctly interpreted, the patient would not have needed the unnecessary medication and referral. Saving time, money and a lot of stress for the patient.
This problem will only be exacerbated by the growing use of wearable devices in non-clinical settings to assess cardiac rhythm. In 2020, there were almost 34 million Apple Watches sold with ECG sensors built in, allowing for recording of ECG data on an unprecedented scale.
At its core, the promise of accurate automated ECG interpretation software remains true, improving patient safety and expediting patient care. However, we must innovate and improve the performance of the interpretation software using more advanced technologies. This case highlights the importance of accurate AF detection for both patients and physicians alike.
PulseAI will be presenting the full results of their clinical validation at this year's International Computing in Cardiology Conference in Tampere, Finland.
The PulseAI neural network technology is available via our cloud platform (dev.pulseaitechnologies.com) and easily integrates with 12-lead, Holter and Smartwatch ECG monitors. For further details, get in touch: info@pulseai.io
References
B. Devine, E. Clark, S. Luo and P. W. Macfarlane, "Critical values in the Uni-G program," 2012 Computing in Cardiology, 2012, pp. 601-604.
Mortara Veritas Physicians Guide, 2015
Smulyan H. The computerized ECG: friend and foe. The American journal of medicine. 2019 Feb 1;132(2):153-60.
Goldberger, A., Amaral, L., Glass, L., Hausdorff, J., Ivanov, P. C., Mark, R., ... & Stanley, H. E. (2000). PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation. 101 (23), pp. E215–e220.
Moody GB, Mark RG. The impact of the MIT-BIH Arrhythmia Database. IEEE Eng in Med and Biol 20(3):45-50 (May-June 2001). (PMID: 11446209)
Luo C, Li Q, Rao H, Huang X, Jiang H, Rao N. An improved Poincaré plot-based method to detect atrial fibrillation from short single-lead ECG. Biomedical Signal Processing and Control. 2021 Feb 1;64:102264.
Mandrola, J. (2017). Fake Atrial Fibrillation – A Growing Patient-Safety Issue. Dr John M. https://www.drjohnm.org/2017/03/fake-atrial-fibrillation-a-growing-patient-safety-issue/
