Authors: Lochana Mendis, Debjyoti Karmakar, Marimuthu Palaniswami, Fiona Brownfoot, Emerson Keenan
Categories: Article, Cardiotocography, deep learning, fetal compromise, fetal heart rate, time-series classification
Source: IEEE Journal of Translational Engineering in Health and Medicine
Continuous monitoring of fetal heart rate (FHR) and uterine contractions (UC), otherwise known as cardiotocography (CTG), is often used to assess the risk of fetal compromise during labor. However, interpreting CTG recordings visually is challenging for clinicians, given the complexity of CTG patterns, leading to poor sensitivity. Efforts to address this issue have focused on data-driven deep-learning methods to detect fetal compromise automatically. However, their progress is impeded by limited CTG training datasets and the absence of a standardized evaluation workflow, hindering algorithm comparisons. In this study, we use a private CTG dataset of 9,887 CTG recordings with pH measurements and 552 CTG recordings from the open-access CTU-UHB dataset to conduct a cross-database evaluation of six deep-learning models for fetal compromise detection. We explore the impact of input selection of FHR and UC signals, signal pre-processing, downsampling frequency, and the influence of removing intermediate pH samples from the training dataset. Our findings reveal that using only FHR and pre-processing FHR with artefact removal and interpolation provides a significant improvement to classification performance for some model architectures while excluding intermediate pH samples did not significantly improve performance for any model. From our comparison of the six models, ResNet exhibited the strongest fetal compromise classification performance across both databases at a downsampling rate of 1Hz. Finally, class activation maps from highly contributing signal regions in the ResNet model aligned with clinical knowledge of compromised FHR patterns, highlighting the model’s interpretability. These insights may serve as a standardized reference for developing and comparing future works in this domain. Clinical and Translational Impact: This study provides a standardized workflow for comparing deep-learning methods for CTG classification. Ensuring new methods show generalizability and interpretability will improve their robustness and applicability in clinical settings.
Cardiotocography (CTG) has become a routinely used technique for intrapartum care around the world. In the USA, 89% of singleton pregnancies are monitored using CTG during labor [1]. This technique simultaneously monitors the fetal heart rate (FHR) and maternal uterine contractions (UC) to identify fetuses that may show signs of hypoxia, which represents insufficient oxygen supply to the fetal tissues [2]. Hypoxia can have serious consequences for fetal development and well-being and may require interventions such as cesarean section or assisted vaginal delivery [2]. In practice, CTG recordings are evaluated visually by obstetricians based on clinical guidelines. However, interpreting CTG recordings remains a challenging task given the heterogeneous and complex nature of these waveforms. Many studies have shown substantial inter- and intra-observer variability in visual CTG interpretation [3], [4], [5]. Additionally, the high false positive rate of CTG monitoring is believed to be associated with the global increase in unnecessary operative deliveries [6]. Notably, the true positive rate (TPR) for detecting fetal compromise in clinical practice remains low, typically ranging between 31% and 48%, with a false positive rate (FPR) of 16%–21% [7], [8]. Therefore, clinicians may significantly benefit if they were supported by a decision support system relying on automatic CTG interpreting algorithms.
With the availability of digital CTG records, initial studies proposed computerized algorithms that mimic expert interpretations based on clinical guidelines by analyzing morphological features of FHR such as the baseline, variability, accelerations, and decelerations [9]. Thereafter, additional FHR features from time, frequency, and non-linear domains were also introduced [10]. Later, these features were used with classical machine-learning algorithms to improve the automated CTG interpretations to detect fetal compromise. Some of these algorithms evolved into computerized CTG decision support systems such as the Porto System [11], [12], Dawes-Redman System [13], INFANT [14], [15] and OxSys [16] which were evaluated in clinical practice. However, to our knowledge, these systems are yet to achieve significantly improved CTG interpretation performance to drive widespread clinical adoption.
In recent years, advancements in larger datasets and computational power have paved the way for modern automated methods, notably deep learning algorithms, which have exhibited superior performance in detecting fetal compromise compared to classical machine learning methods and clinical practice [7], [17], [18], [19], [20], [21]. This advancement was aided by the availability of the CTU-UHB dataset [22] hosted by PhysioNet [23], which is the largest publicly accessible intrapartum raw CTG dataset released to date. The current state-of-the-art model, a stacked multimodal convolutional neural network (MCNN) reported an area under the receiver operating characteristic curve (AUC) of 0.82 on the CTU-UHB after being trained on a private CTG dataset of over 35,000 records [7]. This model uses both the FHR and UC downsampled at 0.25Hz along with a quality vector as input to the model.
However, two major challenges hamper the progress of these modern First, the existing open-access CTU-UHB database is still relatively small, and other larger private datasets are inaccessible to the general public. Therefore, many studies have utilized the CTU-UHB dataset alone to propose novel methods [19], [20], [24]. However, results trained only on this dataset utilize a limited set of compromised cases and therefore may not necessarily perform well on unseen CTG cohorts. Hence, the limited availability of larger clinical datasets makes evaluating the generalizability of algorithms challenging as fetal compromise is rare and signals associated with adverse outcomes are heterogeneous.
Second, many existing approaches have not followed a standardized evaluation workflow for fetal compromise detection as highlighted in a recent review [25]. For instance, in clinical practice, FHR and UC are typically assessed together, as contractions can influence FHR through alterations in uteroplacental perfusion or umbilical cord compression [25]. However, certain computerized CTG classification studies have utilized FHR alone for fetal compromise detection [19], [20], [26], [27], while others have employed both FHR and UC [7], [18], [28]. To our knowledge, no study has comprehensively compared the use of FHR input alone against FHR and UC together for fetal compromise detection.
In addition to input selection, pre-processing also affects deep learning methods. Usually, before inputting CTG recordings into deep learning models, they undergo pre-processing steps including artefact removal and interpolation to address short durations of missing signal values. However, the impact of applying artefact removal and interpolation compared to using raw FHR recordings has not been investigated.
Furthermore, the CTG device records the raw signals at a rate of 4Hz. However, a fetal heart typically beats at a rate of less than 3 times per second (<180 bpm) [7]. This suggests that the raw 4Hz data may contain redundant data points. As a common practice in the literature, the raw FHR signal is often downsampled (i.e. smoothed) to a lower sampling rate to enhance computational efficiency and reduce memory requirements. However, the existing works have used different downsample frequencies for CTG analysis such as 4Hz [19], 1Hz [18] and 0.25Hz [7], [26], [29]. Despite this common downsampling practice, a comprehensive comparison of different downsampling frequencies in the context of detecting fetal compromise is lacking. Therefore, identifying the optimal sampling frequency for deep learning remains an open question [7], [25].
In general practice for fetal compromise detection, the entire dataset, or a subset of input samples are used in the deep learning training process. Some existing studies on deep-learning methods for fetal compromise detection particularly the MCNN have removed cases with intermediate pH values ( \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} pH <7.15) during training to prevent potential labeling errors [7], [26], [29]. However, the difference between these two approaches has not been studied to identify the best approach. As such, differences in the standard approach for input selection, pre-processing method, sampling frequency, and the selection of training data may greatly influence the performance of deep learning model architectures and limit the ability to make meaningful comparisons among them.
Finally, while quantitative performance is crucial for decision-support systems in medical practice, it is important to give attention to the broader concepts of interpretability and robustness. That is, the deep learning model should ideally produce outputs that are meaningful and understandable to humans while maintaining its generalizability for unseen data. Visualizations have been used as a powerful tool to improve the interpretability of these models as they can intuitively link the observed model outcomes with potential hypotheses about the observed data [30]. Such insights may help reduce the barrier of considering these algorithms as uninterpretable by clinicians and may increase the chances of adoption in real-world clinical practice [30], [31], [32], [33].
To address these challenges, this study performs a cross-database evaluation of deep learning methods for CTG classification as follows. Section II introduces a large novel intrapartum CTG dataset. Section III presents comparisons on a standardized workflow for comparing methods for fetal compromise detection. First, we compare the performance of using FHR to that of using FHR and UC combined. Secondly, we investigate whether using artefact removal and interpolation in pre-processing improves classification performance. Finally, we investigate the effect of removing intermediate cases from the training data and compare the effects of different down-sampling frequencies. Section IV presents a cross-database evaluation of six existing state-of-the-art deep-learning-based time series classification algorithms and findings on model interpretability. Section V provides insights into the processing workflow, model generalizability and interpretability, and the clinical implications derived from this study. The main contributions of this study can be summarized as •We provide a comprehensive analysis comparing the incorporation of UC signals with FHR signals, investigating the impact of pre-processing FHR signals, analyzing the effects of removing intermediate cases from training data, and exploring the optimal downsampling rate. These findings will assist in identifying the best-performing approach and establishing it as the standard for utilizing deep learning models in fetal compromise detection.•We implement and compare six different deep learning-based classification algorithms for detecting fetal compromise using a cross-database analysis. Utilizing two intrapartum CTG datasets provides robust evidence to identify the optimal model architecture that achieves superior performance and generalizability for clinical translation.•We demonstrate the interpretability of the optimal deep learning model in detecting fetal compromise, challenging the perception of deep learning as lacking transparency. This bridges the gap between advanced techniques and clinical understanding, increasing the likelihood of clinical adoption.
The MHW dataset introduced in this work comprises data from women who had CTG monitoring during labor at the Mercy Hospital for Women, Heidelberg, Melbourne, Australia from 2010 to 2021. Ethics approval for this dataset was granted by the Mercy Health Human Research Ethics Committee (Approval Number 2020-077). In this study, we selected a subset of 9887 CTG records from the MHW dataset consisting of singleton deliveries at gestation of more than 36 weeks, where CTG recording ended within 1 hour of birth and had a pH evaluation within 40 minutes of birth. We denote this subset of data as “MHW-pH” hereafter. The detailed data selection steps are shown in Fig. 1. FIGURE 1.Flowchart diagram depicting the data selection process for the MHW-pH CTG dataset.
The CTG records include both FHR and UC signals sampled at 4Hz. The FHR signals were measured using different a non-invasive Doppler ultrasound transducer, a direct electrocardiogram using a fetal scalp electrode, or a combination of both. When non-zero FHR values were available on multiple channels, precedence was given to the signals of the fetal scalp electrode when combining the channels to create a single FHR recording. An external tocodynamometer was used to acquire the UC signals. Table 1 provides an overview of participant characteristics for the 9,887 recordings with pH measurements from the MHW dataset.TABLE 1Participant Characteristics for the MHW-pH DatasetMeanMedianMinMaxMaternal age (years)31.96321858.1Parity0.480015Gravidity1.962117Gestational Age (weeks)39.4439.5036.1043.10Arterial cord pH7.257.266.007.52Apgar 1min8.059010Apgar 5min8.919110Neonatal birth weight (g)3383338012935360TypeNumberPercentageNeonatal SexFemale457546.3%Male531053.7%Birth TypeVaginal685269.3%Cesarean303530.7%Acquisition ModeUS441544.6%DECG220922.4%US+DECG326333.0%Records Not Available: Apgar 1min = 3, Apgar 5min = 5, Neonatal Sex = 2 US: Ultrasound, DECG: Direct Electrocardiogram
The open access CTU-UHB dataset [22] comprises a carefully selected set of 552 CTG recordings acquired during labor at the University Hospital in Brno, Czech Republic between 2010 to 2012. The Institutional Review Board of University Hospital Brno approved the data collection. All recordings include FHR and UC signals sampled at a rate of 4Hz, along with outcome measurements such as arterial cord pH (pH). These signals have a maximum duration of 90 minutes and conclude within 30 minutes before birth.
In practice, the arterial cord blood pH, which reflects the fetal blood’s acidity level, is used as the indicator for identifying potentially compromised babies. Hence, pH is the most widely used objective classification criterion used by studies to divide the data into fetal compromise and normal classes [25]. Predominantly in literature, births with pH <7.05 are considered to be compromised and normal otherwise. As shown in Table 2, there is a considerable imbalance between the population of the two classes with a small number of compromised cases, reflecting the clinical reality. Additionally, each dataset has a large difference in the proportion of compromised cases with the CTU-UHB dataset containing 7% compromised cases vs 1% in the MHW-pH dataset.TABLE 2Class Distribution for MHW-pH and CTU-UHB Datasets Using pH as the Classification CriterionDatasetTotal RecordsCompromisedNormalpH<7.057.05 \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} pH<7.15pH \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document}7.15CTU-UHB55240 (7%)65 (12%)447 (81%)MHW-pH9887111 (1%)762 (8%)9014 (91%)
To compare deep learning models and processing workflow, we selected the last 60 minutes of the CTG recordings from both datasets regardless of the stage of labor. The CTG data were used in their entirety without filtering based on signal loss to match clinical reality. The four experiments investigated in this study are as follows.
Firstly, we compare the classification performance of using FHR alone versus using both FHR and UC together to determine whether the inclusion of UC enhances classification performance. Secondly, since the FHR data are contaminated with noise consisting of signal artefacts and signal loss, we assess the impact of pre-processing on FHR data. Specifically, we compare the performance of deep learning models when using artefact removal and interpolation of FHR signals against the utilization of raw, unprocessed FHR signals. The artefact removal of FHR consists of removing outlier FHR values that exceed 200 beats per minute (bpm) and less than 50 bpm as the typical baseline FHR of a normal fetus varies between 110 bpm and 160 bpm. It also includes removing non-physiological abrupt changes in FHR where the difference between two adjacent FHR values exceeds 25 bpm. During interpolation, we linearly interpolated the periods of signal loss less than 15 s and coded the periods of signal loss greater than 15 s as zeros. The sampling rate of FHR was kept intact at 4Hz in this stage. We direct the reader to a recent review [25] for more details on FHR pre-processing steps.
Thirdly, we define an Intermediate class ( \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} pH <7.15, see Table. 2) only for the training data which acts as a middle ground between normal and compromised classes as identified in previous studies [7], [25], [26]. Here we examine the consequences of including or excluding intermediate cases from the training data while keeping the test data intact. In the fourth experiment, we explore the effects of downsampling the FHR recordings at various frequencies and analyze their performance to determine the optimal sampling frequency. Finally, we select the model architecture that shows the best generalization capability. An overview of this evaluation and comparison process is illustrated in Figure 2. FIGURE 2.Comparative decision-making process for identifying the optimal deep learning evaluation procedure for fetal compromise detection. The maximum geometric mean is used as the metric to select the optimal approach in each stage.
In this study, we evaluate six different deep-learning algorithms adapted from the literature for time series classification. These can be broadly categorized as convolutional neural networks, recurrent neural networks, and a combination of both. The convolutional networks can be further subdivided into standard convolutional architectures, fully convolutional architectures, and ResNet-based architectures. An overview of the model architectures used in this study is shown in Fig. 3. FIGURE 3.Overview of the six deep-learning architectures (CNN [17], B-CNN [26], FCN [37], LSTM-FCN [36], LSTM [17], and ResNet [37]) for time series classification used in the comparative analysis. Our implementations closely follow the architectural designs proposed in the original publications and their respective reference implementations. The last fully connected layer (FC) is the output layer of each model consisting of one neuron with a sigmoid activation function. The global average pooling layer is denoted as GAP.
The standard feed-forward 12-layer CNN and the LSTM follow the model architectures proposed by Petrozziello et al. for CTG classification [17]. The 12-layer CNN is the backbone architecture used in the MCNN model. The B-CNN with the global average pooling (GAP) layer was initially proposed in our previous studies [26], [29] for fetal compromise detection. This model uses three parallel branches of convolutional layers of varying kernel sizes which then get concatenated in the filter dimension and passed through two convolutional layers in series.
In addition to the models used in existing works for fetal compromise detection, we evaluated three widely used deep learning models for time series classification validated on multiple time series datasets of different domains from the UCR time series archive [34], [35], [36]. The fully convolutional neural network (FCN) was first proposed by Wang et al. [37] for classifying univariate time series. The main characteristic of this architecture is the replacement of the traditional final fully connected layer of CNN with a GAP layer which drastically reduces the number of parameters. The same author also proposed the Residual Network (ResNet) with a GAP layer following the convolutional layers for time series classification. Using the shortcut to link the output of the residual block to its input enabled the flow of gradient reducing the vanishing gradient effect for deep networks [38]. The LSTM-FCN model was proposed by Karim et al. [36] for time series classification by concatenating the FCN with a dimension-shuffled LSTM. This study adopts the recommended hyper-parameter settings from the corresponding reference studies for each model implementation.
To investigate model interpretability, we employed a technique called the class activation map (CAM) to identify the class-specific regions in the data [39]. The class activation map is a weighted linear combination that takes into account the presence of visual patterns across various spatial locations. We followed the implementation of CAMs for time series classification in [37] and adapted it to suit time series binary classification to visualize the contributing regions for the compromised class. Using the CAMs we investigated the regions of the FHR recording which contributed most highly to the model prediction.
We validated each experiment through cross-database evaluation, which assessed performance between the CTU-UHB and MHW-pH datasets. We trained the models on the CTU-UHB dataset and tested them on the unseen MHW-pH dataset, and vice versa, where we trained on the MHW-pH dataset and tested on the unseen CTU-UHB dataset. It is important to note that we ensured the testing data remained entirely unseen by the models during the training process. Prior to training, the FHR and UC signals in the datasets are individually scaled using MinMax Scaler normalization before training. To address the highly unbalanced class distribution in the datasets, we utilized a weighted binary cross-entropy function as the loss function. The weighting was determined based on the inverse class frequency. This adjustment penalizes misclassifications of the minority class more heavily. During the training process, each dataset was randomly divided into 80% for training and 20% for validation while maintaining the original class distribution. The validation samples for each dataset were only used to determine when to halt the training process. All models are trained with a batch size of 16 for a maximum of 400 epochs, utilizing the Adam optimizer with an initial learning rate reduced by half each time the training loss did not show an improvement for a period of 10 epochs, until a final learning rate of 0.0001. The initial learning rate of 0.001 was used as the default for each model unless stated otherwise in their corresponding original implementations. In addition, we employed Early Stopping when the validation loss did not show a reduction for 100 epochs. After the training, the model at the epoch with the lowest validation loss was selected as the best-trained model for cross-database evaluation. The experiments were carried out on an Intel(R) Xeon(R) Gold 6326 CPU at 2.90GHz with Nvidia A100 GPU and 16 GB of RAM. The models were implemented using the TensorFlow 2.0 framework.
For each cross-database evaluation, we create a receiver operating characteristic curve by plotting the true positive rate against the false positive rate across classification thresholds. The area under the curve (AUC) is then calculated, measuring the model’s ability to correctly make predictions across thresholds. Unlike metrics heavily influenced by class distribution, the AUC is less sensitive to class imbalance, making it a robust measure for datasets with disproportionate positive and negative cases. In this study, we derived an additional metric from the cross-database evaluation by calculating the geometric mean of the AUC values from the two datasets, henceforth referred to as GM AUC. Each experiment was repeated five times, and the average GM AUC values were reported to ensure robustness. To identify the optimal approach at each stage, we selected the results that achieved the highest GM AUC. The Mann-Whitney U test was utilized to assess if the difference in GM AUC for each experiment was statistically significant. The SciPy library version 1.7.3 was used for all comparisons, where a p-value <0.05 was considered significant.
We assessed the effectiveness of using the raw FHR as a single time series feature sampled at 4Hz, as opposed to using both the raw FHR and raw UC sampled at 4Hz. As depicted in Fig. 4, the ResNet model demonstrated the highest GM AUC of 0.71 when using only FHR data. In contrast, the ResNet model achieved a GM AUC of 0.67 when both FHR and UC data were used together. All six models trained solely with FHR data showed improved performance compared to those trained with both FHR and UC data, however, only the performance difference for the B-CNN, FCN, LSTM-FCN, and ResNet models were statistically significant. Therefore, we selected FHR as the only feature for the subsequent analysis based on the highest GM AUC performance. For results presenting FHR+UC in using the remaining workflow steps, please see Supplementary Material 1. FIGURE 4.Comparison of the geometric mean AUC of six deep learning models when using FHR as the sole feature versus using both FHR and UC. Statistical significance is indicated by * (p <0.05), ** (p <0.01), and *** (p <0.001).
We evaluated the six models using both the raw FHR signal and pre-processed FHR signal to determine whether the removal of artefacts and interpolation contribute to improved FHR classification performance. As illustrated in Fig. 5, the ResNet model achieved the highest GM AUC of 0.73 when applying artefact removal and interpolation to the FHR data, compared to 0.71 with raw FHR. In general, pre-processing the FHR data with artefact removal and interpolation showed better GM AUC values across all models, except for the LSTM model. However, only the difference using LSTM-FCN was statistically significant. Following our evaluation criteria, we selected to pre-process the FHR data with artefact removal and interpolation for the subsequent analysis. FIGURE 5.Comparison of geometric mean AUC for six deep learning models using raw FHR signal versus pre-processed FHR (involving artefact removal and interpolation of missing signals lasting less than 15 seconds). Statistical significance indicated by *(p <0.05), **(p <0.01), and ***(p <0.001).
The comparison of the classification performance of deep learning models with the inclusion and exclusion of intermediate cases in the training data is shown in Fig. 6. As can be seen, CNN, LSTM, B-CNN, and LSTM-FCN models showed higher performance when intermediate cases were excluded from the training data. In contrast, FCN and ResNet models exhibited no change. The strongest GM AUC of 0.73 was achieved by the ResNet model; however, none of the differences were statistically significant. As the majority of models showed higher performance when intermediate cases were removed, we selected removing intermediate cases from the training data for the subsequent analysis. FIGURE 6.Comparison of geometric mean AUC of six deep learning models with inclusion versus exclusion of intermediate cases defined by deliveries with pH >7.05 and pH \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} in the training data. None of the differences in geometric mean AUC were statistically significant.
We evaluated the GM AUC performance of six deep learning models at sampling frequencies of 4Hz, 2Hz, 1Hz, 0.5Hz, and 0.25Hz, to determine the most suitable sampling rate. Moreover, we investigated the inference time required to make predictions on a single FHR recording at each of these sampling frequencies as illustrated in Fig. 7. The inference time exhibits a declining trend across all models as the sampling frequency is reduced. This pattern is consistent with the reduction in input data size as the sampling rate decreases. FIGURE 7.Comparison of geometric mean AUC () of the six deep-learning models at downsample frequencies 0.25Hz, 0.5Hz, 1Hz, 2Hz and 4Hz. The secondary y-axis shows the variation of inference time () for a single 60-minute FHR recording at the respective frequencies.
However, the GM AUC did not follow a uniform trend across all models. The GM AUC performance exhibited an increasing trend as the sampling rate decreased from 4Hz to 0.25Hz for the CNN and LSTM models. Whereas for the B-CNN, FCN, LSTM-FCN, and ResNet models, the performance remained relatively the same or marginally increased as the sampling rate was reduced from 4Hz to 1Hz and thereafter showed a declining trend. Interestingly, the lowest performance for the B-CNN, FCN, LSTM-FCN, and ResNet models occurred at 0.25Hz, while the CNN and LSTM models achieved their best performance at the same sampling frequency. Nevertheless, the ResNet model achieved the best GM AUC of 0.73 with the lowest inference time at a sampling frequency of 1Hz. The statistical comparison of GM AUC performance of models at different sampling rates is shown in Fig. 8. For statistical comparisons across sampling rates for each model, we refer the reader to Supplementary Material 1. FIGURE 8.Statistical significance comparison of geometric mean AUC of the six deep learning models at downsampling frequencies 0.25Hz, 0.5Hz, 1Hz, 2Hz and 4Hz, indicated by * (p <0.05), ** (p <0.01), *** (p <0.001), and ns = non-significant (p \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document}).
We evaluated the model that achieves the best performance on both datasets using the selected optimal processing workflow consisting of pre-processing FHR signals, training with intermediate cases removed, and downsampling at 1Hz. The cross-database AUC performance of all models, along with their corresponding GM AUCs, are detailed in Table 3. The ResNet model achieved the highest mean AUC of 0.81 when trained on MHW-pH and tested on CTU-UHB and a mean AUC of 0.65 when trained on CTU-UHB and tested on MHW-pH. The ResNet architecture at 1Hz demonstrated the highest overall GM AUC performance of 0.73, as well as the highest individual cross-database AUCs making it the model with the best generalizability. As shown in Fig. 8, the GM AUC performance of the ResNet model was significantly different compared to all other models except for the LSTM-FCN model.TABLE 3Cross Database AUC Performance of Models at 1HzDL ModelsTrain: MHW-pHTrain: CTU-UHBGeometric Mean (AUC)Test: CTU-UHB (AUC)Test: MHW-pH (AUC)CNN \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document}LSTM \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document}B-CNN \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document}FCN \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document}LSTM-FCN \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document}ResNet \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document}
To assess model interpretability, we visualized the contributing regions of compromised cases in each test dataset using the ResNet model. In clinical practice, abnormal FHR decelerations are carefully inspected as they are likely to indicate the presence of fetal compromise. According to FIGO guidelines, deceleration patterns of main interest are variable decelerations (V-shaped), late decelerations (U-shaped), and prolonged decelerations. Both late and prolonged decelerations have shown some association with fetal hypoxia/acidosis whereas variable decelerations are not typically associated with fetal hypoxia/acidosis [40]. We identified that regions with high intensity of the activation map of compromised cases aligned with deceleration patterns associated with fetal compromise. Particularly the prolonged and late decelerations are identified as the most significant differentiators by the model. Most importantly, these contributing regions remained consistent in cross-database evaluation. Six example 15-minute segments of the FHR CAMs of compromised cases showing deceleration patterns in each test dataset of cross-database evaluation are shown in Fig. 9. For additional CAMs, we refer the reader to Supplementary Material 2. FIGURE 9.Exemplary 15-minute segments of compromised FHR recordings of both test CTG datasets showing activation regions consistent with medical practice (First Row: three 15-minute segments tested from CTU-UHB, Second Row: three 15-minute segments tested from MHW-pH). Prolonged decelerations which are frequently associated with acute hypoxia/acidosis and late decelerations (U-shaped) which are indicative of hypoxemia show high-intensity activations by the model. The variable decelerations which are seldom associated with fetal hypoxia/acidosis show low-intensity regions by the model. Additional 60-minute recordings of CAMs of FHR along with UC are provided in Supplementary Material 2.
The comparative analysis of six deep learning models, as presented in Fig. 4, reveals a higher GM AUC performance when utilizing raw FHR values alone, as opposed to a combination of raw FHR and UC. The potential reason behind this could be the reliability of the majority of UC signals in these datasets, which are captured using an external tocodynamometer (TOCO). In fact, all UC signals from the MHW-pH dataset are derived from an external TOCO.
According to Bakker et al. [41], only 2% of UC recordings captured using an external TOCO were deemed adequate (recognizable and reliable) during the first stage and approximately 30% were adequate quality for the second stage of labor. In clinical practice, the quality of UC is often checked before analysis and if it were poor, only the FHR signal would be analyzed [7]. Similarly, our findings suggest that computerized methods may yield better results by focusing solely on FHR when the reliability of UC signals is not adequate. We recommend future studies be undertaken to validate this finding using other UC acquisition methods such as the intrauterine pressure catheter and electrohysterogram [25].
Following the selection of FHR data over FHR and UC data, our subsequent analysis compared pre-processed FHR signals and raw FHR signals. Our findings which demonstrated generally improved performance with the application of pre-processing align with established practices in deep learning [7], [19]. In particular, the inclusion of artefact removal serves to eliminate unwanted or misleading signals that can distort underlying information in the training data. This step ensures that the model learns from clean and accurate data, preventing the interference of artefacts that may stem from factors such as fetal/maternal movements, transducer displacement, autocorrelation-induced half or double counting in Doppler ultrasound-based CTG monitoring, and noise introduced by vaginal examinations and maternal pushing during CTG monitoring using direct fetal electrocardiogram [42], [43].
In addition to artefacts, signal loss can pose a challenge. Typically, for FHR from Doppler ultrasound-based CTG, median signal loss ranges from 5 to 8% during the first stage of labor and 9 to 20% during the second stage. In contrast, the direct fetal scalp electrode exhibits lower median signal loss, with values of approximately 0.8 to 1% in the first stage of labor and 3 to 4% in the second stage [42], [44], [45]. These gaps in data continuity can disrupt the performance of the model. Interpolation, a technique for estimating missing values based on neighbouring data points, plays a crucial role in maintaining data continuity and enabling the model to capture temporal patterns effectively. Therefore, our experimental results demonstrate that raw FHR data benefits from pre-processing, including artefact removal and interpolation, before integration with deep learning algorithms.
The intermediate samples which consist of deliveries with pH values ranging from 7.05 to 7.15 is a diverse group that poses a detection challenge as it may encompass poor outcome cases that develop during labor, as well as fetuses that have experienced stresses that may not necessarily be abnormal [7]. Consequently, incorporating these cases into the training data could potentially impede the convergence of data-driven methods such as deep learning, given that the labels assigned to these cases may be erroneous. This could potentially be the reason for the slightly increased, but non-significant difference in fetal compromise classification performance shown by removing intermediate cases in our investigation using two independent CTG datasets. However, the fact that the FCN and ResNet models showed no improvement after removing intermediate cases implies that the inclusion of intermediate cases may not affect all model architectures in the same manner.
In general, the findings indicate a similar or slightly improved trend in GM AUC performance across models when the FHR sampling frequency is decreased from 4Hz to 1Hz, with decreasing inference time. This suggests that lower sampling frequency can be used for the implementation of these algorithms on resource-limited devices without compromising performance.
When downsampled from 1Hz to 0.25Hz, all models except for LSTM and CNN, exhibited reduced GM AUC performance. Downsampling lowers temporal resolution, smoothing the data which may eliminate finer-grained details. This smoothing can enhance pattern visibility over longer time scales, explaining improved performance for the simpler CNN and LSTM models. However, excessive downsampling may obscure crucial information, leading to performance drops in the B-CNN, FCN, and ResNet models. Our findings suggest that 1Hz represents a reasonable balance between performance and inference time, however this should be confirmed in future work.
The generalization ability of deep-learning models is critical for consistent performance in clinical settings. Prior research, such as by Vabalas et al. [46], highlights that performance reported on small datasets often yields overly optimistic results. To address this, our study evaluates the generalization capabilities of six deep-learning models using cross-database testing on the MHW-pH and CTU-UHB CTG datasets. By employing an external and unseen testing dataset, we ensure unbiased performance estimates.
Interestingly, the performance on the CTU-UHB dataset consistently exceeded that on the MHW-pH dataset for most models. This discrepancy may be attributed to the specifically curated nature of the CTU-UHB dataset, which includes a higher proportion of compromised cases (7%) compared to the prevalence observed in clinical practice (0.6%–3.5%) [17], [22], [47].
As can be seen from Table 3, the ResNet model achieved the highest GM AUC of 0.73, demonstrating superior generalizability across both datasets. It was closely followed by LSTM-FCN, FCN, and B-CNN, all of which significantly outperformed CNN and LSTM. Notably, these top-performing models incorporate a GAP layer at the classification stage, which minimizes overfitting and likely contributes to their enhanced generalized performance. Additionally, the ResNet’s skip connections facilitating effective gradient flow for deeper layers may have resulted in its increased performance. The strong generalization capability demonstrated by ResNet in FHR time series classification aligns with findings from other studies on time series data [35], [37].
The existing literature consists of a limited number of studies reporting the performance of deep-learning models trained on private CTG datasets and subsequently evaluated on the CTU-UHB dataset, as summarized in Table 4. Petrozziello et al. [17] reported AUCs of 0.82 and 0.81 for CNN and LSTM, respectively, when trained on a private Oxford dataset of over 35,000 records and tested on CTU-UHB. In a separate study [7], the same authors employed two different deep learning model architectures, MCNN, and a stacked MCNN, achieving AUCs of 0.81 and 0.82, respectively. However, the authors state that the MCNN model is superior to the stacked MCNN due to the performance difference on their internal testing set. In another study, Ogasawara et al. [18] trained a novel deep-learning model on their private CTG dataset comprising 2,116 records from Keio University Hospital and reported an AUC of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} on a subset of CTU-UHB with signal loss less than 16%. All these studies used both FHR and UC in their analysis. A recent study by Mccoy et al. [47], conducted a comparative study of six deep learning models trained on a private dataset of 10,182 recordings of FHR recordings with less than 30% signal loss from the University of Pennsylvania. Their best model achieved AUCs of 0.76 and 0.72 on the CTU-UHB dataset, with and without transfer learning, respectively.TABLE 4Performance comparison of related studies externally tested on the CTU-UHBStudyDL ModelInputOutcomeTraining DatabaseTesting DatabaseAUCPetrozziello et al. [17]CNNFHR+UCpH<7.05Last 60 min Oxford at 0.25HzCTU-UHB0.82Petrozziello et al. [7]Stacked MCNNFHR+UC+QualitypH<7.05Last 60 min Oxford at 0.25HzCTU-UHB0.82Petrozziello et al. [7]MCNNFHR+UC+QualitypH<7.05Last 60 min Oxford at 0.25HzCTU-UHB0.81Ogasawara et al. [18]CNNFHR+UCpH<7.2 or Apgar \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document}Last 30 min Keio data at 1HzCTU-UHB(subset of 78 cases) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document}Mccoy et al. [47]InceptionTimeFHRpH<7.05Last 60 min Pennsylvania at 0.25HzCTU-UHB0.72This WorkResNetFHRpH<7.05Last 60 min MHW at 1HzCTU-UHB \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document}
In our work, the top-performing ResNet model, utilizing only FHR data, achieved an AUC of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document} on the CTU-UHB dataset, with performance slightly below that of the state-of-the-art MCNN. The superior performance of the MCNN could be attributed to its use of the substantially larger Oxford training dataset. Another possible explanation is that the MCNN models in their study may have learned different features, as 76% of UC in the Oxford dataset were replaced with zeros due to them being assessed as low-quality [7].
Despite being trained on the MHW-pH dataset, which is about 70% smaller than the Oxford dataset, the ResNet model using only FHR as input demonstrated strong performance compared to existing works. This emphasizes the importance of FHR and the potentially limited contribution of UC for computerized fetal compromise detection. However, the varied input data selection criteria based on signal loss thresholds, input feature choices, and definition of outcomes complicate direct model comparisons. This emphasizes the critical need for standardized processing workflows in CTG analysis, as advocated in this study.
In clinical contexts, healthcare professionals are sometimes reluctant to embrace machine learning, particularly deep learning algorithms. This hesitancy is rooted in the perception that these data-driven algorithms are viewed as complex and challenging to understand, akin to opaque systems [31], [32]. To this end, the inclusion of a GAP layer allows the use of the CAM technique to visualize the contributing regions in the raw data for specific classes [39]. The applicability of using CAMs for visualizing different time series classification datasets has been demonstrated by previous works [35], [37]. Recent work on electrocardiography classification used activation maps to visualize relevance patterns of two classes and show how they aligned with medical knowledge [32]. Similarly, our work is the first to use CAMs to visualize FHR. This provides a way to explain how these models operate during FHR classification and allows clinicians to align indicative features with their medical knowledge.
According to FIGO guidelines, variable decelerations typically do not indicate a significant level of fetal hypoxia/acidosis unless there is reduced variability within the deceleration. However, late decelerations and prolonged decelerations lasting more than 3 minutes suggest the presence of hypoxemia. Decelerations exceeding 5 minutes with FHR less than 80 bpm and reduced variability are often linked to acute fetal hypoxia/acidosis and require urgent intervention [40]. In Fig. 9, it is evident that the CAMs show high-intensity activation in the regions of late and prolonged decelerations compared to variable decelerations. This is especially noticeable in FHR recordings with longer periods of prolonged decelerations. Thus, this demonstrates that the observed patterns in the interpretability outputs may align with standard clinical knowledge for CTG analysis. However, conducting a more comprehensive analysis of the activation maps to determine their relationship with existing clinical knowledge is recommended for validating our findings in future studies.
In clinical practice, CTG is widely used for antepartum and intrapartum fetal monitoring. This study focuses on improving fetal compromise detection during the intrapartum period. To this end, we constructed the private MHW-pH intrapartum CTG dataset, encompassing a heterogeneous population over a period of 11 years. Alongside the open-access CTU-UHB dataset, our cross-database evaluation demonstrated the ResNet model’s generalization to both cohorts. Particularly, it achieved an AUC of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document}, with a TPR of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} \end{document}% at a 20% FPR on the CTU-UHB dataset. This performance surpasses current clinical practice (TPR 31%-48% at FPR 16%-21%) and enables 60-minute recordings to be analyzed in milliseconds, underscoring its clinical potential.
Particularly, integrating visual model predictions via CAMs alongside other clinical inputs has the potential to enhance pattern recognition for clinicians and improve the detection of fetal compromise. Visualizing these CAMs of FHR with UC, even when UC is not used as an input feature, could further boost clinician utility (see Supplementary Material 2).
Additionally, a recent economic analysis reported that a data-driven computerized CTG decision support tool such as our work could provide both cost savings and improvement in health outcomes [48]. While further optimization and prospective patient-level studies are needed to formally evaluate the ResNet model’s effectiveness in clinical practice, its potential to improve intrapartum care is substantial. By enhancing accuracy, consistency, and interpretability, this model represents a meaningful advancement in automated CTG analysis.
This study has several limitations that should be considered. First, while cross-database evaluations were conducted, the datasets were sourced exclusively from Australia and the Czech Republic, limiting generalizability to populations with differing demographics and clinical practices. Validation on more diverse datasets is needed to ensure broader applicability. Additionally, the reliance on FHR signals due to unreliable UC signal quality should be investigated further. Particularly, our findings raise the question of whether UC signals contribute meaningful information for assessing fetal status or if their utility is significantly impacted by their acquisition method.
Another limitation is the overall moderate AUC values reported, which, although consistent with prior research, highlight the need for model optimization and the inclusion of additional clinical features. While CAMs were used to enhance interpretability, they only partially address the challenge of model transparency. Advanced interpretability techniques and exploration of the correspondence between suspicious FHR regions and high-quality UC signals could further improve model understanding and clinical utility. Lastly, this study focused solely on deep learning architectures using CTG signals, without investigating hybrid approaches that integrate deep learning with feature engineering and electronic health records. These limitations emphasize the importance of ongoing research to comprehensively study the workflows utilized for computerized CTG analysis methods.
The automated assessment of cardiotocography recordings addresses an unmet clinical need, enhancing the accurate detection of fetal compromise during labor. However, advancement in this field is currently impeded by the limited CTG databases available for training deep learning models. Additionally, the absence of a standardized evaluation workflow hinders the comparison of optimal model architectures. To address these challenges, we conducted a comparative analysis of various strategies employed in the workflow for training deep learning algorithms for fetal compromise detection and assessed six commonly used models for time series classification. This involved a cross-database evaluation between the CTU-UHB intrapartum CTG dataset and a new private MHW-pH CTG dataset consisting of a total of over 10,000 unique CTG recordings. Our findings indicate that utilizing only the FHR signals and implementing artefact removal and interpolation in pre-processing improves model classification performance. Overall, we observed that the ResNet architecture demonstrates the highest performance and generalizability while also facilitating clinical interpretability for healthcare professionals. These insights may serve as a foundational reference for future works in this domain.
Lochana Mendis and Debjyoti Karmakar have no competing interests to declare. Emerson Keenan, Marimuthu Palaniswami, and Fiona Brownfoot are co-founders and shareholders of Kali Healthcare Pty Ltd.