Authors: Suwan Dai, Yingchun Ren, Lingyan Chen, Min Wu, Rong Wang, Qinghe Zhou
Categories: Research, Catheter-related bladder discomfort, General anesthesia, Machine learning, Nomogram, Prediction model
Source: BMC Anesthesiology
Catheter-related bladder discomfort (CRBD) commonly occurs in patients who have indwelling urinary catheters while under general anesthesia. And moderate-to-severe CRBD can lead to significant adverse events and negatively impact patient health outcomes. However, current screening studies for patients experiencing moderate-to-severe CRBD after waking from general anesthesia are insufficient. Constructing predictive models with higher accuracy using multiple machine learning techniques for early identification of patients at risk of experiencing moderate-to-severe CRBD during general anesthesia resuscitation.
Eight hundred forty-six patients with indwelling urinary catheters who were resuscitated in a post-anesthesia care unit (PACU). Trained researchers used the CRBD 4-level assessment method to evaluate the severity of a patient’s CRBD. They then inputted 24 predictors into six different machine learning algorithms. The performance of the models was evaluated using metrics like the area under the curve (AUC).
The AUCs of the six models ranged from 0.82 to 0.89. Among them, the RF model displayed the highest predictive ability, with an AUC of 0.89 (95%CI: 0.87, 0.91). Additionally, it achieved an accuracy of 0.93 (95%CI: 0.91, 0.95), 0.80 sensitivity, 0.98 specificity, 0.94 positive predictive value (PPV), 0.92 negative predictive value (NPV), 0.87 F1 score, and 0.07 Brier score. The logistic regression (LR) model has achieved good results (AUC:0.87) and converted into a nomogram.
The study has successfully developed a machine learning prediction model that exhibits excellent predictive capabilities in identifying patients who may develop moderate-to-severe CRBD after undergoing general anesthesia. Furthermore, the study also presents a nomogram, which serves as a valuable tool for clinical healthcare professionals, enabling them to intervene at an early stage for better patient outcomes.
The online version contains supplementary material available at 10.1186/s12871-024-02720-5.
Keywords: Catheter-related bladder discomfort, General anesthesia, Machine learning, Nomogram, Prediction model
According to the literature, approximately 15%-25% of hospitalized patients undergo short-term indwelling catheterization [1] for various reasons such as assessing urine output, relieving urinary retention, or performing urological procedures [2]. However, the insertion of a catheter can sometimes result in the involuntary contraction of the muscles in the urethra, leading to a range of clinical symptoms known as Catheter-related bladder discomfort (CRBD) [3]. CRBD can be categorized into four different grades based on the severity of the condition [4]. Grade 0 indicates the complete absence of CRBD symptoms. Grade 1 represents mild discomfort experienced by the patient. For grades 2 and 3, the discomfort is classified as moderate and severe, respectively. The use of urinary catheters is an essential intervention for patients undergoing surgery under general anaesthesia, and this common and painful complication is a recognised risk factor during general anaesthetic awakening [5].
Studies have indicated that the occurrence of CRBD varies between 47% and 90% [3, 6], with the prevalence of moderate to severe CRBD ranging from 27% to 44% [7, 8]. Moderate CRBD is characterized by a feeling of fullness in the lower abdomen or a burning sensation in the urethra [9]. These symptoms significantly impact the patient’s comfort and diminish their perception of the medical care provided. Severe CRBD is commonly associated with aggressive behavioral reactions, including agitation [6] and delirium [5]. These responses can result in significant safety incidents like postoperative bleeding, falling out of bed, and mechanical injury to the urethra [10]. These events not only intensify patient discomfort and lengthen hospital stays [11], but also place additional burden on healthcare professionals [6]. Hence, CRBD following general anesthesia, with a focus on the widely discussed concept of enhanced recovery after surgery (ERAS) [12], deserves significant attention.
Due to the considerable harm caused by moderate-to-severe CRBD, it is essential to urgently tackle effective prevention and reduction of its occurrence within a clinical setting.
The screening and identification of CRBD play a pivotal role in both preventing and treating this condition. A comprehensive review of several prior articles [8, 9, 13, 14] examined the risk factors associated with CRBD and grouped them into four major patient-related factors, surgical factors, anesthesia-related factors, and indwelling catheter-related factors [10]. However, these studies failed to construct predictive models that could effectively identify high-risk groups. In another study by Liang et al. [15], a multivariate logistic regression model was developed. They also created a nomogram to visualize the results. The model achieved an AUC of 0.78, which indicates a moderate level of predictive effectiveness. It is important to note that this model only predicts the likelihood of CRBD occurrence, and not its severity. The study’s limitations include a small sample size and a limited number of risk factors considered. Additionally, the logistic regression algorithm assumes that the variables have independent effects on the outcome, which sometimes may reduce predictive performance for different types of data and complex data [16, 17], so further verification of its effectiveness is still needed. Due to the intricate and diversity of diseases, a number of significant risk factors often go unnoticed. As a result, there is a clear demand within the medical field for a systematic, streamlined, and accurate statistical approach to address these concerns.
Machine learning (ML) is a sophisticated technique for analyzing data that harnesses the capabilities of computers to uncover patterns and insights from vast and complex datasets [18]. It involves automatically building analytical models, improving predictive capabilities, and maximizing the accuracy of predictions [19]. This method utilizes the expertise of algorithms to process and interpret multivariate data effectively [18, 19]. By implementing predictive models in this manner, it becomes possible to proactively identify high-risk groups susceptible to associated illnesses. This enables the implementation of precise interventions, aimed at preventing and minimizing complications. Ultimately, this approach establishes a novel and contemporary management model. Currently, machine learning has demonstrated impressive results in various clinical areas. A meta-analysis summarised studies related to machine learning prediction of hypertension, which showed that AUC ranged between 0.77 and 1.00 [20]. Additionally, Heo et al. [21] conducted a study using machine learning techniques to predict the prognosis of ischemic stroke, achieving a high degree of accuracy with an AUC of 0.89. Such advancements in machine learning not only provide improved diagnostic capabilities but also offer potential for disease prevention [22]. However, it is worth noting that no relevant machine learning studies have been identified thus far regarding the clinical issue of CRBD.
Therefore, the objective of this study was to tackle the common clinical issue of moderate-to-severe CRBD after general anesthesia during the resuscitation period. This was achieved by collecting data in a prospective manner, developing a prediction model with enhanced accuracy using various machine learning techniques, and creating a nomogram for clinical visualization. The ultimate goal was to enable early identification of patients at risk for moderate-to-severe CRBD, thereby preventing and reducing the incidence of this condition. Additionally, the study aimed to facilitate the speedy recovery of patients, as well as enhance their comfort and satisfaction levels.
This study was an observational prospective cohort conducted in a municipal tertiary hospital in Zhejiang Province, China, which was approved by the hospital's medical ethics committee (No. 2023-LY-375).
Patients undergoing post-anaesthetic resuscitation in the post-anesthesia care unit (PACU) between August 2023 and December 2023 were included in this study. Inclusion age ≥18 years; general anaesthesia with indwelling urinary catheter; American Society of Anaesthesiologists (ASA) classification I-III; clear, able to answer questions and express themselves normally; voluntary participation in this study and signing of informed consent; exclusion history of overactive bladder, neurogenic bladder, preoperative urinary tract infection; unplanned admission to the ICU after surgery; removal of the urinary catheter before awakening from general anaesthesia; incomplete relevant information.
CRBD can be categorized into four grades based on its severity, as originally defined by Agarwal [4]. Grade 0 refers to the absence of CRBD, with no urethral or bladder discomfort. Grade 1 indicates mild discomfort that is only reported when specifically asked. Grade 2 represents moderate discomfort, where the patient experiences a sensation of bloating in the lower abdomen or a burning sensation in the urethra without exhibiting any behavioral response. Grade 3 signifies severe discomfort, characterized by the patient expressing a strong urge to urinate, experiencing unbearable pain in the lower abdomen, and exhibiting strong verbal and behavioral reactions, such as fidgeting and attempting to remove the catheter. Trained investigators utilized a grading scale to assess patients in the PACU who met the requirements for indwelling urinary catheters. Because grade 2-3 patients exhibit behavioral symptoms and require clinical intervention, screening is more necessary. According to the classification methods of existing researches [23–25], we divided them into two Grades 0 and 1 indicated no moderate-to-severe CRBD, while grades 2 and 3 indicated the presence of moderate-to-severe CRBD.
The researchers collaborated with the nurses in each department to explain the purpose and importance of the study to patients who were about to undergo general anesthesia and were expected to have indwelling urinary catheters after surgery. The patients were informed about the study and their consent was obtained. To collect the results of the CRBD assessment and related factors, a homemade data collection form was used. Before initiating the formal study, a brief pre-test was conducted to become familiar with and improve the data collection process. To prevent any potential bias, patient evaluations were carried out simultaneously by two investigators, and a third investigator was consulted for any conflicting judgments. To ensure the quality of the study, all collected data were carefully checked and entered by two individuals. Additionally, after data entry, a logical check was performed, where 10% of the data was randomly selected for review to guarantee the accuracy of data entry.
The investigators conducted a thorough search of various databases using specific keywords and subject terms to find relevant predictors based on existing literature. These databases include Pubmed, Cochrane Library, Embase, Web of Science, CKNI, and Wanfang databases. The search terms used were ("risk factor" or "influencing factor" or "predictor") and (catheter-related bladder discomfort or "catheter-related bladder irritation sign" or "CRBD"). Following that, professionals specializing in clinical anaesthesia, urology, and roles related to anaesthesia care were chosen to engage in discussions and adjusted the CRBD predictors obtained from the literature review. These adjustments were made based on factors such as clinical significance and well-established knowledge. In the end, a total of 24 predictors were identified and determined to be relevant. These included six variables of patients' own gender, age, BMI, comorbidity, calcium ions (the blood biochemistry analysis results on the day before surgery), magnesium ions (the blood biochemistry analysis results on the day before surgery). Eight anaesthetic ASA classification, nerve block (intercostal nerve block), postoperative analgesics, anticholinergics, propofol, dezocine, dexamethasone, and dexmedetomidine. Five surgical surgical duration, type of surgery, pneumoperitoneum, hypothermia, and intraoperative fluid infusion volume. Five urinary catheter timing of indwelling urinary catheters, type of urinary catheters, balloon volume, lubricant type, history of indwelling urinary catheters.
The study population was divided into a training cohort (80%) and an internal validation cohort (20%) randomly by the computer. Various algorithms such as Logistic Regression (LR), Random Forest (RF), Decision Tree (DT), Support Vector Machine (SVM), Naive Bayes (NB), Gradient Boosting Decision Tree (GBDT), Categorical Boosting (CatBoost), Light Gradient Boosting Machine (LightGBM), and eXtreme Gradient Boosting (XGBoost) were used to build individual risk class prediction models. Based on evaluation indicators such as AUC and Accuracy, the top 6 models are preliminarily selected as the base classifiers and a multi-model fusion approach based on the stacking strategy algorithm was used to establish the optimal prediction of moderate-to-severe CRBD. The data were employed for model validation through ten-fold cross-validation. The model's accuracy, specificity, sensitivity, positive predictive value (PPV), negative predictive value (NPV), F1-Score, area under curve (AUC), and Brier score were subsequently utilized to evaluate its performance.
Machine learning experiments were conducted using Python version 3.7.7 and the nomogram was drawn using R software. Data preprocessing, including procedures such as data cleaning, missing value filling, and normalization, was performed using the pandas library. The machine learning algorithms were implemented using the scikit-learn library. The Kolmogorov-Smirnov method was used to check the normality of the data. Continuous variables were shown as means ± standard deviations (SD) or medians (interquartile ranges) (IQRs) and compared using student’s t-tests or Mann-Whitney U tests. Categorical variables were shown as frequencies (percentage) and compared using chi-squared tests. P＜0.05 was considered statistically significant. Finally, the LR model indicators were visualized as a nomogram, serving as a clinical tool for visual representation.
This study examined 895 patients who had indwelling urinary catheters and underwent general anesthesia. Ultimately, 846 cases (94.5%) were included in the study. The screening process is depicted in Supplementary material Fig. 1. Stratified based on the presence or absence of moderate-to-severe CRBD and displayed the distribution of different variables in two groups of patients, as shown in Table 1. Of the included patients, 476 were male (56.3%) and 370 were female (43.7%). The training cohort consisted of 676 patients, with 29.1% experiencing moderate-to-severe CRBD, and 56.4% being male. The validation cohort comprised 170 patients, with 34.7% experiencing moderate-to-severe CRBD, and 55.3% being male. The distribution of patient characteristics in the training and validation cohorts is presented in Table 2. There is no statistical difference between patient features.
Table 3 presents the performance evaluation of the six selected models, and the receiver operating characteristic (ROC) curves are shown in Fig. 1. The AUC values range from 0.82 to 0.89, indicating good predictive ability for all models. The RF model performs the best, with a training cohort AUC of 0.89 (95%CI: 0.87, 0.91), an accuracy of 0.93 (95%CI: 0.91, 0.95), 0.80 sensitivity, 0.98 specificity, 0.94 PPV, 0.92 NPV, 0.87 F1 score, and 0.07 Brier score.
Fig. 1 ROC curve of six machine learning models in the training cohort and validation cohort
To identify the most important predictive factors for moderate to severe CRBD, variable importance rankings were done for the LR, RF, CB, XGB, GBDT, and LGBM models. The results indicate that the RF model includes 15 variables, with the top six variables being balloon volume, age, timing of indwelling urinary catheter, gender, surgical duration, and type of surgery (Fig. 2). Crossing all models, balloon volume, age, and timing of indwelling urinary catheter are consistently identified as important variables, See Supplementary material Figs. 2, 3 and 4.
Fig. 2 Feature importance ranking based on Random Forest (RF) in the training cohort
To overcome the limitation of machine learning models in providing clinical visualization, we decided to represent the results of the LR model as a nomogram. It is a practical graphical tool in predictive modeling, which can process clinical data through fast and complex computational processes, and visually represent the influence of each predictor variable on the outcomes [26]. This approach enhances the interpretability of the relationship between each predictor variable and the results. This specific model demonstrated nice predictive ability, as indicated by an AUC value of 0.82 (95% CI: 0.79, 0.85). Thus, it can be considered suitable for clinical guidance and validation. By considering the feature ranking of the LR model alongside other models, we carefully selected five important predictive variables. These variables were utilized to create the final nomogram, which includes gender, age, timing of indwelling urinary catheter, history of urinary catheterization, and balloon capacity (Fig. 3).
Fig. 3 Nomogram for estimating moderate-to-severe CRBD
To our understanding, this is the initial endeavor to establish machine learning-based predictive models of moderate-to-severe CRBD. By leveraging machine learning algorithms, the study developed predictive models that offer higher accuracy in addressing this issue. Additionally, the study introduced the use of a nomogram for clinical visualization, providing a comprehensive and valuable visual representation of the collected data. The study included a total of 846 patients, and six prediction models were LR, RF, GBDT, CatBoost, LightGBM, and XGBoost. These models demonstrated strong predictive ability, with the AUC ranging from 0.82 to 0.89. This suggested that they can effectively identify high-risk patients at an early stage, enabling clinical staff to implement appropriate interventions and reduce the incidence of moderate-to-severe CRBD.
The analysis of the six prediction models consistently highlighted the catheter balloon volume as the strongest predictor. However, it is worth noting that there is currently no standardized criteria for determining the most appropriate balloon volume. A study conducted by Zugail et al. [27] found that reducing the volume of the catheter balloon by half in patients undergoing urological surgery resulted in a notable decrease in pain scores and the severity of CRBD. The analysis suggested that by reducing the volume of the catheter balloon, the number of stimulated receptors is also reduced, leading to a relief in the severity of CRBD symptoms. Additionally, our study data indicated that catheters larger than 18Fr were found to increase the occurrence of moderate-to-severe CRBD, which aligns with findings from previous studies [7, 14]. Danish researchers who controlled for catheter type and utilized 12-14Fr catheters to minimize mechanical stimulation observed a noteworthy decrease in the incidence of CRBD [28]. However, there are fewer studies related to the size of urinary catheters, which are not yet conclusive, and need to be further explored for different populations or types of procedures in the future. The timing of indwelling urinary catheters and a patient’s previous history of indwelling urinary catheters were significant factors to consider when predicting outcomes. When a patient loses consciousness after the administration of anesthesia, the cerebral cortex is unable to form appropriate memories or psychological adaptations [29]. This can make them more susceptible to the stimuli associated with catheterisation when they are awake. On the other hand, individuals who have had previous experiences with indwelling urinary catheters may exhibit psychological adaptation, which can alleviate resistance and fear during catheterization [9]. This potential adaptation can help avoid the negative consequences of moderate-to-severe CRBD.
The results of this study show that age and gender have great potential in building a severe CRBD prediction model. There is currently debate about the factor of age. The study by Sy et al. [13] showed that age ≥ 50 is a protective factor affecting CRBD, speculating that the reason may be that as age increases, pain sensitivity decreases [30], which leads to an increase in the tolerance threshold for CRBD. This is contrary to the results reported by Lim and Yoon [8]. Our study supported that older patients are more likely to experience severe CRBD, but further discussion can be conducted in the future due to the lack of a reference point for age. In addition, compared to females, males have a longer and more complex urethra with two physiological bends, three narrowings, and two dilations, so when a male has an indwelling urinary catheter, a larger area of the urethra is stimulated [10], increasing the likelihood of experiencing moderate to severe CRBD.
Our study found that the type of surgery and the duration of the operation had a considerable impact on the incidence of moderate to severe CRBD. We observed a significant difference in the occurrence rate of moderate to severe CRBD between urological surgeries and non-urological surgeries. One potential explanation for this difference is that urological surgery stimulates the urethral mucosa and increases the reporting rate of CRBD [31]. Our study aims to examine the occurrence of moderate to severe CRBD in different surgical procedures, including those in the urology field. To mitigate its potential impact, future research can exclude urological surgeries and concentrate on other areas for further investigation. Based on the analysis, we concluded that the shorter the operation time, the stronger the feeling of pain and discomfort, and the more likely it is to experience moderate to severe CRBD. This speculation is based on the observation that postoperative analgesics for shorter surgeries are typically administered through a single intravenous injection, which can lead to fluctuating drug plasma concentrations and unstable analgesic effects [32]. It should also be noted that multimodal analgesia, such as combined nerve block, was not utilized during these surgeries [32]. As we all know, peripheral nerve blocks are considered an effective method for treating CRBD, because they reduce stimulation to the urethral and bladder mucosa by blocking the nerve endings [33]. However, the impact of the surgery duration in managing CRBD remains uncertain. Therefore, further evidence and research are required to establish a clearer understanding of this relationship.
In our study, multiple advanced ML algorithms were employed to further validate their predictive performance in actual clinical data, such as LightGBM, XGBoost and CatBoost. But the results showed that classical ML algorithms also demonstrated superior predictive advantages, with RF performing the best. It is an ensemble algorithm consisting of multiple decision trees [34]. It can utilize patterns learned from existing data to make predictions on new data [35]. RF can improve the accuracy of predictions without significantly increasing computational burden and capture complex relationships between variables and extract information on variable importance [36, 37], allowing for reasonable predictions of the effects of a large number of explanatory variables. It is currently one of the most widely used and best performing algorithms [35]. Furthermore, LR exhibited equally remarkable performance in addressing the data characteristics investigated, aligning with the result of a previous meta-analysis [38]. In scenarios where ML and LR demonstrate comparable performance, LR may possess greater advantages in clinical practice due to its visualization and interpretability [38]. Consequently, when constructing predictive models, it is advisable to incorporate sophisticated and complex ML algorithms for technical validation and exercise. However, it is crucial to acknowledge the merits of traditional classical algorithms. This study reinforces the efficacy of LR for research involving small sample sizes and a limited number of predictive variables.
CRBD is a significant clinical issue that impacts patients’ comfort and overall quality of care following surgery. In recent years, healthcare professionals have increasingly recognized its importance. However, most studies on CRBD have only focused on identifying influencing factors. This limited approach can result in overlooking potential factors and significantly hinder the guidance available for clinical decision-making. Furthermore, considering the challenging nature of completely avoiding CRBD, it is more clinically logical to prioritize patients who are experiencing moderate-to-severe symptoms of CRBD. Machine learning is an efficient learning algorithm that enhances the accuracy of data analysis. It excels at capturing potential predictors and analyzing valuable information. When applied in the clinical field, machine learning enables more accurate predictions of various clinical adverse events and aids in efficiently identifying high-risk patients. However, sometimes traditional algorithms such as LR can also demonstrate equally excellent predictive performance and have their unique advantages, presenting the prediction results in a simple and understandable nomogram. The nomogram a practical visual aid for clinical practice that effectively evaluates the health status of patients in a timely and convenient manner. It can be utilized by healthcare professionals to quickly detect potential instances of moderate-to-severe CRBD and mitigate the associated risk factors through suitable interventions. This can help improve patient outcomes and minimize adverse effects.
In addition to the strengths mentioned, our study also has a few limitations. Firstly, our sample size was limited to the same healthcare organisation, resulting in a lack of diversity. Moreover, we only conducted internal validation, which reduced the representativeness of our findings. To further test the effectiveness of this predictive model, it is necessary to conduct future external validation using data from other healthcare institutions. This will help assess the model’s performance and ensure its applicability beyond a single organization. Furthermore, the scope of this study was limited to assessing the severity of CRBD only during the PACU period. However, it did not specifically investigate the condition of the patients once they were transferred back to the ward. To enhance the knowledge in this area, future research can expand the assessment of CRBD severity over various time intervals, allowing for the development of an accurate prediction model based on the severity observed during different time periods. In addition, this study did not encompass a comprehensive range of anesthesia medications. It is recommended that future research incorporates literature reports on other influential drugs in order to explore their clinical significance in enhancing the management of postoperative moderate-to-severe CRBD. Finally, the results of this study need to be validated by other national groups, considering the cultural and physical differences between countries.
The objective of this study was to focus on the population with moderate-to-severe CRBD and identified key factors that can predict its occurrence. Multiple machine learning techniques were used to develop accurate predictive models. The aim was to scientifically determine the incidence of moderate-to-severe CRBD in clinical settings. Furthermore, features with significant influences were chosen to create a visual tool, called a nomogram, which could assist healthcare professionals in making informed clinical decisions.
We appreciate the great support we received from Zhejiang Province Medical and Health Technology Plan Project [grant 2024KY439], the Key Discipline of Anesthesiology of Jiaxing City [grant 2023-zc-001], the Key Discipline of Clinical Nursing Innovation of Jiaxing City [grant 2023-zc-007] and National Oncology Clinical Key Specialty [grant 2023-GJZK-001].
All authors have read and approved the manuscript. SD conducted a literature review, data collection, statistical analysis and wrote the first draft of the paper. YR assisted in statistical analysis and manuscript editing. LC and MW made significant contributions to manuscript revision. RW and QZ helped with manuscript editing and obtained funding.
This study was supported by Zhejiang Province Medical and Health Technology Plan Project [grant 2024KY439], the Key Discipline of Anesthesiology of Jiaxing City [grant 2023-zc-001], the Key Discipline of Clinical Nursing Innovation of Jiaxing City [grant 2023-zc-007] and National Oncology Clinical Key Specialty [grant 2023-GJZK-001].
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
This study was approved by the Affiliated Hospital of Jiaxing University medical ethics committee (No. 2023-LY-375).
Not applicable.
The authors declare no competing interests.
Rong Wang, Email: a043119@163.com.
Qinghe Zhou, Email: jxxmxy@163.com.
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.