Authors: Peng Sun, Jinqiang Wang, Yousheng Yuan, Zihan Chen, Jiayi Liu, Liang Xia, Jun Zhang, Nan Xu
Categories: Article, health sciences
Source: iScience
Authors: Peng Sun, Jinqiang Wang, Yousheng Yuan, Zihan Chen, Jiayi Liu, Liang Xia, Jun Zhang, Nan Xu
Distinguishing lateral malleolar avulsion fractures (LMAFs) from subfibular ossicles (SFOs) on routine ankle radiographs is a clinically consequential challenge, as the two conditions share overlapping radiographic appearances but require distinct management strategies. We developed a two-stage deep learning framework that first localizes perimalleolar bone fragments using RetinaNet and subsequently classifies them as an LMAF or an SFO using a fine-tuned MobileNetV2 classifier. Applied to X-ray images from 2,121 patients across two centers, MobileNetV2 achieved an area under the curve of 0.887 on the external test set, outperforming three comparator architectures and two experienced radiologists. Radiologists provided with AI-generated predictions and saliency maps showed significant improvement in diagnostic accuracy over unaided reading. These findings demonstrate that an integrated detection-classification pipeline can enhance first-visit radiographic triage, offering a practical, lightweight approach to support earlier and more accurate clinical decision-making in acute ankle injuries.
The ankle is the most injury-prone joint in the human body. Approximately 5% of emergency department visits are for acute ankle injuries, accounting for nearly 40% of all sports-related injuries.^1^^,^^2^ The most common mechanisms are inversion and blunt trauma, which typically lead to ligament sprains, muscle strains, or even fractures.^2^ About 15% of patients with inversion sprains have an acute lateral malleolar avulsion fracture (LMAF), which directly influences clinical decision-making. While nearly all simple sprains can achieve satisfactory outcomes with conservative management—short-term bracing, immobilization, swelling control, and functional rehabilitation—missed LMAFs may progress to nonunion, resulting in chronic lateral ankle instability and pain. Consequently, most LMAFs require stricter cast immobilization or surgical treatment.^3^^,^^4^ Free bony fragments adjacent to the tip of the lateral malleolus may also represent a subfibular ossicle (SFO), encompassing accessory bones formed by unfused secondary ossification centers—namely the os subfibulae (OSF)—as well as nonunited avulsion fractures of the distal fibula.^5^^,^^6^ The prevalence of SFOs ranges from 0.2% to 6.7%; they are usually asymptomatic and often incidentally detected on ankle radiographs obtained after inversion injuries or other trauma.^6^ In contrast, SFOs are frequently observed in 10%–38.5% of patients with chronic lateral ankle instability, suggesting an association between SFOs and chronic ankle instability.^7^ Although SFOs have two potential origins, their clinical presentation is similar—lateral ankle instability and pain; so, the primary clinical focus should be on selecting an appropriate treatment strategy, rather than scrutinizing the underlying etiology.^8^^,^^9^
Because fractures and simple sprains present with similar clinical symptoms—and because patient-reported complaints and history are often uncertain—accurate diagnosis based on physical examination alone can be challenging.^10^ Conventional radiography is currently the first-line modality for diagnosing ankle fractures, yet the missed fracture rate ranges from 14% to 85%.^11^^,^^12^^,^^13^^,^^14^ Reasons for missed diagnoses include very small lesions, minimal displacement, fracture lines not perpendicular to the X-ray beam, poor image quality, insufficient clinical information, and overlapping structures.^15^ Compared with radiography, computed tomography (CT) substantially improves the visualization of bony fragments; however, large-sample studies evaluating the diagnostic performance of CT for LMAFs with surgery or arthroscopy as the reference standard are lacking. Yet, morphological analysis can assist in differentiating acute LMAFs from SFOs—for example, LMAFs typically appear as irregular, thin fragments that conform to the anatomic contour of the lateral malleolar tip with sharp margins, whereas SFOs are more often roundish, well circumscribed, and corticated.^16^^,^^17^ Uncertainty increases when the fragment is not fully separated from the fibula, the margins appear smooth, cortical definition is poor, or osteophytes coexist.^18^ MRI can delineate bone marrow edema, ligamentous injuries, and the spatial relationship between the lateral collateral ligament complex and bony fragments through multi-sequence, multi-planar imaging.^19^ However, most institutions acquire ankle MRI with 3–4 mm slice thickness, which may introduce partial volume effects and reduce the detectability of small fragments. In addition, MRI is often unsuitable for emergency settings due to contraindications, limited availability, and prolonged scheduling and examination times.
In recent years, radiomics and deep learning (DL) based on X-ray and CT imaging have been widely applied to diagnose musculoskeletal fractures, including those of the ankle,^20^ femoral neck,^21^ hip,^22^ knee,^23^ and spine.^24^ Diagnostic approaches using deep convolutional neural networks (DCNNs) have also been deployed across various medical imaging tasks, with excellent reported performance. Li et al.^25^ used YOLO to detect and classify acute rib fractures, achieving accuracy comparable to physicians. Fang et al.^26^ applied faster region-based convolutional neural network (R-CNN) for scapular fracture diagnosis, with accuracy significantly outperforming orthopedic surgeons. Zheng et al.^22^ reported automated hip fracture detection and classification using DAMO-YOLO. However, prior studies have mainly relied on single DL models. To our knowledge, no study has reported DCNN-based detection and differential diagnosis of LMAFs versus SFOs. Therefore, we propose and validate a DCNN-based approach to detect and classify LMAFs and SFOs on X-ray images and assess its clinical applicability. We evaluate the detection and classification performance of four different DCNNs to identify the optimal architecture. In addition, we compare the diagnostic performance of the best-performing DCNN with that of radiologists to appraise and discuss its clinical value in real-world settings.
A total of 2,121 patients—1,390 with LMAFs and 731 with SFOs—met the inclusion criteria. Ages ranged from 18 to 50 years, with a mean of 25.9 ± 7.7 years. There were 1,665 male patients and 456 female patients. The training, internal validation, and external test sets included 1,221 cases (LMAFs: 801, SFOs: 420), 523 cases (LMAFs: 342, SFOs: 181), and 377 cases (LMAFs: 247, SFOs: 130), respectively. Table 1 summarizes the demographic characteristics of each dataset.Table 1Baseline characteristic of patients in the training set, internal validation set, and external test setCharacteristicTraining set (n = 1221)Internal validation set (n = 523)External test set (n = 377)**Sex, No. (%)**Male940 (77.0)401 (76.7)324 (85.9)Female281 (23.0)122 (23.3)53 (14.1)**Age (years)**Mean ± SD25.2 ± 7.226.1 ± 8.324.2 ± 8.4
We conducted a comparative analysis of the classification performance of four DCNNs. Detailed predictive performance is shown in Table 2. Figures 1 and 2 display the precision-recall (PR) curves and receiver operating characteristic (ROC) curves for each model, respectively. On the test set, MobileNetV2 achieved a significantly higher area under the PR curve at 0.807, outperforming ResNet18 (0.283), EfficientNet-B0 (0.271), and ResNet34 (0.239), indicating the best overall ability to balance precision and recall across thresholds. The area under the curve (AUC) values of the four DCNNs on the test set were 0.606 for ResNet18, 0.518 for ResNet34, 0.586 for EfficientNet-B0, and 0.887 for MobileNetV2. DeLong’s test showed that all pairwise differences were statistically significant (p < 0.05), indicating that MobileNetV2 had the highest predictive performance for distinguishing LMAFs from SFOs.Table 2The performance of the models used for comparisonModelAccuracyAUC (95% CI)SensitivitySpecificityF1 scoreResNet18Training set0.7860.812 (0.735–0.871)0.8220.7500.791Interval validation set0.7310.758 (0.657–0.860)0.7780.6900.727External test set0.5730.606 (0.513–0.696)0.8330.4840.500ResNet34Training set0.7310.748 (0.661–0.819)0.5330.9240.662Interval validation set0.6790.655 (0.543–0.765)0.4440.8810.561External test set0.4760.518 (0.433–0.614)0.8570.3440.456EfficientNet-B0Training set0.7690.796 (0.712–0.860)0.8220.7170.779Interval validation set0.7180.735 (0.639–0.845)0.7780.6670.718External test set0.5490.586 (0.498–0.676)0.8570.4430.493MobileNetV2Training set0.9340.920 (0.863–0.965)0.9330.9350.933Interval validation set0.9360.907 (0.801–0.988)0.9170.9520.930External test set0.9150.887 (0.801–0.953)0.9290.9100.848AUC, area under the curve.Figure 1Precision and recall curves for the four deep convolutional neural networks(A) Training set.(B) Internal validation set.(C) External test set.The curves are displayed in different colors, with the performance of each model shown. MobileNetV2 achieved the highest performance.Figure 2Receiver operating characteristic curves for the four deep convolutional neural networks models(A) Training set.(B) Internal validation set.(C) External test set.
The calibration curves for the classification performance of the four DCNNs on X-ray images are shown in Figure 3. For the MobileNetV2 model, the calibration curves in the training, internal validation, and external test sets indicated that the predicted probability curves were close to the observed probability curves (Hosmer-Lemeshow χ^2^ = 0.449, 0.027, and 18.241; p = 0.930, 0.999, and 0.440, respectively). Decision curve analysis (DCA) showed that the MobileNetV2 model can serve as a useful tool for distinguishing between LMAFs and SFOs (Figure 4).Figure 3Calibration curve for the prediction model(A) Training set.(B) Internal validation set.(C) External test set.Figure 4Decision curve analysis of the model(A) Training set.(B) Internal validation set.(C) External test set.
Table 3 presents the AUC, sensitivity, specificity, and accuracy for the MobileNetV2 model and two radiologists in diagnosing LMAFs and SFOs. The AUCs for MobileNetV2 and the two radiologists were 0.887, 0.821, and 0.752, respectively. DeLong’s test showed that all pairwise differences were statistically significant (p < 0.05), indicating that the MobileNetV2 model outperformed both radiologists. With AI assistance, both radiologists demonstrated significant improvement in diagnostic performance. Radiologist 1’s accuracy increased to 0.859 (p = 0.012). Similarly, radiologist 2’s accuracy improved to 0.847 (p = 0.008). Notably, while AI-assisted radiologist performance (radiologist 1-AI: AUC = 0.867; radiologist 2-AI: AUC = 0.851) did not surpass that of the standalone MobileNetV2 model (AUC = 0.887), the clinically relevant comparison is between unassisted and AI-assisted radiologist performance. Both radiologists demonstrated statistically significant improvement with AI assistance, confirming the model’s value as a decision-support tool within a human-in-the-loop framework.Table 3Comparison of AUC, accuracy, sensitivity, and specificity between the MobileNetV2 model and radiologists’ readingAUCAccuracySensitivitySpecificityMobileNetV20.887 (0.801–0.953)0.9150.9290.910Radiologist 10.821 (0.754–0.888)0.8540.8230.918Radiologist 20.752 (0.678–0.827)0.7810.8540.651Radiologist 1-AI0.867 (0.786–0.912)0.8590.8810.852Radiologist 2-AI0.851 (0.806–0.923)0.8470.8570.844
Table 4 presents class-specific performance metrics for MobileNetV2 on the external test set. The model achieved balanced performance across both classes, with slightly higher sensitivity for LMAFs (0.929) than for SFOs (0.910) and comparable precision values. The high F1 scores for both classes (LMAF: 0.932; SFO: 0.902) demonstrated robust classification performance despite the moderate class imbalance, indicating that the model does not exhibit significant bias toward the majority class.Table 4Class-specific performance metrics on external test setModelClassSensitivityPrecisionF1 scoreMobileNetV2LMAF0.9290.9360.932SFO0.9100.8950.902
On the independent external test set, RetinaNet achieved strong detection performance across multiple intersection over union (IoU) thresholds. At the standard IoU = 0.5 threshold, the model achieved mean average precision at an IoU threshold of 0.50 (mAP50) of 0.920, indicating high detection sensitivity. Importantly, the model maintained robust performance at stricter thresholds, with mean average precision at an IoU threshold of 0.75 (mAP75) of 0.828 (IoU = 0.75) and mAP50-95 of 0.712 (averaged across IoU = 0.5–0.95). After the final convolutional layer of the model, gradient-weighted class activation maps (Grad-CAMs) were generated and overlaid on the X-ray images to visualize the relevance of specific regions during classification. As shown in Figures 5A and 5B, the model focused on the region containing the avulsed bone fragment when detecting LMAFs, whereas for SFOs, it primarily focused on the smooth edge of the lateral malleolus (Figures 5C and 5D). These results indicated that the model learned to assess the correct features, rather than merely memorizing inter-image correlations.Figure 5Gradient-weighted class activation maps for LMAF and SFO feature assessment(A and B) Radiograph and Grad-CAM for LMAFs.(C and D) Radiograph and Grad-CAM for SFOs.Shown are Grad-CAMs with activation after the last convolutional layer of the MobileNetV2 overlaid, with the radiograph serving as heat maps (red, higher activation; blue, lower activation). The maps show that the neural network focused on the region of the abnormality for its assessment.
Our study is the first to systematically evaluate multiple mainstream DCNNs on ankle radiographs for the binary classification of LMAFs versus SFOs. MobileNetV2 achieved the best discrimination, with an AUC of 0.887 and an accuracy of 0.915 on the external test set—significantly outperforming ResNet18/34 and EfficientNet-B0, and exceeding the reading performance of two radiologists. In parallel, for the object-detection task targeting “perimalleolar free bone fragments,” RetinaNet delivered a strong overall performance, effectively supporting the classifier’s focus on lesion regions and enhancing model interpretability in a synergistic manner, thereby further improving end-to-end clinical usability and workflow readiness. These findings have clear clinical implications. First, they enable rapid and accurate differentiation of perimalleolar free bone fragments at the point of first contact in emergency or outpatient settings, reducing missed or delayed diagnoses and shortening the decision-making time. Second, given the model’s high classification reliability and favorable decision curves, the approach has the potential to standardize referral decisions and subsequent imaging pathways (for example, guiding more appropriate selection between MRI and CT), thereby optimizing resource allocation. Third, for patients with concomitant or strongly suspected chronic lateral ankle instability, the model outputs can serve as quantitative evidence to prompt early referral for orthopedic/sports medicine evaluation and to support individualized intervention strategies. Overall, the combination of a lightweight MobileNetV2 with strong external generalizability and a high-performing lesion localizer (RetinaNet) is complementary, offering a feasible route to embed an integrated “detection-discrimination” tool into routine radiography workflows and substantially improving the efficiency of “localize-and-classify at first visit.” Importantly, our human-AI collaboration experiment demonstrated that the AI model can serve as an effective decision support tool to enhance radiologists’ diagnostic performance.
An important observation is that AI-assisted radiologist performance, while significantly improved over unassisted reading, remained slightly below the standalone AI model’s metrics. This finding is consistent with the well-characterized phenomenon of “algorithm aversion,” whereby clinicians selectively override algorithmically correct predictions, particularly for borderline cases where subjective clinical experience conflicts with model output.^27^^,^^28^ However, this does not diminish the clinical utility of the human-AI collaboration framework. First, fully automated AI diagnosis without human oversight is not currently acceptable in clinical practice due to ethical, legal, and patient safety considerations. Second, the human-in-the-loop design provides a critical safety net for cases where the AI model may fail—for example, images with atypical presentations or distribution shifts not represented in the training data. Third, the statistically significant improvement in both radiologists’ accuracy demonstrates that the AI model effectively augments human decision-making. Future studies employing prospective, randomized crossover designs with larger reader panels are needed to further optimize the human-AI interaction protocol and quantify the residual effect of anchoring bias.
DL applied to routine ankle radiographs has reached a clinically usable level for lesion recognition, subtype classification, and quantitative measurement. When powered by multi-view fusion and high-quality annotations, it markedly improves the detection rate and inter-reader consistency for subtle or occult findings.^29^ For “upstream” tasks related to syndesmotic instability and lateral malleolar pathology, prior work has achieved explainable preoperative identification of distal tibiofibular syndesmotic instability on radiographs, using the intraoperative hook test as the reference standard; the model achieved sensitivity and specificity of approximately ≥0.8, with heatmap attention aligning with key biomechanical landmarks, suggesting utility for preoperative risk stratification and surgical planning.^30^ In fracture detection and classification, multi-view InceptionV3/ResNet-50 has demonstrated near-expert-level radiologist performance under three-view settings and can identify the vast majority of occult fractures.^31^ Automated, X-ray-based objective measurements (e.g., talar tilt and anterior talar translation) have been implemented in an end-to-end manner from region of interest (ROI) detection/segmentation to measurement, showing strong agreement with physician readings and offering direct clinical value for quantifying lateral malleolar avulsion with concomitant ankle instability.^32^ These studies offer direct guidance for “LMAF versus SFO detection and discrimination.” Lateral malleolar avulsion fragments are small and low contrast, and they are easily confused with distal fibular enthesophytes/accessory ossicles, periosteal reactions, or chronic ossicles. Therefore, beyond a simple “fracture vs. non-fracture” binary output, the model should also localize or detect the free fragment to support reliable downstream differentiation. It should also be noted that our model was trained using only anteroposterior (AP) radiographs. Although multi-view radiograph fusion has shown improved performance in several fracture detection studies, the ability of our model to achieve robust discrimination using a single projection suggests that meaningful diagnostic features are already present in the AP view alone.
Methodologically, our study offers several advantages and innovations. First, we developed an end-to-end, two-stage pipeline. Our comprehensive evaluation across multiple IoU thresholds (mAP50, mAP75, and mAP50-95) provides robust evidence of the model’s detection accuracy beyond the commonly reported mAP50 metric. The consistent high performance at stricter IoU thresholds (mAP75 = 0.828) demonstrates that RetinaNet not only identifies the presence of ossicles but also achieves precise spatial localization—a critical capability given that LMAF fragments can be quite small and require accurate delineation for treatment planning. This precision is essential for clinical implementation, where imprecise bounding boxes could lead to diagnostic uncertainty or missed findings. The mAP50-95 metric of 0.712 aligns with the performance standards of state-of-the-art object detectors on medical imaging tasks and confirms the model’s robustness across varying localization requirements.
Upstream, RetinaNet performs lesion localization and automatic ROI extraction, enabling stable and reproducible cropping while markedly reducing reliance on manual annotations and heuristic rules.^33^ Downstream classifiers operate on standardized ROIs, concentrating on key anatomical regions and subtle imaging cues, which enhances density contrast and suppresses background noise. This creates a closed-loop “detection-discrimination” workflow that is amenable to integrated clinical deployment. Second, we validated the suitability of lightweight backbones for this task. MobileNetV2 delivered the best overall performance in ankle radiographs characterized by relatively limited sample size, low contrast, and weak texture, suggesting that its inverted residuals with linear bottlenecks are well aligned with the weak-texture, low signal-to-noise ratio (SNR) properties of osteoarticular X-rays.^34^ This architecture balances model capacity, inference efficiency, and fine-grained edge representation,^35^ supporting rapid deployment in emergency and primary-care settings. Third, from an interpretability perspective, Grad-CAM saliency maps indicate that the model primarily attends to fracture lines, ossicle margins, and adjacent cortical discontinuities—features that align with radiologists’ reading heuristics,^36^ thereby improving transparency and clinical acceptability. Finally, through head-to-head comparisons with three mainstream DCNNs, cross-center external validation, and statistical testing via the DeLong’s test, we provide convergent evidence for the robustness and generalizability of our conclusions.
The moderate class imbalance in our dataset (LMAF:SFO ratio of 1.9:1) reflects the clinical epidemiology where LMAFs are indeed more common than SFOs in patients presenting with ankle trauma. Our model’s performance metrics indicate successful mitigation of potential class imbalance effects through multiple strategies including data augmentation, focal loss implementation, and threshold optimization. The class-specific metrics (Table 4) demonstrate that the model maintains high performance for both LMAF and SFO classes, with no evidence of significant bias toward the majority class. The discrepancy between ROC-AUC (0.887) and PR-AUC (0.807) is within the expected ranges for moderately imbalanced datasets and reflects the different aspects of classification performance that these metrics capture. Importantly, both metrics exceed clinically acceptable thresholds (>0.80), and the model’s balanced sensitivity for both classes makes it suitable for clinical deployment where missing either an LMAF or an SFO would have negative consequences for patient care. Although a discrepancy between ROC-AUC (0.887) and PR-AUC (0.807) was observed, this is expected in moderately imbalanced datasets and does not imply increased clinical risk. PR-AUC is more sensitive to class prevalence, whereas false-negative risk is better reflected by sensitivity. In our study, the sensitivity for SFO reached 0.910 (false-negative rate = 9.0%), comparable to those for LMAF (0.929), indicating no bias toward the majority class and a low miss rate for the minority class.
In sum, under a comprehensive framework of high-precision detection, lightweight discrimination, multi-dimensional interpretability, and rigorous validation, our study delivers an AI model that combines accuracy, efficiency, and trustworthiness. Compared with prior work that largely relied on pure object detection or early-generation classifiers, existing methods often degrade in scenarios with overlapping anatomy and tiny and smoothly marginated fragments.^37^^,^^38^^,^^39^ They also struggle to differentiate among imaging-wise similar entities such as LMAFs and SFOs and are frequently constrained by single-center cohorts, small sample sizes, or lack of reader comparison, limiting generalization and clinical translation. In contrast, our approach explicitly focuses on avulsion fragments in LMAF cases and on the smooth lateral malleolar contour in SFO cases, indicating that the model learns pathophysiologically consistent discriminative features, rather than non-lesion correlates. This confers an advantage in complex, high inter-class similarity tasks and offers a feasible, intelligent solution for triage at the first emergency visit.
Unlike prior studies that have primarily focused on detecting/localizing the presence of fractures, this work is the first to systematically evaluate the clinically consequential fine-grained task of distinguishing LMAFs from SFOs—directly addressing the urgent need for rapid subtyping of “perimalleolar free bone fragments” at the initial emergency/outpatient encounter. In terms of imaging accessibility, while CT/MRI provides superior tissue contrast and anatomical detail, they are constrained by cost, waiting time, and contraindications. Demonstrating usable and robust performance on routine, first-line radiographs fills the “first step” of the clinical pathway and provides quantitative justification for whether CT/MRI should be added, aligning better with real-world practice. Compared with the AUC/accuracy reported in the literature for “ankle fracture presence” detection tasks,^40^ our study maintains a high AUC and good calibration in a more challenging, fine-grained binary classification setting and shows clinical substitutability and complementarity through direct comparison with radiologists. The superiority of MobileNetV2 in our study likely reflects the interaction between model architecture and task characteristics. Although deeper networks such as ResNet and EfficientNet typically provide stronger representation capacity, they may also introduce a higher risk of overfitting when trained on relatively limited medical imaging datasets. In contrast, MobileNetV2 uses inverted residual blocks and linear bottlenecks that preserve fine structural information while maintaining a compact parameter space, which may be advantageous for capturing subtle morphological differences in small ROIs. The markedly low AUC values of ResNet34 (0.518) and EfficientNet-B0 (0.586) on the external test set—despite strong training set performance—are consistent with the overfitting behavior expected when high-capacity networks are trained on small, single-institution datasets. ResNet34 carries approximately 21 million parameters compared with MobileNetV2’s 3.4 million; this capacity mismatch relative to a training set of 1,221 samples likely drove representation collapse during generalization. A similar pattern has been observed in prior medical imaging studies where lightweight networks outperformed deeper counterparts on fine-grained binary classification tasks with limited data.^41^^,^^42^
Under real-world X-ray constraints, it achieves more robust generalization with fewer parameters and stronger inductive bias, reducing overfitting risk. Its inverted residuals with linear bottlenecks preserve edges and subtle textures more effectively, matching the nuanced difference between the “sharp fracture line” of LMAFs and the “smooth cortical boundary” of SFOs. In a class-imbalanced scenario where clinical priority is on high-quality positive case recognition, PR-AUC is more informative than ROC-AUC for summarizing the precision-recall trade-off. The markedly higher PR-AUC of MobileNetV2 further supports its superior ability to balance precision and recall, which is particularly important in clinically imbalanced classification scenarios. Notably, its PR-AUC (0.807) was substantially higher than those of ResNet18 (0.283), ResNet34 (0.239), and EfficientNet-B0 (0.271), providing strong empirical justification for selecting MobileNetV2 as the final classification architecture. The marked advantage of MobileNetV2 in PR-AUC indicates fewer missed diagnoses while maintaining high specificity and efficient alerting. Favorable probability calibration suggests that model scores can be directly used to set clinical thresholds and interface with triage/referral protocols. DCA shows net benefit across a broad threshold range, supporting the integration of model scores into clinical decision support—guiding stratified “CT/MRI add-on” strategies and serving as a quantitative trigger for early referral to orthopedics/sports medicine in patients with suspected chronic lateral ankle instability.
Our study proposes an X-ray-based pipeline combining RetinaNet for detection and the lightweight MobileNetV2 for classification, enabling reliable automatic detection and discrimination between LMAFs and SFOs. The approach outperforms multiple comparator networks and experienced radiologists across several metrics, while demonstrating favorable probability calibration and net clinical benefit on DCA. The model is poised to support “early recognition, early triage, and early intervention” at the first emergency/outpatient encounter, offering a practical technical foundation to help reduce adverse outcomes such as chronic lateral ankle instability and post-traumatic osteoarthritis.
This study has several limitations. First, the retrospective two-center design may introduce selection bias and limit generalizability across institutions, imaging protocols, and populations; prospective multi-center validation is warranted.^43^ Second, although CT/MRI served as the reference standard, the lack of surgical confirmation and long-term follow-up may leave residual diagnostic uncertainty in some cases. Third, the model was developed using only AP radiographs, rather than the full ankle series, which may limit the detection of certain fracture characteristics better visualized on other projections. Fourth, Grad-CAM-based interpretability remains a post hoc approach and may not fully reflect causal feature attribution, particularly under distribution shifts.^44^ Fifth, the study population was restricted to patients aged 18–50 years, with a male predominance, which may limit applicability to other demographic groups.^45^ Finally, the human-AI reader study included only two radiologists and may still be affected by residual recall and anchoring biases. Future studies with larger, more diverse cohorts, multi-view imaging, and expanded reader panels are needed.
Requests for further information and resources should be directed to and will be fulfilled by the lead contact, Jun Zhang (zhangjun21416@hotmail.com).
Iteration of the trained DL model is available on request from the lead contact. There are restrictions to the availability of patients’ medical images used in training of the DL model due to institutional and legal regulations over patients’ confidentiality and imaging.
•Patients’ medical images used in training of the DL model in this study cannot be deposited in a public repository due to institutional and legal regulations over patient confidentiality and imaging. Request for their access should be directed to the lead contact.•Code used in this study is publicly available at GitHub (https://github.com/410312774/PixelMedAI) and has been archived at Zenodo: https://doi.org/10.5281/zenodo.20414980.•Any additional information required to reanalyze the data reported in this paper is available from the lead contact upon request.
This work was supported by the Medical Scientific Research Project of the 10.13039/100017962Jiangsu Provincial Health Commission (grant no. M2024055). We would like to express our great appreciation to the editor and anonymous reviewers for their comments, which helped us improve the quality of our paper. We would like to thank American Journal Experts (www.aje.com) for editing the language of our manuscript draft. For advice regarding the code used in this revised manuscript, we thank PixelmedAI platform and its developers.
Conceptualization, P.S., J.W., Y.Y., Z.C., J.L., L.X., and J.Z.; Z.C. and L.X.; software, validation, and formal analysis, L.X. and N.X.; investigation, resources, and data curation, P.S., J.W., Y.Y., J.Z., and N.X.; writing – original draft, P.S., J.W., and Y.Y.; writing – review & editing, P.S., J.W., Y.Y., Z.C., J.L., L.X., J.Z., and N.X.; visualization, P.S., J.W., Y.Y., J.L., J.Z., and N.X.; supervision, J.L. and L.X.; project administration and funding acquisition, J.Z.
The authors declare no competing interests.
REAGENT or RESOURCESOURCEIDENTIFIERDeposited dataRaw X-ray images dataThis paperN/ACode for model and network training and analysisThis paperhttps://doi.org/10.5281/zenodo.20414980Software and algorithmsPython (version 3.8)Python Software Foundationhttps://www.python.org/R (version 4.1.0)R softwarehttp://www.r-project.org/RetinaNet(Lin et al., 2017)^46^arXiv:1708.02002MobileNetV2(Sandler et al., 2018)^47^arXiv:1801.04381ResNet18 & ResNet34(He et al., 2016)^48^arXiv:1512.03385EfficientNet-B0(Tan et al., 2016)^49^arXiv:1905.11946
This paper uses de-identified medical images of patients who had ankle radiographs performed at two institutions within the study period.
Description of overall age and gender is described in Table 1 of the manuscript.
Waiver of full ethical deliberation was provided by the host institutional review boards and ethics committees, with the study conducted using deidentified data.
This study developed a two-stage deep learning framework consisting of lesion detection and regions of interest (ROI)-based classification. First, a RetinaNet model was used to automatically detect perimalleolar bone fragments and generate ROIs from ankle radiographs. The cropped ROIs were then fed into a MobileNetV2-based classifier to differentiate LMAF from SFO.
Our study used medical imaging data from two hospitals in China. With approval from the respective institutional review boards and ethics committees, the retrospective design exempted the requirement for individual informed consent. For the development (training) and internal validation cohorts, we collected imaging examinations of patients with ankle sprains treated at Center I (Air Force Medical Center, Air Force Medical University) between January 2014 and December 2024. Inclusion criteria (1) 18 ≤ age ≤ 50 years with an ankle sprain occurring within the preceding 3 days, to reduce confounding effects from severe degenerative changes, multiple fractures and osteoporosis commonly observed in older patients; (2) a single perimalleolar free ossicle on radiographs (size ≥ 1 cm); (3) complete raw data for both radiography and CT/MRI, with an interval of less than 1 week between the two examinations; and (4) complete clinical data (sex, age). Exclusion criteria (1) suspected pathological fractures related to infection or tumor; (2) hereditary skeletal disorders or skeletal developmental anomalies; (3) prior ankle surgery; and (4) poor image quality or foreign-body artifacts. An independent external test cohort was assembled from Center II (Sir Run Run Hospital, Nanjing Medical University) between January 2017 and December 2024, applying the same inclusion and exclusion criteria. As illustrated in the screening flowchart (Figure S1), a total of 2,121 patients were included (1,390 LMAFs and 731 SFOs). The training and internal validation sets were split 3, comprising 1,221 and 523 patients, respectively; the external test set included 377 patients.
Patient age and sex were extracted from the electronic medical records. Imaging devices and acquisition parameters for radiography, CT, and MRI are provided in Tabel S1. Before X-ray examination, metallic objects on the ankle and distal lower leg and thick socks were removed. When necessary, a brace was used for immobilization. Patient positioning was adjusted gently to avoid secondary injury. For anteroposterior (AP) ankle radiographs, patients were positioned supine or seated. The examined limb was extended with the toes pointing upward and slightly internally rotated. The ankle was dorsiflexed to align the plantar surface parallel to the detector. The central beam was directed perpendicular to the ankle joint, centered at the midpoint between the medial and lateral malleoli. Our study performed processing and analysis based on AP ankle radiographs.
We chose to use only AP radiographs for the following reasons. First, LMAF and SFO are predominantly located at the lateral aspect of the distal fibula. While the AP view provides visualization of this anatomic region, we acknowledge that mortise and lateral views may offer superior detection of certain fracture patterns, particularly nondisplaced oblique fractures. However, the AP view remains a standard component of ankle imaging and provides adequate visualization of the fibular tip and associated osseous fragments for developing an initial screening tool. Second, in many emergency and primary-care settings—particularly in resource-limited environments—imaging protocols vary considerably. Ottawa Ankle Rules help clinicians determine whether radiography is indicated, and in some settings where complete three-view series are not consistently performed, the AP view may be the most readily available projection.^50^ Therefore, developing an AI model based solely on AP radiographs addresses a practical clinical scenario and has broader applicability. Third, from a technical perspective, training deep learning models on standardized single-view images reduces variability in input data, simplifies annotation, and facilitates more robust feature learning. Multi-view integration would require complex fusion architectures and substantially larger annotated datasets, which were beyond the scope of this initial study. In addition, to ensure a fair comparison between the AI model and radiologists, both were restricted to the same imaging input (AP radiographs only). This design avoided potential bias that could arise if radiologists had access to additional projections that were not available to the model.
In our study, differentiation between LMAF and SFO was established on CT or MRI, which served as the diagnostic reference standard. CT-based criteria,^45^ LMAF: Irregular fragments with sharp fracture lines and margins, accompanied by signs of acute soft-tissue injury (indirectly reflected by soft-tissue swelling or hemarthrosis), and a “recently separated” relationship from the parent bone. SFO: Small, dense, smoothly contoured ossicles with rounded margins and intact cortex, sharply demarcated borders, and often a “flat, symmetric, joint-like” gap or fibrous connection with the parent bone, without ancillary signs of acute soft-tissue injury. MRI-based criteria,^45^ LMAF: Bone marrow edema involving the distal fibula and the corresponding free fragment region, commonly with soft-tissue swelling or evidence of ligamentous injury. SFO: Absence of bone marrow edema in the corresponding region and lack of surrounding soft-tissue injury, consistent with a chronic, stable ossicle. Image interpretation on CT/MRI was performed independently by two senior musculoskeletal radiologists, each with over 20 years of experience. Discrepancies were resolved by consensus; if consensus could not be reached, a third experienced musculoskeletal radiologist was consulted, and the final diagnosis was determined by majority opinion.
For the external test set, each case’s radiographs were independently reviewed by two Radiologists 1 (associate chief radiologist with 15 years of musculoskeletal imaging experience) and radiologists 2 (attending radiologist with 7 years of experience). During interpretation, both readers were blinded to CT and MRI findings and performed assessments independently. Each reader, based on their clinical experience, independently recorded a categorical diagnosis of LMAF or SFO. To compute diagnostic performance metrics, each reader’s diagnosis was compared against the reference standard defined by CT or MRI. Finally, we directly compared each reader’s performance on the external test set with that of the AI model to evaluate whether the AI outperformed human assessment.
X-ray images were acquired in Digital Imaging and Communications in Medicine (DICOM) format to ensure high resolution and rich grayscale depth. For automated analysis of ankle radiographs, we first employed a RetinaNet object detection model pretrained on the Microsoft Common Objects in Context dataset. Through transfer learning on manually annotated ankle radiographs, RetinaNet automatically detected perimalleolar bone fragments and generated bounding boxes corresponding to ROIs. The detected ROIs were then cropped, underwent contrast stretching, and were resized to 224 × 224 pixels. These standardized ROI images were subsequently used as inputs for the downstream classification model. For the classification stage, we adopted MobileNetV2 pretrained on ImageNet as the backbone network, which was fine-tuned to differentiate LMAF from SFO (Figure S2). Gradient-weighted class activation maps (Grad-CAM) were generated from the final convolutional layer to visualize the model’s decision process.
In our study, we selected MobileNetV2 as the backbone neural network model to distinguish LMAF from SFO. MobileNetV2 is an efficient and lightweight convolutional neural network architecture designed for mobile and resource-constrained scenarios. Its technical core is the inverted residual structure and linear bottlenecks. This structure helps the network maintain strong feature extraction capability while significantly reducing the number of parameters and computational cost.
Following automated ROI extraction by RetinaNet, the MobileNetV2 backbone receives the cropped lateral malleolus ROIs as input. First, at the input layer to the MobileNetV2 backbone, the ROIs output from RetinaNet are already resized to 224 × 224 pixels and normalized to single-channel grayscale format to match MobileNetV2's default input requirement. Next, the main network adopts a backbone composed of 17 inverted residual modules. The model’s initial parameters are transferred from the large-scale ImageNet dataset and fine-tuned for the task in this study to enhance adaptability to medical imaging details. The inverted residual structure is MobileNetV2’s core it first compresses the number of channels through a small fully connected layer (bottleneck), then restores the channels through expansion convolutions, and introduces linear activation in between, which ensures smooth information flow and effectively avoids loss of feature representation and overfitting.^47^ The backbone is followed by a global average pooling layer to further compress spatial features. This structure both reduces the number of subsequent parameters and improves the model’s generalization performance. Finally, in the classification layer, the feature vector passes through a single-neuron fully connected layer and uses a sigmoid activation function to achieve binary probabilistic output for LMAF and SFO. This structure balances lightweight design with efficient feature representation and is well-suited to small-sample analysis scenarios in medical imaging.
During training, the loss function was binary cross-entropy, the optimizer was Adam, the initial learning rate was set to 0.0001, and the batch size was 16. To improve model generalization, various data augmentation methods were applied during training, such as random rotation, scaling, flipping, and brightness and contrast adjustments. During model training, the internal validation set was used to monitor loss and accuracy in real time, and the best-performing model weights were automatically saved. The detailed network architecture and preprocessing pipeline are described in Table S2.
To rigorously evaluate the effectiveness of the MobileNetV2 model, we also compared MobileNetV2’s classification performance with mainstream backbone networks including ResNet18, ResNet34, and EfficientNet-B0. Each of these models has distinct ResNet18/34 mitigate the vanishing gradient problem in deep networks through residual connections and possess strong feature extraction capabilities. EfficientNet-B0 achieves a balance between computational complexity and performance through efficient convolutions and a compound scaling strategy. The comparative experiments were conducted using the same training set, data augmentation, and optimization strategies to systematically assess the applicability and performance of different network architectures for the LMAF vs. SFO classification task. To ensure a fair and reproducible comparison, all four architectures (MobileNetV2, ResNet18, ResNet34, and EfficientNet-B0) were trained under strictly identical the same training set and class-stratified splits, the same data augmentation pipeline (random rotation, scaling, flipping, brightness and contrast adjustment), the same optimizer (Adam), the same initial learning rate (0.0001), the same batch size (16), the same loss function (binary cross-entropy), and the same early-stopping criterion (patience = 10 epochs based on validation loss), with the best-performing checkpoint saved for each model. All models were initialized with ImageNet-pretrained weights and fine-tuned end-to-end. A Table S3 all shared hyperparameters explicitly. The final selection of MobileNetV2 as the classification backbone was based on both empirical performance and task-specific considerations. Although deeper architectures such as ResNet and EfficientNet are known for strong feature representation capability, our comparative experiments demonstrated that MobileNetV2 achieved the best overall performance on the external test set, with substantially higher AUC and PR-AUC values than the other evaluated networks.
The detection and classification networks were trained separately but integrated into a unified inference pipeline. Our dataset exhibited moderate class imbalance (LMAF:SFO ratio ≈ 1.9:1; 420, 181, 130). To mitigate this imbalance, we implemented several (1) differential data augmentation with more aggressive augmentation applied to the minority class (SFO); (2) focal loss in the detection stage to down-weight easy examples and emphasize hard cases; and (3) comprehensive evaluation using both ROC-AUC for threshold-independent assessment and PR-AUC for precision-recall focused evaluation, which is more sensitive to minority class performance. For detection performance, we evaluated mean average precision (mAP) across multiple IoU thresholds to comprehensively assess both detection sensitivity and localization accuracy. Specifically, we (1) mAP50 (IoU=0.5), which emphasizes detection sensitivity; (2) mAP75 (IoU=0.75), which requires more precise bounding box localization; and (3) mAP50-95, the average mAP across IoU thresholds from 0.5 to 0.95 with 0.05 increments, following the COCO evaluation protocol. This multi-threshold evaluation is particularly important for small fracture fragments where precise localization is clinically critical. We also calculated class-specific precision, recall, and F1 score at IoU=0.5 for both LMAF and SFO detection.
For classification performance, we computed class-specific metrics including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy for both LMAF and SFO. ROC curves were plotted and AUC values calculated to assess discriminative ability. PR curves were generated to visualize precision-recall trade-offs across decision thresholds. DeLong's test was used to compare AUCs between models and between the two radiologists. Model calibration was evaluated using calibration curves and the Hosmer-Lemeshow test, and clinical utility was assessed through decision curve analysis (DCA). Gradient-weighted class activation maps (Grad-CAM) were employed to visualize model decision-making processes.^23^ Statistical analyses were performed using R software (version 4.0.2) with RMS and rmda packages. Two-sided P < 0.05 was considered statistically significant.
For the external test set, the human–AI collaboration experiment was conducted after a 6-month washout period to minimize recall bias. In the AI-assisted session, radiologists were provided with the following AI outputs for each (1) the binary classification result (LMAF or SFO), (2) the predicted probability score, and (3) a Grad-CAM heatmap highlighting the discriminative region. To mitigate anchoring bias, a two-stage reading protocol was each radiologist first independently recorded a preliminary diagnostic impression without AI input, and then reviewed the AI output and recorded a final decision. Radiologist 1 and 2 were explicitly instructed that they could accept or override the AI recommendation based on their own clinical judgment. Both reading sessions were performed under blinding to CT/MRI reference standards.
The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study was approved by the Institutional Ethics Committee of the Sir Run Run Hospital, Nanjing Medical University (2025-SRFA-626), and Air Force Medical Center, Air Force Medical University (2025-11-PJ01). The institutional ethics committees waived written informed consent in view of the retrospective nature of the study. All the procedures performed were part of routine care. All images were deidentified before use to protect patient privacy.