Diagnostic Efficiency of ACR-TIRADS Score for Differentiating Benign and Malignant Thyroid Nodules of Various Pathological Types

Introduction

Thyroid nodules have a prevalence of up to 65% in the general population [1,2]. The incidence of thyroid cancer has increased at rate of 3% per year, and the incidence of advanced papillary thyroid carcinoma (PTC) and the associated mortality has also increased [3]. This increase could be attributed to the widespread use of medical imaging and medical health surveillance, as well as to improvements in healthcare [4]. Thyroid ultrasound imaging can be used to identify patients at low risk for cancer [5–7] as well as those requiring fine-needle biopsy [8]. Fine-needle biopsy and molecular testing can reduce unnecessary testing and the related expenses to a great extent. However, the inappropriate use of fine-needle biopsy has the potential for increased medical risk, as does the cytological uncertainty that warrants surgery. Hence, appropriate ultrasonic examination is a key step in the effective monitoring and management of thyroid cancer [9,10].

The American College of Radiology Thyroid Imaging Reporting and Data System (ACR-TIRADS) assigns a score to thyroid nodules based on their ultrasonic features; the sum of the scores determines the nodule’s risk level and the need for biopsy or follow-up [11,12]. Hence, this system is highly effective and easily available [13–15]. Grani et al [15] compared 5 internationally recognized ultrasound classification systems, the American Thyroid Association (ATA), American Association of Clinical Endocrinologists (AACE), American College of Radiology (ACR), European Thyroid Association, and Korean Society of Thyroid Radiology systems, and found that ACR-TIRADS was most effective in ruling out malignant nodules and in differentiating benign and malignant nodules, thus reducing unnecessary fine-needle aspiration [16–18]. In another multicenter study, ACR-TIRADS was found to be more effective than the ATA guidelines and the Korean Society of Thyroid Imaging Reporting and Data System (KTIRADS) in reducing the number of unnecessary biopsies [19,20]. Therefore, ACR-TIRADS grading is a valuable tool for the diagnosis of thyroid nodules and is worthy of adoption in clinical practice [21].

Most current classification systems, including ACR-TIRADS, focus on PTC because of its high incidence. However, the annual incidence of medullary thyroid cancer (MTC) and follicular thyroid cancer (FTC) has also increased significantly [11,22], and the prognosis of different pathological types of thyroid cancer varies greatly [23]. Those with poor prognosis, such as anaplastic thyroid cancer (ATC), have almost no treatment options [24]. Many studies have used the ACR-TIRADS scoring system to evaluate benign and malignant thyroid nodules, but there are relatively few studies on the different pathological classifications of malignant thyroid nodules [25]. In this study, we analyzed and compared ultrasound images of benign and malignant thyroid nodules and different pathological classifications of malignant nodules based on the ACR-TIRADS scoring standard, evaluated its diagnostic performance for benign and malignant thyroid nodules, and evaluated the different pathologies of malignant nodules, which will in turn provide a solid basis for clinicians in diagnosis and treatment.

Material and Methods

PATIENT SELECTION:

A retrospective study was conducted in patients with thyroid nodules who underwent surgery in the hospital and had histopathology results from June 2009 to December 2022. To make a more in-depth study of different pathological types of patients, considering the low proportion of MTC, FTC, and undifferentiated carcinoma, this study included all patients with these cancer types in the past 10 years and included some patients with PTC. A total of 1614 patients were included in the study, including those with benign nodules (n=799) and malignant nodules (n=815), including PTC (n=631), MTC (n=85), FTC (n=80), and undifferentiated carcinoma (n=19). This retrospective study was approved by the Zhejiang Cancer Hospital, and the requirement for informed consent was waived by the Zhejiang Cancer Hospital and Zhejiang Rongjun Hospital and Shanghai University of Traditional Chinese Medicine.

Inclusion criteria were (1) patients who received preoperative ultrasonic examination in our hospital with complete ultrasound data and (2) patients who underwent surgery at our hospital with clear pathological findings.

Exclusion criteria were (1) patients for whom no ultrasonic examination data or images were available preoperatively; (2) the anatomical location of available images was not consistent with the description of pathological reports; and (3) there was a history of previous thyroid surgery.

ULTRASONIC EXAMINATION:

The Logiq E9 (GE Healthcare, USA), Toshiba 790A (Toshiba Corp, Japan), Esaote Mylab90 (Esaote S.P.A, Italy), and Philips iU22 (Philips, Netherlands) ultrasonic systems were used in this study. The probe types included ML6-15, PLT-805AT, LA523, and L12-5, and the probe frequency was 5–13 MHz.

The patient was in the supine position with neck hyperextension, and the transverse and longitudinal sections of the thyroid gland and adjacent tissues were carefully scanned. Information on the location, size, number, echogenicity, composition, morphology, margin, and calcification of the thyroid nodules was recorded. When evaluating nodules, each nodule feature was assigned a score according to the ACR-TIRADS guidelines criteria [13], as follows: (1) composition: cystic or almost completely cystic 0 points; spongy 0 points; mixed cystic and solid 1 point; solid or almost completely solid 2 points; (2) echo: no echo 0 points; high echo or equal echo 1 point; low echo 2 points; very low echo 3 points; (3) shape: wider than high 0 points, taller than wide, 3 points; (4) edge: smooth 0 points, unclear edges 0 points, lobulated or irregular 2 points, extrathyroidal extension 3 points; and (5) echo focus: no or large comet tail artifacts are scored as 0 points, coarse calcifications are scored as 1 point, peripheral calcifications are scored as 2 points, and punctate strong echoes are scored as 3 points.

Based on the ultrasound images of each thyroid nodule, the ACR-TIRADS score was used to analyze the sonographic characteristics of benign and malignant thyroid nodules and different pathological classifications [13]. Pairwise analysis and comparison between groups was performed on the 4 pathological types of malignant thyroid nodules according to the ACR-TIRADS scoring criteria. Finally, the diagnostic ability of the ACR-TIRADS score on benign and malignant thyroid nodules and different pathological classifications was determined through the area under the receiver operating characteristic (ROC) curve (AUC) and cut-off value.

DATA ANALYSIS:

SPSS (version 23) was used for statistical analyses. Descriptive variables are displayed as the mean±standard deviation values, and categorical variables are displayed as a percentage. The measurement data were compared by independent-sample t tests, and counting data were compared by the chi-square test. The differences between the groups were compared and analyzed by a nonparametric test (Kruskal-Wallis rank-sum test). That is, the ACR-TIRADS score was used to conduct pairwise comparisons between groups of 4 different pathological classifications of malignant thyroid nodules to evaluate whether the differences between different groups were statistically significant. The AUC of ROC curves was used to judge the diagnostic efficacy of this score for benign and malignant thyroid nodules and malignant nodules of different pathological types. The Youden index can be calculated based on sensitivity and specificity (Youden index=sensitivity+specificity-1). The best cut-off, sensitivity of and specificity were obtained according to the Youden index. A value of P<0.05 was considered to indicate statistical significance. When constructing the prediction model, univariate logistic regression analysis was first performed, and then significant variables with P values less than 0.1 were included in the multivariable logistic regression analysis. R-STUDIO software (version 4.1.3) was used for statistical analysis, and P<0.05 was considered statistically significant.

Results

PATIENT SCREENING:

In this study, 1675 thyroid nodules in 1614 patients (1235 women and 440 men; average age=49.1±12.91 years; age range=12–86 years) were involved. Among them, 846 thyroid nodules were benign, including 637 nodular goiters, 191 adenomatous hyperplasia nodules, and 18 adenomas. However, 829 thyroid nodules in 815 patients were malignant, including 631 PTC, 96 MTC, 81 FTC, and 21 ATC. The benign and malignant nodules varied significantly in terms of patient age, maximum diameter, location, composition, echogenicity, morphology, margin, calcification, ACR-TIRADS score, and classification (P<0.05). No differences were found by sex (P>0.05). The details are shown in Table 1.

ULTRASONIC FEATURES OF MALIGNANT THYROID NODULES OF DIFFERENT PATHOLOGICAL TYPES:

According to the ultrasonic features of malignant thyroid nodules of the 4 pathological types, taller-than-wide shape was commonly found in PTC (384, 60.9%). Mixed cystic and solid, and hyperechoic or isoechoic were common proportions in FTC (Figure 1), at 9.9% and 4.9%, respectively. Fine dot-like calcifications and extreme hypo-echogenicity were more common in MTC (Figure 2), at 39.6% (n=38) and 16.7% (n=16), respectively. ATC was characterized mainly by extracapsular thyroid invasion, in 47.6% (n=10), as detailed in Table 2.

COMPARISON OF MALIGNANT THYROID NODULES OF 4 PATHOLOGICAL TYPES BASED ON THE ACR-TIRADS CRITERIA:

Table 3 describes the distribution of ACR scores in 4 pathological types of thyroid malignant nodules. Based on the results of pairwise comparison with benign nodules at the controls between groups, we concluded that the ACR scores of FTC were significantly different from those of PTC, MTC, and thyroid failure (P<0.05). The difference in differentiated cancer (ATC) was statistically significant, P<0.05; the difference in ACR-TIRADS scores between PTC and MTC was also statistically significant (P<0.05; Figure 3, Table 4).

COMPARING DIAGNOSTIC ACCURACY OF ACR-TIRADS SCORE FOR BENIGN AND MALIGNANT THYROID NODULES AND MALIGNANT NODULES OF DIFFERENT PATHOLOGICAL TYPES BASED ON AUC:

As shown in Figure 4, the AUC of the ACR score for the diagnosis of malignant thyroid nodules was 0.942 (95% confidence interval [CI]: 0.932, 0.953). The AUC for the diagnosis of PTC was 0.955 (95% CI: 0.946, 0.965), the AUC for the diagnosis of MTC was 0.914 (95% CI: 0.886, 0.942), the AUC for the diagnosis of FTC was 0.877 (95% CI: 0.843, 0.912), and the AUC for the diagnosis of undifferentiated carcinoma was 0.952 (95% CI: 0.917, 0.988). For all thyroid nodules, the Youden index, cut-off, sensitivity of and specificity of ACR score in the diagnosis of thyroid malignant nodules and different pathological types are shown in Table 5.

LOGISTIC REGRESSION SCREENING VARIABLES AND CONSTRUCTING PREDICTION MODELS:

Through single-factor and multi-factor logistic regression analysis, it was determined that age, max_size, composition, echogenicity, margin, point, and TIRAD risk score (TR) were independent predictors of malignant thyroid nodules (Table 6). Composition, echogenicity, margin, echogenic_foci, point, and TR were independent predictors of PTC (Table 7). Composition, echogenicity, margin, shape, and TR were independent predictors of MTC (Table 8). Max_size, composition, echogenicity, margin, and TR were independent predictors of FTC (Table 9). Age, max_size, margin, and TR were independent predictors of ATC (Table 10). The predictors were jointly used to construct a prediction model and were combined with the prediction model constructed using the ACR-TIRADS score.

The AUC of ROC curve of the thyroid malignant nodule prediction model constructed by combining predictors was AUC of 0.961 (95% CI: 0.952–0.969), with sensitivity of 0.924, and specificity of 0.877. The AUC of the model constructed using the ACR-TIRADS score was 0.942 (95% CI: 0.932–0.952), with sensitivity of 0.929, and specificity of 0.826 (P>0.05).

The AUC of ROC curve of the PTC prediction model constructed by combining predictors was 0.966 (95% CI: 0.958–0.975), with sensitivity of 0.957, and specificity of 0.865. The AUC of the prediction model constructed using ACR-TIRADS scores was 0.955 (95% CI: 0.946–965), with sensitivity of 0.930, and specificity of 0.875 (P=0.0001).

The AUC of ROC curve of the MTC prediction model constructed by combining predictors was 0.939 (95% CI 0.915–0.964), with sensitivity of 0.975, and specificity of 0.729. The AUC of the prediction model constructed using ACR-TIRADS scores was 0.914 (95% CI: 0.887–940), with sensitivity of 0.880, and specificity of 0.760 (P>0.05).

The AUC of ROC curve of the FTC prediction model constructed by combining predictors was 0.891 (95% CI: 0.858–0.923), with sensitivity of 0.723, and specificity of 0.914. The AUC of the prediction model constructed using scores was 0.877 (95% CI: 0.844–0.910), with sensitivity of 0.647, and specificity of 0.951 (P>0.05).

The AUC of ROC curve of the ATC prediction model constructed by combining predictors 0.982 (95% CI: 0.969–0.996), with sensitivity of 0.907, and specificity of 0.99. The AUC of the prediction model constructed using scores was 0.952 (95% CI: 0.917–0.987), with sensitivity of 0.930, and specificity of 0.857 (P=0.02).

Discussion

Thyroid nodules have a prevalence of up to 65% in the general population. Hence, appropriate ultrasonic examination is a key step to effectively monitor and manage this disease. Ultrasound examination is the first choice in the evaluation of thyroid nodules and is the primary tool for cancer risk stratification and in ascertaining the need for fine-needle biopsy [1,26]. The ACR-TIRADS risk stratification system enables assessment of all thyroid nodules [27,28]. The system is relatively simple to use and can be applied in practices with a varying number of patients with thyroid nodules as well as in practices with differing levels of expertise [29]. A comparative analysis of the 4 risk stratification systems, namely, ATA, ACR-TIRADS, EU-TIRADS, and KWAK-TIRADS, showed that ACR-TIRADS exhibited optimum diagnostic performance (AUC=0.879) and specificity [30]. The best algorithm was selected among the 5 thyroid nodule ultrasound scores (KTIRADS, ATA, AACE/ACE-AME, EU-TIRADS, and ACR-TIRADS), and the ACR-TIRADS total score demonstrated the highest accuracy, at 0.647 (95% CI: 0.625–0.669) [31]. Although the ACR-TIRADS ultrasound risk stratification system is primarily designed to detect PTC, it also appears to provide reliable recommendations for fine-needle biopsy testing in FTC [23] and MTC [25]. Jin et al [32] used a score of 4.5 as the optimal cut-off value and obtained an accuracy of 75.6%, sensitivity of 85.0%, and specificity of 71.6% in predicting malignant tumors. In our study, the specificity was comparatively higher. It has also been reported that with a cut-off value of 4, the accuracy, sensitivity, and specificity were 87.56%, 92%, and 83.49%, respectively [33]. Using ACR-TIRADS criteria, Hoang et al unanimously recommended biopsy in 55 of 100 nodules, with sensitivity, specificity, and accuracy of 87%, 51%, and 56%, respectively [34]. Considering the high relative prevalence of FTC, MTC, and undifferentiated carcinoma in our study, compared with that of other studies, the results could have differed due to interobserver heterogeneity and differences in study populations and sample sizes.

Reports have stated that fine-needle aspiration will not be performed based on the ultrasound patterns in at least 31% of FTC cases. As cytology has certain limitations, great caution should be exercised in using ultrasound as a follow-up tool for indeterminate nodules [30,35]. While fine-needle aspiration can accurately diagnose PTC, FTC is always classified in the uncertain fine-needle aspiration category [36]. Despite the presence of some malignant signs in FTC, this classification system cannot be fully applied to predict FTC [21,37]. In our study, the AUC for the diagnosis of FTC was the lowest, at 0.877 (95% CI: 0.843, 0.912), and the specificity was also only 64.8% when the cut-off score was 3.5. This result can be related to its ultrasonic features, which were found in 9.9% and 4.9% of the patients with cystic solid and hyper-/iso-echogenicity in our study, from the 4 pathological types. This observation is also in line with the claim that cystic changes are associated with FTC [21]. Although the diagnostic specificity and Youden index of ultrasound images are not high, a study has combined preoperative thyroglobulin levels and other risk factors to create a model for predicting FTC, in which the accuracy, sensitivity, and specificity were improved to 89.2%, 90.2%, and 87.7%, respectively [24]. This finding demonstrates the need for better predictive models to diagnose FTC. In addition, in our study, we found that the main features of MTC were extremely hypoechoic in 16.7% (16) and fine dot-like calcifications in 39.6% (38) of cases, which is consistent with the fact that one-third of all MTC cases analyzed by Kim et al [38] exhibited ultrasonic features similar to the benign nodules. Furthermore, it has been reported that the sensitivity of ultrasound for the diagnosis of MTC was 75.3%, specificity was 93.1%, and the overall accuracy was 80.4% [39]. Hence, the sensitivity was comparable to that of our study, but the accuracy in our study was higher, suggesting that our system was reliable in detecting MTC. In our study, we used multiple indicators to build a diagnostic MTC prediction model and used the ACR-TIRADS total score to build a prediction model. The AUC was 0.939 and 0.914, the sensitivity was 0.975 and 0.880, and the specificity was 0.729 and 0.760, respectively. The prediction model can produce a higher sensitivity in diagnosing MTC and was more conducive to screening out positive nodules; however, the specificity was low, which can be due to different sample sizes and selection bias.

ATC is an aggressive but rare malignant thyroid tumor that accounts for only 1% to 2% of all thyroid cancers [40]. However, this subtype is associated with a poor prognosis, and the therapeutic options are limited [41]. Survival remains low regardless of surgery, radiotherapy with or without chemotherapy, or palliative and symptomatic treatment [42,43], and the underlying mechanism is variable, from signaling pathways to target gene regulatory networks [44–46]. Nearly 50% of ATCs occur pathophysiologically in the context of preexisting or concurrent differentiated thyroid carcinoma, suggesting that a significant proportion of ATC is caused by the dedifferentiation of preexisting differentiated thyroid carcinoma into a more aggressive phenotype. Hence, the diagnostic efficacy of ATC on a graded scale is close to that of differentiated thyroid carcinoma, which is in line with the results of the present study. The main features of ATC in our study were extracapsular thyroid invasion in 47.6% (10) of cases, which is consistent with the fact that patients with ATC usually present with a rapidly enlarging neck mass and clinical symptoms related to the compression of local structures (eg, neck vessels, esophagus, or trachea) [47]. Also, some studies have shown that in the evaluation of the accuracy of ACR-TIRADS, the diagnostic accuracy of PTC is the highest, although the reliability of detecting MTC, FTC, and other malignant tumors is not high. It has therefore been suggested to modify the mode and cut-off point of fine-needle aspiration or to combine ultrasound with other techniques [48]. Multimodal ultrasound examinations, such as combined contrast and elastography, can be recommended clinically for patients with suspected malignant thyroid nodules [49], or other techniques, such as molecular gene modification can be recommended [50]. In summary, our study suggested a valuable diagnostic efficiency of PTC based on the ACR-TIRADS score, which will be very useful in the clinic for the pathological classification of malignant nodules in patients.

Conclusions

Thyroid ultrasound examination based on the ACR-TIRADS score is helpful in the diagnosis of benign and malignant thyroid nodules, and has certain diagnostic value for different pathological types of malignant thyroid nodules. Although this score is more conducive to the diagnosis of PTC and ATC, the prediction model constructed through multiple indicators has better diagnostic performance. The prediction model for diagnosing FTC and MTC constructed through multiple indicators was compared with the model constructed based on the ACR-TIRADS total score. The diagnostic efficiency was improved, but the difference was not statistically significant. Due to the imbalance of data, prospective, multi-center studies are needed to verify the results obtained.