External Quality Assessment of Maternal Serum Levels of Alpha-Fetoprotein, Free Beta-Human Chorionic Gonadotropin, and Unconjugated Estriol in Detecting Down Syndrome and Neural Tube Defects in the Second Trimester of 87 Maternal Serum Samples, Based on 105-139 Days

Background

Alpha-fetoprotein (AFP), free beta-human chorionic gonadotropin (free β-hCG), and unconjugated estriol (uE3) are used in the maternal prenatal screening of trisomy 21 (T21, also known as Down syndrome) and neural tube defects (NTD) in the second trimester of pregnancy [1–3]. A meta-analysis showed that double and triple tests involving AFP, uE3, total hCG, and free β-hCG were significantly better than single marker tests. When the false positive rate (FPR) is 5%, 60% to 70% of Down syndrome could be detected [4]. Fetuses with T21 are relatively immature compared with fetuses without T21, so the level of hCG will be higher than expected, while the levels of AFP and uE3 will be lower than expected [5]. Studies have shown that the best time to measure maternal serum AFP for detecting NTD is at 16 to 18 weeks of pregnancy [6]. Individuals with Down syndrome, which is the most common chromosomal abnormality, have a significantly higher risk of congenital cardiac disease, Alzheimer disease, and autoimmune disease [7]. It has been demonstrated that the prevention, screening, and diagnosis of NTD can significantly improve the outcome of mothers and infants. However, mothers and infants can be harmed if the patient refuses routine ultrasound scans or delays critical care milestones for T21 and NTD screening, adversely affecting the patient, infant, patient’s family, and healthcare system [8].

Prenatal screening findings can affect the decision of whether to continue a pregnancy [9]. As such, it is the physician’s responsibility to provide sufficient information so that an informed decision can be made by the patient and the patient’s family. Thus, prenatal screening has an essential role in identifying common genetic disorders [10–12].

Clinical research laboratories require high-quality and high-quantity specimens to generate reliable test results [13]. Quality assurance refers to the steps taken to ensure the reliability and improve the accuracy of the test results according to national and international laws as well as other standards and guidelines. These steps include internal quality control procedures and external quality assessment (EQA) methods that involve the pre-analytic process, target analyte type, specimen type, platform type, test components, and positive and negative controls [14,15].

To ensure the accuracy and the reproducibility of the test results, EQA should be implemented from the pre-analytic stage to the analytic stage [16]. To examine the accuracy and the reproducibility of T21 and NTD screening results across 6 laboratories using the same instrument and the same analysis, we measured the concentrations of AFP, free β-hCG, and uE3. The results and risk values were compared and studied to provide a foundation for a standardized management system for maternal prenatal screening in Hangzhou.

Material and Methods

SCREENING LABORATORY CONDITIONS:

The 6 laboratories were designated as A, B, C, D, E, and F. Laboratory A had the largest sample size and was one of the earliest laboratories in the region, which was identified by the regional pre-production screening Quality Control Management Center. The laboratory quality assessment results of the national inspection center are qualified every year. Thus, laboratory A was the reference laboratory with which all other laboratories were compared. The second trimester maternal serum levels of AFP, free β-hCG, and uE3 were measured.

REAGENTS AND INSTRUMENTS:

The 1235 automatic time-resolved fluorescence analyzer (PerkinElmer, Shelton, CT, USA) and Lifecycle 4.0 software were used in all laboratories. An AFP/free β-hCG double-labeling kit and uE3 reagent supporting Wallac software (Wallac Oy, Turku, Finland) were used in conjunction with quality control samples (batch no. UF2181101, Hangzhou Biozone Technology Co., Ltd., Hangzhou, China).

INCLUSION AND EXCLUSION CRITERIA:

There were 87 randomly selected cases of prenatal screening in laboratory A. The inclusion criteria were single pregnancy and voluntarily serological screening in the second trimester. There were specimens from pregnant women from gestational age of 119 (105–139) days, maternal age of 28.13 (19.52–39.12) years, and maternal weight of 57 (44.52–74.20) kg. Among them, there were 4 women with serum samples with T21 and 5 with NTD and 78 were normal. The exclusion criteria were as follows: double or multiple pregnancies, smoking, diabetes, history of chromosomal abnormalities and congenital malformation, in vitro fertilization, and pregnancy complicated with medical and surgical diseases. The study was approved by Hangzhou Women’s Hospital [2022] Medical Ethics Review K (1)-09. Patient consent was not required because this was a retrospective study.

COMPARISON OF SPECIMEN SOURCES:

Serum samples from 78 women with normal pregnancies were collected from May to August 2018, and serum samples from 4 pregnant women with T21 and 5 pregnant women with NTD were collected from September 2016 to June 2017. Eighty-seven serum specimens collected from women at 15 to 20 weeks and 6 days of pregnancy, which had previously undergone measurement of AFP, free β-hCG, and uE3 concentrations were randomly selected and briefly stored at −80°C. After thoroughly thawing and mixing, each specimen was divided into six 250 μL fractions, placed into an ice box, and transported to the 6 laboratories at 2 to 8°C for testing.

DETERMINATION OF MULTIPLE OF THE MEDIAN VALUES FOR AFP, FREE β-HCG, AND UE3:

We determined the multiple of the median (MoM) values for maternal weight and gestational age at testing to replace the concentrations of AFP, free β-hCG, and uE3 to reduce the deviation caused by gestational age and maternal weight. In brief, the MoM values were normalized to maternal weight and gestational age at testing [17] and subsequently adjusted according to the median equation of each laboratory.

The definition and calculation of MoM value was:

where Original Conj. was the original concentration values of AFP, free β-hCG, and uE3 and Median represented the median of the original concentration value of the corresponding indicator.

QUALITY CONTROLS:

All 6 instruments were maintained by trained users on a daily, weekly, and monthly basis, and the manufacturer provided unified quarterly maintenance to make each instrument reach the best state. Also, 3 different quality control serum specimens were conducted as part of internal quality control in the 6 laboratories. Each laboratory was required to participate in the EQA activities organized by the Clinical Laboratory Center of China Health Commission every year. In addition, all personnel were required to show proof of unified pre-job training and completed qualification certificates from health authorities.

EVALUATION STANDARDS:

All laboratories used Life Cycle 4.0 software to calculate the risk values of T21 and NTD based on maternal age, gestational age at testing, and maternal weight. The cut-off values according to the 2010 China Ministry of Health Guidelines for maternal prenatal screening in the second trimester [18–19] were as follows: high risk (T21 ≥1: 270, T18 ≥1: 350, NTD AFP MoM ≥2.50) and low risk (T21 <1: 270, T18 <1: 350; NTD AFP MoM <2.50).

PRENATAL DIAGNOSIS:

The indications were as follows: pregnant women over 35 years old with high risk of fetal chromosomal abnormalities screened before delivery and who had given birth to children with chromosome diseases; pregnant women with suspected fetal chromosomal abnormalities by prenatal ultrasound examination; the husband and wife were carriers of chromosomal abnormalities; doctors considered it necessary to carry out prenatal diagnosis in other cases. Pregnant women at 17 to 20 weeks of pregnancy underwent ultrasound-guided amniocentesis with informed consent as follows. After placing of the amniotic fluid tank, a 20 G puncture needle was used to collect 30 mL of amniotic fluid from the abdomen and place it in a sterile special centrifuge tube for examination. It was then centrifuged at 2500 r/min for 10 min, cell microspheres were cultured (37°C, 5% CO2), and cell growth was observed. Thirty mitotic phases were counted under the microscope, 5 karyotypes were analyzed, 400 G-banded chromosomes were produced, and karyotypes were analyzed according to the international nomenclature system of human genetics. The number of abnormal karyotypes was increased, and C and N bands were added if necessary [20]. Ultrasound diagnosis was performed in accordance with the guidelines for prenatal ultrasonography [21].

STATISTICAL ANALYSIS:

R software (version 4.0.3; https://www.R-project.org) was used for drawing, statistical data analysis, and visualization. Lilliefors (1967) modified test (Shapiro-Wilk test) was used to determine whether the data was normally distributed. AFP and free β-hCG concentrations showed a skewed distribution; therefore, they were expressed as medians and percentiles (M [P_2.5, P_97.5]). Skewed data was compared with the laboratories that used the Kruskal-Wallis H (K) test or the Mann-Whitney U test. The linear correlation analysis of MoM values was done with Pearson linear correlation, whereas the high-risk positivity rate and the detection rate was analyzed using the paired-sample chi-squared test (McNemar test). The Kappa consistency test was used to analyze the accuracy and the reproducibility across the different laboratories. P values <0.05 indicated that the difference was statistically significant. The sensitivity, specificity, positive predictive value, negative predictive value, FPR, and false negative rate (FNR) were used to evaluate the screening efficacy of T21 and NTD. To reduce the cross-correlation of AFP, free β-hCG, and uE3 concentrations between T21 and NTD, the corresponding cases were excluded when the detection rate was calculated.

Results

COMPARISON OF DIFFERENT LABORATORY SCREENING RESULTS:

The 6 laboratories analyzed 87 serum specimens. The lowest AFP concentration (30.52 [22.96–44.42] U/mL) was reported by laboratory F, and the difference was statistically significant (P<0.01) (Figure 1A). There were no statistically significant differences in free β-hCG and uE3 concentrations among the 6 laboratories (P>0.05) (Figure 1B, 1C). The MoM values for AFP, free β-hCG, and uE3 were also determined by the 6 laboratories (Figure 2). The MoM values for AFP and uE3, as detected by laboratory F, were the lowest, and the differences were statistically significant (P<0.001; P=0.008) (Figure 2A, 2C). However, there was no statistically significant difference in the free β-hCG MoM values among the 6 laboratories (P>0.05) (Table 1, Figure 2B).

CORRELATION ANALYSIS OF THE MOM VALUES FOR AFP, FREE β-HCG, AND UE3:

The MoM values for AFP, free β-hCG, and uE3, as determined by the other 5 laboratories were comparable to those determined by laboratory A, with good linear correlation. The correlation coefficients for the 3 MoM values were all >0.900 (P<0.05) (Figures 3–5). Whereas the correlation coefficient for the uE3 MoM value was the lowest at 0.834, as detected by laboratory F (Table 2, Figure 5; Supplementary Tables 1–3).

COMPARISON OF THE T21 POSITIVITY RATE AND DETECTION RATE:

Laboratories A and F detected the most cases of high-risk T21, with 6 (7.32%) and 7 (8.54%) cases, respectively, whereas the other 4 laboratories detected 4 cases (4.88%) each (Table 3). The McNemar paired chi-squared test showed that there was no significant difference between the T21 positivity rate and clinical diagnosis (P>0.05). The kappa consistency test was used to analyze the consistency of the different laboratory results. For laboratories B, C, D, and E, the experimental results had good consistency (kappa=0.737); however, for laboratories A (kappa=0.575) and F (kappa=0.515), the experimental results had mediocre consistency (Table 4).

COMPARISON OF THE NTD POSITIVITY RATE AND DETECTION RATE:

McNemar’s paired chi-squared test showed that laboratories C and F detected 1 case (1.20%) each of high-risk NTD, which was less than the 3 cases (3.61%) detected by the other 4 laboratories. However, there were no statistically significant differences in the positivity rate and detection rate of NTD (P>0.05) (Table 5). For laboratories A, B, D, and E, the experimental results had good consistency (kappa=0.738); however, for laboratories C (kappa=0.320) and F (kappa=0.320), the experimental results had poor consistency (Table 6).

COMPARISON OF THE T21 AND NTD SCREENING EFFICACY:

The highest FPR for T21 screening was detected by laboratory F (5.13%), followed by laboratory A (3.85%) and the other 4 laboratories (1.28%) (Table 7). For NTD, the lowest sensitivity of NTD screening were detected by laboratory C (20.00%); the highest FNR of NTD screening were detected by laboratory F (80.00%).

Discussion

EQA IN DIFFERENT LABORATORY DETECTION SYSTEMS:

The 6 laboratories in this study determined and calculated the risk of T21 and NTD using maternal serum specimens in which the concentrations of AFP, free β-hCG, and uE3 were measured. EQA can objectively and retrospectively compare the results of different laboratories through proficiency testing, which is maintained by external institutions. The main purpose of EQA is to establish a robust comparability between laboratories and methods and to reach agreement with established reference values [22].

The automatic time-resolved fluorescence analyzer used in this study employed a standardized protocol of loading specimens and reagents uniformly, incubating and shaking specimens for fixed periods of time, washing plates consistently, and adding equivalent volumes of enhancement reagent [23]. Yee et al reported that it is best to define as many parameters as possible, such as the pre-analytic process, specimen type, platform type, and test components, as well as positive and negative controls [24]. A previous study showed the 5420 pregnant women were enrolled for triple test investigations in the second trimester, which can be used as reference population – specific median values in India [25]. EQA is often used in the determination of biomarker concentrations, which is critical in the diagnosis and prognosis of certain diseases, such as T21 and NTD, as well as in the prediction of treatment outcomes [26,27]. Our results revealed that the AFP concentration and AFP MoM value, as determined by laboratory F, were the lowest among the 6 laboratories, and the difference was statistically significant. This might be related to the newer version of the instrument system tested in the laboratory and the relatively small daily sample size of the laboratory.

COMPARISON OF AFP, FREE β-HCG, AND UE3:

Although there is an international reference value for AFP, the AFP detection method across different laboratories is different, which renders the reference value useless in analysis. As such, laboratories should use quality control serum specimens that are not included in commercially-available EQA kits [28]. The simultaneous quantification of AFP and free β-hCG levels is common in the screening of T21 in the second trimester of pregnancy, and its efficacy is assessed by the FPR. Any bias in MoM values may result in the incorrect application of median equations, and regular monitoring of these indicators may be essential for quality control in prenatal screening [29]. Mannings et al reported that the AFP concentration, as measured by the automatic time-resolved fluorescence analyzer, can indeed lower the AFP concentration due to complement interference, which can increase the calculated risk of T21 [30]. Our results showed that there was no statistically significant difference in the free β-hCG MoM value determined by the 6 laboratories. However, it is important to mention that laboratories implementing the new platform, which employs the new assay, will need to determine new reference values (medians), although this is not expected to have a marked effect on the maternal prenatal screening of T21 [31].

There was no statistically significant difference in the uE3 concentrations measured by the 6 laboratories. However, the uE3 MoM value determined by laboratory F was the lowest, and the difference was statistically significant, indicating that the MoM value, which was normalized against maternal weight and gestational age at testing, was more accurate than the concentration. Thus, it is necessary to simultaneously monitor 2 or more assays, which can be easily accomplished by monitoring the MoM values for each assay performed in the maternal prenatal screening of T21. The monitoring of MoM values can identify factors that inappropriately alter critical variables. For instance, the integration of MoM values can increase the detection rate when interpreting the test results of T21 screening [32,33].

For specific individuals, the use of localized parameter calculations can be more accurate because these calculations are significant in maternal prenatal screening [34]. Nix et al reported that a 10% bias in a single marker can lead to a 1% to 2% increase in the FPR and that quality control measures should be established to control the magnitude of the bias associated with the MoM value or the magnitude of the bias associated with the calculation of patient-specific risks [35]. The uE3 MoM value was 1.05, as determined by laboratory C, which was higher than that of the other 5 laboratories. This might have been related to the fact that the uE3 assay was just performed and the median equation for uE3 was not yet adjusted by the 6 laboratories.

COMPARISON OF T21 AND NTD DIAGNOSED CASES:

The MoM values for AFP, free β-hCG, and uE3 determined by the 5 laboratories were comparable with those determined by laboratory A, with good linear correlation. The correlation coefficient was >0.900, and the difference was statistically significant, indicating that the test results met the clinical requirements. Laboratories A and F detected the most cases of high-risk T21. The positivity rate of T21, as determined by laboratories A and F, was higher than that of the other 4 laboratories, whereas the positivity rate of NTD, as determined by laboratories C and F, was lower than that of the other laboratories, although the differences were not statistically significant. Furthermore, the results of the kappa consistency test revealed that the findings of most laboratories had good consistency. However, the accuracy and the reproducibility of the maternal prenatal screening for T21 in the first trimester in China should be further improved [36].

It is not advisable to improve the accuracy and the reproducibility of test results by requiring that all laboratories participate in EQA programs, although such programs may quickly identify issues with the procedures or the methods of laboratories. Inaccurate and non-reproducible test results should be promptly addressed by appropriate measures [37,38]. For instance, EQA monitoring can identify issues that are not readily detected with conventional quality control measures. There are different calibration system errors and individual differences of operators in different laboratories to facilitate the homogenization management of prenatal screening reports among different laboratories. As the quality control management center in the region, we compared the results of 6 laboratories using the same samples and laboratory equipment.

LIMITATIONS:

This study had some limitations. It was limited to 87 serum samples, including only 4 confirmed cases of T21 and 5 confirmed cases of NTD, while cases of T18 were not included. There may be bias in the comparison of AFP, free β-hCG, and uE3 among the 6 instruments based on such a small sample. Further validation will be done by increasing the sample size in later studies.

Conclusions

In summary, the MoM values of AFP, free β-hCG, and uE3, as determined by the 5 laboratories, were comparable with those determined by laboratory A, with good linear correlation. The correlation coefficient was >0.900, and the difference was statistically significant, indicating that the test results met the clinical requirements, except that the kappa consistency of a few laboratories was poor. After improving the quality control of various factors that can affect test results, the median equation calibration work should be performed according to the comparison results to reduce all errors to their lowest levels. The values of AFP, free β-hCG, and uE3 measured in 5 laboratories were comparable to those measured in laboratory A and had good linear correlation. The results of prenatal screening for pregnant women with T21 and NTD could meet the clinical requirements of their institutions.