02 March 2021: Clinical Research
Comparison of the Performance of 24 Early Warning Scores with the Updated National Early Warning Score (NEWS2) for Predicting Unplanned Intensive Care Unit (ICU) Admission in Postoperative Brain Tumor Patients: A Retrospective Study at a Single CenterLingli Peng12AEFG, Zhen Luo3CE, Lingling Liang4BG, Mingming Liu3B, Lingyao Meng3B, Jianwen Tan5BB, Lili Song3BF, Yan Zhang3AD*, Lixiang Wu1AD
Med Sci Monit 2021; 27:e929168
BACKGROUND: There have been few studies to evaluate early warning score (EWS) systems, or track and trigger systems (TTS), to identify early clinical deterioration in patients following brain tumor surgery who are admitted to the Intensive Care Unit (ICU). The National Early Warning Score (NEWS2) is an established method used in the U.K. National Health Service to improve care for in-hospital patients. This retrospective study from a single center aimed to compare the performance of NEWS2 with 24 other types of EWS to evaluate unplanned ICU admissions within 72 h after brain tumor surgery.
MATERIAL AND METHODS: A total of 326 patients with brain tumors were included in the study. Patients who experienced unplanned ICU transfer after surgery (69 cases) were diagnostically matched with patients who did not require intensive care (257 controls). We collected the physiological variables to calculate the area under the receiver operator characteristic curve (AUROC), sensitivity, specificity, Youden index values, cutoff values, positive predictive values, and negative predictive values.
RESULTS: The NEWS2 identified postoperative brain tumor patients with AUROC (0.860, p=0.000). The Patient-At-Risk (PAR) score was higher than NEWS2 in terms of AUROC value (0.870, P=0.000), Youden index (0.589 vs 0.542).
CONCLUSIONS: The findings showed that although the NEWS 2 performed well when used to evaluate unplanned ICU admissions within 72 h of postoperative brain tumor patients, the PAR score was also an accurate EWS.
Keywords: Decision Support Techniques, Intensive Care Units, Aged, Area Under Curve, Case-Control Studies, Critical Care, Early Warning Score, Forecasting, Hospital Mortality, Hospitalization, Middle Aged, Patient Admission, Postoperative Complications, Predictive Value of Tests, ROC Curve, Retrospective Studies, Risk Assessment, Risk Factors, Severity of Illness Index
Postoperative patients with brain tumors in general hospital wards may suffer clinical deterioration due to underlying conditions [1,2]. Unnoticed patient deterioration and failure to intervene early may then cause severe adverse events (SAEs) , such as cardiac arrest or unplanned admissions to the Intensive Care Unit (ICU), which are associated with prolonged hospital stays, unfavorable prognosis, and even death [4,5]. Of all patients, however, those undergoing neurologic surgery have more than twice the average risk of perioperative mortality, according to Noordzij et al . Even upon successful surgery, brain tumor patients may experience clinical deterioration within 72 h after surgery , so early recognition and timely intervention are essential for preventing in-hospital SAEs in this patient population.
According to one study’s statistics, 368 117 primary brain and other central nervous system tumors were diagnosed in the United States from 2009 to 2013 . However, the characteristics of severe illness and rapid disease progression in postoperative brain tumor patients, along with some patients’ unconsciousness or unclear communication, make it difficult to detect patient deterioration [7,9]; consequently, SAEs can occur, which may contribute to a high mortality rate among these patients . Nuño et al found that the mortality rate of brain tumor patients who suffer postoperative SAEs increases by 7.6 times , and Gold et al similarly reported that the in-hospital mortality rate associated with unplanned neurological ICU transfers was 17% . These data thus urgently call for enhanced screening of these patients.
Despite the difficulties identifying them, research has shown that SAEs do not happen suddenly or are impossible to detect . Some surveys have even suggested that about one-third of potentially preventable deaths in the U.K. involve poor clinical monitoring . Substantial evidence from many studies has indicated that there are changes in physiological parameters in the hours preceding clinical deterioration [12,13]; therefore, SAEs could be prevented if changes in physiological parameters are monitored . In response, early warning scores (EWSs), or track and trigger systems (TTS), using several physiological variables and established thresholds have been designed to identify patients at risk of deterioration at an early stage and have been adopted worldwide . Gerry et al pointed out that due to the limitations in methods and reports of previous validation studies on various EWSs, the effect of various EWSs in clinical practice is inconsistent with that reported in the literature . Therefore, based on the existing evidence, it is impossible to determine which EWS is the most effective and which is the most suitable for postoperative patients with intracranial tumors.
The National Early Warning Score is one of the most commonly used early warning system in the world . In December 2017, the Royal College of Physicians updated the National Early Warning Score (NEWS) to the National Early Warning Score 2 (NEWS2) to standardize acute illness severity assessment in hospitals . The NEWS2 includes 7 physiological parameters: respiratory rate, oxygen saturation, supplemental oxygen, systolic blood pressure, pulse, temperature, and consciousness level (Table 1). Based on NEWS, NEWS2 added a new assessment point: new confusion or delirium, to the alert, verbal, pain, unresponsive (AVPU) scale for the alert, new confusion, or delirium, voice, pain, unresponsive (ACVPU) scale; and NEWS2 set a new SpO2 scale for patients with known hypercapnic respiratory failure by adjusting oxygen saturation to 88–92% .
NEWS/NEWS2 is mainly used in hospitalized patients, but also has been evaluated in emergency and ambulance environments [19,20]. The application of NEWS/NEWS2 in patients with sepsis, acute pancreatitis, and liver disease has been reported . Tan et al used NEWS to evaluate the predictive value of unplanned ICU transfer and death in patients with acute pancreatitis . Hydes et al used NEWS to evaluate the predictive value of unplanned ICU transfer, death, and cardiac arrest in liver disease patients within 24 h of observation . In addition to independently evaluating NEWS/NEWS2, Smith et al compared NEWS with 33 other EWSs to analyze their predictive value in unplanned ICU transfer, death, and cardiac arrest of emergency patients . Le Lagadec et al compared 12 EWSs, including NEWS, for predicting severe adverse events in in-hospital patients . Silcock et al compared the predictive value of NEWS and quick Sepsis-related Organ Failure Assessment for ICU transfer and death of emergency patients . Martín-Rodríguez et al reported the better performance of NEWS2 among 4 EWSs in predicting ICU admission of patients with brain trauma .
To the best of our knowledge, there have been few studies evaluating EWS systems to identify early clinical deterioration in patients following brain tumor surgery who are admitted to the ICU. The updated NEWS2 is an established method used in the National Health Service in the U.K. to improve care for in-hospital patients. Therefore, this retrospective study from a single center aimed to compare NEWS2 with 24 other types of EWSs to evaluate unplanned ICU admissions within 72 h after brain tumor surgery.
Material and Methods
Ethics approval was obtained from the Xiangya Hospital, Central South University Research Ethics Committee (201911778). The requirement for patient informed consent was waived because our study did not involve performing interventions. This research conformed to the provisions of the 1995 Declaration of Helsinki. Anonymity and confidentiality (access to records, data encoding, and information archiving) were guaranteed throughout the entire research process.
STUDY DESIGN AND SETTING:
This retrospective study followed a case-control design. The study was implemented in a university-affiliated tertiary hospital with 3500 beds in Hunan Province, China.
EARLY WARNING SCORE SELECTION:
Based on 33 EWSs reviewed by Smith et al , 24 EWSs were available when we searched the literature databases. In addition to the NEWS2, 24 other EWSs were assessed in our study (Table 2), including some commonly used EWSs like the Modified Early Warning Score , the NEWS , and the Patient-At-Risk (PAR) EWS [28,29].
All patients who were diagnosed with a brain tumor and had an unplanned ICU admission within the first 72 postoperative hours in the neurosurgical general ward from June 1, 2014 to June 1, 2019 were reviewed for eligibility. Patients who had an unplanned ICU admission in the operation theater, anesthesia recovery room, or emergency department were not considered, nor were children (<16 years of age) and pregnant patients during the study period. The included patients with unplanned ICU admissions (cases) were matched to those without unplanned ICU admissions (controls) with the same hospital, admission year, ward, sex, and admission diagnosis based on the International Classification of Diseases, Tenth Revision (ICD-10) .
Per the formula n=(Zα/2/δ)2 P (1–P), we determined that the NEWS2’s sensitivity and specificity in the literature are 0.907 and 0.578, respectively . The required case and control group sizes were estimated to be 60 and 83 samples, respectively, where α and δ were 0.05 and 0.10, respectively. We initially selected 92 cases and 271 controls. According to the exclusion criteria, 25 patients were excluded for being under 16 years of age, 9 were excluded for being transferred to the ICU from the emergency department, and 3 were excluded for being transferred to the ICU from the anesthesia recovery room. This yielded 69 cases and 257 controls for the study, which was adequate (see Figure 1). Of these 326 patients, 156 were male and 170 were female.
Demographic and diagnostic information and physiological variables were collected from patient electronic medical records. The worst physiological variables prior to ICU admission within 72 h after brain tumor surgery were collected. These variables included body temperature, respiration rate, oxygen saturation, supplementary oxygen mode, heart rate, blood pressure, urine output, and level of consciousness. Unanticipated ICU admissions were identified from the ICU admissions database. For patients who experienced multiple unplanned ICU transfers during the same hospitalization, only the first admission was included in the analysis. Only fully recorded physiological variables were analyzed, though if there were missing data during the data collection, the latest recorded data were taken, as per Parshuram et al . All data were collected and confirmed independently by 2 researchers.
The physiological parameters collected were applied to the NEWS2 and other EWSs. All statistical analyses were performed using IBM SPSS Statistics version 22.0 (IBM, Armonk, NY, USA). The demographic data were analyzed using simple descriptive statistics and an independent-samples
The area under the receiver operating characteristic (AUROC) curve was analyzed by receiver operating characteristic (ROC) analysis to assess the ability of EWSs to distinguish between patients at risk and not at risk of having unexpected ICU admissions within 72 h after surgery . An AUROC value of 0.7–0.8 is considered to have a fair discriminatory ability, and a value above 0.8 is considered good . The Youden index values, optimal cutoff values, sensitivity, specificity, positive predictive values (PPV), and negative predictive values (NPV) were then calculated for comparison. The Youden index indicates an EWS’s total ability to identify patients at risk of experiencing unplanned ICU admissions and patients who are likely to have uneventful hospital stays; the higher the value, the better the EWS’s effectiveness and the greater the authenticity. The Youden index’s maximum value was chosen as the optimal cutoff value; that is, the point closest to the AUROC curve in the upper left corner with the lowest total number of false positives and false negatives. Sensitivity indicates the EWSs’ ability to accurately identify deteriorating patients – that is, those with high EWS scores – who may experience unplanned ICU transfers. Specificity represents the EWSs’ ability to accurately identify patients whose condition is not deteriorating (ie, patients with low EWS scores who may not require an ICU transfer) . The PPV refers to the proportion of true positives in the total number of unanticipated ICU admissions the EWSs detected. The NPV represents the ratio of true negatives in non-deteriorating patients the EWSs identified. A value of P<0.05 was considered statistically significant.
A total of 326 patients were included, of which 69 were case patients; 156 (47.90%) were male and 170 (52.10%) were female. The case patients’ average age was significantly higher than that of the control patients [52±16.32 vs 48±12.72, year, P=0.007], and their length of stay (LOS) was significantly longer than the control patients’ [23±10.92 vs 15±5.86, day, P=0.000]. Data on 4 co-morbidities were collected, of which the most common was hypertension (case=27.5%, control=19.1%). Heart disease significantly differed between the case and control patients (10.1% vs 2.7%, P=0.018). In addition, 9 types of brain tumor were identified, meningioma being the most common (case=34.78%, control=16.34%; Table 3). There were also significant differences in oxygen saturation (92.7±12.81 vs 97.9±1.56, P=0.000), systolic blood pressure (131.2±27.25 vs 116.5±19.53, P=0.000), heart rate (111.9±26.78 vs 94.4±18.78, P=0.008), respiratory rate (21.3±7.70 vs 19.6±5.86, P=0.000), temperature (37.7±0.90 vs 37.1±0.73, P=0.001), and consciousness (84.1% vs 23.7%, P=0.000) between the 2 study groups (Table 4).
PROGNOSTIC SCORE ACCURACY:
The AUROC curve, calculated from the total 25 EWSs using case-control (ie, unplanned ICU admission or no unplanned ICU admission) as a criterion, ranged from 0.737 to 0.870 (Table 5). The AUROC value for the NEWS2 (0.860, P=0.000) was higher than that of the NEWS (0.852, P=0.000). The PAR EWS had the highest AUROC value (0.870, P=0.000), while centile-based EWS had the lowest value (P=0.000) .
The Youden index values of the NEWS2, NEWS, and PAR score were 0.542, 0.513, and 0.589, respectively, and their corresponding cutoff values were 7, 7, and 4. Using these values as thresholds, the sensitivity of these 3 EWSs scores were 0.768, 0.739, and 0.710, respectively, while their specificity scores were 0.774, 0.774, and 0.879. The NPV values of the NEWS2, NEWS, and PAR score were 0.926, 0.917, and 0.919, respectively (see Table 5).
To the best of our knowledge, this is the first study to investigate whether the NEWS2 specifically accurately identifies the risk of postoperative brain tumor patients’ unplanned ICU admissions. Our finding showed that NEWS2 performed well, with an AUROC of 0.860. The PAR score had a higher AUROC (0.870) than NEWS2.
Regarding unexpected ICU transfer predictions, we found that the NEWS2 and NEWS had greater discriminatory power in brain tumor patients than other EWSs. These results are similar to the findings of Smith et al for emergency patients within 24 h of ICU admission (AUROC=0.857)  and to the results of Martín-Rodríguez et al for brain trauma patients transferred to the ICU (AUROC=0.888) . The NEWS2 showed high sensitivity (0.768) and specificity (0.774), meaning that it can identify 76.8% of the patients at risk of being admitted to the ICU after brain tumor surgery and 77.4% of the patients not at high risk of clinical deterioration.
In contrast, Teasdale found that the NEWS was not suitable for assessing acute brain damage patients, arguing that its 2-tier AVPU system is inadequate for assessing consciousness level and that this system should include eye, verbal, and motor responses, limb movements, and pupil reactivity . This difference could be explained by the fact that 50.56% of the unplanned ICU admissions in our study were due to low oxygen saturation, and the SpO2 scale in the NEWS2 can be a good indicator of deterioration in postoperative brain tumor patients.
The major complications in postoperative brain tumor patients include hematomas and cerebral edema , which can lead to increased intracranial pressure and Cushing reflex. As a result, these patients’ consciousness worsens, their heart rates and respiration slow, and their blood pressure rises. Although the NEWS2’s 2-tier ACVPU system does not fully reflect consciousness like the Glasgow Coma Scale, the NEWS2 uses a physiological composite score to comprehensively diagnose changes in patients’ vital signs. Once a patient’s consciousness worsens, other physiological parameters change and the overall NEWS2 score increases, indicating the patient’s deterioration . In addition, as new confusion or delirium in the NEWS2’s ACVPU scale is allocated 3 points and postoperative brain tumor patients commonly have confusion or delirium, this can explain the NEWS2’s specificity in this patient population.
Our study showed that the NEWS2’s cutoff point was 7, which means postoperative brain tumor patients with a NEWS2 score ≥7 may have a high risk of unplanned ICU admission within 72 h. This is consistent with the clinical response to the NEWS2’s risk classification in the guideline published by Royal College of Physicians , which indicate that patients with a NEWS2 score ≥7 need to be considered for transfer to a high-level care facility, such as the ICU. This is in accordance with the results of Martín-Rodríguez et al for brain trauma patients within 48 h of an ICU transfer (cutoff=7) . This finding indicates that 7 may be a reasonable NEWS2 threshold value for medical teams to identify postoperative brain tumor patients’ deterioration and increase their care level.
Compared to the 24 other EWSs, while the NEWS2 has the higher AUROC value (0.860) and Youden index (0.542), these values are still lower than the PAR EWS’s (AUROC=0.870, Youden index=0.589). This might indicate that the PAR score has the greatest discriminatory power. The PAR score was developed by Goldhill et al and then used in the Royal London Hospital [28,29]. It has been reported to prevent surgical deaths  based on 7 variables: temperature, heart rate, systolic blood pressure, ventilatory frequency, oxygen saturation, consciousness, and urine output. Unlike the NEWS2, it incorporates urine output as a physiological parameter, which is an important observation index, as patients with sellar tumors may postoperatively generate a large amount of urine due to intraoperative injury to the hypothalamus or pituitary gland. Moreover, the PAR score’s 4-tier ACVPU system can assess consciousness level more accurately than the NEWS2’s 2-tier ACVPU system. These factors may account for why the PAR score worked better than the NEWS2 for postoperative brain tumor patients. However, even without the urine volume parameter, the NEWS2’s 2-tier ACVPU system can make it easier and more accurate to calculate patient deterioration in a busy clinical environment. Therefore, clinical workers can carefully choose EWSs based on their clinical practice. Overall, this indicates that vital signs combined with other specific neurosurgical variables, suitable parameter bounds, and weighted scores can identify deteriorating brain tumor patients more accurately.
In addition, a systematic review reported that the actual effect of EWSs in clinical application was not as good as the present evidence . This suggests that the clinical use of EWSs may not be solely dependent on its predictive results without clinical nurses’ professional judgment. Meanwhile, Nielsen et al developed the Individual Early Warning Score (I-EWS), which is the combination of NEWS and clinical assessment of clinical nurses, and its prediction ability is better than NEWS alone . The I-EWS allows clinical nurses to adjust the NEWS score according to the clinical evaluation according to nurses’ experience and patients’ condition, and the I-EWS score can be modified range from −4 to +6 points . This suggests that I-EWS could be used and further validated in predicting deterioration of postoperative brain tumor patients by combining EWSs with individual clinical evaluation of nurses to improve the accuracy.
This was a retrospective study at a single center and these factors may have introduced study bias. Importantly, as shown in the recent systematic review and appraisal published by Gerry et al , although EWSs are now widely used in clinical practice to identify early clinical deterioration in patients when in hospital, most EWS systems have methodological weaknesses that could adversely affect patient care. For this reason, these systems should continue to be tested, developed, and modified.
The findings from this retrospective study showed that although the NEWS2 performed well when used to evaluate unplanned ICU admissions of patients with brain tumors within 72 h after surgery, the PAR score was also an accurate EWS.
TablesTable 1. The National Early Warning Score 2. Table 2. Early warning systems included in the study. Table 3. Patient demographics between the case and control groups. Table 4. The NEWS2 physiological parameters between the case and control groups. Table 5. Evaluation value of the NEWS2 and the 24 other EWSs in the study.
1. Weenk M, Koeneman M, van de Belt TH, Wireless and continuous monitoring of vital signs in patients at the general ward: Resuscitation, 2019; 136; 47-53
2. Nuño M, Carico C, Mukherjee D, Association between in-hospital adverse events and mortality for patients with brain tumors: J Neurosurg, 2015; 123; 1247-55
3. Alam N, Vegting IL, Houben E, Exploring the performance of the National Early Warning Score (NEWS) in a European Emergency Department: Resuscitation, 2015; 90; 111-15
4. Jäderling G, Bell M, Martling C-R, ICU admittance by a rapid response team versus conventional admittance, characteristics, and outcome: Crit Care Med, 2013; 41; 725-31
5. Renaud B, Brun-Buisson C, Santin A, Outcomes of early, late, and no admission to the Intensive Care Unit for patients hospitalized with community-acquired pneumonia: Acad Emerg Med, 2012; 19; 294-303
6. Noordzij PG, Poldermans D, Schouten O, Postoperative mortality in The Netherlands: A population-based analysis of surgery-specific risk in adults: Anesthesiology, 2010; 112; 1105-15
7. Nittby HR, Maltese A, Ståhl N, Early postoperative haematomas in neurosurgery: Acta Neurochir (Wien), 2016; 158; 837-46
8. Ostrom QT, Gittleman H, Xu J, CBTRUS Statistical Report: Primary brain and other central nervous system tumors diagnosed in the United States in 2009–2013: Neuro-oncology, 2016; 18; v1-75
9. Budėnas A, Tamašauskas Š, Šliaužys A, Incidence and clinical significance of postoperative delirium after brain tumor surgery: Acta Neurochir (Wien), 2018; 160; 2327-37
10. Gold CA, Mayer SA, Lennihan L, Unplanned transfers from Hospital Wards to the Neurological Intensive Care Unit: Neurocrit Care, 2015; 23; 159-65
11. Hogan H, Healey F, Neale G, Preventable deaths due to problems in care in English acute hospitals: A retrospective case record review study: BMJ Qual Saf, 2012; 21; 737-45
12. Weller RS, Foard KL, Harwood TN, Evaluation of a wireless, portable, wearable multi-parameter vital signs monitor in hospitalized neurological and neurosurgical patients: J Clin Monit Comput, 2018; 32; 945-51
13. Taenzer AH, Pyke JB, McGrath SP, Impact of pulse oximetry surveillance on rescue events and Intensive Care Unit transfers: A before-and-after concurrence study: Anesthesiology, 2010; 112; 282-87
14. Rafter N, Hickey A, Condell S, Adverse events in healthcare: Learning from mistakes: QJM, 2015; 108; 273-77
15. Ludikhuize J, Hamming A, de Jonge E, Rapid response systems in The Netherlands: Jt Comm J Qual Patient Saf, 2011; 37; 138-44
16. Gerry S, Bonnici T, Birks J, Early warning scores for detecting deterioration in adult hospital patients: systematic review and critical appraisal of methodology: BMJ, 2020; 369; m1501
17. Royal College of Physicians: National Early Warning Score (NEWS) 2: Standardising the assessment of acute-illness severity in the NHS, 2017, London, RCP https://www.rcplondon.ac.uk/projects/outputs/national-early-warning-score-news-2
18. O’Driscoll BR, Howard LS, Earis J, British Thoracic Society Guideline for oxygen use in adults in healthcare and emergency settings: BMJ Open Respir Res, 2017; 4; e000170
19. Silcock DJ, Corfield AR, Staines H, Superior performance of National Early Warning Score compared with quick Sepsis-related Organ Failure Assessment Score in predicting adverse outcomes: A retrospective observational study of patients in the prehospital setting: Eur J Emerg Med, 2019; 26; 433-39
20. Corfield AR, Lees F, Zealley I, Utility of a single early warning score in patients with sepsis in the Emergency Department: Emerg Med J, 2014; 31; 482-87
21. Tan J-W, Zhang X-Q, Geng C-M, Development of the National Early Warning Score-Calcium Model for predicting adverse outcomes in patients with acute pancreatitis: J Emerg Nurs, 2020; 46; 171-79
22. Hydes TJ, Meredith P, Schmidt PE, National Early Warning Score accurately discriminates the risk of serious adverse events in patients with liver disease: Clin Gastroenterol Hepatol, 2018; 16; 1657-66.e10
23. Smith GB, Prytherch DR, Meredith P, The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated Intensive Care Unit admission, and death: Resuscitation, 2013; 84; 465-70
24. Le Lagadec MD, Dwyer T, Browne M, The efficacy of twelve early warning systems for potential use in regional medical facilities in Queensland, Australia: Aust Crit Care, 2020; 33; 47-53
25. Martín-Rodríguez F, López-Izquierdo R, Mohedano-Moriano A, Identification of serious adverse events in patients with traumatic brain injuries, from prehospital care to Intensive-Care Unit, using Early Warning Scores: Int J Environ Res Public Health, 2020; 17; 17051504
26. Smith GB, Prytherch DR, Schmidt PE, Review and performance evaluation of aggregate weighted ‘track and trigger’ systems: Resuscitation, 2008; 77; 170-79
27. Subbe CP, Kruger M, Rutherford P, Validation of a modified Early Warning Score in medical admissions: QJM, 2001; 94; 521-26
28. Goldhill DR, Preventing surgical deaths: Critical care and intensive care outreach services in the postoperative period: Br J Anaesth, 2005; 95; 88-94
29. Goldhill DR, McNarry AF, Mandersloot G, A physiologically-based early warning score for ward patients: The association between score and outcome: Anaesthesia, 2005; 60; 547-53
30. World Health Organization, 2019 https://icd.who.int/browse10/2019/en#/I
31. Pimentel MAF, Redfern OC, Gerry S, A comparison of the ability of the National Early Warning Score and the National Early Warning Score 2 to identify patients at risk of in-hospital mortality: A multi-centre database study: Resuscitation, 2019; 134; 147-56
32. Parshuram CS, Duncan HP, Joffe AR, Multicentre validation of the bedside paediatric early warning system score: A severity of illness score to detect evolving critical illness in hospitalised children: Crit Care, 2011; 15; R184
33. Prytherch DR, Smith GB, Schmidt PE, ViEWS – towards a national early warning score for detecting adult inpatient deterioration: Resuscitation, 2010; 81; 932-37
34. Tarassenko L, Clifton DA, Pinsky MR, Centile-based early warning scores derived from statistical distributions of vital signs: Resuscitation, 2011; 82; 1013-18
35. Teasdale GM, National Early Warning Score (NEWS) is not suitable for all patients: BMJ, 2012; 345; e5875
36. Lonjaret L, Guyonnet M, Berard E, Postoperative complications after craniotomy for brain tumor surgery: Anaesth Crit Care Pain Med, 2017; 36; 213-18
37. Nielsen PB, Schultz M, Langkjaer CS, Adjusting Early Warning Score by clinical assessment: A study protocol for a Danish cluster-randomised, multicentre study of an Individual Early Warning Score (I-EWS): BMJ Open, 2020; 10; e033676
38. Duckitt RW, Buxton-Thomas R, Walker J, Worthing physiological scoring system: Derivation and validation of a physiological early-warning system for medical admissions. An observational, population-based single-centre study: Br J Anaesth, 2007; 98; 769-74
39. Chatterjee MT, Moon JC, Murphy R, The “OBS” chart: An evidence-based approach to re-design of the patient observation chart in a district general hospital setting: Postgrad Med J, 2005; 81; 663-66
40. Paterson R, MacLeod DC, Thetford D, Prediction of in-hospital mortality and length of stay using an early warning scoring system: Clinical audit: Clin Med (Lond), 2006; 6; 281-84
41. Heaps N, Thorley K, Langley S, Critical care outreach: Creating a safe culture: Br J Nurs, 2005; 14; 1208-11
42. Allen K, Recognising and managing adult patients who are critically sick: Nurs Times, 2004; 100; 34-47
43. Hancock HC, Durham L, Critical care outreach: The need for effective decision-making in clinical practice (part 2): Intensive Crit Care Nurs, 2007; 23; 104-14
44. Smith AF, Oakey RJ, Incidence and significance of errors in a patient “track and trigger” system during an epidemic of Legionnaires’ disease: Retrospective casenote analysis: Anaesthesia, 2006; 61; 222-28
45. Andrews T, Waterman H, Packaging: A grounded theory of how to report physiological deterioration effectively: J Adv Nurs, 2005; 52; 473-81
46. Priestley G, Watson W, Rashidian A, Introducing Critical Care Outreach: A ward-randomised trial of phased introduction in a general hospital: Intensive Care Med, 2004; 30; 1398-404
47. Ryan H, Cadman C, Hann L, Setting standards for assessment of ward patients at risk of deterioration: Br J Nurs, 2004; 13; 1186-90
48. Odell M, Rechner IJ, Kapila A, The effect of a critical care outreach service and an early warning scoring system on respiratory rate recording on the general wards: Resuscitation, 2007; 74; 470-75
49. von Lilienfeld-Toal M, Midgley K, Lieberbach S, Observation-based early warning scores to detect impending critical illness predict in-hospital and overall survival in patients undergoing allogeneic stem cell transplantation: Biol Blood Marrow Transplant, 2007; 13; 568-76
50. Gardner-Thorpe J, Love N, Wrightson J, The value of Modified Early Warning Score (MEWS) in surgical in-patients: A prospective observational study: Ann R Coll Surg Engl, 2006; 88; 571-75
51. Wright MM, Stenhouse CW, Morgan RJ, Early detection of patients at risk (PART): Anaesthesia, 2000; 55; 391-92
52. Smith GB, Prytherch DR, Schmidt P, Hospital-wide physiological surveillance – a new approach to the early identification and management of the sick patient: Resuscitation, 2006; 71; 19-28
53. Petraschka M, Gilbert T, Westenbroek R, Validation of a Modified Early Warning Score (MEWS) in Emergency Department observation ward patients: Journal of Pain, 2006; 13; 24-30
54. Riley B, Faleiro R, Critical care outreach: Rationale and development: Bja Cepd Reviews, 2001; 1; 146-49
55. Rees JE, Mann C, Use of the patient at risk scores in the Emergency Department: A preliminary study: Emerg Med J, 2004; 21; 698-99
56. Cooper N, Patient at risk!: Clin Med (Lond), 2001; 1; 309-11
26 January 2023 : EditorialEditorial: The XBB.1.5 (‘Kraken’) Subvariant of Omicron SARS-CoV-2 and its Rapid Global Spread
Med Sci Monit In Press; DOI: 10.12659/MSM.939580
19 January 2023 : Clinical ResearchEvaluation of Health-Related Quality of Life and Mental Health in 729 Medical Students in Indonesia During ...
Med Sci Monit In Press; DOI: 10.12659/MSM.938892
19 January 2023 : Clinical ResearchDetermining the Impact of the COVID-19 Pandemic on Dental Care in the Serbian Adult Population: An Online Q...
Med Sci Monit 2023; 29:e939238
27 Jan 2023 : Database AnalysisAssociation Between Neutrophil-Lymphocyte Ratio and All-Cause Mortality in Critically Ill Patients with Chr...
Med Sci Monit In Press; DOI: 10.12659/MSM.938554
27 Jan 2023 : Clinical ResearchProposal for a Simple Equation for Limb Muscle Weight Calculation
Med Sci Monit In Press; DOI: 10.12659/MSM.938606
26 Jan 2023 : EditorialEditorial: The XBB.1.5 (‘Kraken’) Subvariant of Omicron SARS-CoV-2 and its Rapid Global Spread
Med Sci Monit In Press; DOI: 10.12659/MSM.939580
Most Viewed Current Articles
13 Nov 2021 : Clinical ResearchAcceptance of COVID-19 Vaccination and Its Associated Factors Among Cancer Patients Attending the Oncology ...
Med Sci Monit 2021; 27:e932788
30 Dec 2021 : Clinical ResearchRetrospective Study of Outcomes and Hospitalization Rates of Patients in Italy with a Confirmed Diagnosis o...
Med Sci Monit 2021; 27:e935379
08 Mar 2022 : Review articleA Review of the Potential Roles of Antioxidant and Anti-Inflammatory Pharmacological Approaches for the Man...
Med Sci Monit 2022; 28:e936292