Comparing the National Institutes of Health Stroke Scale Scores between emergency medicine physicians and neurologists for timely decision‑making for alteplase administration

Abstract

OBJECTIVES: The National Institutes of Health Stroke Scale (NIHSS) is used to determine the severity of the disease and to make treatment decisions in ischemic stroke patients. However, the need for a neurologist to assess NIHSS before thrombolytic therapy may prolong the treatment process.

METHODS: This prospective, single center, observational, planned study included patients who presented to the emergency department in the first 24 h after the onset of symptoms and were diagnosed with ischemic stroke between September 2022 and June 2023. The NIHSS was evaluated by the emergency medicine physicians and neurologists who evaluated the patients in the emergency department, and the decisions on whether to administer thrombolytics and the time taken for this decision were recorded and compared.

RESULTS: A very high agreement was found when the total NIHSS scores of emergency medicine physicians and neurologists were compared (intraclass correlation coefficient = 0.947 [95% confidence interval 0.92–0.96]). Emergency medicine physicians and neurologists showed high agreement with thrombolytic therapy decisions. In patients given thrombolytic therapy, emergency medicine physicians made the decision earlier than neurologists, and there was a significant difference of 14 ± 12 min between the decisions of emergency physicians and those of neurologists.

CONCLUSIONS: There is high agreement between emergency medicine physicians and neurologists in the NIHSS evaluation and thrombolytic decisions for patients with acute ischemic stroke. According to our results, emergency medicine physicians can provide thrombolytic treatment in accordance with neurologists, thus shortening the time for thrombolytic treatment.

Introduction

Stroke is the second leading cause of disability and death worldwide.[1] Stroke can be classified into two main categories: hemorrhagic and ischemic strokes. Ischemic strokes account for nearly 80% of all strokes. In the case of ischemic stroke, the most effective treatment methods supported by research and currently in use are thrombolytic therapy and mechanical thrombectomy. Both treatment approaches have specific time limits from the onset of the disease. This underscores the importance of not delaying the diagnosis and initiation of treatment for ischemic stroke.

In some developing countries, the guidelines state that only neurologists can calculate the National Institutes of Health Stroke Scale (NIHSS) and make decisions regarding thrombolytic treatment for ischemic stroke patients.[2] However, the need for a neurology specialist to assess NIHSS before thrombolytic treatment can prolong the treatment process. Moreover, it may not be feasible to have a neurology specialist available for emergency consultation at every healthcare facility.

Patients with stroke symptoms often present to emergency departments because their complaints typically have an acute onset. This situation increases the role and importance of emergency physicians in the diagnosis and treatment of strokes. Our study was designed with the hypothesis that emergency physicians can accurately and consistently calculate the NIHSS score and that having emergency physicians assess NIHSS can reduce the time required to initiate thrombolytic treatment.

Material and Methods

The study was conducted as a single center, prospective, observational clinical study. Ethical approval for this study was obtained from the ethics committee of the University of Health Sciences—Trabzon Faculty of Medicine at Kanuni Training and Research Hospital in Turkey on September 12, 2022, with approval number of 2022/50. Patients who presented to the emergency department with a preliminary diagnosis of acute ischemic stroke were evaluated between September 15, 2022, and June 15, 2023. The hospital where the study was conducted serves approximately 1 million patients annually, with around 300,000 of them being emergency department visits, and it functions as a stroke center in the region.

Study population

The study population included patients aged 18 and above who presented to the emergency department with suspected acute ischemic stroke and were examined between September 15, 2022, and June 15, 2023. These patients agreed to participate in the study and signed an informed consent form. Patients in whom the NIHSS was not calculated by emergency physicians and neurologists and those with a time interval of symptom onset exceeding 24 h from their emergency department admission were excluded from the study.

Study protocol

Patients who were initially examined in the emergency department with suspected acute ischemic stroke or who were referred from external centers were evaluated by an emergency physician following the national guidelines for the diagnosis and treatment of acute ischemic stroke. Neurological assessments were conducted, and necessary tests (i.e., complete blood count, biochemistry, coagulation profile) were ordered. Noncontrast brain computed tomography (CT) was performed for all patients presenting with suspected stroke symptoms, primarily to exclude intracranial hemorrhage, large established infarcts, or other contraindications to thrombolytic therapy. CT scans were initially interpreted by the emergency physician and subsequently confirmed by the radiologist. The diagnosis of acute ischemic stroke was based on clinical presentation in combination with imaging findings and symptom onset time. Patients with clinical features of stroke and no contraindication on CT were considered eligible for thrombolytic therapy. Diffusion weighted magnetic resonance imaging was not routinely performed before treatment initiation. Neurology consultation was requested following the initial assessment.

Immediately after requesting a neurology consultation, the emergency physician calculated the NIHSS for the patients and decided on thrombolytic treatment. The neurologist who came for consultation in the emergency department assessed the patients at bedside and independently calculated the NIHSS score based on the emergency physician’s NIHSS calculation and thrombolytic decision. The treatment process continued based on the decision made by the neurologist to manage the patients.

The patients’ demographic information, comorbidities, biochemical test results, NIHSS values calculated by emergency physicians and neurologists, thrombolytic treatment decisions, and decision times were recorded in the relevant sections of the data form prepared for the study.

In the study, the group referred to as emergency physicians consisted of 15 resident physicians with at least 2 years of training. The group referred to as neurologists consisted of six experienced neurology specialists working at the stroke center.

Statistical analysis

Statistical analysis was performed using SPSS 25.0 software (IBM Corp. Released 2017. IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp.). The normality of the data was assessed using the Kolmogorov–Smirnov test. Categorical data were presented in frequency tables, while ordinal data were presented with mean, standard deviation, and min–max values. Student’s t test was used for the statistical analysis of normally distributed data, and the Chi square test was used for nonparametric categorical data. The Mann–Whitney U test was used for nonnormally distributed nonparametric data. The intraclass correlation coefficient (ICC) test assessed the agreement between the total NIHSS scores among the groups. The weighted kappa test was used for the agreement analysis of ordinal data. Cohen’s kappa analysis was used for the agreement analysis of dichotomous or nominal data, and the agreement strength was evaluated using the kappa coefficient (K). A paired t test was used to compare the time intervals between patient admission and tissue plasminogen activator (tPA) decisions in both specialties. A value of P < 0.05 was considered statistically significant.

Results

Our study was conducted with 113 patients who met the inclusion criteria. One patient had missing data, while another exceeded the first 24 h from symptom onset at admission; thus, they were excluded from the study. As a result, 111 patients were included in the study. The clinical and demographic characteristics of these patients are presented in Table 1.

Table 1. Demographic and clinical characteristics of patients (n=111)

According to the results, when the total NIHSS scores calculated by emergency physicians and neurologists were compared, very high agreement was observed in all patients (ICC = 0.947 [95% confidence interval (CI) 0.92–0.96]). When the NIHSS parameters were evaluated individually, a high level of agreement was observed in most of the parameters [Table 2].

Table 2. Interobserver agreement for National Institutes of Health Stroke Scale

In the analysis, in which the total NIHSS score calculated by emergency physicians was subtracted from the total NIHSS score calculated by neurologists, the total NIHSS scores were equal for 51 (45.9%) patients. For 29 (26.1%) patients, the calculated total NIHSS score had a difference of 1. In 80 (72.0%) patients, the difference in the calculated total NIHSS score was <2. In terms of the calculated total NIHSS scores, the largest difference was in two patients, in which emergency physicians calculated 6 points more than neurologists.

The patients were categorized into three groups based on the total NIHSS scores calculated by emergency physicians and neurologists. Those with scores of 0–6 were classified as mild, scores of 7–16 as moderate, and scores of 17–40 as severe stroke [Table 3]. In examining the stroke severity categorizations based on the total NIHSS scores calculated by emergency physicians and neurologists, a high level of agreement was also observed (K: 0.888 [95% CI 0.81–0.95]).

Table 3. Stroke severity groups determined by National Institutes of Health Stroke Scale

In our study, emergency physicians recommended thrombolytic treatment for 61 patients (54.9%), while neurologists recommended thrombolytic treatment for 58 patients (52.2%). According to the clinical follow up guidelines, thrombolytic treatment was planned for 58 patients by neurologists, and it could be applied to 52 of them (46.8%). Out of the six patients who remained, thrombolytic treatment consent could not be obtained for three of them, while thrombolytic agents were not available at the healthcare facility for the other three. Thus, tPA treatment could not be administered.

Data related to the thrombolytic decisions of emergency physicians and neurologists are presented in Table 4. This table shows a high agreement between the thrombolytic decisions of emergency physicians and neurologists (K = 0.873, P < 0.001). In total, different decisions were made for seven patients. Emergency physicians recommended tPA for five patients, while neurologists recommended not administering tPA. The reasons for neurologists’ decision not to administer tPA were relative contraindications in four patients and the presence of a sizeable ischemic area on the brain CT in one patient. For two patients for whom neurologists recommended tPA, emergency physicians recommended not administering it. The reasons for the decision of emergency physicians not to administer tPA were relative contraindications in one patient and the presence of a sizeable ischemic area on the brain CT in one patient.

Table 4. Details of thrombolytic treatment in patients

Among the patients who received thrombolytic therapy, emergency physicians made the decision for thrombolytic treatment significantly earlier than neurologists, with a mean difference of 14 ± 12 min (P < 0.001).

Discussion

Ischemic stroke, one of the most urgent health problems in the world, is an important cause of mortality and morbidity. Although there may be differences from country to country, the management of ischemic stroke, especially in developing countries, is governed by a protocol in which only neurologists can calculate the NIHSS and decide on thrombolytic therapy. It is thought that this process may prolong the initial stage of thrombolytic therapy or mechanical thrombectomy, which plays a vital role in the management of ischemic stroke. However, the evaluation performed by emergency physicians in the emergency department, which is the first point of contact for these patients, can contribute to stroke management if it is correct and shortens the process. In this regard, a high level of agreement was found in the calculation of NIHSS and the decision to administer thrombolytic therapy for patients who presented to the emergency department with a preliminary diagnosis of acute ischemic stroke. Furthermore, emergency medicine physicians decided to administer thrombolytic therapy more promptly in harmony with neurologists. As previous studies have demonstrated a correlation between the early initiation of thrombolytic therapy and favorable clinical outcomes, as well as the reduction of potential complications, adjustments can be made to algorithms in the management of ischemic stroke, taking into account the competence of emergency medicine physicians in making these critical decisions.

According to the results, a very high level of agreement was observed between neurologists and emergency medicine physicians in evaluating the total NIHSS score obtained by scoring 11 main categories and 15 parameters. It is possible to come across studies investigating the agreement in NIHSS assessments by different healthcare professional groups, often because of concerns such as the unavailability of access to neurologists or the desire to expedite the process. A recent study by Larsen et al., in 2022, examined data from 274 patients and found an agreement in the total NIHSS evaluation between paramedics certified for NIHSS assessment and stroke specialists.[3] In this study, Larsen et al. found moderate agreement between stroke specialists and certified paramedics regarding NIHSS assessment (K = 0.58).[3] Cummock et al.’s study in 2023 retrospectively investigated 129 patients with acute ischemic stroke with two observers, one being an emergency physician evaluating the patients in the emergency department and the other being a neurologist, and found a very high agreement in total NIHSS calculations (ICC = 0.95).[4] Larsen et al. examined the agreement between certified paramedics, while Cummock et al. compared emergency physicians in an emergency department of a hospital with a stroke center with neurologists, similar to our study.[3,4] In another study, Katz et al. reported a high agreement (ICC = 0.82) in the total NIHSS assessment between emergency physicians and neurologists for 340 acute ischemic stroke patients who presented to the emergency department.[5] To the best of our knowledge, apart from these three studies, no other studies in the literature have examined agreement in the total NIHSS assessment. When the results of our study are evaluated alongside the results of Cummock et al.’s and Katz et al.’s studies, it can be concluded that emergency medicine physicians who frequently encounter and actively manage ischemic stroke patients receive adequate education as part of their training in critical areas, such as NIHSS assessment, ischemic stroke management, and administration of thrombolytics, and can make decisions similar to neurologists.[4,5]

Katz et al. evaluated the agreement in the total NIHSS assessment and separately assessed the agreement of the individual component parameters of the NIHSS. They found a high agreement for consciousness questions and speech and a low agreement for gaze, visual field, sensation, ataxia, and neglect.[5] It is possible to find other studies in the literature that examined the individual components of the NIHSS score and their agreement. Dewey et al. analyzed the agreement in NIHSS assessment using 31 patients diagnosed with ischemic stroke, two neurologists, and two certified nurses. They found a high interobserver agreement for most parameters when both observers were neurologists but found a low agreement between nurses and neurologists.[6] Goldstein et al. examined the agreement in NIHSS assessment among four stroke experts and reported a high agreement for the language and motor sections but a low agreement for consciousness commands, speech, and others.[7] Brott et al. analyzed 24 ischemic stroke patients and found a high agreement between neurologists and neurology nurses in NIHSS assessment.[8] In our study, the parameters related to consciousness evaluation and motor examinations stood out with high agreement, followed by speech and dysarthria parameters with high agreement levels. Conversely, the neglect parameter was noticeably marked with very low agreement. Although moderate and high agreement was detected in almost all parameters, the very low agreement in the neglect parameter suggests significant differences between emergency medicine physicians and neurologists in its evaluation. This indicates that this parameter may not be well understood or overlooked. The reason for this significant difference should be examined by both physician groups, who need to have the same perspective and speak the same language in the management of ischemic stroke. In other studies, a higher level of agreement was observed in this parameter. Although our study found low agreement in the neglect parameter, the total NIHSS difference was ± 6, consistent with studies comparing emergency physicians with neurologists in the literature. Cummock et al. reported a total NIHSS difference of ±5 in their study.[4]

In a retrospective study, Comer et al. examined neurologists and NIHSS certified emergency department nurses who evaluated NIHSS scores for 438 patients and found that in 286 (65.3%) patients, the neurologist–nurse NIHSS score difference was <2.[9] In our study, in examining the emergency physician–neurologist NIHSS score difference, the difference was <2 in the majority of the patients (72%). As there were no studies for such a comparison, a detailed comparison could not be made. The data showed similarities to Comer et al.’s study.

In addition to the high agreement found between emergency medicine physicians and neurologists in NIHSS assessment, there was also high agreement in deciding to administer thrombolytic treatment. Difficulty accessing neurologists is a significant problem worldwide in the management of ischemic strokes. According to data published by the American Academy of Neurology, even in the United States, which has one of the most advanced healthcare systems, a neurologist shortage of 19% is predicted by 2025.[10] However, due to reasons such as the extension of the time it takes for thrombolytic decisions with telestroke management and the door to needle time not being at the desired levels, many studies have been conducted to determine whether emergency medicine physicians and neurologists agree in making decisions regarding thrombolytic treatment. The results of these studies show that in many centers, emergency medicine physicians can make agreeable and safe thrombolytic decisions with neurologists.[11,12] Berekashvili et al. conducted a study in 2019 with 60 patients diagnosed with acute ischemic stroke, compared the thrombolytic treatment decisions of emergency medicine residents, vascular neurology specialists, and vascular neurology faculty members, and found a high level of agreement.[11] Mecozzi et al. examined 23 emergency department doctors, 25 emergency medicine residents, 11 neurology residents, and 5 stroke specialists. The decision of the stroke specialist was considered the gold standard, and the thrombolytic treatment decision of emergency physicians was compared with that of neurologists, revealing a similar high level of agreement.[12] As a result of these and similar studies, there has been a focus on the more active role of emergency medicine physicians in the management of acute ischemic stroke patients. The effect of the decision to administer or not administer thrombolytic treatment promptly on patients’ access to thrombolytic treatment has been investigated, and it has been demonstrated that the inclusion of emergency medicine physicians in stroke management allows for a reliable and rapid access to thrombolytic treatment.[13] Heikkilä et al. conducted a study in 2016 in which the management of acute ischemic stroke patients in the emergency department was organized according to the decision of emergency department physicians, and they found that the average door to needle time was reduced from 54 min to 28 min.[14] Similarly, in 2019, Heikkilä et al. demonstrated that emergency medicine physicians could safely administer thrombolytic treatment in ischemic stroke patients, which shortened the time to access thrombolytic treatment by approximately 23 min. Our study also showed that emergency physicians could make thrombolytic decisions in agreement with neurologists and significantly shorten the time to access thrombolytic treatment.

In light of all these studies and the knowledge pool available in the literature, many developed countries have changed their stroke management algorithms, and emergency medicine physicians have become one of the most important decision makers in stroke management within various protocols.[13] However, in some developing countries, the role of emergency medicine physicians in decision making and administering thrombolytic treatment in the management of ischemic stroke is still not clearly defined in the current guidelines. Considering the literature and the results of our study, it is evident that similar regulations should be implemented promptly in all countries with emergency medicine physicians.

Limitations

As this is a single center study, it was conducted with a low number of patients. In our study, the physicians referred to as emergency medicine physicians consisted of emergency medicine residents who were undergoing emergency medicine specialization training. Another limitation is that the emergency medicine and neurology physicians included physicians with different experience levels within their respective groups. As the hospital where the study was conducted serves as a stroke center, the emergency medicine physicians who participated in the study could have been more experienced in diagnosing and treating acute ischemic stroke. Thus, our results may not be applicable to all emergency department physicians.

Conclusion

The emergency physicians and neurologists in this study found a very high level of agreement in the calculated total NIHSS assessment and the decision to administer thrombolytic treatment. Emergency physicians made thrombolytic treatment decisions earlier than neurologists, and the time difference was significant. According to the results of this study and the literature, emergency physicians can safely and effectively make and administer thrombolytic treatment decisions. In ischemic stroke patients, in which every minute counts, it is necessary for emergency medicine physicians to have a more prominent role in the diagnosis and treatment algorithm for the administration of thrombolytics.

How to cite this article: Tecir O, Çiçek M, Çekiç ÖG, Ardıç Ş, Akpınar R, Usta NC, et al. Comparing the National Institutes of Health Stroke Scale Scores between emergency medicine physicians and neurologists for timely decision-making for alteplase administration. Turk J Emerg Med 2025;25:223-9.

The ethical approval of this study was authorized by the University of Health and Science, Trabzon Faculty of Medicine, Kanuni Training and Research Hospital Clinical Research Ethics Committee (Trabzon, Turkey) with decision number 2022/50 on September 12, 2022.

Osman Tecir: Conceptualization, methodology (lead), software (lead), investigation (lead), resources, writing – original draft (lead), writing – reviewing and editing (supporting). Mustafa Çiçek: Data curation, methodology (supporting), writing – review and editing (lead), supervision (lead). Özgen Gönenç Çekiç: Data collection (lead), visualization, investigation (equal), project administration. Şenol Ardıç: Data collection (supporting). Ramazan Akpınar: Software (supporting), reviewing. Nuray Can Usta: Software (supporting), formal analysis, validation. Süleyman Türedi: Writingreviewing and editing (supporting), supervision (supporting).

None declared.

None.

References

1. GBD 2016 Neurology Collaborators. Global, regional, and national burden of neurological disorders, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol 2019;18:459–80.
2. Republic of Turkey Ministry of Health, General Directorate of Health Services, Department of Research, Development and Health Technology Assessment. Acute Ischemic Stroke Diagnosis and Treatment Guideline. 1st ed. Ankara, Turkey: Republic of Turkey Ministry of Health; 2020.
3. Larsen K, Jæger HS, Hov MR, Thorsen K, Solyga V, Lund CG, et al. Streamlining acute stroke care by introducing National Institutes of Health Stroke Scale in the emergency medical services: A prospective cohort study. Stroke 2022;53:2050 7.
4. Cummock JS, Wong KK, Volpi JJ, Wong ST. Reliability of the National Institutes of Health (NIH) Stroke Scale between emergency room and neurology physicians for initial stroke severity scoring. Cureus 2023;15:e37595.
5. Katz B, Khoury J, Alwell K, Moomaw C, Kleindorfer D. Interrater reliability of the National Institutes of Health Stroke Scale: Rating by emergency room physicians and neurologists in a population based study (P5.140). Neurology 2015;84 Suppl 14.
6. DeweyHM, DonnanGA, FreemanEJ, SharplesCM, MacdonellRA, McNeil JJ, et al. Interrater reliability of the National Institutes of Health Stroke Scale: Rating by neurologists and nurses in a community based stroke incidence study. Cerebrovasc Dis 1999;9:323 7.
7. Goldstein LB, Bertels C, Davis JN. Interrater reliability of the NIH Stroke Scale. Arch Neurol 1989;46:660 2.
8. Brott T, Adams HP Jr., Olinger CP, Marler JR, Barsan WG, BillerJ, et al. Measurements of acute cerebral infarction: A clinical examination scale. Stroke 1989;20:864 70.
9. Comer AR, Templeton E, Glidden M, Bartlett S, D’Cruz L, Nemati D, et al. National Institutes of Health Stroke Scale (NIHSS) scoring inconsistencies between neurologists and emergency room nurses. Front Neurol 2022;13:1093392.
10. Dall TM, Storm MV, Chakrabarti R, Drogan O, Keran CM, Donofrio PD, et al. Supply and demand analysis of the current and future US neurology workforce. Neurology 2013;81:470 8.
11. Berekashvili K, Zha AM, Abdel Al M, Zhang X, Soomro JH, Prater SJ, et al. Emergency medicine physicians accurately select acute stroke patients for tissue type plasminogen activator treatment using a checklist. Stroke 2020;51:663 5.
12. Mecozzi AC, Brown DL, Lisabeth LD, Barsan WG, Silbergleit R, Hickenbottom SL, et al. Determining intravenous rt PA eligibility in the emergency department. Neurocrit Care 2007;7:103 8.
13. Jauch EC, Holmstedt CA. Fast protocol for treating acute ischemic stroke by emergency physicians: What took so long? Ann Emerg Med 2019;73:113 5.
14. Heikkilä I, Kuusisto H, Stolberg A, Palomäki A. Stroke thrombolysis given by emergency physicians cuts in hospital delays significantly immediately after implementing a new treatment protocol. Scand J Trauma Resusc Emerg Med 2016;24:46.