Scholarly impact of the dissertation requirement for postgraduate medical education and factors affecting transformation into publication: A bibliometric analysis of 2434 dissertations in the field of emergency medicine

Abstract

OBJECTIVES: In Turkey, conducting research for a dissertation is necessary to obtain a specialist degree, but publication of this research is not mandatory. Previous studies have shown a low rate of publication for dissertation derived research. The aim of this study was to determine publication rates, factors affecting the transformation of the dissertations into high-quality publications, and bibliometric analysis of published articles in the field of emergency medicine (EM).

METHODS: This was a retrospective bibliometric study of EM dissertations submitted between 1998 and 2021 to the National Thesis Center. Research characteristics, publication status, journal characteristics, indexing, citation analysis, and institution characteristics were recorded. Journals indexed in the web of science (WOS) were defined as high quality journals. A logistic regression was performed to identify factors affecting publication in high quality journals.

RESULTS: A total of 2434 dissertations were included. Of these, 864 (35.5%) were published and 474 (54%) were published in WOS indexed journals. The most common area of research was trauma (n = 150, 17%), and the most common journal was the American Journal of EM (n = 74, 8%). Prospective data collection (odds ratio [OR] = 2.5, 95% confidence interval [CI] = 1.8–2.5), experimental design (OR = 2, 95%, CI = 1.3–3), university type residency program (OR = 1.4, 95% CI = 1.02–2.1), and duration between year of graduation and publication (OR = 0.9, 95% CI = 0.84– 0.95) were associated with publishing in WOS indexed journals.

CONCLUSION: EM is a relatively successful specialty for publishing dissertation derived studies. Prospective and experimental research design, graduation from a university-type residency program, and shorter duration between the graduation and publication may increase the chance of publishing in high quality journals.

Introduction

In Turkey, specialization in medicine is considered to be the equivalent of a Doctor of Philosophy degree.[1] During the residency training, in addition to the standardized residency curriculum, successfully conducting and defending a research dissertation is required to obtain the specialist degree.[2] The aim of this process is to ensure that residents possess a high level of comprehension of scientific principles, literature review, research methodologies, statistical analysis, and writing of scientific articles.[3] However, subsequent publication of the dissertation research in scientific journals is at the discretion of the graduating resident or the faculty member who supervised the research.[3] Therefore, transforming research dissertations into publications is dependent, in part, on academic motivation of the graduating resident or supervisor.

Emergency medicine (EM) is a relatively young specialty in Turkey. Although it has been officially recognized since 1993, the first residents graduated in 1998.[4] To date, 105 EM residency programs in both university and community hospitals have been established and approximately 3500 EM specialists have graduated across the country.[5] However, the rapid increase in the number of available residency training positions in the last decade has caused an unbalanced resident/faculty ratio and may have affected the scientific quality of research topics in EM. Therefore, the number and quality of published articles derived from residency dissertation research may be compromised.

Several studies that have evaluated the publication rates of dissertations have shown poor publication rates in many specialties.[3,6 9] including EM.[10] However, evidence concerning factors affecting the publication rate and the scientific impact of the published articles derived from the dissertations is very limited. Therefore, the aim of this study was to investigate publication rates of research dissertation derived articles in the field of EM, identify factors affecting the transformation of the dissertations into high quality publications, and bibliometric analysis of published articles.

Material and Methods

Study design

This was a retrospective, bibliometric study of dissertations in the field of EM in the National Thesis Center of Turkey.[5] Institutional review board approval was obtained for the study on October 13, 2022 (GOKAEK 2022/17.17).

Selection of sample

All dissertations in EM, submitted between 1998 and 2021, were identified in the National Thesis Center of Turkey. The dissertations that the public access is not permitted by the principal investigator and the principal investigator who was not graduated from an EM residency program were excluded from the study.

Study protocol

All dissertations submitted during the study period to the National Thesis Center were filtered by subject area and those in the field of EM were selected. Then, dissertations that met the eligibility criteria were included. The author’s name, sex, institution, dissertation title, and graduation year were recorded in an electronic data collection form. Publication status was investigated using the author’s name, dissertation title and abstract in Google Scholar (GS), PubMed, web of science (WOS) Core Collection, and TRdizin. In published dissertations, the digital object identifier number, date of publication, duration between graduation and publication, name of the publishing journal, number of authors, research field, keywords, research design and the number of citations in GS (for all journals) and WOS (for WOS indexed journals) were also recorded. Studies published in the WOS indexed journals were assumed to be high quality publications.[11] For evaluation of the individual host institutions’ scholarly productivity, the number of publications in EM derived from dissertation research was divided by the total number of dissertations reported, and this was defined as the program productivity index.

Outcome measures

The primary outcome was the publication rate of dissertations and citation metrics of published articles derived from the dissertations. Secondary outcomes included the factors affecting the publication of dissertations in WOS indexed journals, word co occurrence network of keywords and area of research of published articles.

Data analysis

For statistical analyses, IBM SPSS Statistics version 26.0 (IBM Corp., Armonk, NY, USA) was used. The Kolmogorov–Smirnov test was used to assess the distribution of normality. Continuous variables are expressed as mean and standard deviation or median and interquartile range (IQR) according to the distribution of normality. Categorical variables are expressed as numbers (n) and percentage (%). Student’s t test or Mann–Whitney U tests were used to compare continuous variables, as appropriate, and Chi square test was used to compare categorical variables. Multivariate logistic regression testing was used to investigate factors affecting publication in WOS indexed journals. The significance was set at a P < 0.05. Decimals were rounded for better read.

Results

After excluding 19 dissertations, a total of 2434 dissertations were included in the study. Of these, 864 (35.5%) were published in scientific journals [Figure 1]. Of the published studies, 657 (75%) were written in English, 642 (74%) were prospective, and 458 (53%) were cross sectional. The most common area of research was trauma (n = 150, 17%). Four hundred and seventy four (54%) articles were published in WOS indexed journals. The median number of citations of WOS indexed articles was 5 (IQR = 15) but was only 0.5 (IQR = 3) in non WOS indexed articles (P < 0.01). Other descriptive data are presented in the Table 1. The word cloud matrix for keywords and research field was presented in the Figure 2.

Figure 1. Flowchart of the included dissertations and their methodology

Figure 2. Word cloud matrix for keywords and research field. WOS: Web of science

Table 1. Descriptive characteristics of published articles derived from dissertations

Univariate regression analysis was used to identify significant factors affecting publication. Subsequently, multivariate regression analysis was used to identify factors associated with successful publication of EM dissertations in WOS indexed journals. The Hosmer and Lemeshow test indicated that regression results were acceptable (P = 0.9). Multivariate analysis identified the following significant factors: Prospective data collection (odds ratio [OR] = 2.5, 95% confidence interval [CI] = 1.8–2.5, P < 0.01), experimental design (OR= 2, 95%, CI = 1.3–3, P = 0.001) university type residency program (OR = 1.4, 95% CI = 1.02–2.1, P = 0.04) and shorter duration between the year of dissertation completion and publication (OR= 0.9, 95% CI = 0.84–0.95, P < 0.01).

Among Turkish EM residency programs, the Dokuz Eylul University EM residency program had the highest number of reported dissertations (n = 152, 5%) followed by Akdeniz and Cukurova Universities (n = 101, 4%, and n = 80, 3%, respectively). However, Kocaeli University had the highest program productivity index (70%), including WOS indexed publications [64%, Table 2].

Table 2. Turkish emergency medicine residency program dissertation publication productivity showing the top five institutions in descending order from first to fifth

Discussion

This study showed that only 35% of EM dissertations were published in any scientific journals, and only 18% (474/2581) were published in WOS indexed journals. Those published in WOS indexed journals had a significantly greater number of citations than those not published in this group of journals; factors most strongly associated with publication in WOS indexed journals were prospective data collection and experimental design.

Several studies from various specialties have previously reported descriptive features of publications derived from dissertations, such as publication rates, citation analysis, research fields, research methodologies, or journal indexes. In 2014, Cevik et al., reporting on 579 dissertations in the field of EM between 1998 and 2013, found a publication rate of 27%, of which 14.9% were in Science Citation Index Expanded (SCI E) indexed journals.[10] Most of these studies were prospective observational, which is consistent with our findings. However, since 2014, we found that both the number and publication rates of dissertations in EM have markedly increased. This may be due to increased availability of academic EM in Turkey.[12] The publication rate varies between the other specialties. A publication rate of 39.1% in anesthesiology,[6] 37.9% in neurosurgery,[8] 37.7% in psychiatry,[13] and 20.5% in general surgery[3] have been reported. In this study, we found an overall publication rate of 35.5% in the field of EM. Although these studies are roughly indicative of the publication productivity of each of these medical specialties, reported publication rates may differ within the same specialty.[3] There are several reasons for this including different study methodology and the range of publication databases used. WOS is one of the largest and most trustworthy global citation databases. WOS includes the SCI E and Emerging Source Citation index (ESCI) in medical fields.[14] In Turkey, publishing in SCI E indexed journals is one of the major criteria for academic promotion in medicine. Therefore, there will be a national bias towards publishing in SCI indexed, and therefore WOS indexed, journals. However, in contrast to previous studies, we did not classify articles into SCI E or ESCI indexes. Instead, we used the catch all grouping of WOS indexed publications as a study marker of high quality publication. It is notable that all the criteria related to quality of journal are the same in both SCI E and ESCI indexes, the only difference between the two groupings being related to impact.[14]

In comparing the study characteristics of dissertations between WOS and non WOS indexed publications, we found that the median number of citations were higher in WOS indexed journals. This result was expected since WOS indexed high quality journals prefer to publish better designed articles that have higher citation potential. Publication in an indexed journal also provides greater article visibility and citation potential. Although not statistically significant, the median duration between dissertation and publication year was lower in articles indexed in WOS compared to non WOS indexed publications; a median of 2 versus 3 years, respectively. The average duration between residency graduation and publication of dissertation research is fairly similar in other specialties such as 3.83 ± 2.98 years in surgery,[3] 3.46 ± 2.62 in anesthesiology,[6] 3.8 ± 2.7 in neurosurgery.[8] This result can also be predicted since the gap duration between graduation and publication years could be caused by repetitive rejection from the journals in poor designed studies or the journal’s reluctance to publish older data.

Several factors can affect the likelihood of a dissertation being published in a high quality journal. These include the quality of research, design, contribution to the field, or writing quality. Most of these factors are closely related to mentorship, research environment, and access to the resources during residency training. We investigated factors associated with publishing in WOS indexed journals, in contrast to earlier articles in the field of EM. Factors which emerged included prospective data collection and good experimental design. These factors are mainly associated with the quality of data. Graduating from a university type residency program was also associated with publication in the high quality journals. This may be due to the better research environment, and access to resources and mentorship in academic university hospitals. Although the gap between years of graduation and publication was not statistically significant in the univariate analysis, we included this variable in the multivariate analysis since the novelty of data was an essential factor for publishing in high quality journals. Our results showed that the likelihood of publishing in the high quality journal decreased by 10% for each year after the completion of dissertation study.

Limitations

This study has several limitations. First, it was a retrospective study so that some probably significant variables could not be evaluated. Second, analyses were used to differentiate between studies published in WOS and other indexes, but unpublished dissertations were not analyzed. Therefore, we were unable to comment on factors associated with failure to publish in the medical literature. Third, the characteristics of the dissertation authors, advisors or host institutions were not evaluated. Academic and sociodemographic characteristics of these variables will almost certainly be significant in affecting the chance of high quality publication. Fourth, high quality publication was defined as journals indexed in WOS. Although WOS requires journals to have some determined high quality standards, it is not a universally accepted definition. Fifth, we did not differentiate WOS indexed journals by their impact factors. Finally, being indexed in SCI E could indicate a higher quality of the publication but we included this index in the overall grouping of WOS.

Conclusion

EM is one of the most productive disciplines contributing to scientific literature from residency dissertations. Conducting dissertation research prospectively with an experimental design increases the likelihood of publication in high quality journals. Moreover, completing the residency in a university type program and earlier submission to scientific journals were also associated with high quality publication.

How to cite this article: Özturan İU, Sarbay İ. Scholarly impact of the dissertation requirement for postgraduate medical education and factors affecting transformation into publication: Abibliometric analysis of 2434 dissertations in the field of emergency medicine. Turk J Emerg Med 2023;23:219-24.

Institutional review board approval was obtained for this study on October 13, 2022. (Kocaeli University Non‑Interventional Clinical Research Ethics Committee ‑ GOKAEK‑2022/17.17).

•  İbrahim Ulaş Özturan: Conceptualization, methodology, formal analysis, investigation, Writing ‑ Original Draft, Review and Editing, Supervision, Project administration
•  İbrahim Sarbay: Conceptualization, methodology, investigation, software, resources, data curation, visualization, writing – review and editing.

None Declared.

None.

References

1. Turkish Higher Education Law No. 2547. Web Site. Available from: https://www.mevzuat.gov.tr/mevzuat?MevzuatNo =2547&MevzuatTur=1&MevzuatTertip=5. [Last accessed on 2023 Feb 14].
2. Regulation on Residency Education in Medicine and Dentistry. Web Site. Available from: https://www.resmigazete.gov.tr/ eskiler/2022/09/20220903-2.htm. [Last accessed on 2023 Feb 14].
3. Ferhatoğlu MF, Kıvılcım T, Kartal A, Filiz Aİ, Kebudi A. An analysis of general surgery theses set up between years 1998-2018 in Turkey: Evidence levels and publication rates of 1996 theses. Turk J Surg 2020;36:9-14.
4. Ersel M, Yürüktümen A, Özsaraç M, Kiyan S, Aksay E. International publications of Academic Emergency Medicine Departments in Turkey: 15th year analysis. Turk J Emerg Med 2010;10:55-60.
5. National Thesis Center. Web Site. Available from: https://tez. yok.gov.tr/UlusalTezMerkezi/. [Last accessed on 2023 Feb 14].
6. Küpeli İ, Yazıcı Kara M, Kayhan A. From thesis to publication – Analysis of 2580 theses in the field of anesthesiology and reanimation. Acta Med Cordoba 2022;53:144-9.
7. Koca K, Ekinci S, Akpancar S, Gemci MH, Erşen Ö, Akyıldız F. An analysis of orthopaedic theses in Turkey: Evidence levels and publication rates. Acta Orthop Traumatol Turc 2016;50:562-6.
8. Sarica C, Sayman OA. Analysis of research productivity of neurosurgical residents in Turkey and publication rates of theses. Turk Neurosurg 2020;30:673-8.
9. Mayir B, Bilecik T, Çakır T, Doğan U, Gündüz UR, Aslaner A, et al. Analysis of the publishing rate and the number of citations of general surgery dissertations. Turk J Surg 2017;33:33-6.
10. Cevik E, Karakus Yilmaz B, Acar YA, Dokur M. Systematic analysis of theses in the field of emergency medicine in Turkey. Turk J Emerg Med 2015;15:28-32.
11. Chavarro D, Ràfols I, Tang P. To what extent is inclusion in the web of science an indicator of journal ‘quality’? Res Eval 2018;27:106-18.
12. Bildik F, Aslaner MA, Kılıçaslan İ, Keleş A, Aksu ŞH, Yazla M, et al. Trends in medical student’s career choice of emergency medicine: A 14-year study. Turk J Emerg Med 2020;20:75-80.
13. Erim BR, Petekkaya S. Retrospective analysis of psychiatry specialization theses made between 1981-2018 in Turkey. Turk Psikiyatri Derg 2020;31:1-8.
14. Web of Science Core Collection. Web of Science Journal Evaluation Process and Selection Criteria. Clarivate. Web Site. Available from: https://clarivate.com/products/scientific-and-academicresearch/research-discovery-and-workflow-solutions/web-ofscience/core-collection/editorial-selection-process/editorialselection-process/. [Last accessed on 2023 Feb 14].

We thank Prof. Nurettin Özgür Doğan, MD, and Dr. İsmail Sarbay, MD, for their helps as we were conducting this study.