MULTIPLE-CRITERIA EVALUATION MODEL FOR MEDICAL PROFESSIONALS ASSIGNED TO TEMPORARY SARS-COV-2 HOSPITALS

Hospitals around the world, as health institutions with a key role in the health system, face problems while providing health services to patients with various types of diseases. Currently, those problems are intensified due to the pandemic caused by SARS-CoV-2 virus. This pandemic has caused an extreme spread of the disease with constantly changing needs of patients which impacts the capacities and overall functioning of hospitals. In order to meet the challenge of the COVID-19 (COronaVIrus Disease- 2019) pandemic, health systems must adjust to new circumstances and establish separate hospitals exclusive for patients infected with SARS-CoV-2 virus. In the process of creating COVID-19 hospitals, health systems face a shortage of medical professionals trained for work in COVID-19 hospitals. Using this as a starting point, this study puts forward a two-phase model for the evaluation and selection of nurses for COVID-19 hospitals. Each phase of the model features a separate multiple-criteria model. In the first phase, a multiple-criteria model with a dominant criterion is formed and candidates who meet the defined requirements are evaluated. In the second phase, a modified multiple-criteria model is formed and used to evaluate medical professionals who do not meet the requirements of the dominant criterion. By applying this model, two groups of medical professionals are defined: 1) medical professionals who completely meet the requirements for working in COVID-19 hospitals and 2) medical professionals who require additional training. The criteria for evaluation of medical professionals in this multiple-criteria model are defined based on research conducted on medical professionals assigned to the COVID-19 Crisis Response Team during the COVID-19 pandemic in the Republic of Serbia. The model was tested on a real example of evaluating medical professionals assigned to the COVID-19 hospital in Sombor. The model for evaluating medical professionals presented in this paper can help decision makers in hospitals and national policy makers to determine the readiness level of hospitals for working in the conditions of the COVID-19 pandemic, as well as underline the areas in which hospitals are not ready to meet the challenges of the pandemic. © 2020 Regional Association for Security and crisis management. All rights reserved.


Introduction
) is a disease caused by a novel virus from the group of coronaviruses, first isolated in 1962. Since then, it is known that some viruses from the coronavirus group, infect only certain animals, some humans, while some can breach the barrier between species, causing states ranging from a mild cold to severe acute respiratory syndrome (SARS). A new, so far unknown coronavirus, SARS-CoV-2, the cause if COVID-19 disease, belongs to the same subgroup as MERS-CoV and SARS-CoV, and was first detected in the Chinese city of Wuhan, in the province of Huubei, the ground zero of the epidemic.
WHO (World Health Organisation) declared a global pandemic in March 2020 and started a research mission into it. The COVID-19 pandemic came as a surprise and brought with it many challenges to health systems all over the world, including our own. The immediate demand for staff training, repurposing structures, equipment acquisition, communication management, continuous supervision, healthcare availability to special groups of population under new conditions etc. has created numerous operational, logistical, organizational and ethical tasks for managers, medical professionals and associates.
Success and weaknesses of the health systems during the pandemic, is graded by the global health safety index, used for evaluating readiness in prevention, detection, fast response to high-risk environments, and following international protocols in new conditions. Due to the specifics of the COVID-19 pandemic, continuous analysis of all advantages and weaknesses of health systems and planning based on adaptive models of operation and providing services are necessary because the immediate demands and the unpredictability of the COVID-19 infection requires agility and flexibility of all services in regards to timely responses, especially in the case of medical professionals.
Previous research on evaluating the readiness of hospitals and medical professionals for performance in crises and disasters is very limited. This applies especially to the application of multiple-criteria decision models in this field, which limited the analysis of already available literature. Most authors consider the application of two concepts for solving this problem: 1) application of multiple-criteria decision making for evaluating readiness of hospitals in crises and 2) application of research based on surveys and statistical analysis. Nekoie-Moghadam et al. (2016) present a comprehensive review of literature with different methodologies used for evaluating hospitals for work in crises. In their review, they considered the most important topics that relate to evaluating health systems, such as logistics, planning, human resources, communication, management and control, training, evacuation, disaster recovery, coordination, transport, safety (Fallah-Aliabadi et al. 2020;Verheul and Duckers, 2020;Nekoie-Moghadam et al., 2016). Of the 15 papers considered in total by Nekoie-Moghadam et al. (2016), all consider the application of research based on statistical analysis of data collected in surveys. In some situations, authors used statistical methodologies, such as the Delphi technique and similar tools (Rezaei and Mohebbi-Dehnavi, 2019). A comprehensive review of literature that considers the application of statistical tools for hospital readiness analysis in crises is presented in Fallah-Aliabadi et al. (2020), Verheul and Duckers (2020), Alruvaili et al. (2019) and Nekoie-Moghadam et al. (2016). In their study Tabatabaei and Abbasi (2016) carried out risk assessment during crises based on the Hospital Safety Index. The Safety Index was defined based on comprehensive research with statistical data processing. Samsuddin et al. (2018) identified key factors to determine the readiness level of hospitals for work in crises. The results have demonstrated that human resources, their training and ability to adapt in a timely fashion, are the crucial factor. Marzaleh et al. (2019.) put forward an approach where the Delphi technique is used to evaluate the readiness of emergency services in hospitals for working in crises. Their study identified 31 criteria grouped in 3 clusters. The results demonstrate that training of medical professionals has the highest priority. A similar approach for analyzing hospital capacities using the Delphi technique was demonstrated by Shabanikiya et al. (2019.).
However, aside from statistical processing of factor weight used for evaluating hospitals in crises, previous research also contains a number of papers based on multiple-criteria techniques. For example, Mulyasari et al. (2013) have applied multiple-criteria techniques for ranking eight hospitals in Iran. The ranking was carried out using factors grouped in four clusters, based on which hospital structural and functional readiness in crises was evaluated. Following this study, Hosseini et al. (2019.) developed a model for ranking hospitals based on the TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution) multiple-criteria technique. The study identifies 21 factors grouped in 4 clusters. However, this study used direct evaluation of survey participants instead of subjective/objective methods for determining weight coefficients of criteria. Also, Ortiz- Barrios et al. (2017) demonstrate the possibility of applying analytical multiple-criteria approach for analyzing the readiness of particular wards/clinics in hospitals during crises. The approach is a hybrid model that consists of: 1) applying the AHP (Analytic Hierarchy Process) and the DEMATEL (Decision Making Trial and Evaluation Laboratory) methods for determining weight coefficients of criteria and their mutual relationships and 2) applying the TOPSIS method for evaluating hospital capacities. Unlike previously listed studies, Ortiz-Barrios et al. (2017), in addition to defining weight coefficients of criteria, also put forward a methodology for defining mutual relationships and impact of factors used for evaluation. Following Ortiz- Barrios et al. (2017), Roy et al. (2018) identified key factors for evaluating hospital capacities using the DEMATEL method. However, unlike Ortiz- Barrios et al. (2017), Roy et al. (2018) used rough numbers to exploit the lack of certainty and precision found in expert preferences.
Considering that the crisis caused by the COVID-19 pandemic is still ongoing, there is no research that considers the problem of evaluating the training of medical professionals for working in COVID-19 hospitals. There is a limited number of papers that consider the application of multiple-criteria tools for solving problems caused by the COVID-19 pandemic. Sarkar (2020) maps the areas susceptible to COVID-19 infection in Bangladesh. The AHP method, in conjunction with GIS (Geographic Information System) spatial analysis were used for area mapping. Sangiorgio and Parisi (2020) used Artificial Neural Network and GIS for mapping COVID-19 infection risks in urban zones in Italy. Their study showed spatial analyses of a total of 257 city districts. Nardo et al. (2020) demonstrated the application of multi-criteria decision analysis for determining weights for eleven criteria in order to prioritize COVID-19 non-critical patients for admission to hospitals in healthcare settings with limited resources. Yildirim et al. (2020) evaluated the available COVID-19 treatment options in hospitals. For their evaluation, they used modified PROMETHEE (Preference Ranking Organization METHod for Enrichment of Evaluations) and VIKOR (VIseKriterijumska Opitimizacija i Kompromisno Resenje in Serbian) technique by applying fuzzy numbers.
As we can see from the above-mentioned studies, there are numerous approaches relating to evaluating readiness of hospitals in crises. Most papers contribute by putting forward methodological frames that require application of research featuring statistical processing of data collected via surveys. On the other hand, based on the above-mentioned literature, we can see the significance of multiple-criteria techniques for researching topics pertaining to evaluating readiness of hospital capacities in crises. We can note a wide spectrum of multiple-criteria techniques used in literature and applied in various fields. However, the number of papers that apply these methods for evaluating hospital capacities is limited. Furthermore, the number of papers that consider the application of multiple-criteria methods for evaluating hospitals and medical professionals in the conditions of the COVID-19 pandemic is especially limited. Therefore, it is the aim of this paper to develop a multiple-criteria model for selection and evaluation of medical professionals for working in COVID-19 hospitals in the conditions of a pandemic. The suggested model deals with assessing the training of medical professionals for working in the conditions of the COVID-19 pandemic. Furthermore, the suggested model has the following advantages that improve the literature relating to the application of multiple-criteria techniques in the field of healthcare: 1. An original, multiple-criteria methodology for evaluating propriety of medical professionals for working in COVID-19 hospitals has been developed. The demonstrated methodology is conducted in two phases. A separate multiplecriteria model has been developed for each phase. The criteria and criteria value scales have been defined following months of research with the participation of medical professionals from the COVID-19 Crisis Response Team of the Republic of Serbia. 2. The demonstrated methodology is not limited to application in the healthcare field and is applicable in other fields due to its adaptability 3. The suggested methodology provides a new, clear and concise frame for resource management. In order to illustrate the effectiveness of the suggested methodology, an empirical study of the application of this multiple-criteria methodology is presented in the paper. 4. The approach presented in this paper can solve the problem of selection of medical professionals in the conditions of the COVID-19 pandemic in a systematic and analytical way. The developed model was implemented and tested in a case study of the COVID-19 hospital in Sombor (Serbia). 5. The model for evaluating medical professionals presented in this paper can help decision makers in hospitals and national policy makers to determine the readiness level of hospitals for working in the conditions of the COVID-19 pandemic, as well as underline the areas in which hospitals are not ready to meet the challenges of the pandemic. The paper is structured into four sections. After the introduction that presents the problem and analyzes the existing literature, the second section mathematically formulates the multiple-criteria model for evaluating medical professionals in the conditions of the COVID-19 pandemic. The third section presents the implementation of the multiple-criteria model on a real example of evaluating medical professionals in the COVID-19 hospital in Sombor, the Republic of Serbia. The fourth section of the paper presents the conclusion and directions for future research.

Multiple-criteria Evaluation Model for Medical Professionals Assigned to COVID Hospitals
Let us assume a multiple-criteria model with defined criteria   12 , ,..., where n stands for the number of criteria used in the multiple-criteria model. Also let us assume a set of alternatives where m stands for the number of alternatives to be ranked in the model. We can define the decision matrix mxn whose elements ij a stand for value of j criterion for i alternative.  In multiplecriteria models with a dominant criterion, there is a problem where it is necessary to choose more than one alternative, for example p alternative (p<m) while criterion Cs is met only with q alternatives, where q<p. This case is characteristic of crises where it is necessary to choose a certain number of candidates (alternatives) who are not sufficiently prepared for working in such conditions. This example considers the crisis caused by SARS-CoV-2 virus. This paper develops a model that makes it possible to evaluate medical professionals for assignment in COVID-19 hospitals. The multiplecriteria evaluation model is conducted in two phases, Figure 1.  The model is conducted in two phases. A separate multiple-criteria model has been developed for each phase. For evaluating the training of medical professionals in the first phase, a multiple-criteria method with a dominant criterion has been developed. Medical professionals who meet the requirements of the evaluation defined by the dominant criterion are directly assigned to COVID-19 hospitals. In the second phase, medical professionals who do not meet the requirements of the first phase are evaluated. For evaluating medical professionals in the second phase, a simple additive weighting operator was used. Medical professionals selected during the second phase undergo additional training for work in a COVID-19 hospital. Below, we present the multiple-criteria models in the first and second phases.  Step 2: Determining weight coefficients of the criteria In this paper, the authors used Non-Decreasing Series at criteria significance Levels (NDSL) (Zizovic et al., 2020;Pamucar et al., 2020a) model for determining weight coefficients of criteria, Figure 2.
The determination of the most significant criterion from within the set of the criteria The ranking of the criteria in accordance with the experts' preferences: The grouping of the criteria according to the significance levels: Level L 1 : Defining the boundary values of the criteria significance(s): Level L 1 : Defining the functions of the criteria significance(s) The calculation of the optimal values of the weight coefficients of the criteria Figure 2. The NDSL model (Zizovic et al., 2020) A detailed presentation of phases of the NDSL model can be found in (Zizovic et al., 2020).
Step 3 (1) and (2) where ij x stands for elements of the preliminary decision matrix ij mn Xx     .
Step 5: Evaluation and selection of candidates that meet additional criterion s C . The evaluation is conducted by applying expression (3) (Pamucar et al., 2020b(Pamucar et al., , 2020c). If we assume that we chose as the dominant criterion, the final grade of the candidate is defined by applying the following expression:  (3), is selected, and there is no need for the second phase of the model. However, if the number of the needed candidates who meet the dominant criterion is lower than p, the second phase of the model is carried out. The remaining (q) candidates, who do not meet the dominant criterion, are ranked in the second phase, presented below.
Second phase: Forming a modified multiple-criteria model. In the modified multiple-criteria model, we adapt the starting set of criteria and evaluate q candidates as per newly defined set of criteria. Let us assume that the first criterion is dominant, i.e. that Step 2), here too, we use the NDSL model for determining weight coefficients of criteria (Behzad et al., 2020), Slika 1.
Step 2: Forming the preliminary decision matrix .
ij mn Yy     . where we select q ( 1 qm  ) candidates for assignment to a COVID-19 hospital.
Step 3 where ij y stands for elements of the preliminary decision matrix ij mn Yy     .
Step 4: Evaluation and selection of candidates. Each candidate is assigned a value of the criteria function based on which the evaluation is carried out. The criteria function is defined by applying the following expression (6)

Results
This chapter considers the case study of organizing hospitals specially prepared for the admission of patients suffering from a SARS-CoV-2 infection. Such hospitals in the Republic of Serbia were organized as separate parts of existing hospital capacities or were repurposed as entire hospitals to admit only patients diagnosed with COVID-19 during the crisis. Simultaneously, there was a need for medical professionals trained for working in the newly formed medical institutions. There was a need for priority treatment of patients suffering from a largely unknown, high-risk disease, with too few qualified medical professionals in the field. Additionally, there was a need for selection of qualified medical professionals to carry out the listed duties, as well as simultaneously train other medical professionals that, at the moment, were not trained for assignment to COVID-19 hospitals.
This study presents the application of the model for evaluating medical professionals for working in COVID-19 hospitals on the example of the COVID-19 hospital in Sombor, Republic of Serbia. The study features part of the general hospital in Sombor repurposed for treatment of COVID-19 patients. Eight criteria were identified that were used for evaluating nurses in two phases of the model. Criteria and criteria value scales for evaluation of medical professionals were defined based on a survey with members of the COVID-19 Crisis Response Team and with medical doctors who participated in the operation of the COVID-19 hospital in Sombor. All candidates for assignment in COVID-19 hospitals were psychologically tested and interviewed by teams from the COVID-19 Crisis Response Team (teams of medical doctors). Data obtained in this way served to form evaluations for criteria. Below, we present eight criteria with their value scales. The presented criteria are applied for evaluation in the first phase of the model. C1 -Experience in working with infectious and pulmonary diseases in hospital treatment (C1). Value scale:  Nurse with work experience at an infectious or pulmonary diseases ward in hospital treatment -5 points;  Nurse with work experience at internal medicine ward in hospital treatment -4 points;  Nurse with work experience at a different ward in hospital treatment -2 points;  Nurse with no work experience in hospital treatment -1 point. C2 -Professional training of the candidate for working with COVID-19 diagnosed (Covid+) patients in "COVID-19" zone:  Professional training for immediate work with COVID-19+ patients, use of personal protective equipment and materials in COVID-19 conditions, knowledge of work organization in COVID-19 conditions, training for movement through safety zones -5 points  Professional training for immediate work with COVID-19+ patients, use of personal protective equipment and materials in COVID-19 conditions, knowledge of work organization in COVID-19 conditions -4 points;  Professional training for immediate work with COVID-19+ patients, use of personal protective equipment and materials in COVID-19 conditions -3 points;  Professional training for immediate work with COVID-19+ patients -2 points;  No professional training -0 points. C3 -Health risk of the candidate:  No health risk -10 points;  Diseases and injuries that do not significantly affect ability -8 points;  Physical injuries that partially affect mobility -7 points;  Lower risk chronic diseases in COVID-19 conditions -6 points;  Single parent of a child up to 12 years old -3 points;  Parent of a child up to 3 years old -2 points;  Age (over 60) -1 point;  Chronic diseases like: diabetes, psychosomatic diseases, autoimmune diseases, malignancy, diseases or treatment with a negative influence on the immune system, pregnancy -0 points. C4 -Physical evaluation of candidates for work in difficult working conditions and in shifts:  Mandatory vaccination (with BCG+POLIO) -5 points;  Mandatory vaccination (without BCG/POLIO) -4 points;  Basic vaccination -2 points;  None -1 point. After defining the criteria, below we present the application of the model on the selection of medical professionals.
First phase: Defining an additional criterion from the set of criteria and forming a multiple-criteria model with a dominant criterion.
Step 1 Step 2: Determining criteria weight coefficients j w ( 1, 2,...,8 j  ). As noted above, for determining criteria weight coefficients we used the NDSL model (Žižović et al., 2020). Determining criteria weight coefficients using the NDSL model is presented below in steps 2.1 -2.5.
Step 2.1: Determining the most important criterion from the criteria set   1 2 8 , ,..., j C C C C  and ranking criteria. As C1 was defined as the dominant criterion, C1 is also the most important criterion in the set j C . Based on evaluations of experts, the criteria from the set j C were ranked as follows: C1>C7>C3>C4>C5>C6>C2>C8.
Step 2.2: Grouping criteria into significance levels. The criteria were grouped into four levels, as follows: Level Step 2.3: Based on relations for defining border values of the criteria ( ) the values i  were defined for criteria relating to significance levels: f Step 3: Forming the preliminary decision matrix. The Crisis Response Team carried out the evaluation of medical professionals from the pulmonology and infectious wards of the general hospital in Sombor. These two wards counted in total 43 nurses who were candidates for working in the COVID-19 hospital, i.e. for working in the "red" and "orange" zones. A selection of 29 nurses was considered of the available 43 candidates. Based on the defined evaluation criteria and available number of medical professionals, the preliminary decision matrix was formed Table 3.  Alt  C1  C2  C3  C4  C5  C6  C7  C8  A1  5  5  10  5  4  4  5  5  A2  2  5  10  5  5  4  5  5  A3  5  5  10  5  5  4  4  5  A4  1  2  0  1  1  5  1  5  A5  5  5  10  4  5  5  5  4  A6  5  5  10  3  5  5  5  5  A7  5  5  7  3  5  3  5  5  5  5  5  5 Step 4: Normalization of elements of the preliminary decision matrix. Since all the considered criteria fall under the max type (higher value is better), for the normalization of values we used the expression (1). The elements of the normalized matrix are shown in Table 4. Step 5: Evaluation and selection of medical professionals who meet the dominant criterion 1 C . Medical professionals who have the value of the dominant criterion 1 4 C  enter the evaluation process in the first phase of selection. Border value for the dominant criterion is defined based on the evaluation of experts. The remaining candidates who do not meet the requirement move on to the second phase of selection. Based on the analysis of the preliminary decision matrix (Table 3) we see that 26 candidates meet the requirement of 1 4 C  . By applying the expression (3) we evaluate and rank the candidates, Table 5.
In a similar way, the remaining candidates' grades were attained, as shown in Table  5.
The mandatory criteria were met by 26 candidates, who were ranked in Table 5. Since 29 candidates were needed for the COVID-19 hospital, the remaining 17 candidates were ranked in the second phase of the model with the aim of selecting 3 additional, most appropriate candidates.
Second phase: Forming a modified multiple-criteria model. In the modified multiple-criteria model, we adjust the starting set of criteria. Criteria C1 and C2 are eliminated and in their place, we insert new ones: * 1 C -Evaluation of speed of acquiring knowledge and skills for working in COVID-19 institutions:  Very quick and safe start of work activities -5 points;  Sufficiently quick and safe start of work activities -4 points;  Satisfactorily quick and safe start of work activities -3 points;  Slow but safe start of work activities -2 points;  Slow and barely safe start of work activities -1 point. , ,..., were ranked as follows: C1 * >C7 * >C4 * >C6 * >C3 * >C8 * >C5 * >C2 * .
Step 1.2: The criteria were grouped into sets of four levels, as follows:   Step 2: Forming the preliminary decision matrix. Since the minimum criteria were not met by 17 of 43 candidates, we form the preliminary decision matrix of rank , Table  7. Steps 3 and 4: Normalization of elements of the preliminary decision matrix is done by applying the expression (4). After the normalization of the decision matrix elements, by applying expression (6) we define the grade for each candidate. The final grades and candidate ranking are shown in Table 8. After the evaluation of candidates (Table 8), three best ranked candidates were selected, and after completing a training cycle, were assigned to a COVID-19 hospital. The training program is defined by the Crisis Response Team. The remaining 14 candidates also completed the training program but are currently not assigned to the COVID-19 hospital. They are available for assignment in case of assigned staff being removed from the team for self-isolation. Self-isolation may be a consequence of accidental exposure (human error, breakdown of equipment, etc.) or disease.

Conclusions
Management of human resources is a key segment that affects the efficacy of the health system of every country. This is especially obvious in crises, like the COVID-19 pandemic. This is why it is necessary to efficiently manage human resources in hospitals, to reduce as much as possible the dangers caused by the COVID-19 pandemic. As far as the authors are aware, there are no current models for considering the evaluation of medical professionals' training for working in crises, so the motivation for a study such as this is logical.
In this paper, we put forward a multiple-criteria model that allows decision makers in medical institutions and national crisis response teams to evaluate the training of medical professionals for working in the conditions of the COVID-19 pandemic. For the needs of this multiple-criteria model, we defined criteria based on which we evaluate medical professionals. The criteria and criteria evaluation scales were defined after months of research with participation from health institution managers and members of the Crisis Response Team of Serbia. The developed multiple-criteria model is conducted in two phases. The first phase evaluates medical professionals according to one or more dominant criteria. Medical professionals who meet the conditions defined in the first phase, meet the conditions necessary for working in a COVID-19 hospital. Medical professionals who do not meet the conditions defined in the first phase, move on to the second phase of evaluation. After completing the second phase of evaluation, the staff who partially meet the conditions are identified and they undergo training for working in COVID-19 hospitals. This methodology was applied to the example of the COVID-19 hospital in Sombor.
The suggested methodology can be used for other decision problems, by adapting the criteria according to the nature of the decision problem. The basic advantage of this study is application, i.e. testing of the suggested methodology on objective data in the conditions of the COVID-19 pandemic. This demonstrates, on a real example, all advantages of this methodology. Future research should be directed towards implementing the suggested methodology in the conditions of uncertain input model parameters (Ecer and Pamucar, 2020). Uncertainty in future research can be exploited by applying various uncertainty theories such as fuzzy theory or rough theory.
Author Contributions: Each author has participated and contributed sufficiently to take public responsibility for appropriate portions of the content.
Funding: This research received no external funding.

Conflicts of Interest:
The authors declare that there is no conflict of interest.