COMPARATIVE ANALYSIS OF NORMALIZATION TECHNIQUES IN THE CONTEXT OF MCDM PROBLEMS

: Normalization is an essential step in data analysis and for MCDM methods. This study aims to outline the positive and negative features of the normalization techniques that can be used in MCDM problems. In order to compare the different normalization techniques, fourteen sets representing different scenarios of decision problems were used. According to the results, if the decision-maker chooses to take the alternative with the highest value in the criteria and avoid the one with the lowest value, or vice versa, optimization-based normalization techniques should be preferred, whereas the reference-based normalization techniques are considered appropriate for situations where there are ideal values determined by the decision-maker for each criterion. However, if the decision-maker believes that the values in the criteria do not represent the monotonous increasing or decreasing benefit/cost, then non-linear normalization techniques should be used. Also, in the event of a change in the conditions mentioned above, the decision maker may opt for mixed normalization techniques. However, some data structures, such as the presence of zero, and negative values in the decision matrix, can prevent the use of some normalization techniques. The choice of the normalization technique may also be affected by the problem of rank reversal, the range of normalized values, obtaining the same optimization aspect for all criteria, and the validity of results.


Introduction
In quantitative research, researchers often try to use methods appropriate for the data structures. To do this, the data is first collected and then compiled. In the compilation process, it is always important to create the data structure required by the relevant method using scaling techniques.
In the scaling process, the unit of measurement, the size, and the level of the criteria are changed alongside one or more of the transformations, re-measurement, normalization, or weighting operations. The differences in various features of the criteria such as measurement levels, size of the range, importance levels and reflecting the decision maker's preferences effectively are prominent reasons for scaling. The other reason for scaling is the need to meet the assumptions of the method used for the research or the decision problem. In this context, the primary purpose of scaling is to provide the appropriate measurements or data structure for the proper method or analysis. Normalization is one of the critical processes used in scaling data (Jensen, 1984;Roberts, 1984;Lootsma, 1999;Tavşancıl, 2006;Kainulainen et al., 2009;Sarraf et al., 2013;Jahan & Edwards, 2015;Podviezko & Podvezko, 2015;Gardziejczyk & Zabicki, 2017).
There are numerous application areas for normalization including data mining, multivariate statistics and multi-criteria decision making (MCDM) among others. This study will however focus on the effects of the normalization processes on solutions obtained using MCDM methods.
Normalization is used to obtain criteria that have the same weight, are dimensionless, and are suitable for compensatory processes in MCDM problems. Normalization also enables the decision maker to show his preferences regarding the problem to a certain extent. There are many normalization techniques in the literature to achieve this. The choice of the normalization technique depends on the structure of problems and the assumptions of MCDM methods. Although not yet sufficient, studies on the comparison of normalization techniques have been increasing in the past decades. These studies, however, usually include a small number of techniques. Similarly, studies considering the selection of the normalization technique, and the criteria to be used in this selection process are also limited. Another important issue is that every normalization technique cannot be suitable for all decision problems. It is, therefore, necessary to investigate the extent to which the normalization techniques have achieved their purposes of development, their roles in the problem, and the MCDM method, their dimensionlessness, and comparability. In this context, this study will examine the practical comparisons of the normalization techniques, determine the positive and negative features, and outline the selection process of normalization techniques suitable for different data structures. The purpose of the study is thus to provide different perspectives on normalization techniques and a holistic framework for researchers and decision makers.

Normalization
Normalization is a scaling process used to make the criteria comparable by eliminating the optimization orientation (benefit, cost), the unit of measurement, and the variation range. Through normalization, the data is converted to a specific norm or standard. Another term often used interchangeably with normalization is standardization. However, standardization is a normalization process that eliminates unit differences and transforms values to a specific range, such as 0-1 in all criteria. In general, normalization techniques are expected to equalize the effect levels of all criteria (regardless of the weighting process), process the zero and negative values, generate the same normalized value for different units of measure that can be converted into each other (as in the case of g / cm 3 -kg / m 3 ), and not cause rank reversal problems while also ensuring symmetry in the cost and benefit optimization orientation. The normalization technique that has these features is considered successful (Pavličić, 2001;Jahan & Edwards, 2015;Podviezko & Podvezko, 2015).
In the analysis of normalization techniques in the MCDM literature, the decision matrix given in equation (1) will be used. The rows of the decision matrix contain alternatives while the columns carry the criteria. Each of its cells/elements shows the quality, feature, or performance value of each alternative in the relevant criterion. The elements of the decision matrix in equation (1) are expressed as xij. xij is the performance or result value of alternative i in criteria j. The following section of the study outlines the classifications of normalization techniques.

Classification of Normalization Techniques
Classification of normalization techniques makes it easier to identify the similarities and differences of the techniques, standardize the concepts in the field, and examine the increasing number of techniques. Various approaches can be used in the classification of normalization techniques. However, the most common classifications in the literature are done according to the distance measurements, the linearity of the normalization process or the optimization orientation of the criteria (Milani et al., 2005;Yoon & Kim, 1989;Zeng et al., 2013;Jahan & Edwards, 2015).
Distance measurements are the most commonly used in the normalization process. The distance-based normalization is the ratio of the distances of the alternatives from the starting point (vector 0) to the sum of distances of all alternatives from the starting point in the relevant criterion. In the distance-based normalization processes, Eq. (2) is used in the Lp metric. (Yoon & Kim, 1989, p. 22 Eq. (2) is used in the benefit criteria. For the cost criteria, the values are converted to benefit with the transformation of 1/xij values. In Eq. (2), Manhattan distance normalization is performed for p = 1, with Euclidean distance normalization for p = 2 and Tchebycheff distance normalization for p = ∞ (Yoon & Kim, 1989). In the literature, Manhattan distance normalization is called Sum-Based Linear Normalization, while Euclidean distance normalization is known as Vector Normalization.
For, normalization processes that are not based on distance, a specific value is used. Often, these values are the maximum and the minimum in the criterion. Similarly, reference values or large fixed numbers can be used.
The linearity of the normalization process is that the utilities or values in the criterion increase or decrease monotonously in a specific direction. In non-monotonic normalization processes, there is no continuous increase or decrease of acceptable performance values within the criterion in a certain direction. For example, in the criterion with a normal distribution, the linear normalization process will not be appropriate if the desired/ideal values are within three or four standard deviations of the mean. Z score normalization, some reference-based normalizations, and nonlinear normalizations are some examples of non-monotonic normalization techniques (Zavadskas & Turskis, 2008;Zeng et al. 2013).
We can divide normalization techniques into two fundamental classes based on whether they consider the optimization orientation of the criteria. However, some normalization techniques provide a mixed/integrated normalization process with the idea that the optimization orientation and reference value are vital for different criteria that can be found at the same time in the decision problem. The following section will examine the normalization techniques which depend on the optimization orientation, those that are independent of the optimization orientation, and those that have a mixed structure.

Normalization Techniques Depending on the Optimization Orientation
Most normalization techniques provide normalization according to the optimization orientation of the criteria. The optimization orientation is divided into two -benefit and cost. The benefit optimization orientation implies that the increase in the performance values of the alternatives evaluated in criterion j is preferred to the decrease. The cost optimization orientation is that the decline in the performance values of the alternatives in criterion j is preferred to the increase. In general, we can say that the highest (maximum) value in benefit-orientation criteria and the lowest (minimum) value in cost-orientation criteria are preferred.
Normalization techniques depending on the optimization orientation are given in Table 1 (Brauers & Zavadskas, 2006;Zavadskas & Turskis, 2008;Fayazbakhsh et al., 2009;Jahan & Edwards, 2015;Gardziejczyk & Zabicki, 2017). These techniques mainly use performance value totals, the maximum value, and the minimum value in a criterion.  Gardziejczyk & Zabicki (2017) It is possible to further divide the normalization techniques depending on the optimization orientation into four sub-classes: sum-based, a maximum or minimum value-based, range-based, and others. From the techniques in Table 1, while N1, N2, and N3 are sum-based, N4-N9 are maximum-minimum value-based, N10 is rangebased, and N11 is evaluated under the other category.
In sum-based normalization techniques, the sum of performance values within the criterion is used. It seems that the normalization techniques in this class may lead to the rank reversals problem due to the changes (adding or removing alternatives) in the alternative set. For example, when the alternative performance with the highest performance value is removed from criterion j, which has a benefit optimization orientation, the maximum value used in N4 will change. Change of the maximum value in criterion j will require the recalculation of normalized values. Also, when the numerator or denominator values change, all normalized values to be generated by the techniques in Table 1 will change in criterion j.
Another critical issue is the presence of the criterion that has negative values in the decision problem. The presence of negative values in a criterion can prevent effective solutions in N1 and N2 depending on the nature of the problem. The negative and positive values in N1 can be said to be mutually offsetting. Also, the negative performance values (xij) in N2 cause negative normalized values. And these situations can prevent the realization of effective solutions.
The range of normalized values obtained by the normalization techniques in Table  1 are different from each other. The sum of all normalized values is equal to 1 in N1 for benefit-orientation criteria and N3 for all type criteria. Also, N1, N2, N3 are expected to generate nij in the range of 0-1. From these techniques, N3 was found to be useful if the values in a criterion are quite different from each other (Jahan & Edwards, 2015, p. 338).
Maximum/minimum value-based normalization techniques are said to be less successful than the sum-based normalization techniques in handling the scale effect. Also, some of the techniques in this class cannot be effectively applied to cost criteria. In some cases, the normalized values may be higher than 1 in the techniques of this category. This situation is generally undesirable in some MCDM methods (Jahan & Edwards, 2015, p. 338). Among the techniques in this class, it is aimed to provide normalization in the range of 0-1, with the highest value being 1 in N4 and the lowest value being 1 in N5. After normalization using N6, the best performance value depending on the optimization orientation of criteria is expected to equal 1 while all normalized values are expected to fall within the range of 0-1. In N7, the normalized values are expected to be in the range of 0-1 while the best value is equal to 1. Since N8 produces large normalized values, it does not seem appropriate for most MCDM methods. The range of normalized values obtained in N9 can be expected to be smaller than the normalized values created by most techniques.
N10 is one of the most used techniques in the normalization steps of MCDM methods. Providing range-based normalization, N10 is successful in handling the scale effect (Jahan & Edwards 2015, p. 338). Normalized values in N10 are expected to be in the range of 0 and 1. Z score normalization is frequently used in the application of multivariate statistical methods. Fayazbakhsh et al. (2009) used the Z score normalization as dependent of the optimization orientation. Optimization orientation dependent Z score normalization can generate negative values, but normalized values are usually around 0. This situation restricts N11's usage in MCDM methods (Jahan & Edwards, 2015).
Normalization techniques were developed to be used for specific purposes or expectations. We have highlighted these purposes and expectations for the optimization orientation dependent normalization techniques. On the other hand, we will test whether the intended or expected normalized values of these normalization techniques will always be obtained in the application section.

Normalization Techniques Independent of the Optimization Orientation
For normalization techniques independent of the optimization, a specific reference value/range or a constant is used instead of the optimization orientation of the Comparative analysis of normalization techniques in the context of MCM problems criteria. For the techniques in this category, one or more of maximum value, minimum value, mean, standard deviation, reference (ideal/target) value/range, adjustable constant number, and data distributions are used in the normalization process (Wu, 2002;Shih et al., 2007;Jahan et al., 2011;Jahan et al., 2012;Alpar, 2013;Saranya & Manikandan, 2013;Jahan & Edwards, 2015;Gardziejczyk & Zabicki, 2017;Aytekin, 2020).
Such values as maximum and minimum are also used in the normalization of optimization orientation-dependent techniques. The normalization techniques independent of the optimization orientation have no formula change based on the optimization orientation and use the same equation for all criteria types. Normalization techniques independent of the optimization orientation are given in Table 2.  Alpar (2013) In the normalization techniques in Table 2, Rj shows the reference value determined for criterion j. The reference value is the base, source, or guide point that reflects the decision maker's preferences in the decision problem. The reference value can be identified subjectively by the decision-maker, or it can be determined with the help of scientific tools or techniques (Aytekin, 2020). The arithmetic mean is generally used as a reference value in N12. The normalized values obtained in N12 are mainly in the range of 0 to ± 3. Jahan et al. (2011) used N13 in the extension of VIKOR. N14 and N15 allow the decision-maker to adjust the performance values within the criteria according to the average or standard deviation. The ρ parameter in N16 is determined by the digit value of the largest absolute number in the decision matrix. Thus, N16 can generate normalized values are between -1 and +1. N17, N18, and N19 are techniques that provide normalization based on reference. Jahan et al. (2012), in their study on material selection, proposed N17, which is the extension of Weitendorf's Linear Normalization, based on the reference value. Wu (2002) suggested using N18 in the Gray Relational Analysis if the reference value is determined among the maximum and minimum values. Aytekin (2020) proposed N19 by integrating reference-based and decimal normalization processes. In this study, N19 is revised to equate the bestnormalized value to 1, and the worst normalized value to 0 according to the reference value. N20 and N21 provide normalization based on range. It is aimed to obtain normalized values between -1 and +1 in N20, and 0-1 range in N21.

Integrated-Mixed Normalization Techniques
The different structures of MCDM problems always force researchers to seek new solutions. In MCDM problems, the optimization orientation-based or reference-based approaches are generally adopted for solutions. However, it may be possible for decision-makers to solve the problem using reference values for some criteria and optimization orientations for others. Similarly, some of the criteria in the decision matrix may be monotonously increasing or decreasing, while others may not. In these cases, for instance, it is possible to determine the maximum or the minimum value according to the optimization orientation as a reference value or to create a solution by applying transformations to existing techniques. In such a case, there are normalization techniques developed for these integrated/mixed situations (Zhou et al, 2006;Zeng et al., 2013;Jahan & Edwards, 2015). The integrated-mixed normalization techniques are given in Table 3.   Zhou et al. (2006) to determine the ratio of xij to the reference value or maximum/minimum value. Zeng et al. (2013) developed N23 for an extension of VIKOR to provide practical solutions to problems in the health sector. Zeng et al. (2013) stated that the criteria having normal distribution are mostly used in the issues in the health sector and that an absolute deviation from the average is acceptable in these criteria. Also, they stated that normalization could be achieved with N23, including monotonic increasing or decreasing criteria.
Apart from the normalization techniques examined in the study, there are still more normalization techniques developed for different purposes like membership functions (rough numbers, triangle, trapezoidal, etc.) which are used in the normalization processes within the framework of fuzzy logic or rough sets (Sharma et al., 2018;Vafaei et al., 2018a;Roy et al., 2019). A comparative review of the normalization techniques given in Table 1, Table 2, and Table 3 will be carried out in the following section of the study.

Comparisons of Normalization Techniques in the Literature
Normalization techniques have an essential role in the solution of MCDM problems. Normalization processes used in the vast majority of MCDM methods enable the criteria of various structures to be dimensionless so that they can be directly compared. However, every normalization technique cannot be suitable for all decision problems. For example, some techniques do not provide eligible normalization for criteria with negative values or 0. Other than this, the expected range of normalized values and the rank reversal problem likely to be encountered are among the other determining factors in the selection of the normalization technique. On the other hand, it is not possible to evaluate MCDM methods independently of the normalization techniques they contain. Changing the normalization process included in an MCDM method results in the creation of a new extension/derivative of the method.
There are many normalization techniques and MCDM methods in the literature. The structure of problems and the assumptions of MCDM methods are prominent factors for choosing the normalization technique. In this context, although not yet sufficient, studies on the comparison of normalization techniques have been increasing in the recent past. These studies, however, usually include a small number of techniques and exclude most. Similarly, those that consider the selection of the normalization technique, and the criteria to be used in this selection process are limited.
In one of the prominent studies in comparing normalization techniques, Çelen (2014) examined the deposit banks using the FAHP-TOPSIS integrated model. The study compared N1, N2, N4, and N10. The consistency and validity of the normalization results were evaluated under four conditions. According to the first of these conditions, the distribution of the normalized values should be similar when compared to other techniques. Under this condition, inferences could be made by looking at the mean, standard deviation, smallest-largest values, and Kolmogorov-Smirnov normality test. In the second condition, the first three and the last three of the values to be obtained by normalization techniques should be the same. The third condition states that the correlations of these rankings should be high while the fourth condition emphasizes the need for the normalization techniques to produce similar scores, and this condition was examined with correlation coefficient values. The Pearson correlation coefficient was used for measuring similarities (Çelen, 2014, p. 201-203).
The use of the Kolmogorov-Smirnov normality test and Pearson correlation coefficient in the study carried out by Çelen (2014) can be seen to be unsuitable for MCDM problems. This is because MCDM problems mostly contain data structures that are not normally distributed, and the normal distribution is not generally sought in MCDM problems. Normalized values provided by normalization techniques may also have different structures. While some normalization techniques limit the data to values in a particular range, some ensure that obtained values are close to zero, while others provide only positive one-way data. It is thus clear that the structures of the decision problem and the MCDM method have a direct influence on the determination of the normalization process. The primary purpose of the normalization process performed in MCDM problems is not to obtain data with normal distribution, but to get comparable data with equal weight. However, it should be accepted that Çelen (2014) gives a different perspective on the comparison of normalization techniques. Chakraborty and Yeh (2007) suggested RCI (Ranking Consistency Index) to compare normalization techniques. In RCI, a normalization technique is evaluated based on ranking consistency with other normalization techniques. For this, it is necessary to simulate the decision matrix of at least 4x4 and at most 20x20. Finally, the results of the different normalization techniques are analyzed.
In another study conducted for the selection of normalization techniques, Vafaei et al. (2018a) discussed the appropriate normalization technique for TOPSIS using the reviews previously held in the literature. Accordingly, RCI, mean and distribution measures of normalized values, Kolmogorov-Smirnov normality test, ranking consistency of normalization techniques in terms of the first three and last three rows, Pearson and Spearman correlations were used in the comparison of normalization techniques and order of suitability for TOPSIS. The authors concluded that Vector Normalization is the best technique for TOPSIS in similarity with the study of Chakraborty and Yeh (2009). They also stated that comparison based on the normal distribution is questionable (Vafaei et al., 2018a).
Some of the processes mentioned in the comparison of normalization techniques do not always seem to be possible due to the structural features of MCDM problems. It is often not possible for the criteria to have a normal distribution. Furthermore, when the number of normalization techniques to be compared is high, and the size of the decision matrix exceeds 20x20, the use of RCI may not be effective. Another critical problem is that using the Pearson correlation coefficient to examine the correlation of the rankings does not give accurate results. Consequently, in the application part of this study, the Spearman rank correlation coefficient will be used to analyze the rank correlations of the normalization techniques. The ability to give effective normalization in different criteria structures, differences between normalized values, and their use in MCDM methods will also be examined.
Normalization is one of the essential process steps in MCDM methods, and it directly affects the solution to the problem. The normalization process used in the MCDM methods should not be considered independent of the method itself. For example, TOPSIS uses Euclidean distance in solving the decision problem. In this context, vector normalization, which is the second moment according to the starting vector (0), is used in the normalization process in TOPSIS. Like TOPSIS, many other MCDM methods also have normalization procedures for the intended solution. Table  4 shows the MCDM methods and the normalization techniques used in the original forms of these methods (the first form, not extension form) and the studies comparing them depending on different normalization techniques.
As seen in Table 4, the most recommended normalization techniques are Sum-Based Linear Normalization and Vector Normalization. The normalization technique should be chosen by considering the nature of the decision problems. In the following section of the study, an applied comparison of the normalization techniques will be given.

Applied Comparative Analysis of Normalization Techniques with Different Scenarios
In this part of the study, a comparison of normalization techniques will be highlighted. The twenty-three normalization techniques in the second section will be compared over a randomly and purposefully generated data set for a valid comparison. These sets were created by considering different scenarios to reflect the general structure of MCDM problems.
The literature and scenarios were taken as a basis in determining the number of criteria and alternatives for decision matrices. To this end, studies that conducted the literature review of MCDM methods were examined. The conclusion is that the most observed numbers were four for criteria, and five for alternatives (Durucasu et al., 2017). The numbers in this study were considered important as they provide guidance. For this study, however, it was decided that it would be appropriate to have six criteria and six alternatives to reflect the structural differences of scenarios and decision problems to be used in comparing normalization techniques.
After determining the number of criteria, and the number of alternatives, the different scenarios to be created were decided. Each scenario is named with a different set. To ensure that the criteria in Set 1 are different from each other, K1, whose variation range is quite wide compared to other criteria, K2 with a variation range of 0-1, K3 containing 0 and positive values, K4 containing negative values and 0, K5 containing negative, 0 and positive values together, and K6 whose values are all negative were established. While the optimization directions of all criteria in Set 1 were determined as benefit (maximum), it was evaluated as cost (minimum) in Set 2. In set 3, a scenario was created where all criteria have the benefit optimization orientation and where the ranges do not intersect was created. Accordingly, the ranges are 1-10 for K1; 11-100 for K2; 101-1,000 for K3; 1.001-10.000 for K4, 10.001-100.000 for K5 and 100.001-1.000.000 for K6. The optimization orientations of all criteria were set as benefit in Set 3 whereas it was determined as the cost in Set 4 which has the same decision matrix. In Set 5 to Set 11, the aim is to investigate the effects of adding and removing alternatives from the decision problem. Set 12 was created to examine whether units that can be converted into each other are normalized with the same values. In Set 12, the optimization orientations of the criteria are determined as benefit, while they are cost in Set 13.
Set 14 was created based on the values obtained in scaling techniques commonly used in MCDM problems. In this regard, the ten-point direct rating scale for K1, Saaty's Fundamental (Linear Priority) Scale for K2, Likert type scale for K3, DEMATEL Scoring Scale for K4, Semantic Scale for K5, and the hundred-point direct rating scale for K6 were used to determine the values of the alternatives. The ten-point direct rating scale allows alternatives to be evaluated in the range of 1-10. Alternatives are evaluated in the range of 1-9 by pairwise comparisons using Saaty's Fundamental (Linear Priority) Scale. DEMATEL Scoring Scale is based on determining the interactions between the alternatives by pairwise comparisons in the range of 0-4 (1-5 in some research). Semantic Scale generates values in the range of 0-100 using binary comparisons of alternatives in the range of 0-6 (1-7 in some research). The hundred-point direct rating scale allows alternatives to be evaluated in the range of 1-100 (Saaty, 1977;e Costa & Vansnick, 1994;Wu, 2008).
The fourteen sets in Table 5 will be used in the comparison of the normalization techniques. These sets contain decision matrices created for different scenarios. There are six criteria and six alternatives in these matrices. MS Excel was used to generate the performance values of the alternatives randomly. For this purpose, the formulas = RANDBETWEEN (lower_bound_value; upper_bound_value) and = RAND () were used. However, some values were determined purposely; for instance, in set 1, to examine the effects of 0, and in set 3 to check the effects of rank reversals in sets 6-11. The values that could be converted into each other were also purposely assigned in Set 12. To examine the rank reversal problem, SAW, which has one of the simplest and basic forms of MCDM methods, was used to solve the decision problems. Besides, MS Excel, SANNA, and SPSS 25.0 were used in the analysis. In Table 5, to show the effect of removing an alternative from the decision matrix in Set 5, A6, which ranked first in the solutions obtained with SAW in Set 3, was removed from the decision problem. In Set 6, A5, which took the last place in the solutions obtained with SAW in Set 3, was removed from the decision problem. In Set 7, both A5 and A6 alternatives in Set 3 are excluded from the decision matrix. In Set 8, A7, a new alternative with the best values in all criteria, was added to the decision matrix specified in Set 3. In Set 9, A8, a new alternative with the worst values in all criteria, was added to the decision problem in Set 3. Set 10 was created by simultaneously adding A7 and A8 to Set 3. Set 11 was created by changing the optimization orientations of Set 10. The values in Set 12 and Set 13 are assigned to represent values that can be converted into each other. In Set 14, the values of K2, K4, and K5 should be created by pairwise comparison. Values ranging between 0 and 1 were obtained for each alternative as a result of the operations performed using Saaty's Fundamental Scale in K2 and DEMATEL Scoring Scale in K4. These values were assigned following the structure of these scales; thus, multiple way performance analysis and comparison of normalization techniques can be made with Set 1-14.
To examine the rank reversals problem, the ranking of the alternatives was obtained via SAW using the normalized values. In the ranking process, the criteria were considered to have equal weights. The best values in criteria depending on the optimization orientation are used as reference values for reference-based normalization techniques.
The issues determined in the applications of normalization techniques in Sets 1-14 are shown in Table 6. The table contains information on the techniques regarding the sets in which the normalization process is not completed, the number of rank reversals, the maximum-minimum normalized values observed in the benefit/cost criteria, the ability to cope with the same values expressed in different units, and providing the same optimization orientation.
The completion of the normalization process, which is one of the issues in Table 6, in all possible data types in a meaningful way (by cleaning all units and making them dimensionless) shows the robustness and usability of the technique. At the end of the normalization process, normalized values between 0 and 1 are preferred while different ranges of normalized values across criteria are undesirable. For example, having normalized values between -1 and +1 in one criterion, and between 0 and 10 in another is not desirable. Such a structure will change the effects/weights of the criteria on the decision problem and make the solution of the decision problem to become invalid. The success of the normalization techniques will be examined by looking at the maximum and minimum normalized values. Also, normalization techniques are expected to handle unit differences successfully. For example, a distance may be expressed in any of the units of kilometers-hectometers-decametersmeters-decimeters-centimeters. The unit of measurement used does not change the length of the distance. The normalization technique should therefore generate units that can be converted to each other and measure the same thing, with the same normalized values. This standpoint is considered when examining how the techniques cope with different units. The last issue in Table 6 is the test of whether the technique gives the same optimization orientation, which is often the benefit orientation, an indication of whether the normalization technique gives one-dimensional values in all criteria. The first column of Table 6 shows the sets in which normalization techniques could not complete all the operations. N2, N10, N11, N12, N13, N14, N15, N16, N17, N19, N20, N21, and N23 completed the normalization process in all criteria in all sets. N3, N4, N5, N6, N7, N8, N9, and N22 could not complete normalization processes in all or some of the criteria because of a zero value in K3; zero and negative values in K4, and negative, zero and positive values in K5. N1, N5, N6, and N22 could not create normalized values for A3 in Set 2 due to the error of dividing by zero. The normalization process with N3 could not be completed because of the zero value in K4 in Set 14. Normalization of the benefit criteria could not be ended with N18. The main reason for this situation in N18 is the selection of reference values depending on the optimization orientation.
When the normalization techniques were analyzed in terms of rank reversals for Sets 1-14, only N16, N18, and N19 were found not to have rank reversal problems. However, a dramatic change in ρ value in only a specific criterion can cause rank reversals in N16. N18, which achieved normalization in only four sets, is subject to rank reversals due to maximum and minimum values in the decision matrix. In N19, if the reference and ρ values are determined depending on the first decision matrix or independently from the decision matrix, rank reversals are not observed. However, if the reference and ρ values change with changes in the decision matrix, rank reversals should be expected. Rank reversal problems were observed in other normalization techniques. Among these, N12 had the highest number of rank reversals. Table 6 also shows the range of the values obtained by normalization techniques in benefit criteria can also be observed. Accordingly, techniques providing normalization in the range of 0-1 are N3, N10, N12, N13, N17, N19, N21, and N23. N2 and N16 give normalization between -1 and +1. The sum of normalized values from the normalization performed in N1 on the benefit criteria is expected to be 1. On the other hand, N1 generated 1.406 and -0.891 normalized values in K5 in Set 1. The following techniques were also seen to have negative normalized values; N1, N2, N4, N5, N6, N7, N8, N9, N11, N14, N15, N16, N20, and N22.
In general, normalized values are expected to be within a certain range in all criteria as a result of normalization. At this point, it is more preferred that the normalized values are in the range of 0-1. In Set 1-Set 14, the average of normalized values obtained by normalization techniques was 0.7057, except for N8 and N9, which can produce structurally high normalized values. For example, an examination of the normalized values in benefit criteria reveals that N4, N5, N6, N8, N9, N14, N15, and N22 could produce normalized values larger than five by absolute value. From these techniques, N8 and N9 produced large normalized values; 2033,33 and 413,44, respectively.
Set 12 and Set 13 were created to examine whether units that could be transformed into each other were normalized in the same way. Table 6 shows that technique N3 failed in this regard. Another important comparison point is that normalized values can be used as-is in MCDM methods without additional processing. Although a significant majority of MCDM methods involve processing steps according to the costbenefit optimization orientation, the normalization technique should shorten this process. In this context, techniques providing one-dimensionality; N1, N2, N3, N4, N5, N6, N7, N8, N10, N11, N13, N19, N22, and N23.
The applications performed in Sets 1 -14 provide an opportunity to see the positive and negative features of normalization techniques. In the following section, a general evaluation of each technique will be given. N1 does not provide useful normalization in the cost criteria as well as it does in the benefit criteria. N1 cannot complete the normalization process for cells that have 0 in the cost criteria. Besides, N1 produced quite large normalized values, such as 108 and -16, in cost criteria. However, as can be seen in K5 in Set 1, if negative, 0, and positive values are included in the benefit criteria, N1 can produce negative normalized values or values greater than 1. Also, N1 is prone to rank reversal problems. N1 can give values corresponding to the performances of the alternatives in the range of 0-1 in cases where the optimization of the criterion benefit orientation and all values are positive. Thus, the weight of an alternative in the relevant criterion can be easily determined. Besides, N1 was also found to have successfully removed unit differences.
N2 produced negative values in Set 1 in the normalization of K4, K5, and K6. It also generated normalized values larger than 1 in Set 2. The main reason for this is the negative values in the decision matrix. N2 is also prone to rank reversal problems as it uses the sum of squares of the decision matrix elements. On the other hand, N2 completed the normalization processes in all criteria. N2 was also found to have successfully removed unit differences and gave normalized values in the benefit orientation.
N3 could not complete the normalization of the criteria having zero and negative values. Although N3 provided normalization in the range of 0-1, most of the values tended to be closer to zero. The rank reversal problem in N3 was less than many other techniques. It was determined that N3 could not successfully remove unit differences. On the other hand, N3 can give normalized values in the benefit orientation. Another critical point is that N3 provides non-linear normalization. Most normalization techniques have a monotonous increasing / decreasing structure. N3 can be said to be an option for decision problems with a different structure.
N4 cannot perform normalization in criteria where zero is the maximum value. Also, it produces negative normalized values in criteria having zero, negative, and positive values together. N4 gives quite large normalized values only for criteria including negative values. There may be rank reversals with N4. On the other hand, it was seen that N4 successfully removed unit differences and gave normalized values in the benefit orientation.
In the cost criteria, N5 cannot complete normalization when 0 is the minimum value whereas it gives normalized value as 0 for all other values in the decision matrix. The use of the minimum value in the decision matrix in the normalization process with N5 can lead to rank reversal problems. N5 can produce quite large normalized values in criteria that have negative and zero values. However, it gives the same normalized values if the same value is expressed in different units. It also gives all normalized values in the benefit orientation.
N6 could not complete normalization operations for K4 containing negative values and zero in Set 1. Similarly, in Set 2, N6 could not provide normalization for zero value in K3, K4 and K5 due to cost optimization orientations of these criteria. Also, it gave quite large normalized values such as 20,33 in K6, which had negative values in Set 1 and Set 2. Another negative feature of N6 is that it is prone to the rank reversal problem. However, N6 was found to provide normalization in the range of 0-1 in criteria where all values are positive. It is also successful in coping with different units and providing all normalized values in the benefit orientation.
At first glance, N7 can be assumed not to produce negative normalized values because it operates with absolute value. On the other hand, negative normalized values for K5 and K6 were observed in Set 1 and Set 2. Normalized values such as -18.33 produced in the mentioned sets are also quite high. N7 provided normalization in the range of 0-1 in other sets. As N7 is dependent on the maximum and minimum values in the decision matrix in the normalization processes, it is prone to the rank reversal problem. On the other hand, N7 is successful in dealing with the unit's differences and providing all normalized values in the same optimization orientation.
N8 did not provide normalization for all cells in K4 in Set 1, and the cells have zero values in K4, K5, and K6 in Set 2. Also, while N8 is expected to produce 100 as normalized values for the best value according to the optimization orientations of the criteria, quite large normalized values such as -356.25 and 2033.33 values were observed in Set 1 and Set 2. The rank reversal problem was also identified in applications done in Sets 1-4. Outstanding positive features of N8 include removing the unit's differences and giving all normalized values in the same optimization orientation.
Another technique that is prone to the problem of rank reversal is N9. N9 could not complete normalization in K4, where zero is the maximum value in Set 1 and Set 2. However, the normalized values created for Set 1 and Set 2 had quite high values, such as 413.44 and 8406.70 and negative values. N9 was found to be able to remove the unit's differences. On the other hand, it was unable to provide the benefit orientation in all criteria. In this context, the same ranks were seen for Set 12 and Set 13, in which their optimization orientations are different. Another critical point is that N9 produces non-linear normalized values. It can therefore be stated that N9 is an option for decision problems with different structures.
N10 provided normalization in the range between 0 and 1 in all sets. N10 was also successful in dealing with the unit's differences and providing benefit orientation values. Among the normalization techniques, N10 is one of the most robust and successful, but it can cause rank reversals.
N11 provided normalization in all sets. The maximum and minimum normalized values observed with N11 are 2.03 in Set 9 and -1.91 in Set 4. Also, the rank reversal problem was observed in normalizations with N11. On the other hand, it was successful in removing unit differences and generating normalized values in benefit criteria. N11 is one of the techniques with a non-monotonous structure. It can be stated that N11 creates an option for decision problems with different structures with these features.
Techniques N1 to N11 perform the normalization process according to the optimization orientation, while N12 to N21 provide normalization independent of the optimization orientation. In normalization techniques independent of the optimization orientation, specific parameters such as reference, mean, digit value, range, standard deviation, or fixed values are used for normalization. However, this study used the maximum/minimum values as reference values following the optimization orientations of the criteria. Furthermore, when the references are determined independently from the decision matrix, the reactions of the techniques will be reported.
The arithmetic mean is used as the reference value in normalization processes performed in Sets 1 -14 with N12. In this case, normalized values ranged from 0-1 were obtained in all sets. In some sets, the same normalized values were generated for criteria with different values. At this point, it should be noted that the values in certain ranges under the normal distribution curve have the same normalized values, and a non-monotonic process is performed. Considering that most normalization techniques have a monotonous increasing/decreasing structure, N12 can be said to differ significantly from other techniques. On the other hand, these features, which distinguish N12 from other techniques, prevent it from being successful at giving all values in the benefit orientation. Furthermore, N12 was the technique in which the rank reversal problem was common in Sets 1-14. On the other hand, N12 has the capability of removing unit differences.
N13 was observed to cause the rank reversal problem. On the other hand, it provides normalization in the range of 0-1 in all sets and successful in coping with unit differences and providing all values in benefit criteria. It gave the minimum normalized value as 0.3678 in all sets. N14 performs the normalization by equating the average value to 1. If a criterion has a large range, normalized values will not be acquired within the desired range using N14. Also, normalization based on average value can lead to rank reversal problems. N14 cannot achieve normalization in all benefit criteria. It however has the capability of removing unit differences. N15 shares the same features as N14 except for the normalization by providing the standard deviation equal to 1.
In N16, the digit value of the largest absolute number in the decision matrix is taken as the basis. When the maximum or the minimum value changes dramatically in a criterion, N16 can cause the rank reversal problem. However, no rank reversal was observed in comparisons made in Sets 1-14 with N16. N16 cannot provide all values in the benefit orientation, but it produces normalized values between -1 and +1 in all sets. Also, N16 is successful in removing unit differences.
Although N17 provides normalization based on reference, it also uses the maximum and minimum values in the criterion. This situation can cause the rank reversal problem. N17 produces normalized values in the range of 0-1, succeeds in coping with unit differences, and provides all normalized values in the same optimization orientation.
N18 was the technique with the highest number of processing errors in Sets 1-14 since the reference values were determined depending on the optimization orientation of the criteria. Revisions were made on Set 1 and Set 3 to measure the reaction of N18 and other reference-based techniques, N12, N13, N17, and N19. For this purpose, the reference values were different from the best values in the criteria. To do this, new reference values were determined to be 10% better and 10% worse than the best values in the criteria, and normalized values were examined. N18 could not complete the normalization process if the references were 0. Also, if the reference was higher than the maximum value in the criterion, N18 produced negative and large normalized values such as -8.85 and -9.37. For the other techniques, no situation other than the issues determined in the context of Set 1-14 was encountered. Also, the fact that N18 uses the maximum value in the criterion in the normalization process can lead to the rank reversal problem. N18 can remove unit differences. On the other hand, it gives the values in the benefit orientation only if the reference is less than the minimum value in the decision matrix.
N19, one of the reference-based normalization techniques, produces normalized values in the range of 0-1 in all sets, but it never creates exactly normalized values as 0. N19 does not cause the rank reversal problem if the reference and ρ values are determined depending on the first decision matrix or independently from the decision matrix. In applications carried out in the context of Sets 1-14, the rank reversal was not observed in N19. Besides, the N19 can remove unit differences and provide all values in the benefit orientation.
N20 uses the average of the maximum and minimum values and the range in the normalization process. These features, on the other hand, cause the rank reversal. Unable to provide all values in the benefit orientation, N20 produces normalized values between -1 and +1. Also, N20 succeeded to cope with unit differences. N21 applies the formula used for the benefit orientation of N10 in the same way in all criteria. Being independent of the optimization orientation prevents N21 from providing all values in the benefit-orientation. N21 provided normalization in the range of 0-1 in all sets and N21 caused the rank reversal problem at the same time, but it managed to cope with unit differences.
Normalization techniques from N1 to N21 provide normalization either by considering the optimization orientation of the criteria or reference or specific values. The emergence of integrated-mixed normalization techniques in the literature has given researchers different perspectives. In integrated-mixed normalization techniques, the optimization orientations of the criteria or reference/specific values are used under certain conditions. With N22, which is one of the integrated-mixed normalization techniques, normalization could not be achieved in cells that had zero value in Set 1 and Set 2. Also, quite large normalized values, such as 20.33 and -3.56, were observed in normalization depending on the optimization orientation, similar to the techniques using maximum and minimum values in criteria. N22 can cause the rank reversal problem, but it can remove unit differences and gives all normalized values in the benefit orientation.
N23 gives normalized values in the range of 0-1 in all sets and is also successful in dealing with unit differences and providing all normalized values in the benefit orientation. Additionally, it produces non-monotonic normalization, but it can cause the rank reversal problem.
obtained by normalization techniques. The similarities between these rankings were examined using the Spearman rank correlation coefficient, and the results are presented in Table 7.
An examination of the correlation values in Table 7 reveals that the techniques that did not complete all normalization process in criteria having the benefit optimization orientation have a low or meaningless correlation with the other techniques. The same can be said for the techniques not having a linear structure. Among the twenty-three techniques, N12 has the lowest correlation with others. N3, N9, N14, N15, N16, N20, and N21 have a low level of correlation with the other techniques. As seen in Table 7, the technique that has the most inverse correlation with other techniques is N18. This can be attributed to the low number of normalization processes completed by N18 in Sets 1-14. The other techniques were found to provide a high level of correlation (rs> 0.8) with each other in general.

Conclusions
Normalization is a scaling process that is frequently used in creating the data structure required for the method to be used in quantitative studies. The normalization process directly affects the results of the analyzes to be performed. However, it wouldn't be right to use traditional normalization techniques for such a process without question.
In MCDM methods, normalization is used to obtain criteria that have the same weight, are dimensionless, and are suitable for compensatory processes. Although the vast majority of MCDM methods provide normalization within themselves, the nature of the decision problems makes it necessary to review these processes. On the other hand, changing the normalization process of an MCDM method partially leads to the emergence of a new extension of the relevant method. To this end, this study aimed to outline the positive and negative features of the normalization techniques that are widely used and can be used in MCDM problems, thus giving decision-makers or researchers an insight into the situations in which the considered normalization techniques can work well and where they should not be used.
Eleven techniques, based on optimization orientations of criteria, ten techniques independent of optimization orientations, and two integrated-mixed techniques were evaluated in fourteen sets, reflecting different criteria and data structures considered for normalization. In the comparison, the emphasis was made on the ranges of normalized values obtained by the techniques, the presence of the rank reversal problem, normalization performance in criteria with different structures, the capacity to remove unit differences, the ability to give all normalized values in the same optimization orientation, and completion of normalization in all sets. The comparisons in Sets 1-14 showed the importance of the normalization technique selection process. Normalization may not be completed if the normalization technique is not chosen according to the data structure, and should that be the case then the validity of the results may be disputable. The results showed that some normalization techniques are not able to achieve the purposes of development or use in some cases. N1, N3, N4, N5, N6, N7, N8, N9, and N22 could not complete the normalization procedures in all or part of the criteria with zero or negative values in the decision matrix. N1, N4, N5, N6, N7, N8, N9, N14, N15, and N22 produced quite large normalized values. Also, they did not produce normalized values within a certain range, such as 0-1, in all criteria. These techniques provide very high normalized values due to the criteria having a different structure in Set 1 and Set 2. The rank reversal problem was not observed in techniques N16 and N19 in Sets 1-14. Indeed, if the reference and ρ values cannot be changed after being determined at the beginning of the problem, N19 stands out as the only technique that does not cause any rank reversal problems. Also, N19 does not use values such as maximum, minimum that will change with the change in the decision matrix.
Normalization techniques have different structures. However, the nature of the decision matrix, preferences of the decision-maker, and the properties of the MCDM method to be used in the solution of the decision problem should also be considered in the selection of the normalization technique. If the aim is to choose the highest performance value in a criterion and avoid the alternative with the lowest performance value, or vice versa, it would be correct to use techniques that provide normalization depending on the optimization orientation. If the decision-maker has reference/ideal/utopic values determined for each criterion, it will be necessary to use reference-based normalization techniques. If the decision-maker has the opinion that the values in the criteria do not represent the monotonous increasing/decreasing benefit/cost, then the non-monotonic or non-linear techniques should be used. However, some data, that have zero and negative values, may prevent the use of some normalization techniques, as determined in the application section. Also, the problem of rank reversal, the ranges of normalized values, the capability of removing unit differences, and the ability to give all normalized values in the same orientation, robustness and validity of the results will affect the choice of the normalization technique.
This study presents a comparison of normalization techniques in different criteria structures. It sought to highlight the positive and negative features of the techniques in question thereby guiding decision-makers or researchers on the selection of techniques. The study is also envisioned to provide a perspective on the development of new normalization techniques and the creation of new extensions of MCDM methods by replacing the normalization techniques originally included in the MCDM methods with other techniques. Normalization techniques continue to be applied in most areas where data analysis is required. It is, therefore, necessary to conduct comparisons in other fields such as data mining as well. It may also be beneficial for future research to examine the normalization processes associated with fuzzy and rough sets used in MCDM problems.
Alternatives in Sets 1-14 have been ranked by SAW using the normalized values Table  7. Rank