SPECIFIC CHARACTER OF OBJECTIVE METHODS FOR DETERMINING WEIGHTS OF CRITERIA IN MCDM PROBLEMS: Entropy, CRITIC, SD

. The comparative analysis of objective methods for determining the weights of criteria in the problems of multi-criteria decision-making is carried out. It is shown that the use of methods for determining the weights of criteria, based on formal processing of the decision matrix (Entropy, CRITIC, Standard deviation) for MCDM problems in some cases is not correct. It is demonstrated that the Entropy weighting method (EWM) is highly sensitive to evaluation of probabilities of states based on the decision matrix. For the Entropy method two modifications of estimation of probabilities of states are proposed that partially eliminate the contradictions of the basic EWM method. The first modification (EWM.df) is based on a statistical approach and it estimates the probabilities of states based on attribute distribution function. The second modification (EWM.dsp) estimates the probabilities of states based on the relative dispositions of attributes. Two options both have their supporting rationale. The analysis of integrated weighing methods is carried out and various options for aggregation of weights are given. An integrated EWM-Corr-method is proposed which allows to re-allocate the weights obtained by the Entropy method among correlated criteria.


Introduction
A significant number of multi-criteria decision-making methods (MCDM) constructs the performance indicator of alternatives, taking into account the weights of the criteria (Hwang & Yoon, 1981;Triantaphyllou, 2000;Tzeng & Huang, 2011;Pamucar & Savin, 2020). The weights of the criteria quantify their importance and can significantly affect the outcome of the decision-making process. Many authors note the high sensitivity of the solution to variations in the weights of the criteria (Barron & Schmidt, 1988;Mareschal, 1988;Wolters & Mareschal, 1995;Li et al., 2013;Karande et al. 2016;Mukhametzyanov & Pamučar, 2018;Zolfani et al., 2020). Given the importance of the weighing procedure, the balanced choice requires comprehensive analysis. As you know, the weights of the components of a complex system can be obtained in different ways. One of the generally accepted classifications of methods for assessing criteria weights divides methods into three categories based on subjective, objective and integrated or combined approach to weighing (Goodwin & Wright, 1998, Bobko et al., 2007Ginevicius & Podvezko, 2005;Jahan et al., 2012, Odu, 2019Ali et al., 2021). The determination of the subjective weight is based on the opinion of experts or expert groups representing the views of various stakeholders. These are such methods as the direct ranking method (DR) (Goodwin & Wright, 1998;Roberts & Goodwin, 2002;Von Winterfeldt & Edwards, 1986), the point allocation (PA) method (Doyle et al., 1997;Roberts & Goodwin, 2002), the ranking method (Ahn & Park, 2008;Barron, 1992Barron, , 1996Roberts & Goodwin, 2002;Solymosi & Dombi, 1986;Milosevic et al., 2021), methods of programming (Pekelman & Sen, 1974;Shirland et al., 2003;Deng et al., 2004), Delphi method (Hwang & Yoon, 1981), pair-wise comparison (AHP) (Saaty, 1980;Takeda et al., 1987), step-wise weight assessment ratio analysis (SWARA) (Kersuliene et al., 2010), full consistency method (FUCOM) Pamucar & Ecer, 2020), Level Based Weight Assessment (LBWA) (Žižović & Pamucar, 2019). One of the important problems of subjective methods is an assessment of the consistency of expert opinions. For example, the AHP-method defines a consistency index, which improves the reliability of the weight estimates. Other procedures for assessment of the consistency of expert judgments are based on statistical methods and correlation. The category of objective assessment methods is based on the use of information about the criteria and their interactions contained in the decision-making matrix. These are such methods as entropy weighting method (EWM) (Lotfi & Fallahnejad, 2010;Wu et al., 2011;He et al., 2016), CRiteria Importance Through Inter-criteria Correlation (CRITIC) (Diakoulaki et al., 1995), standard deviation (SD) and their modifications (Jahan et al., 2012;Žižović et al., 2020;Alosta et al., 2021). For these methods, there is no answer to the question of how fully and objectively the limited sample of attributes of alternatives describes the value of the criteria. Obviously, the result is completely determined by the decision matrix. In particular, this article provides examples of how strongly the weights obtained for this category of methods for specially constructed decision-making matrices can differ. This paper analyzes the impact of normalization of the decision matrix and inversion of cost attributes on the weights estimation results. The hypersensitivity of the Entropy method to the estimation of state probabilities based on the decision matrix is shown. For the Entropy method, a statistical approach to assessing the probability of states is proposed, which partially eliminates the contradictions of the formal method. The essence of the integrated methods consists in combining the weighing results obtained by different methods with the subsequent use of the normalization of values by the Sum-method (Ma et al., 1999;Xu, 2004;Ustinovičius, 2001;Jahan et al., 2012;Vinogradova et al., 2018;Đalić et al., 2020;Singh et al., 2020). It is believed that integration can overcome the disadvantages of subjective and objective weighing methods. The integration of weights obtained by different methods takes different forms. In some cases, the combination procedure strengthens (weakens) the weights for "strong" ("weak") criteria, when the criterion had a greater (lower) weight in both weighing methods, in other cases, the weight of the criteria is smoothed out when the criterion had a larger one in one of the methods and less weight in a different weighing method. The paper presents various options for aggregating weights. Numerical examples show the importance of preliminary analysis of the results when choosing the final version of the aggregation procedure, taking into account the peculiarities of the decision-making problem. An integrated EWM-Corr method is proposed, which allows redistributing the weights obtained in the Entropy method between correlated criteria. The idea that the assessment of weights is purely technical in nature and therefore objective is erroneous. It is rather difficult to make them objective, since there can be different problems with different data structures, varying degrees of uncertainty, etc. The existence of many methods and their modifications leading to different results indicates that the estimation of weights requires serious analysis. Since the design and determination of weights can always be interpreted in terms of value judgments, the procedure should also include the subjective opinions of individual experts. The differencies in assessments of the surrounding world, as noted by the famous subjective idealist George Berkeley, are due to the fact that people's perceptions are far from identical. Their correct agreement limits the decision obtained by majority rule. The purpose of this work is to show the characteristic features, shortcomings and possible contradictions of some methods for evaluating the weights of criteria for complex and critical analysis when solving problems of making multi-criteria decisions.

Preliminaries. Objective weighting methods: Entropy, CRITIC, SD
The category of objective assessment methods is based on the use of information about the criteria and their interaction contained in the decision-making matrix. The standard deviation or an "entropy" measure of importance have been proposed for quantifying contrast intensity and thus deriving objective weights of criteria (Zeleny, 1982). For objective weight estimation methods, it is believed that the greater the scatter of the values of the attribute of alternatives (the greater the difference between the values of the elements in the column, or the greater the variance, or the less entropy), the greater valuable information the criterion (indicator) contains and the higher the criterion weight. The criterion in which all alternatives have the same performance does not offer any additional information. The general structure of objective methods for assessing weights is shown in the scheme in Figure 1. Data transformation (or normalization) is intended to co-ordinate the measurement scales of individual attributes. In order to maintain the proportions between natural and normalized values, linear transformations are used. Entropy -EWM-method, multiplication of standard deviation and correlation -CRITIC-method and standard deviation -SD-method are used as key indicators. The weights of the criteria are determined as the intensities of the key indicators using the Sum-normalization. Below are step-by-step weight estimation algorithms using the three most commonly used methods for solving MCDM problems -Entropy, CRITIC and SD.

Nomenclature:
Ai alternatives (objects) (i=1,…, m) Cj + , Cjcriteria or objects properties (j=1,…, n), (+)benefit, (-) cost aij elements of decision matrix (DM) rij normalized elements of decision matrix ̅ j average value of j-th criterion aj max maximum element in criteria j aj min minimum element in criteria j wj weight or importance of criteria (j=1,…, n)

Entropy weighting method (EWM)
The values of the decision matrix are transformed into the segment [0; 1] using Max-Min normalization (1) with simultaneous inversion (2) of cost criteria values: max max min , The intensity (p ij ) of the j-th attribute of the i-th alternative is calculated for each criterion (Sum-method): Linear data transformation and inversion of the cost attributes Calculation of the key indicator Calculation of the weight as intensity of q a ij r ij q j w j =q j /∑ q j (3) To calculate the entropy (ej) and the key indicator (qj) of each criterion: To calculate the weight of each criterion: The entropy of the attributes of alternatives for each criterion is a measure of the significance of this criterion. It is believed that the lower the entropy of the criterion, the more valuable information the criterion contains.

CRiteria Importance Through Inter-criteria Correlation (CRITIC)
The values of the decision matrix are transformed based on the concept of the ideal point. To determine "best" (B=b j ) and "worst" (T=t j ) solution ([1 x n]-vector) for all attributes and determine relative deviation matrix V[m x n]: To determine standard deviation (s) ([1 x n]-vector) for colls of V: To determine the linear correlation matrix (c jk ) ([n x n]-matrix) for colls of V is the ( correlation coefficient between the vectors r j and r k ): To calculate the key indicator and weight of criteria by the formula (6): In the CRITIC method, the standard deviation s j is a measure of the significance of this criterion. Allowance for the relationship between the criteria is determined through the correlation matrix, which allows you to distribute the weight between the correlated criteria through the coefficients of reduction (1-c). The amount shown in expression (10) is a measure of the conflict created by j-th criterion in relation to the rest of the criteria. Finally, the amount of information contained in the j-th criterion is determined using multiplicative aggregation of measures by the formula (10). The Spearman rank correlation coefficient could be used instead of c jk , in order to provide a more general measure of the relationship between the rank orders of the elements included in the vectors r j and r k .

Standard deviation (SD)
The values of the decision matrix are transformed into the segment [0; 1] using Max-Min normalization with simultaneous inversion of cost criteria values by formulas (1) and (2). To calculate the key indicator and weight of criteria by expression (6): The standard deviation of r j is a measure of the value of that criterion to the decision making process.

Estimating the probabilities of the object states in the Entropy method
One of the main notes for the Entropy method is that the p ij in the entropy method determines the probabilities of m possible and independent states (alternatives) of an object. It is not clear how the probabilities of states and intensities, obtained by the Sum-method according to expression (3), are related. If we combine expressions (1)-(3) for profit criteria (similarly for cost criteria), we get: The p ij values are the relative distances of the j-th attribute of alternatives to the alternative with the smallest value. Obviously, a discrete random variable is determined by a pair (x i , p i ) and the values of p i are needed to calculate the entropy. However, expression (3) determines p i with respect to x i . In Sections 4.2 and 4.3, to determine the probabilities of possible states of alternatives, it is proposed to use the distribution functions of the attributes of alternatives built on the values of the decision matrix and to use relative dispositions of attributes instead of p ij .

Difference in weights of criteria when changing many alternatives
Consider a decision-making problem in which some or all of the set of alternatives are different, and the attributes are assessed within the same criteria. Taking the principle of uniformity and continuity of changes in the properties of an object, such groups of alternatives exist and have attributes from a certain interval of the area of change. Table 1 presents decision matrices for two groups of alternatives to one problem in the context of the same criteria. Let's form a mix-matrix of solutions DM-3 obtained by replacing the first alternative A 1 of the matrix DM-1 with the first alternative B 1 from DM-2, and form more one decision matrix DM-12 of dimension [16x5], obtained by the concatenation of the matrices DM-1 and DM-2. Both sets of alternatives A i and B i are related to the context of the problem: alternatives are objects of the same nature, Table 1. Decision matrices DM-1 and DM-2 for two groups of alternatives Ai and Bi in the context of the same criteria. having similar properties within the same criteria and having approximately the same range of variation. It should be expected that the criterion weights calculated by the same method for different decision matrices will be approximately the same. However, the weights of the criteria (and the ranks of alternatives) differ significantly (Table 2) not only for different methods, but within the same method for a different set of alternatives.
To characterize the deviation of the weights obtained in various examples and methods, the relative error is used (the calculation is made for each j-th components) δ s-k =|w (s) -w (k) |/w (s) ·100,%. In this particular example, the largest variation in weights in the Entropy method is 185%, in the CRITIC method -52% and SD -24%. Such a wide spread leads to a change in the ranking of alternatives (Simple Additive Weighting method -SAW). On the one hand, all three matrices represent assessments of alternatives within the same criteria. However, the relative difference in the estimates of the weights for the given example for some criteria is up to 50% and more. Even when replacing one alternative, the weights of the criteria change significantly.

Decision matrix transformations and influence on key parameters of CRITIC and SD methods
Since key indicators (q) should not depend on scales of measurements of the attributes, the attributes should be normalized or reduced to dimensionless values.
Considering that the number of methods of normalization and inversion totals about 20 (Jahan et al., 2015), we should expect the ambiguity of the results. 6 basic linear methods of normalization, presented in Table 3, have a meaningful interpretation. The IZ-method (Mukhametzyanov, 2018(Mukhametzyanov, , 2019(Mukhametzyanov, , 2021) is a generalization of the Max-Min method and will be used below in the Entropy-method.
The short name of the normalization methods is determined by the semantic value of the compression ratio k. The method abbreviation is also used as the name of a function that converts values in accordance with the normalization method. For example, rij=Max(aij)= aij/aj max . dSum-method is a combination of Max-Min and Sum methods (see the formula (12)) and 1-r inversion. In the IZ-method, I and Z define a segment of normalized values that is fixed for all criteria (0 ≤ I < Z ≤ 1).
The use of linear normalization methods is due to the fact that they retain the relative dispositions between the natural and normalized values of the alternatives: where rng(rj)=rj max -rj min is the range of values for each attribute.
Preserving the "dispositions" of alternatives after normalization means that the normalized values preserve information about the system. For linear normalization without displacement, one degree of freedom is lost, and for normalization with displacement, two degrees of freedom are lost. Any nonlinear normalization leads to information distortion. When using a general linear transformation of natural values rij=( aij-aj * )/kj, the standard deviation is scaled: and the matrix of pair-wise correlations does not change (invariant): This means that the result of CRITIC and SD depends on the choice of the decision matrix normalization method, since the proportions between k j in different normalization methods for different criteria will be different, and the intensity of the q j key indicators will be different.  Of the six normalization options in Table 3, the Max-Min and IZ method is preferable, since in this case the range of values of all attributes is the same and the standard deviation for the normalized values s j (a ij )/k j of various attributes does not depend on the range and has a natural interpretation as the degree of dispersion of values. In all other cases of normalization, the key indicators q j = s j for various criteria are not consistent with each other, and the result depends on the transformation method of criteria and measurement scales. The numerical results of the example under consideration are shown in Table 4. The results demonstrate a significant difference in criterion weights for different methods, which indicates the importance of choosing the correct normalization method. For the Max-Min and IZ normalization methods, the standard deviation is proportional to the factor (Z-I). Therefore, the criteria weights will be equal (highlighted in the table). Table 4. The values of the standard deviation and weights of the criteria of the decision matrix DM-1 for various transformation methods.

Standard deviation
Weights of criteria For the CRITIC method, the general transformation (7), in contrast to the Max-Min normalization by the formula (1) is not correct, because the "best" (B) and "worst" (T) solution may differ from the maximum and/or minimum value. For example, if "best" and "worst" are absolute (from the entire set of possible alternatives, and not from a private sample in the investigated decision-making problem), then the normalized values are within the interval (0; 1), and for each criterion the domain range will be different. The normalized values will be offset relative to the segment [0; 1] with all the consequences of changing the standard deviation of the attributes of individual criteria and changing their weight. Therefore, the correct application of the CRITIC method is possible only with the Max-Min transformation. To invert the attributes of cost criteria, it is recommended the ReS-algorithm in two forms (Mukhametzyanov, 2020): 1) for natural values of attributes 2) for the normalized values of attributes where Norm() is one of the linear normalization methods applied to both benefit and cost attributes; the j* index meets the cost criteria. The abbreviation of the normalization method (Table 3) or inversion is also used as the name of the function that transforms values in accordance with the selected method: rij=Vec(rij), vij=ReS(rij). ReS-algorithm is universal for any linear and non-linear transformations and ReSinversion algorithm preserves dispositions of natural and normalized attribute values. For the Max-Min normalization method, the inversion by the expression (2) and the inversion using the ReS-algorithm have the same results. According to the standard deviation expression, inverting cost attributes does not affect the outcome. However, in the matrix of pair-wise correlations, the sign of some of the coefficients changes according to the rule: where  is the element-wise product of two matrices of dimension [n x n]; 1 2 (sgn , sgn ,..., sgn ) For example, for the test matrix DM-1 used in this paper: (1 1 1 1 1) 1 1 1 1 1 1 1 1 1 1 1 1 Negative correlation should not increase the weight of the key factor in expression (10). Therefore, in expression (10), the correlation must be absolute and this is the correct expression for CRITIC.

How much do the weights of the Entropy method vary depending on the distribution in the data?
To answer this question, let us turn to the graphical illustration (Fig. 3) (4), the maximum value of the criterion entropy is equal to 1, and the criterion weight is equal to 0. This state is achieved when all probabilities p ij = 1/m for a fixed j. This is possible provided that all alternatives have the same value for the j-th attribute p ij = Const. The greater the difference between the values of the j-th attribute for various alternatives, the lower the value of the criterion entropy and, in accordance to the expression (5), the greater the value of the criterion weight. This is due to the specifics of data normalization by the Sum-method.
Therefore, the criterion j, the attributes of the alternatives of which are concentrated closer to the smallest value a j min , will have more weight. This is believed to indicate that the criterion contains valuable information. A very controversial assumption. The weak point of the entropy method for assessing weight is the high sensitivity (hypersensitivity) of weight to the values of the entropy of various criteria. This hypersensitivity is due to the exponential behavior of the logarithm in the vicinity of 0. Indeed, the rate of change of entropy along the i-th component is determined by expression: therefore, small changes in the values of the j-th attribute of the i-th alternative can change the weights. For example, (see Table 2 and Fig. 3) the entropies for criteria 2 and 3 differ by 1.5%, while the weights differ by 8%. A slight error in the estimates of the attribute of alternatives will lead to a significant change in the estimates of the weights. Another weak point of the entropy method is the sensitivity to the distribution of data in the domain of p ij values. The concentration of the normalized values of one of the attributes in the vicinity of 0 sharply decreases the entropy and leads to an increase in the weight of this criterion. Figure 4 illustrates this feature. When replacing only one element a 12 in the decision matrix DM-1 from 85 to 72, entropy of the first criterion decreased from 0.8897 to 0.8065, and the weight of the first criterion increased from 0.146 to 0.231 (by 158%). The ranking of alternatives has also changed. Note that when the data changed, the standard deviation for the first criterion increased from 5.120 to 5.125 (natural). The SD-method shows weight changes of only 0.1% -from 0.2102 to 0.2104. This example demonstrates the high sensitivity (hypersensitivity) of EWM-method to the distribution of data in the domain.
The apparent efficiency of the entropy method can most likely be attributed to a situation in which the attributes of the alternatives have approximately the same distribution and approximately the same variance. In such a situation, the weights of the criteria differ less significantly.

Do you need to transform (normalize) the natural values of the attributes before applying the Entropy method?
Some studies when estimating weights do not use preliminary data transformation (by formulas (1) and (2)), believing that it is obvious that the transformation Sum (formula (3)) is sufficient, or they use normalization methods other than Max-Min. For example, in (Wu et al., 2011;Li et al., 2011), values are normalized and inverted for cost criteria using the Max and Vec methods. The peculiarity of the transformation according to the expression expression (3) (Sum-method) is that the normalized p ij values are converted into the interval (0; 1) with the preservation of the dispositions of natural values, but the domains for different criteria have different sizes and are shifted relative to each other. This is clearly seen in the diagrams for p ij in Figure 5. For more details on the displacement of domains during normalization, see the study by Mukhametzyanov (2020Mukhametzyanov ( , 2021. The displacement of the p ij values by the j-th criterion, according to the analysis carried out above in Section 3.4, strongly affects the entropy of the e j criteria, which is reflected in the weights. Normalization based on linear methods without displacement (Max, Sum, Vec) fully preserves the proportions of natural and normalized values of attributes. However, after the re-normalization, the equalities are valid (Mukhametzyanov, 2020): pij =Sum(aij/kj)=Sum(Max(aij))=Sum(Sum(aij))= Sum(Vec(aij))= Sum(aij). (20) Therefore, the result of calculating the weights of the criteria will not change. In Figure 4, in particular, p ij coincides for the case when the preliminary data transformation is not performed and for the cases of linear transformation without displacement (1 fragment). Normalization based on linear methods with displacement (Max-Min, dSum) retains only the dispositions of the values. However the probabilities change: pij =Sum( (aij-aj * )/kj ) # Sum(aij), and therefore the entropies of the criteria will change.
In a significant part of the studies ( The subsequent transformation by the Sum-method leads to different values of the intensities (probabilities) p ij than for the natural data of the decision matrix. At least one of the values after transformation by the Sum method is equal to 0. The transformed system is not identical to the original one. Despite the fact that agreement of domains for different criteria has been achieved, within the domain there is a shift in data towards a decrease in the contribution of individual attributes to the entropy of the criterion and a decrease in the entropy of the criterion. At least one of the values after the stepwise transformation by the Max-Min and Sum method is equal to 0. The transformations are uneven for various criteria (except for the distribution in the data that is symmetric with respect to transformation). Therefore, the previously transformed system is a different system, not identical to the original one, and the results of the weight estimation differ significantly. Table 5 and Figure 4 shows an example of such calculations for the first criterion of the DM-1 matrix. The values of the entropy and weights of the criteria calculated after performing the preliminary data transformation can significantly differ from the analogous values without transformation. The largest variation in weights in the Entropy method is over 400%, with no strong case for data transformation.

Do you need to apply value inversion for cost criteria before applying Entropy?
The answer to this question is not obvious. On the one hand, the criteria are independent and the entropy (e j ) of the criterion is not related to the direction of improving the values of alternatives. On the other hand, attribute values are ordered from lowest to highest value. When the direction of the coordinate axis and the origin is changed, the state probability distribution function is reflected. If the distribution does not have symmetry with respect to such a transformation, then formally the entropy of the system will change due to the relative displacement of values within the domain (even in the case of using the best of the inversion algorithms -ReS (Mukhametzyanov, 2020), which preserves the positions of the boundaries of the domain of inverted values). For example, for the example considered in Table 2, the weights calculated without preliminary inversion of the values of the cost criteria (Table 6) will differ from the weights calculated with preliminary inversion. Thus, the second argument is strongly pointing in favor of pre-inverting the cost attribute values. For the inverse of values of cost criteria, the ReS-algorithm presented in 3.3 above is recommended. It is not desirable to use non-linear inversion. For example, in the works of Li, et al., (2011), Wu, et al., (2011, a nonlinear transformation iMax = a ij * min /a ij * , is used, which transforms the original system to another system that is not equivalent to the original one, the weights of which are different.

IZ-transformation of natural values of attributes for the Entropy method
Applying linear operations of shift to a fixed point, tension-compression and subsequent displacement, it is possible to transform the natural values of all criteria into a fixed (or predetermined) interval of values [a; b] (Mukhametzyanov, 2018(Mukhametzyanov, , 2019(Mukhametzyanov, , 2021 while maintaining the relative dispositions of natural values for each attribute. A special case of such a transformation is the Max-Min normalization method, which converts the attribute values for all criteria into the segment [0; 1]. An example of IZ-transformation of the matrix of solutions DM-1 into the segment [0.2; 0.7] is shown above in Figure 2. IZ-transformation is in no way inferior, and even more correct than transformation in [0; 1], which has problems with null values. For example, there is no contribution of min-value of attributes to entropy. As a result of the IZ-transformation, in comparison with the Max-Min transformation, the standard deviation and the matrix of pairwise correlations will not change. This means that such data transformation does not affect the result of the CRITIC and SD methods. However, p ij will change, and due to the nonlinearity of the entropy components (-p ij ·lnp ij ), both the entropy and the weights of the criteria will change. It is natural to ask how the choice of the IZ-transformation interval [a; b] affects the entropy and weights of the criteria? Due to the strict monotonicity of the entropy components on the interval [0, 1/e] (see Fig. 3), all changes will also be strictly monotonic, except for the cases when any of the values of p ij are greater than 1/e. Figure 6 shows a color map of entropy values  The conclusion from the analysis is not encouraging. When manipulating data transformation in the Entropy-method, you can manipulate the criteria weights.

Integrated methods
The essence of integrated methods (or aggregation methods) is to combine weighing results obtained by different methods. For example, weights obtained using subjective and objective weighing methods such as AHP and Entropy are often corrected. It is assumed that the combined weighing method will reduce the potential bias of a single subjective or objective weight, or can make up for the deficiency of the subjective weight. Let the weights w j (1) of the j-th criterion be obtained by one of the available methods, and the weights w j (2) be obtained by another method. A common aggregation procedure has the following multiplicative form: (1) (2) j j j q w w  Aggregation (21) strengthens (weakens) the weights for "strong" ("weak") criteria, when the criterion has a greater (lower) weight in both weighing methods. In other cases, the weight of the criteria is smoothed, when the criterion had greater weight in one of the methods and less weight in the other weighing method. In some cases, such a procedure is dangerous from the point of view of amplifying the error. The q j value can be arranged in various forms, for example, as different variants of the average value in N weighing methods: harmonic mean (HM) arithmetic mean (AM) The weights are calculated as the intensities of the integrated indicator q j by formula (6): For different means, the following inequalities hold: The choice of the method for the mean is not formalized and is determined by the context of the problem. The weights aggregation method is compensating. It compensates for low criterion weights in some intermediate weighing methods with high values in others. If compensation is acceptable, then a higher weight value for the same criterion in a different weighing method will be required to obtain a higher weight value. With multiplicative aggregation, a higher value of the weight of one of the criteria can be obtained by reducing the weight of the other criterion. Given the variability of aggregation and compensation, the integrated methods are not general. Rather, they give the researcher a formal opportunity to eliminate the discrepancy between the criteria weights in a particular problem.

Modification of the entropy method using correlation: EWM-Corr
As discussed in section 3.8 above, key indicators of different methods can be combined to assess criterion weights. The combination of entropy and correlation in the following multiplicative form is relevant: where ej is the entropy of the criterion determined by formulas (1)-(4), cjk is the matrix of pairwise correlations determined by the formula (9). The first factor determines the dispersion of the attribute values, and the second factor allows the weight to be distributed among the correlated criteria in terms of reduction factors (1-c).
The weights of the criteria are determined by the intensity of the key indicator q j using the Sum-normalization by the formula (6). Table 7 shows the results of calculating the weights by the proposed EWM-Corr method. Analysis of the results shows the redistribution of weights due to correlation between the attributes of the alternatives. There was an increase in the weight of 3rd criterion (weak correlation of attributes with the rest) and a weakening in the weight of 1st criterion (significant correlation of attributes with the rest). Comparison of the results with similar estimates by the Entropy and CRITIC methods (Table 2) does not reveal any definite pattern. Since there are no criteria for the adequacy of various weight assessment methods, the effectiveness of the EWM-Corr modification is relative. The choice of a particular weighing method and its appropriateness is assessed by the presence of arguments "for and against". This applies to all methods without exception. In our case, the argument is the redistribution of weights between correlated and uncorrelated attributes.

Assessment of the entropy of the criterion using the distribution function of the attribute of alternatives: EWM.df modification
After the decision matrix is normalized using the Max-Min transformation, the domains of the normalized values of attribute are consistent with each other. The priority of individual criteria due to different measurement scales is eliminated.
Information about the states of an object is determined by the distribution of values within the domain, not by the values themselves. Therefore, the calculation of entropy by values is not correct. A discrete random variable is determined by the pair (x i , p i ), and the values of the probabilities p i are needed to calculate the entropy. To estimate the probabilities of possible states of alternatives for a fixed criterion, it is necessary to construct a discrete distribution using a statistical approach by performing the transition to frequency characteristics. For the case of small samples, the distribution of a random variable can be determined based on estimates of the parameters of a hypothetical distribution of a random variable (for example, normal). The distribution assessment includes a statistical test for testing a hypothetical distribution law (for example, the Jarque-Bera test for checking normality (Jarque & Bera, 1987 where f is the density of the feature distribution, θ1, θ2, ... are the distribution parameters.
Below is a fragment of the calculation procedure for the case of a normal feature distribution law, performed in MatLab using built-in functions: % define DM(m,n) mu=mean(DM); sigma=std(DM); mx=max(DM); mn=min(DM); h=(mx-mn)/(m-1); for j=1:n [hJB, pJB] = jbtest(DM(:,j)); % Jarque-Bera test if hJB==0 xN=mn(j)-h(j)/2 : h(j) : mx(j)+h(j)/2; xN1=xN(1:m); xN2=xN(2:m+1); pd = makedist('Normal',mu(j),sigma(j)); y = cdf(pd,xN2)-cdf(pd,xN1); % cumulative distribution functions end pij(:,j)=y'; end The calculations and illustration for the described algorithm are shown below in Figure 8 and Table 8. When calculating the probability of possible states of alternatives, the mean value, standard deviation, the smallest and largest values of the attribute of alternatives are used. The results of evaluating the weight of the same criteria based on a different set of alternatives (DM-1, DM-2, DM-3, DM-12) are not so much different, as in the case of calculating the probabilities of possible states of alternatives using expression (1), in which a ij is used to determine p ij .
What to do in the absence of information about the distribution of features or rejection of the statistical test 0 hypothesis of distribution? It remains to apply an expert approach. The subjectivity of the assessment with a good expert is minimal. The expert's preferences are a reflection of the frequency of the feature, which is formed intuitively on the basis of experience. Expert scores are, in our opinion, better than statistics, which rarely have integrity and comprehensive coverage.
Fugure 8. Probabilities of states of attributes of alternatives of the first criterion (normal distribution, Decision Matrix from Table 1).
then the entropy of the j-th criterion is ej = 1, and its weight is 0. The order "in the line" is ideal when the distance between neighboring elements is equal. Any violation of the order leads to a decrease in entropy, 2) in accordance with expression (12), the pij values are the relative distances of the ordered list of alternatives of the j-th attribute to the alternative with the smallest value, i.e. tied to a reference point. In contrast to this, dj pq characterizes not the position, but the state, and do not depend on the position of the point on the scale [0; 1], 3) if the distribution of system states is uniform, then the values pij determine the probability that the random value X of the j-th attribute belongs to the interval [aj min ; aij]:  and correspond to the concept of the probability of a state as the probability of belonging to a localized range of values, 4) as noted above, the relative disposition of the attribute of alternatives is invariant with respect to linear data transformation (according to the formula (13)) or retains almost all information about the system (minus two degrees of freedom), 5) each of the elements of the set (33) represents the intensity of the state 6) set (33) represents a differentiated series or the relative rate of change of values in individual sections, 7) unlike pij by the formula (12) the sets {dj} for different j are not subject to displacement relative to each other, and the size of the domains does not depend on the normalization of Sum, 8) set (33) can be considered as an analogue of the distribution histogram.
Thus, to calculate the entropy of each criterion, we use the quantities {dj}: where mj is dimension of the set {dj pq } of different attribute (without 0) values by the j-th criterion (without zeros).
Next, we calculate the weights of the criteria using expression (6). Notes: i. in the EWM.dsp method we used the concept that the probability of a state is proportional to the length of the localized gap. This concept is adequate for the even distribution of attributes. If the distribution is not uniform (or very different from uniform), then it is better to use the EWM.df approach described in section 4.2 above. In any case, EWM.dsp is better than estimates of the probability of states using expression (35) for the basic EWM-method. ii. the proposed approach does not require inversion of values for cost criteria, iii. if e j = 1 for all j, then the weights should be taken equal, iv. if k alternatives have the same attributes, then d j pq = 0. (k-1) values are excluded from the calculations and the entropy is calculated from the (m-k) values. The results of evaluating the entropy and weight of criteria by the proposed EWM.dsp method for matrices DM-1, DM-2, DM-3, DM-12 are presented in Table 9. The weights of the criteria are not as different as in the case of using the basic EWMmethod (Table 2). Figure 9 shows the summary probability diagrams of alternative states and entropy components (-p•lnp) for the decision matrix DM-1, obtained in various modifications of the EWM-method. Due to the equality of the attributes for some alternatives, in the calculations of the EWM.dsp method of the presented example, the number of states involved is less than (m-1). The problem under study, defined by the DM-1 matrix, is weakly sensitive to weight variation. Therefore, for all modifications of the EWM-method, the ranking for the studied example is the same.

Conclusion
The importance of the weights when aggregating the attributes of alternatives is such that alternatives with a priority on the attribute and a higher weight will receive priority in the performance indicator. Therefore, the quality of weight estimation for MCDM tasks is critical. The conducted comprehensive analysis shows that the rationality of all objective methods for evaluating the criteria weights for MCDM tasks is questionable. Specific algorithms for objective methods for evaluating the weights of criteria still require further study. The first decision that needs to be made and which will greatly affect the final results, is the choice between equal and different weights. Equal weighting is the preferred procedure in most applications and makes the weights estimation less subjective. Differential weighing requires choosing the most appropriate approach for determining the weights. The solution should be supported by theoretical considerations that give meaning to each indicator or take into account its effect on synthesis in accordance with the structure of the problem. It is important to compare weighting results for different methods. Since there are no criteria for the effectiveness of weighing methods, the discrepancy in the results of weighting requires serious analysis. The design and determination of weights can be interpreted in terms of value judgments, that methods based on the subjective opinions of individual experts are preferred. It is obvious that the use of various objective (and other classes) methods and modifications lead in many cases to completely different values in the estimates of the weights of the criteria. There are also no criteria for the effectiveness of the methods. Therefore, further research, in our opinion, can be aimed at a constructive solution to the problem -building a decision support system (DSS) for weight estimation, including a wide range of methods, a knowledge base and an intelligent system for analyzing and synthesizing of results. In fact, it is an extended component of DSS for multi-criteria decision support systems. All the algorithms described in this paper are implemented by the author in the MatLab system, posted in a file hosting service "File Exchange MathWorks" and are available for free use at the link (Math Works, 2021).
Funding: This research received no external funding.

Conflicts of Interest:
The authors declare no conflicts of interest.