INVESTIGATION INTO THE EFFICIENCIES OF EUROPEAN FOOTBALL CLUBS WITH BI-OBJECTIVE MULTI-CRITERIA DATA ENVELOPMENT ANALYSIS

: A financially successful football club can achieve sporting achievements as well as become financially stable. In other words, the success of football clubs depends on both financial and sportive success. Contrary to the studies in the literature that focus on financial and sportive success separately, the present study aimed to examine the 5-season activities of 10 football clubs in the big-five league, which are the top leagues of Europe, by using financial and sports criteria. Bi-objective multi criteria data envelopment analysis (BiO-MCDEA) was used for the efficiency analysis. In the study, the number of social media followers, the average number of viewers and total market value were used as input, and the UEFA club score and total revenues were used as output. As a result, Arsenal, Paris Saint-Germain, and Juventus were determined as efficient in the 2015-2016 season, Paris Saint-Germain and Liverpool in the 2016-2017 season, Manchester United, Paris Saint-Germain and Chelsea in the 2016-2017 season, Manchester United, Real Madrid, Bayern Munich and Arsenal in the 2018-2019 season, Manchester United, Paris Saint-Germain and Chelsea. The reasons why PSG was the most successful club in the efficiency analysis (efficient in four out of five seasons) were examined. In addition, in the sensitivity analysis conducted to determine the effect of inputs and outputs on the model, it was concluded that efficiency was highly related to financial data.


Introduction
Football is the most popular sport in the world. Although there are many factors that underlie this popularity, the simplicity of the rules and the low cost can be considered as the most important factors (Galariotis et al., 2018). However, in professional football, which has undergone a great transformation since the early 1990s, footballer salaries have started to increase exponentially (Dobson & Goddard, 2011). The Bosman ruling introduced by the European Court of Justice in 1995 had a significant impact on the future of European football. The Bosman ruling was named after the Belgian midfielder Jean-Marc Bosman's lawsuit that was filed for blocking his transfer from Belgium to France at the end of his contract. The Bosman ruling included the liberalization of the immigration of professional athletes within the EU and the abolition of transfer fees after the expiry of contracts. In addition, restrictions on the number of EU players that clubs can have playing on the field were also considered illegal according to Bosman ruling (Marcén, 2019). After the Bosman ruling was recognized by UEFA in March 1996, the transfers of football players between teams began to be carried out at astronomical figures. In addition, the fact that broadcasting contracts yielded an unimaginable scale of revenue just a few years ago, the complete reconstruction of many football fields, and the immeasurable increase in the importance of commercial sponsorship and merchandising increased the importance of football's financial infrastructure (Dobson & Goddard, 2011). Football clubs are no longer organizations that only provide emotional and symbolic satisfaction to their supporters and focus on sporting success without profit. Instead, football clubs have become a complex system in which investors invest capital and expect financial returns (Miragaia et al., 2019). This development in professional football has turned football from being not only a sport branch in Europe but also an industry branch. The revenues and brand values of football clubs have become competitive with many industries and brands. Spain (La Liga), England (Premier League), Italy (Serie A), Germany (Bundesliga) and France (Ligue 1), which are called the "big five league", constitute a large part of the world football industry. The big five league, which has gained great value in the last 20 years, increased its total value from EUR 2.95 billion in 1998 to EUR 26.8 billion in 2021EUR 26.8 billion in (transfermarkt.com, 2021. However, these financial values are not governed by all the clubs in the big five league, but only the top 10 clubs in Europe in terms of both sporting success and financial standing. Manchester United, Real Madrid, FC Barcelona, Bayern Munich, Manchester City, Arsenal, Paris Saint-Germain (PSG), Chelsea, Liverpool and Juventus have a value of EUR 7.96 billion, which is almost one third of all other clubs total value of the big five league (transfermarkt.com, 2021).
The growth of the football industry at this scale in as little as 20 years has brought along both control and financial difficulties. Although football clubs have many financial resources, these resources are largely related to sporting success. In other words, football clubs must be continuously successful in order to avoid experiencing financial difficulties, which is not possible. As the financial difficulties experienced by football clubs are beginning to become continuous, UEFA has brought some restrictions on clubs with a regulation called Financial Fair Play (FFP). FFP, which entered into force in 2009 and is updated every three years, basically aims to improve the economic and financial capabilities of the clubs and increase both their transparency and reliability. At the same time, thanks to the FFP, which aims to bring more discipline and rationality to club football financing, the ratio of net debts of clubs to their income has decreased from 65% to 35% in a short period of time (UEFA, 2021). In order to achieve this financial success, UEFA has imposed many restrictions on clubs and imposed severe penalties such as the deletion of points, transfer restrictions and ban from tournaments, for those who do not comply with these restrictions. Football clubs, which are suppressed by UEFA, are also trying to meet the sportive success expectations of the stakeholders. It does not seem realistic to evaluate these two processes independently from each other in football clubs where financial success supports sportive success. Some studies in the literature have carried out financial evaluations by only considering the financial data of clubs (Pradhan et al., 2016;Chelmis et al., 2019), some have focused only on sportive success (Rossi et al., 2019;Salabun et al.2020) and others have tried to associate financial success with sportive success (Sakınç et al., 2017;Galariotis et al., 2018). However, the success of football clubs is possible with the realization of the financial and sportive success together in this cycle. The aim of this study, which was designed with the motivation of the idea of realizing financial and sportive success together and the lack in the literature, was to investigate the efficiency values for three seasons of 10 football clubs that are at the top in terms of both sport and finance in Europe. When conducting the effectiveness analysis, the bi-objective multiple criteria data envelopment analysis (BiO-MCDEA) method, recommended by Ghasemi (2014) to eliminate the low discrimination problem of the classical data envelopment analysis (DEA), was used.
The paper begins with a detailed literature review in Section 2. In the Section 3, firstly, the classical DEA and multiple criteria data envelopment analysis (MCDEA) methods that form the basis of the BiO-MCDEA model are introduced and the BiO-MCDEA model is shown. In the data title at the end of the Section 3, how the criteria used in this study were determined, the source of the data used as criteria and the criteria values are shown. In Section 4, the findings of the study are presented and in Section 5, the findings are discussed. In Section 6, the sensitivity analysis is given to determine the contribution of each criterion to the model. In the last section, the conclusions, advantages and limitations of the study and managerial implications are given.

Literature Review
The popularity of football around the world and the huge budgets managed by football clubs have made the football industry the subject of many academic studies conducted to examine the sportive or financial performances of national and international leagues, clubs and even players. In many of these studies, MCDM methods have been used for performance evaluation. Pradhan et al. (2016) investigated the financial performance of Italian clubs using gray relation analysis (GRA), Galariotis et al. (2018) determined the business, financial and sports performance of clubs in the French league using the PROMETHEE II method, Sakınç et al. (2017) studied the financial and sporting performance of 22 European clubs using the TOPSIS method, Chelmis et al. (2019) investigated the financial, commercial and sporting performance of clubs in the Greek league using PROMETHEE II and Salabun et al. (2020) determined the performance of football players using the characteristic objects method (COMET) and TOPSIS method. In addition to these methods used, the most used MCDM method is DEA which was developed by Charnes et al. (1978). In recent years DEA has been used in many decision problems such as the effectiveness of agricultural practices (Angulo-Meza et al., 2019), financial performance assessment (Anthony et al., 2019), hospital efficiency assessments (Kohl et al., 2019), sustainability assessment of the water sector (Lombardi et al., 2019), bank activities assessments (Kamarudin et al., 2019), sustainable supplier selection (Rashidi & Cullinane, 2019), efficiency assessment of railway enterprises (Blagojević et al., 2020), assessment of medium and large-sized industries in the diversity sector (Hassanpour, 2020).
DEA studies in the literature generally consist of efficiency analyses conducted for all teams in the league of a specific country. DEA was used to determine the efficiencies of the teams in England's Premier league (Pestana Barros & Leach, 2006;Guzman & Morrow, 2007;Haas, 2003a;Kern et al., 2012), Germany's Bundesliga (Haas et al., 2004), France's Ligue 1 (Jardin, 2009), USA's Major League Soccer (MLS) (Haas, 2003b), Italian Serie A (Rossi et al., 2019), and Brazil's Serie A (Pestana Barros et al., 2010). In addition, the efficiencies of European clubs (Halkos & Tzeremes, 2013;Miragaia et al., 2019) and national teams participating in EURO 2012 (Rubem & Brandao, 2015) were determined using DEA. However, no study examining the 5season efficiency values of 10 top European clubs which make up half of the total value of the big five league were found in the literature. Furthermore, classical DEA was used in almost all efficiency studies in the literature. Although classical DEA is a widely used nonparametric efficiency instrument, it has the disadvantage of low discrimination power. In order to avoid this disadvantage, the BiO-MCDEA model, which was developed by Ghasemi et al. (2014) and has been used in decision problems such as the equipment efficiency assessment for automotive industry (da Silva et al., 2017), port efficiency assessment (de Andrade et al., 2019), electrical distribution units efficiency assessment (Ghofran et al., 2021), was used in this study.

Material and Methods
BiO-MCDEA is a goal programming based efficiency determination model developed by Ghasemi et al. (2014) in which the DEA model aims to improve the discrimination power. Bal et al. (2010) proposed the goal programming data envelopment analysis (GPDEA) model which is based on goal programming that would eliminate the problem of discrimination power and weight distribution of the DEA model. The GPDEA model is based on solving unwanted deviations using equal weight. BiO-MCDEA was used in the present study to exclude classical DEA and thus avoid the disadvantage of low discrimination power and because its solution steps are easier.

Multiple Criteria Data Envelopment Analysis (MCDEA)
Classical DEA is a widely used non-parametric analysis for efficiency analysis, used especially in social sciences. The conversion of the classical DEA method into a linear programming form proposed by Charnes et al. (1978) is shown below.
Where; j is the number of decision-making units (DMU), r is the number of outputs, i is the number of inputs, is the value of the rth output for the jth DMU, is the value of the ith input for the jth DMU, is the weight of the rth output, is the weight of the ith input and ℎ 0 refers to relative efficiency. In this model, any DMU must be ℎ 0 = 1 in order to be effective (Charnes et al., 1978;Despic et al., 2019) .
Although classical DEA is an efficiency measurement method, Li & Reeves' (1999) MCDEA model is based on ineffectiveness. 0 , which is limited to the [0, 1] range can be considered a measurement of "ineffectiveness" and is defined as ℎ 0 = 1 − 0 . Therefore the smaller the 0 value, the less ineffective (and therefore more effective) DMU is. In the method of Li & Reeves (1999), besides the minimization of d0, which is the measure of ineffectiveness, there are two independent objective functions, namely, minimizing maximum deviation and minimizing the sum of deviations. Their model is as follows: The MCDEA model was proposed primarily as a tool for improving the discrimination power of the classical DEA model. In the solution procedure, MCDEA was proposed as an interactive approach to solve three objectives. The first objective accommodates the classical RIA solution within a set of MCDEA solutions. The other two objectives, Minimax and Minsum, provide more restrictive or lax efficiency solutions, respectively. This model proves that a wider solution is possible to achieve more reasonable input and output weights (Ghasemi et al., 2014).

A Bi-Objective Multiple Criteria Data Envelopment Analysis (BiO-MCDEA)
The MCDEA model consists of three independent objective functions: 0 , and ∑ as defined in Model 2. In a weighted model, these three independent objective functions can be weighted as 1 0 + 2 + 3 ∑ into a single-objective problem. Different efficiency scores can be achieved by changing the Investigation into the efficiencies of European football clubs with bi-objective multi-criteria data envelopment analysis 111 weights ( = 1,2,3). However, since the first objective function ( 0 ) has the same meaning as the classical DEA model, it can be removed from the MCDEA model as the discrimination power of the second ( ) and third ( ∑ ) objective functions has been proved to be higher than the 0 objective (Li & Reeves, 1999;San Cristobal, 2011;Hatami-Marbini & Toloo, 2017). Therefore, only the and ∑ objectives are weighted in the BiO-MCDEA model, which is shown below: The constraints of the BiO-MCDEA model consist of the same constraints as the MCDEA model of Li & Reeves (1999). Only the and variables are constrained by the constant . Although Ghasemi et al. (2014) used = 0,0001 in the samples they solved, they did not suggest an approach to find a suitable value for the constant . In addition, the BiO-MCDEA model is still robust if = 0 in the sample solved using a different data set. In this study, = 0 was used as in the original model.

Data
The data of this study was obtained from the 2020, 2019, 2018, 2017 and 2016 Deloitte Football Money League reports and transfermrkt.com, which regularly collects data on the European football industry every year. In the study, the number of social media followers (v1), average number of viewers (v2) and total market value (v3) were used as input, while the UEFA club score (u1) and total revenue (u2) variables were used as output.
In their studies Aichner (2018) Table 1. The number of people following the clubs on facebook, instagram and twitter (*10 6 ) Average number of viewers ( 2 ) The average number of people who came to the stadium as a spectator in matches hosted by clubs Total market value ( 3 ) The sum of the market values of the club's footballers (*10 6 €) UEFA club score ( 1 ) The total points the club has obtained from all matches during a season Total revenue ( 2 ) The sum of club's matchday revenues, broadcasting revenues and commercial revenues (*10 6 €) Pearson correlation coefficients are widely used when choosing input and output in DEA (Lewin et al., 1982;Thanassoulis et al., 1987;Golany & Roll, 1989;Friedman & Sinuany-Stern, 1998;Dyson et al., 2001). Lewin et al. (1982) argued that inputs should not be highly correlated with other inputs and outputs should not be highly correlated with other outputs. They also stated that if the inputs and outputs are negatively correlated with each other, these variables may be excluded from the model since the increase in inputs will affect the output negatively. The Pearson correlation coefficients of the data used in the present study are shown in Table 2. According to the results given in Table 2, none of the Pearson correlation coefficients had a very high, very low or negative value. Therefore, no input or output variable was excluded from the model.
In this study, an analysis of the efficiency of five seasons of 10 top European football clubs was performed. The values of input and output variables selected to determine the effectiveness of the 2015-2016, 2016-2017, 2017-2018, 2018-2019 and 2019-2020 seasons are shown in Table 3. The reason why the 10 clubs were included in the efficiency evaluation is that these 10 clubs were ranked in the top 10 for five seasons in the Deloitte Football Money League report, which was the main data source of this study. The Deloitte Football Money League report publishes data for the 20 top financially successful clubs each season. However, 10 clubs other than the top 10 change at a certain rate each year. As data of some of the clubs other than the top 10 clubs from different sources could harm the homogeneity of the data, the clubs not included in the top 10 clubs were excluded from the scope of the study.

Results
The MCDEA and BiO-MCDEA efficiency scores of the football clubs were calculated using LINDO w32 software. The first three columns in Table 4    This shows that the financial and sporting success of the clubs affects their rankings in different seasons.
Spearman rank correlation was commonly used in the literature to test the relationship between DMU rankings (Haas et al., 2004;Bal et al., 2010;Örkcü & Bal, 2011). In this study, the relationship between the ranks determined for three different seasons as a result of the BiO-MCDEA model was tested with spearman rank correlation. When the results of the Spearman rank correlation were examined, no statistically significant relationship was found between the rankings for the three seasons. This result supports the idea that the financial and sporting achievements of the clubs affect their rankings in different seasons. In other words, the clubs achieved a ranking according to how successful they were in sports or financial terms. Although the model was created for three consecutive seasons, the rankings differed greatly. The Spearman correlation values are shown in Table 5.

Sensitivity Analysis
These remarkable results raise the question of how much input and output variables contribute to the model when determining the BiO-MCDEA model ranking. Therefore, a sensitivity analysis was performed to determine which input or output contributed to the model. To determine the contribution of each input and output variable, BiO-MCDEA efficiency values including all the variables and BiO-MCDEA efficiency values calculated by excluding each input and output variable were examined. In addition, Pearson correlation coefficients were examined to determine the relationship between the efficiency values of the model including all input and output variables and the efficiency values when each variable was excluded from the model. The sensitivity analysis results and Pearson correlation coefficients are shown in Table 6. When the results of the sensitivity analysis were examined, a significant correlation was observed between the BiO-MCDEA efficiency scores, which included all inputs and outputs, and the BiO-MCDEA efficiency scores, where two inputs and one output were excluded from the model. In particular, when the number of social media followers (v1) input variable was excluded from the model, an excellent correlation (r1=0.943) was observed between the obtained efficiency values and the activity values in which all variables were included in the model. That is to say, this variable did not contribute to the model. In the same way, a statistically significant and strong relationship (r4=0.760) was observed between the efficiency values obtained by excluding the UEFA club score output (u1), and the BiO-MCDEA model in which all variables were included. It was found that these variables did not contribute to the model. The biggest contribution to the model was average number of viewers (v2) and the total market value (v3) inputs and the total revenue (u2) output. In other words, mainly the financial variables influenced the ranking of the BiO-MCDEA model of the clubs.

Discussion
In the analysis made using the data of the 2015-2016, 2016-2017, 2017-2018, 2018- PSG was determined as an efficient club in four out of five seasons included in the analysis. In other words, PSG was the most successful club among the analyzed clubs. This may be attributed to the sale of the club to a Qatari fund group in 2011. Correspondingly, the market value of the club increased with the large expenditures made for transfers immediately after the sale. With this acceleration, the club which had only won two championships in the France Ligue 1 since 1970, became the champion seven times in eight seasons after the 2012 season. The club, which became the champion almost every season after the 2012 season, increased its UEFA club points by joining the Champions League every year and achieved a financially stable structure. In other words, the club, which was financially supported after the 2011 season, increased its sporting achievements, which in turn stabilized its financial support. Among the 10 clubs included in the analysis, Manchester City, which was purchased by a funding organization like in the case of PSG, was not as efficient as PSG according to the BiO-MCDEA model. In the 2008 season when Manchester City was acquired, it was financially supported, similar to PSG. However, Manchester City has not been as successful as PSG. The reason for this is that Manchester City cannot dominate the Premier League as PSG dominates Ligue 1, as 5 of the top 10 clubs at the top of the Big-Five League compete in the Premier League. This suggests that financial support alone does not have an impact on success.
In this study, a sensitivity analysis was performed to measure the sensitivity of efficiency measurement results according to different input/output combinations. Each input and output was removed from the model, which was then resolved and the behavior of the model against the extracted variable was monitored. While a perfect correlation was found between the model created by subtracting the "the number of social media followers" input and the original BiO-MCDEA model, statistically significant correlations were observed between the model created by subtracting the "UEFA club scores" output from the model and the BiO-MCDEA model. This means that while the "the number of social media followers" input makes almost no contribution to the model, the "UEFA club scores" output provide relatively less contribution to the model than other inputs and outputs. No relationship was found between the original model and the BiO-MCDEA model created by excluding the "average number of viewers" and the "total market values" inputs and the "total revenues" output from the model. These variables were determined as the main determinants of the model. This suggests that the variables that contribute to the model are mainly financial variables. However, especially considering the use of social media in the 21st century, it is noteworthy that the "the number of social media followers" variable is not determinative in terms of the model.

Conclusion
The purpose of this study was to determine the efficiencies of the top 10 clubs in the Big-Five League, which make up the largest share of the world football industry. The analysis of efficiency for only 10 clubs can be counted among the limitations of this study. The reason for the inclusion of these 10 clubs in the efficiency review was that although the rankings of the clubs have changed, they are still in the top 10. The Deloitte Football Money League report, from which most of the data in this study was obtained, publishes data on the top 20 clubs in terms of finance every year. While the clubs in the top 10 almost never change, the clubs in the last 10 can enter and exit the list. Only 10 clubs were included in the analysis to obtain consistent data over the entire five years of the analysis. Another limitation of this study was that the analysis was carried out with only quantitative data. However, this analysis could be supported by qualitative data obtained from football professionals including club managers, sponsors, etc. In future studies, the number of football clubs included in the analysis can be increased by using more resources and time, and the obtained quantitative findings can be supported by qualitative findings.
The BiO-MCDEA model, which is an efficiency determination method based on linear programming, was used in the efficiency analysis. It can be said that using this model was the most obvious advantage of the study. The reason for selecting the BiO-MCDEA model was to prevent the low discrimination problem of classical DEA. The findings of the study also included the results obtained with classical DEA. When the classical DEA findings were examined, it was concluded that a very high number of clubs were efficient. In this case, it will be difficult to distinguish between clubs. Moreover, useful information for decision-makers cannot be obtained. "Super efficiency" models can be used to determine which of the efficient clubs are more efficient. In this case, it will produce more complex results for both decision makers and analysts. In addition, the BiO-MCDEA model has easier solution steps compared to other methods such as MCDEA and GPDEA that aim to eliminate the low discrimination power problem of classical DEA. Another advantage of this study is that sports and financial data were used together. This is because financial success is to be used as leverage for sportive success. In this respect, instead of evaluating and associating financial and sports data separately, this study included both in the same model. As financial and sportive success can only be achieved through successful management practices, some managerial implications were made in line with the findings of the study. In order to examine the contribution of the criteria to the model, a sensitivity analysis was conducted in which each criterion was removed from the model, which was then solved again. According to the results of this analysis, the criteria of average audience number, total market value and total income were determined as the criteria that made the greatest contribution to the model. Although the criteria for total market value and total income are direct financial criteria, the average number of viewers seems to be a non-financial criterion. This is because the matchday revenues are at the lower ranks among the revenue items of football clubs. However, bringing fans to the stadium does not only contribute to the clubs as ticket revenue but also to the sales in commercial products and to sponsors spending more on stadium advertisements. In addition, the fact that football clubs achieve more sportive success in the home field can be explained with the support of the fans. From this point of view, club management can implement various practices to make stadiums more attractive to the fans. Among the practices that increase the attractiveness of stadiums are the club management selling tickets at lower prices, facilitating access to the stadium, and creating areas where families can spend time in the stadium.
Although the 10 clubs analyzed are not in the same league, they are constantly in competition as they participate in international tournaments every year. In order to keep competition alive, the financial resource must be sustainable. In order for the financial resource to be sustainable, clubs want to continuously participate in international tournaments, which are one of the most revenue generating elements of the industry. The financial benefits of a successful season will only benefit the club in the next season. Although Real Madrid was champion in the champions league in the 2017-2018 season, according to the analysis conducted in this study, it was found to be an efficient club in the 2018-2019 season, not the 2017-2018 season. Similarly, Chelsea, which was champion in the European League in 2018-2019, was only found to be an efficient club in the 2019-2020 season. As these examples show, clubs can only provide sustainable financial resources with sustainable sportive success. Moreover, they can transfer talented players who can participate in international tournaments every year to make financial resources sustainable, or they can invest in their academies to produce their own qualified football players.
Clubs that make their financial resources sustainable are referred to as "big clubs". It can be said that achieving sportive success is easier for these clubs compared to other clubs, as big clubs are more advantageous in terms of attracting talented and qualified players. However, financial sustainability depends on a number of factors that are not constantly under control. For example, some penalties imposed by UEFA in accordance with FFP policies harm the financial sustainability of clubs. In addition to the clubs' efforts to cope with the FFP limitations, the covid-19 pandemic, which emerged in the province of Wuhan in China in December 2019 and spread all over the world in a short time, led to huge decreases in the revenues of the clubs. Due to Covid-19, some countries have suspended their leagues for a long period of time, broadcasting agreements were interrupted and stadium revenues were not obtained. To avoid the effects of factors such as these that could harm financial sustainability, clubs sometimes turn to finding new sources of funding. For example, clubs may try to provide additional financing with initiatives such as the "European Super League", which was established on April 19, 2021 and was dissolved after only 48 hours. However, for a sport whose rules and organizations are deeply rooted, such initiatives may cause clubs to disconnected from other clubs. Therefore, in order to make the financial resource sustainable, clubs should make their sportive success sustainable in every platform.
Funding: This research received no external funding.

Conflicts of Interest:
The author declare no conflicts of interest.