Efficiency of credit unions in Brazil: an analysis of the evolution in financial intermediation and banking service

Manuela Gonçalves Barros (Department of Accounting Sciences, School of Administration and Accounting Sciences, Federal University of Mato Grosso, Cuiabá, Brazil)
Marcelo Botelho da Costa Moraes (School of Economics Administration and Accounting of Ribeirão Preto, University of São Paulo, Ribeirão Preto, Brazil)
Alexandre Pereira Salgado Junior (School of Economics Administration and Accounting of Ribeirão Preto, University of São Paulo, Ribeirão Preto, Brazil)
Marco Antonio Alves de Souza Junior (School of Economics Administration and Accounting of Ribeirão Preto, University of São Paulo, Ribeirão Preto, Brazil)

RAUSP Management Journal

ISSN: 2531-0488

Article publication date: 5 June 2020

Issue publication date: 4 September 2020

1841

Abstract

Purpose

The purpose of this paper is to evaluate the efficiency in financial intermediation and the cost efficiency in banking service of credit unions in Brazil, based on essentially accounting variables, and to analyze the temporal evolution of the efficiency of these cooperatives.

Design/methodology/approach

With a sample of 315 cooperatives over the period from 2007 to 2014, this research uses a two-stage process: application of regression models with panel data to verify which variables are related to the defined outputs, with the reduction of 31 variables to 8 variables in both models; and application of the data envelopment analysis method to obtain an analysis of credit unions’ efficiency.

Findings

The results demonstrate a high level of efficiency in financial intermediation, with low variation over time, associated with a low efficiency in the banking service, in which few cooperatives have remained efficient over time. In addition, the cooperatives with highest efficiency in financial intermediation were also the most efficient in providing services.

Research limitations/implications

This research has some limitations about the capacity of the proxies used to capture the real effect of the variables and assumptions of economic relations resulting in restrictions to generalize the results.

Practical implications

Cooperatives are usually analyzed under just one dimension. By separating the analysis into financial intermediation and banking services, cooperatives that are more efficient in each dimension can be identified, in addition to analyzing the evolution over time. The authors found that efficiency tends to be lower in banking services, and few cooperatives remain at the highest level of efficiency over time in both models.

Social implications

Credit unions provide an important service in the banking and credit market. Therefore, understanding its operation and the characteristics that influence its efficiency allows a better management of the cooperatives themselves and a greater understanding of this important segment of the financial market.

Keywords

Citation

Barros, M.G., Botelho da Costa Moraes, M., Salgado Junior, A.P. and de Souza Junior, M.A.A. (2020), "Efficiency of credit unions in Brazil: an analysis of the evolution in financial intermediation and banking service", RAUSP Management Journal, Vol. 55 No. 3, pp. 289-308. https://doi.org/10.1108/RAUSP-06-2018-0029

Publisher

:

Emerald Publishing Limited

Copyright © 2020, Manuela Gonçalves Barros, Marcelo Botelho da Costa Moraes, Alexandre Pereira Salgado Junior and Marco Antonio Alves de Souza Junior.

License

Published in RAUSP Management Journal. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) license. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this license may be seen at http://creativecommons.org/licences/by/4.0/legalcode


1. Introduction

Credit cooperatives are financial institutions based on cooperation, which act in the capture and distribution of financial resources and in the transfer of values between economic agents and which make up the Brazilian National Financial System (Sistema Financeiro Nacional). These organizations are institutions that collect deposits (BACEN, 2015) and have an activity similar to commercial banks, which meet the credit demand of borrowers and the expectation of attracting savers and provide financial services to their clients, such as bank accounts, insurance, transfers, money orders and more.

According to McKillop, Glass and Fergunson (2002), the greatest strength of these organizations stems from their philosophy and objectives having a worldwide appeal coming from a diversity of people who see an advantage in achieving greater self-sufficiency in the management of their financial affairs. Such organizations aim to meet the social and economic objectives of their membership, and the excess money generated by business activities belongs to their associate members (Mckillop & Wilson, 2011).

The performance and efficiency of credit unions have been assessed in different dimensions by different authors, such as Vilela, Nagano, and Merlo (2007); Ferreira, Gonçalves, and Braga (2007); Cook and Bala (2007); Glass, Mckillop, and Rasaratnam (2010); Silva, Gollo, and Rodrigues (2013), Carvalho, Diaz, Bialoskorski Neto, and Kalatzis (2015) and Bittencourt, Bressan, Goulart, Bressan, Costa, and Lamounier (2017). Some of these studies proposed methods for evaluating efficiency through models of global efficiency, price efficiency, technical efficiency, among other approaches, but they do not use the separation of efficiency analysis between different models. Still, Porter and Scully (1987) affirm the existence of a common line that efficiency results from the appropriate choice of the cooperative’s objectives, and this is measured by matching the marginal benefits for the cooperative association with the cooperative’s marginal costs.

Abreu, Kimura, Araújo Neto and Peng (2018) indicate the need for specific models with relevant results for cooperatives. Thus, having credit unions as the main objective of generating benefits for the members, this study contributes to assess the efficiency of these organizations in Brazil under two aspects – in financial intermediation and in the costs of providing services – based on accounting variables, on their performance for the benefit of the members, considering the period from 2007 to 2014.

Therefore, the aim of this study is to answer the following question:

Q1.

How does the efficiency of financial intermediation and provision of banking services of Brazilian credit unions behave throughout time?

Efficiency in financial intermediation reflects the financial services including better prices to the members in terms of borrowing and fundraising rates, which leads to a spread reduction but does not disregard the need to generate income from the activity.

On the other hand, cost efficiency in providing services relates to a greater share of service revenue – considering a greater share of services is not linked to financial intermediation –to contribute to the generation of surplus in the cooperative, associated with lower costs related to these services.

The Brazilian studies that addressed the efficiency of credit unions from the perspective of the cooperative members focused on the volume of credit operations practiced as the main socioeconomic benefit outcome. However, the importance of including aspects that are not yet considered and that directly impact the income of the cooperative members is highlighted, in addition to the consideration of the performance of these organizations aimed at providing banking services and variables related to the prices practiced by these organizations in international literature.

Thus, the results of the present study contribute to the literature in the following aspects:

  • to evaluate the efficiency of credit unions in the perspective of generating benefits to members in Brazil;

  • to use variables that represent benefits in the aspects of pricing policy, banking service and capacity to generate income and adding proxies already used in international literature; and

  • to provide a temporal analysis of the evolution of efficiency in this industry.

2. Credit unions

Cooperative members choose to join for economic reasons, with benefits of economic value including services, products and prices available to those who buy from or sell to the cooperative (Barton, 1989). In the case of credit unions, Rubin, Overstreet, Beling and Rajaratnam (2013) consider that the quality of membership gives members the right to the potential monetary benefits of competitive prices for loans, deposits or services. For Barton (1989), the benefits also include the distribution of surpluses to eligible or participating in the form of refunds and dividends on invested capital paid to members and the salaries paid to employees. For the author, because it is the members who organize and control the cooperatives, it is usually for them that the benefits are distributed. Such factors, together, constituted the foundation of the efficiency concept used in this work.

We assume that credit unions can generate economic benefits to members through three practices. First, the pricing policy, which assumes that a cooperative should seek the following:

  • to provide a higher rate of return on deposits; and

  • charging lower interest rates for loans, resulting in a lower spread (Emmons & Schmid, 2002).

Second, providing of banking services, which is common in commercial banks, with presumably lower bank fees than other financial institutions. Third refers to the distribution of income (Emmons & Schmid, 2002).

Accepting savings from members, credit unions have characteristics of producer cooperatives, while in providing loans to members, they have characteristics of consumer cooperatives (Fried, Lovell, & Eeckaut, 1993; Taylor, 1971). This suggests higher interest rates on deposits for saving members conflict with the lower interest rates on loans for borrowing members. To Fried et al. (1993), the lack of unity of interests makes cooperative models of producers and consumption inappropriate for credit unions.

As the interests of the saving members are neither more nor less important than the interests of the borrowing members, it is desirable to avoid attributing, initially, weights for savings and loan services that a credit union offers to its members (Fried et al., 1993). In principle, both services are of equal importance in terms of financial intermediation performance.

However, there are models on the behavior of credit unions that place a strong emphasis on how the balance in the distribution of benefits among their borrowing and saving members is achieved, some of which are presented by McKillop and Wilson (2011). According to the authors, Taylor’s (1971) approach, which focuses on the extent to which credit unions balance the interests of saving members and borrowers, creates a problem, because while saving members want the greatest possible return on invested capital, borrowing members want the lowest cost of credit possible.

Although there are reportedly three possibilities for users in a credit union – the fundraiser, the investor and the “fundraiser and investor” of financial resources – the economic aspects of conflict in these cooperatives are less of an issue than in a common cooperative (Taylor, 1971). The first reason pointed out by the author is that although there are similarities in relation to conflicts between current and new fundraisers and between current and new savers, the relationship between current funders and new savers and between current savers and new funders is complementary. This is because the demand for long-term loans represents the maximum source of financial resources for paying interest to savers, while savers’ financial resources are the lowest cost source for borrowed resources (Taylor, 1971).

The second reason given by Taylor (1971) is that in credit unions, although members initially unite for the purpose of lending or saving, they usually do a little of both operations while they are associated with the cooperative, with the principle of self-help prevailing over self-interest.

Given the objective of acting in favor of the members’ benefit and the coexistence of members who borrow and save, the credit cooperative will have a greater efficiency in generating benefits to the members, in terms of financial intermediation, if it seeks the best interest to both types. If the cooperative seeks to offer a higher return for investors and a lower rate for borrowers, in a neutral way, it is not possible to provide the maximum price benefits for users who are only savers or borrowers. Such an assumption justifies, in this study, the consideration of two variables of generation of economic benefits to the members related to the price policy.

This idea agrees with the theoretical analysis of McKillop and Wilson (2011) on the work of Taylor (1971) that credit unions that seek the simultaneous benefit of both types of members are more efficient. For the authors, neutrality is less likely to create incentives for credit unions to discourage the participation of new members and helps to maintain the institution’s vitality.

Although financial intermediation is the main service provided by cooperatives to their members, with regard to performance in providing banking services that do not generate interest, as a way of benefiting members and as financial institutions that they are, credit unions can offer other services, such as custody, funds transfer, bank accounts, among others. As the general objective of credit unions also involves the provision of services for the benefit of members, it is understood that the performance assessment in these organizations should be done considering this.

Regarding the concept of providing banking services other than financial intermediation, the approach used in relation to the definition of inputs and outputs stands out. According to Holod and Lewis (2011), owing to different possible approaches to the definition of inputs and outputs in financial institutions, there is a disagreement between the current models of bank efficiency regarding the use of the variable deposits as input or as output of the models – a fact which gives rise to the financial intermediation approach and the production approach. The present study uses the production approach, in which deposits are treated as output, as they are a service provided by a bank to its customers. For Berger and Humphrey (1997), deposits also have this characteristic, as they are associated with a substantial amount of liquidity, custody, services and payments.

We also believe that the need for surplus generation by the credit union should not be ignored. Given that, in cooperatives, operations are carried out with their own members, these organizations do not seek to maximize operational waste. This is because the “profits” are first distributed in the form of reduced prices for any services that the organization provides to its members (Milgrom & Roberts, 1992), which requires operational and financial advantages in providing of services and in the financial intermediation operations.

Although the capacity to generate income is reduced in cooperatives, a positive generation of earnings is necessary to enable them to have a good financial status, survival and growth. According to Quiroga, Bressan and Braga (2005), higher the credit union’s capitalization better will be the cooperative’s financial status. For Bauer (2008), the relationship between capitalization and the cooperative’s financial health occurs because the cooperative’s growth is limited to the required reserves, and the only way to increase these reserves is through the retention of income. Although this retention postpones the allocation of the benefit, it reduces the risk of bankruptcy of the cooperative (Rubin et al., 2013).

We emphasize that the income implies decisions regarding their explicit destination for the constitution of mandatory and nonmandatory reserves (Brazil, 1971), considering possible financial losses resulting from insufficiencies in the contribution of members and in decisions regarding the return to the cooperative members, which must be made in proportion to the operations carried out by them – which strictly speaking – must be approved by the general meeting.

Thus, we assume in this work that there is a widespread practice of distributing income to cooperative members; however, retention also reverts to benefits as a form of reinvestment, favoring the maintenance of credit union activities in the long run.

3. Methodology

Initially, we present the main aspects concerning the technique used for the efficiency assessment, followed by the presentation and justification of the inputs and outputs used in the construction of the efficiency assessment models.

3.1 Data envelopment analysis

For the evaluation of efficiency, data envelopment analysis (DEA) was used, a nonparametric technique that allows the identification, within a set of comparable decision-making units (DMUs) of the units of production with the best practices, which form an efficient frontier from the optimization of each individual information (Cook & Seiford, 2009). Thus, each DMU has its efficiency rating obtained in an iterative way, which is necessary so that the performance of each firm is compared with that of the other components of the sample (Soares Mello, Ângulo Meza, Gomes, & Biondi Neto, 2005).

The efficiency measured by the DEA reflects the ability of a DMU, from a given set of inputs, to obtain the maximum product or, differently, from a given level of product, to obtain the lowest use of inputs (Ferreira, 2005).

Despite calculating relative efficiency using a relationship between inputs and products, the DEA makes no assumptions as to how a DMU converts inputs into outputs (Holod & Lewis, 2011). According to the authors, this characteristic is especially attractive for financial institutions, which do not have a well-defined production function.

One of the requirements for using DEA is that DMUs belong to a homogeneous group, which performs the same tasks with the same objectives and operates in the same industry, given the same market conditions (Soares Mello et al., 2005). Given the objective of evaluating the efficiency of credit unions, within their own group in relation to those that present the “best practices,” the DEA is applicable to the research problem.

In view of the different variables defined as representative of the economic benefits to cooperative members, owing to the fact that credit unions produce multiple products through loans and savings activities and provide banking services, the adequacy of the technique is justified.

The classic DEA models are the Charnes, Cooper and Rhodes or constant returns to scale (CRS) and the Banker, Charnes and Cooper or variable returns to scale (VRS). Both have variations resulting from the orientation toward maximizing outputs or minimizing inputs.

In this study, the DEA model used is the VRS owing to the difference in technology used in small and large credit unions. According to Coelli, Rao, O’Donnell, and Battese (2005), the CRS assumption is appropriate when all firms operate at the optimum scale, which is not the case in the used sample.

Despite the differences between the types of credit unions, we decided to not separate the different classifications and systems of cooperatives. This option does not impact the results, as the use of the regression step with panel data for the selection of variables in the DEA models proves to be statistically significant for the sample, in a way that validates the homogeneity of the sample for measures of performance used in regressions, allowing its extrapolation to efficiency models with application of DEA, in an approach similar to Bittencourt et al. (2017).

We use the VRS orientation for outputs, considering that efficient cooperatives, given the available inputs, will produce the largest possible product in relation to the others. The mathematical representation of the model is presented in equation (1):

(1) Max h0Subject toxiok=1nxikλk0,  ih0yjok=1nyjkλk0,  jk=1nλk=1λk0,  k
where:

ho = efficiency (ho = 1 Effo) of the DMU0;

Eff0 = efficiency of the DMU0 in analysis;

xio and yjo = inputs i and outputs j of the DMU0, respectively;

xik and yjk = inputs i and outputs j, respectively, of the DMUk, n, k =1,…; and

λk = contribution of the DMU k in the target of the DMU0.

The first restriction, as in the CRS model with product orientation, ensures that the increase in outputs does not change the current level of inputs; the second restriction ensures that the increase in each of the outputs does not exceed the frontier defined by the efficient DMUs and the third restriction is related to the border convexity.

Regarding the efficiency results, the DMU’s efficiency score ranges from 0 to 100, with the score 100 being interpreted as the maximum relative efficiency and indicates that the DMU in reference is a benchmark for the others in the analyzed group. Scores below 100 reveal relative inefficiency. The degree of inefficiency given by (100/score −1) × 100, indicating the proportion of increase in product that the inefficient DMU could have, maintains the amount of inputs, when compared to the reference DMUs.

For the composition of the efficiency assessment models, the definition of the outputs of each model, considering the generation of benefits to the members, and the selection, using regressions with panel data, of the inputs for each model are described in subsection 3.2 and Section 4, respectively.

3.2 Definition of the outputs of the efficiency models

We selected the following variables from the existing literature related to efficiency and performance in credit unions so that it sought to reflect the economic benefits to the T1 cooperative, as shown in Table 1.

The efficiency in financial intermediation reflects the provision of banking services and these services at more affordable prices to the members in terms of fees and does not disregard the need for surplus generation by the cooperatives.

The efficiency in financial intermediation was carried out using the five proxies as output variables, which are as follows:

  1. interest rate on loans;

  2. interest rate on deposits;

  3. volume of credit operations with members;

  4. volume of deposits from members; and

  5. gross income from financial intermediation.

The fact that the cooperative members are associated with their credit union, continuously capitalizing it, facilitates the reduction of the union’s spread and the better exploitation of its operational capacity (Ferreira et al., 2007). Thus, following Fried et al. (1993), Wheelock and Wilson (2013) and Frame, Karels, and Mcclatchey (2003), the interest rate on loans and the interest rate on deposits were used as proxies of the price policy in the financial intermediation activity.

Both proxies were used under the assumption, based on McKillop and Wilson (2011) and Fried et al. (1993), given the coexistence of the two types of members; if the cooperative seeks the simultaneous interest of both types in a neutral way, it will act more efficiently. It is noteworthy that in this way it would not be possible to provide the maximum price benefits for members who are only savers or borrowers.

About the loans, savings and banking services provided by the credit union, these must be maximized for the benefit of the members. Thus, the greater the volume of credit operations with members and the volume of deposit from members the better will be the cooperative will be in its self-sufficiency.

The volume of credit operations is commonly used as an output, generally as the main one, in studies about the efficiency of credit unions, such as Fried et al. (1993), Ferreira et al. (2007); Vilela et al. (2007); Bressan, Braga, and Bressan (2010); Silva et al. (2013), Wheelock and Wilson (2013); Glass et al. (2010) and Cook and Bala (2007).

The volume of deposits collected from cooperative members, following Fried et al. (1993) and Cook and Bala (2007), was used as an output because of the adoption of the production approach of efficiency.

The variable gross profit from financial intermediation represents the cooperative’s ability to generate positive earnings for the credit union through financial intermediation. The capacity to generate income as an essential factor for maintaining the cooperative’s activity in the long run is corroborated by Ferreira et al. (2007).

The efficiency of credit unions in providing services that do not generate interest was carried out considering the capacity to generate surplus through this activity and the practice of lower costs. As shown in Table 2, only one proxy was used as an output – the share of service revenue.

For Bressan, Braga, Bressan, and Resende Filho (2011), the importance of service revenue lies in the fact that the greater the relationship between services and administrative expenses, the greater the efficiency of the institution. Matias, Quaglio, Lima, and Magnani (2014) found that, in credit unions, the fees charged for financial management, such as bank account services, credit cards, credit operations and guarantees provided, among other services provided to customers, have evolved significantly in the past years.

Thus, the greater the share of service revenue in total revenue and the lower the administrative expenses, the better will be the credit union performance in the generation of income. Silva, Leite, Guse, and Gollo (2017) demonstrate the relationship between the efficiency of credit unions in Brazil with the use of financial measures, especially those of performance.

3.3 Sample composition

This study used secondary data as a research strategy, and credit unions were analyzed at the firm level. The initial database is composed of the credit union components of the database of the Central Bank of Brazil that presented operations throughout the period from 2007 to 2014. Despite the objective of efficiency in the perspective of generating benefits to the cooperative, there was no access to the data of the cooperative members, so the assessment was made from the accounting data of the cooperatives.

The sample composition process is summarized in Table 2.

The elimination of cooperatives that presented negative values for the variable gross income of financial intermediation occurred because the DEA does not work with negative values. Portela, Thanassoulis, and Simpson (2004); Freaza, Guedes, and Gomes (2006) and Lee, Joo, and Park (2017) present other possibilities to work around this limitation.

We choose to keep using the classic DEA models and considering that, in the case of a transformation of variables, the variable that would need to change is an output variable – which would require the use of the DEA model as a guide for inputs (Lovell & Pastor, 1995 and Pastor, 1996) – and as the guidance used in this work is for outputs, it was decided to reduce the sample. According to Freaza et al. (2006), if the number of units under evaluation is large, the simple exclusion of units that have negative values in resources and products can be adopted.

To identify outlier firms, we used the interquartile interval owing to the nonnormality of the analyzed data, and the elimination of outliers occurred in order not to harm the results of the regressions with panel data (Gonçalves, Braga & Ferreira, 2012) and to adapt the sample to the use of the DEA technique, whose main weakness is the sensitivity to this type of observation (Cook & Seiford, 2009).

Thus, the results found refer to the range of credit unions in the final sample after the elimination of probable outliers, whose descriptive statistics of the main sets of accounts are presented in Table 3:

The indicators that affect the output variables, or performance, in the financial intermediation activity and in the banking service provision activity of the credit unions under study were identified based on the evaluation of the impact of the accounting indicators selected to compete as inputs in the models.

Thus, a set of accounting indicators commonly used to assess banking activities was applied to assess their significance in the performance of credit unions in generating benefits for members. We used regressions with panel data for each model, with the output variables being the dependent variables and the possible inputs being the independent variables.

As the set of indicators presents a wide variety and the results of performance and efficiency are the same objectives of the research, this step allows an analysis on the determining factors of the outputs in the DEA model (Table 1). The difference between performance and efficiency is found in the fact that performance is an absolute measure obtained through regressions and efficiency is relative, obtaining a result of maximizing the outputs considering the same inputs.

We do not give any specific treatment to credit unions that may have incorporations, that have discontinued their activity or that have changed their profile to free admission association process during the period under analysis, which is pointed out as a limiting of this research. Previous studies that examined the effects of mergers and incorporations of international credit unions on their efficiency show that the effects were mixed, with improvement and degeneration (Yamori, Harimaya, & Tomimura, 2017).

4. Analysis of results

In Section 4, we present the steps to define the input variables and operationalize the models, in addition to discussing the results found.

4.1 Input variables of efficiency models

We verified the performance-determining indicators in the financial intermediation activity and in providing banking service activity of the credit unions represented by the output variables described in Table 1 by the impact of the accounting indicators that contributed to input type in the model.

Initially, we calculated 31 indicators considered relevant and commonly used for banking activity: capitalization, leverage, fixed assets, floating funding, tax and labor liabilities, investments in credit operations, investments in treasury, coverage with services, funding costs, administrative costs, profit sharing of other operating revenues, operating income generation, operating efficiency, operating margin, nonproblematic credits [1], default, insolvency, provisioning, adjusted net equity commitment, net equity commitment, demand deposits and savings, funding by long-term deposits, share of revenue from lending and leasing operations, share of treasury result, spread, general liquidity, asset mismatch, liabilities mismatch, administrative expenses/total assets, net income/total assets and return on assets.

To avoid multicollinearity issues, indicators that showed a Spearman’s correlation greater than |0.80| were eliminated owing to nonnormality in some of the variables used, leaving 14 indicators. Subsequently, the variance inflation factor test corroborates the absence of multicollinearity.

The input variables used in the model were those that were significant at 99% significance in the results of the regressions with panel data and that influence at least one of the dependent variables with the appropriate sign for the cooperative’s objectives. For example, the more efficient the cooperative will be, the smaller will be the interest rate on loans, so the variables that affected the output with a negative sign were defined as inputs. This step allows the identification of the determinants of the outputs, considering as inputs only the statistically significant variables.

For all regression models, the results of which are shown in Table 4, the Chow test rejected the null hypothesis that “the groups have a common intercept.” By the Breusch–Pagan test, we found that the variance of the residuals rejects the null hypothesis that the “variance of the unit-specific error = 0,” so that the pooled ordinary least square model is not indicated. The Hausman test also rejects the null hypothesis that “the generalized least squares estimates are consistent,” indicating that the random effects model is not indicated, in this case, considering Wooldridge’s (2008) indicative that the fixed effects model is preferable.

Although the variable share of the result of other operating revenues, operating efficiency, operating margin and share of revenue from services has been indicated by the results of the regressions, we understood that they are result indicators and do not fit the definition of inputs, as they are more related to products of cooperative activities. This interpretation culminated in their elimination from the model. In addition, the funding by long-term deposits was also eliminated, as the production approach was adopted. Table 5 shows the final composition of the models.

For each of the models, the variable selected as input for one of the output variable components of the same entered as input for the model, even though it was not selected for another component variable of the same model.

Table 6 presents the final input variables that make up each of the efficiency models and their formulas.

The following variables are inputs to the model for evaluating efficiency in financial intermediation: capitalization, fixed assets, insolvency, liabilities mismatch and administrative expenses/total assets. In the model for efficiency in providing banking services, the variables are inputs: fixed assets, administrative expenses/total assets and funding by floating.

The software used for the processing of models by DEA was Frontier Analyst®, a tool from Banxia® Software Ltd., Glasgow, Scotland.

For data analysis in DEA and to allow greater discrimination between cooperatives, we multiplied all input and output variables by 100,000 avoiding scale problems, as DEA models are sensitive to scale differences and variables with reduced scale of variability. With this, we choose the adoption to replace the zeros with the decimal value 0.01, a substitution that does not change the classification of DMUs but that makes the solution of the model possible (Kassai, 2002). This resource was adopted because many cooperatives had a zero value for the liabilities mismatch and insolvency variables, and the model used does not work with a zero value.

4.2 Cooperative efficiency results

The DEA analysis of different periods allows the perception of the evolution of the relative efficiency of the credit unions over time. For the analysis of the results, the ranking of the cooperatives by efficiency level was made for each year. The most efficient cooperatives in the activity of financial intermediation and in banking services were grouped, according to the concepts adopted.

Table 7 shows the main descriptive statistics of Model 1, of efficiency in financial intermediation.

There were expressive results of efficiency, and the descriptive statistics of the results were similar for the period from 2007 to 2014, showing homogeneity in the levels of efficiency in financial intermediation during the period. The average of the efficiency results presented its lowest value, 86.37, in 2007 and its highest value, 91.43, in 2009, showing a difference of approximately 5 basis points. The analysis of the coefficient of variation, less than 15% in all years, also indicates low dispersion of the results, with the data in each year varying slightly around the average.

Figure 1 allows the visualization of the evolution of the number of cooperatives regarding the efficiency in financial intermediation over the period under study.

We found that the results of efficiency in providing banking services were different from those found on Model 1. The average of the results of efficiency varied between 41.01 and 57.17, showing a difference of more than 16 basis points.

The result presented is expressive when compared to other studies that evaluated efficiency using a different set of inputs and outputs in other samples; however, it was an expected result, considering that the financial products with which credit unions operate, especially the financial commodities, with high standardization and low financial spread. This characteristic explains the high efficiency and low differentiation between credit unions in financial intermediation.

Only 20 credit unions were efficient in the intermediation activity in all the period, although a reasonable number of cooperatives are efficient each year, few are consistently efficient throughout time.

The degrees of inefficiency, shown in Table 7, indicate that it is possible to reduce the consumption of inputs without changing the product or increasing the product while maintaining the same quantities of inputs in the proportions presented each year.

Table 8 presents the main descriptive statistics of Model 2, of efficiency in banking services.

We found that the results of efficiency in providing banking services were different from those found on Model 1. The average of the results of efficiency varied between 41.01 and 57.17, showing a difference of more than 16 basis points.

The results of minimum efficiency were very low in all the years analyzed, with 75% of the credit unions showing minimum scores below 38. Furthermore, the coefficient of variation presents values above 40% for the entire period under study, indicating a high dispersion of the results.

Figure 2 indicates that many cooperatives had efficiency scores below 50.

For the credit unions evaluated as efficient, with a score of 100, we observed that for each year analyzed, this number was lower than 32, less than 11% of the sample. No cooperative showed results of efficiency in banking services for the entire period, but 230 cooperatives, 73.01% of the sample, presented inefficiency consistently, throughout the period.

This result was also expected by the fact that the main activity of credit unions in Brazil is in fact financial intermediation, and providing banking services is not their main goal. Based on Ferreira et al. (2007, p. 438), the considerable space for efficiency gains in terms of banking services found is “associated with the underutilization of resources.”

There is no standardization in banking services such as that existing in funding and investment products; the result of inefficiency found alerts to the need for better performance related to the management decisions and administrative structure of these cooperatives regarding banking services.

All cooperatives that have remained efficient in financial intermediation activity in all years have also been efficient in providing banking services in at least two years of the same period, an indication that, apparently, efficient cooperatives in the main activity have also developed in banking services.

However, when comparing the evolution of average efficiency in the models, we can see (Figure 3) that efficiency scores in financial intermediation increase, but they are not necessarily followed by efficiency in banking services.

While the literature in Brazil has been using only one efficiency model (financial intermediation), which does not consider the activity of providing banking services or jointly with financial intermediation (which presents a superior comparative dimension), the results presented emphasize the importance of using different efficiency models for both activities.

5. Conclusions

This study developed a new model for assessing the efficiency of credit unions, from the perspective of the members, from a breakdown of their performance in two different ways: financial intermediation and banking services. Although both activities are typical of financial institutions, they are affected differently by the financial resources available.

We adopted the assumption that, when seeking to generate benefits for the members, the credit union should act with a neutral behavior of dominance, and the efficiency models were built under the same assumption. We also analyzed the temporal evolution for eight consecutive years.

The results indicated what we expected when the proposal to divide the model into two aspects – the two activities presented independent and well-differentiated results. There was low efficiency in banking services, with no cooperative rated as efficient during the analyzed period and 73% of the sample showing inefficiency in this activity throughout the period. Thus, in banking services, credit unions have acted in an unsatisfactory manner, with great scope for improvement in the use of resources – a result already expected, given the greater dependence on decisions and administrative structure.

In financial intermediation, cooperatives have been inefficient over time, despite the low degree of inefficiency each year. Only 20 cooperatives have been identified as a benchmark in this activity consistently over time. Considering each year of the studied period individually, the number of efficient cooperatives was significant.

The research has limitations commented on the work and restrictions on the ability to generalize the results. We recommend for future research to validate the variables used in the efficiency models, which could be done through interviews with managers, who naturally have a more faithful view of what happens in the market and who can aggregate the results of research academic.

Figures

Evolution of the number of cooperatives regarding efficiency in financial intermediation

Figure 1.

Evolution of the number of cooperatives regarding efficiency in financial intermediation

Evolution of the number of cooperatives regarding efficiency in banking services

Figure 2.

Evolution of the number of cooperatives regarding efficiency in banking services

Comparison of the evolution of average efficiency

Figure 3.

Comparison of the evolution of average efficiency

Outputs of efficiency models

Outputs Formula Referenceb Impact on efficiency
Outputs for efficiency in financial intermediation Interest rate on loansa
Income from Credit OperationsCredit Operations
Fried et al. (1993), Wheelock and Wilson (2013) Negative
Interest rate on deposits
Funding ExpensesBank Deposits(short and long term) +Obligations for Special Deposits and Program Funds
Fried et al. (1993), Wheelock and Wilson (2013) Positive
Volume of credit operations with members
Credit OperationsTotal Assets
Vilela et al. (2007), Silva et al. (2013); Ferreira et al. (2007), Bressan et al. (2010); Glass et al. (2010), Wheelock and Wilson (2013) Positive
Volume of deposits from members
DepositsTotal Assets
Fried et al. (1993), Cook and Bala (2007) Positive
Gross income from financial intermediation
Gross Income fromFinancial IntermediationTotal Assets
Ferreira et al. (2007)c Positive
Outputs for efficiency in providing of banking services Service revenue share
Service RevenueTotal Revenue
Bressan et al. (2011), Ferreira et al. (2007)c Positive

Notes: aFor the purposes of applying the DEA model, the variable was transformed (x − 1) to transform the relationship of all variables into a positive effect on efficiency; breference referring to concepts; cthe authors defend the ability to generate income as an essential factor for maintaining the cooperative’s activity, despite not being the main focus of these organizations; however, the proxies for the capacity to generate income were not presented by them

Sample composition

Initial sample of credit unions
Credit unions in the Central Bank’s database with operations throughout the period 1,068
(−) Credit unions with no available data (282)
(=) Initial sample balanced for panel data application 786
(−) Outliers (433)
(=) Sample after eliminating outliers 353
(−) Credit unions with negative values for the variable gross profit from financial intermediation in at least one of the years (5)
(−) Credit unions with zero for the variable share of revenue from services (33)
(=) Final sample (number of credit unions) 315

Descriptive statistics of the final sample

Statistics Current and noncurrent assets Permanent assets Total assets Current and noncurrent liabilities
Mean 36,875,141 1,693,577 142,942,817 30,567,926
Standard deviation 47,403,171 1,998,699 185,421,285 40,892,194
Minimum 1,049,212 11,067 3,555,562 769,226
Maximum 298,428,347 16,286,264 1,308,177,118 270,001,377
Statistics Equity Liabilities + equity Gain (P&L) Losses (P&L)
Mean 7,510,165 142,942,817 8,934,135 (7,895,433)
Standard deviation 8,872,691 185,421,285 11,311,600 10,383,243
Minimum 145,570 3,555,562 81,555 (69,582,106)
Maximum 48,409,368 1,308,177,118 73,825,161 (95,023)

Results of regressions with panel data

Financial intermediation model Banking service model
Significant variables for the composition of the models Interest rate on loans Interest rate on deposits Volume of credit operations with members Volume of deposits from members Gross income from financial intermediation Share of service revenue
Constant −+9.76655e+09 0.161095 0.0697621 0.183369 −0.0121091 0.235379
Treasury applications −0.212269***
Capitalization 0.0329965a*** −0,208663***
Fixed assets 0.0178827a*** −0,056198*** 0,0030852a*** 0,042498a***
Floating funding −0.10889*** 0,072994a***
Operational efficiency −0.0228684*** 0.0261345a*** −0,027768*** 0,0170609a*** −0,029902***
Profit sharing in other operating income −0.110455a*** −0.0543849*** −0.069574*** −0,017472*** −0,223693***
Operating margin 0.100818*** 0.0726487a*** 0,0490426a*** 0,147049a***
Financial default 0.0351485*** −0.023958***
Insolvency 0.508263*** −0.189153*** 0,0270948a***
Funding by long term deposits −0.070064a*** −0.0241461*** 0.046002a*** 0,065801a***
Liabilities mismatch −0.351679a*** −0.0928804*** 0.865781a***
Administrative expenses by total assets 2.40137*** 0.218882a*** 2.88737a*** 3,81923a*** 0,944032a*** 0,803892a***
Return on assets 1.39255a*** 1,73647a*** 0,193536a*** −0,466562***
Service revenue share −0.21982a*** −0.063166*** −0.183041*** −0,073304***
Interest rates on deposits −0,15407 7***
I***nterest rates on loans −0,089664***
R-square 0.888349 0.600088 0.947134 0.934198 0.963534 0.859767
P-value (F) 0.000000 5.5e-298 0.000000 0.000000 0.000000 0.000000

Notes:aVariables that affected the output with the expected signal; *; **; *** statistical significance at the levels of 1%, 5% and 10%, respectively

Final models composition

Input variables Output variables
Model 1: financial intermediation Model 2: banking services
1 2 3 4 5 6
Capitalization X
Insolvency X
Liabilities mismatch X X
Administrative expenses by total assets X X X X X
Fixed assets X X X
Floating funding X

Notes: 1 – interest rate on loans; 2 – interest rate on deposits; 3 – volume of credit operations with members; 4 – volume of deposits from members; 5 – gross income from financial intermediation; 6 – share of service revenue; X – indicates that the variable is input for a given output

Input calculation formulas

Inputs for the efficiency financial intermediation model Capitalization
EquityCurrent and Non-Current Liabilities
Fixed assets
Fixed AssetsEquity
Insolvency
Operations with Risk E, F, G, HCredit Classification Portfolio
Liabilities mismatch
Interbank Deposits + Obligations for Committed Operations+ Interfinancial Relationships + Obligations for Loansand lendings + Derivative Financial InstrumentsTotal Assets
Administrative expenses by total assets
Administrative ExpensesTotal Assets
Inputs for the efficiency banking services model Floating funding
Demand deposits + Collection and Collection of TaxesCurrent and Non-Current Liabilities
Fixed assets
Fixed AssetsEquity
Administrative expenses by total assets
Administrative ExpensesTotal Assets

Descriptive statistics of efficiency results – financial intermediation

Statistics 2007 2008 2009 2010 2011 2012 2013 2014
Mean 86.37 91.43 87.98 87.03 90.02 88.33 87.75 89.02
Coefficient of variation (%) 13.6 10.4 12.8 13.1 11.1 12.0 12.7 10.7
Minimum 52.60 65.45 53.95 59.08 64.15 60.96 57.47 57.25
1st quartile 78.35 84.30 79.00 77.68 82.36 79.51 79.28 81.70
Median 85.69 94.07 89.67 87.26 92.12 89.59 87.67 89.26
3rd quartile 100 100 100 100 100 100 100 99.07
Maximum 100 100 100 100 100 100 100 100
Degree of inefficiency* 15.8 9.4 13.7 14.9 11.1 13.2 14.0 12.3
Note:

*Calculated based on average efficiency

Descriptive statistics of efficiency results – banking services

Statistics 2007 2008 2009 2010 2011 2012 2013 2014
Mean 41.01 52.23 53.55 54.87 52.16 57.17 55.26 52.73
Coefficient of variation (%) 58.8 49.4 48.9 48 47.4 47.3 44.1 41.7
Minimum 0.03 3.21 3.82 3.13 1.66 0.28 2.97 6.05
1st quartile 22.47 31.27 33.68 34.37 31.23 34.99 37.78 36.32
Median 37.56 52.56 52.99 55.27 54.15 59.75 56.33 53.89
3rd quartile 51.78 69.12 72.69 71.73 65.95 76.84 68.80 65.17
Maximum 100 100 100 100 100 100 100 100
Degree of inefficiency* 143.8 91.5 86.7 82.2 91.7 74.9 81.0 89.7
Note:

* Calculated based on average efficiency

Note

1.

Refer to the ratio between level AA and A in risk operations and the credit portfolio classification.

References

Abreu, E. S., Kimura, H., Araújo Neto, L. M., & Peng, Y. (2018). Efficiency of the Brazilian credit unions: A joint evaluation of economic and social goals. Latin American Business Review, 19, 107129. 10.1080/10978526.2018.1479640.

Banco Central do Brasil. (2015). Composição: Composição e evolução do SFN. Brasília: BACEN. Disponível em: Retrieved from www.bcb.gov.br/?SFNCOMP (acessed 09 March 2015).

Barroso, M. F. G., & Bialoskorski Neto, S. (2010). Distribuição de resultados em cooperativas de crédito rural no estado de São Paulo. Organizações Rurais e Agroindustriais, 12, 290307.

Barton, D. (1989). What is a cooperative?. In D. W. Cobia, Cooperatives in agribusiness. (pp. 120) Nova Jersey: Regents/Prentice Hall.

Bauer, K. (2008). Detecting abnormal credit union performance. Journal of Banking & Finance, 32, 573586. 10.1016/j.jbankfin.2007.04.022.

Berger, A. N., & Humphrey, D. B. (1997). Efficiency of financial institutions: International survey and directions for future research. European Journal of Operational Research, 98, 175212. 10.1016/S0377-2217(96)00342-6.

Bittencourt, W. R., Bressan, V. G. F., Goulart, C. P., Bressan, A. A., Costa, D. R. M., & Lamounier, W. M. (2017). Rentabilidade em bancos múltiplos e cooperativas de crédito brasileiros. Revista de Administração Contemporânea, 21, 2240. 10.1590/1982-7849rac2017150349.

Brasil, Lei n° 5.764, de 16 de dezembro de. (1971). Define a política nacional de cooperativismo, institui o regime jurídico das sociedades cooperativas, e dá outras providências. Diário Oficial da União, Brasília, DF. Recuperado de: Retrieved from www.planalto.gov.br/ccivil_03/Leis/L5764.htm

Bressan, V. G. F., Braga, M. J., & Bressan, A. A. (2010). Eficiência e economia de escala em cooperativas de crédito: uma abordagem de fronteira estocástica de custo com dados em painel. Advances in Specific and Applied Accounting, 3, 335352.

Bressan, V. G. F., Braga, M. J., Bressan, A. A., & Resende Filho, M. D A. (2011). Avaliação de insolvência em cooperativas de crédito: Uma aplicação do sistema pearls. Ram. Revista de Administração Mackenzie, 12, 113144. 10.1590/S1678-69712011000200006.

Bressan, V. G. F., Bressan, A. A., Oliveira, P. H. M., & Braga, M. J. (2014). Quais indicadores contábeis financeiros do sistema PERLS são relevantes para a análise de insolvência das cooperativas centrais de crédito no Brasil?. Revista Contabilidade Vista & Revista, 25, 7498.

Carvalho, F. L., Diaz, M. D. M., Bialoskorski Neto, S., & Kalatzis, A. E. G. (2015). Saída e insucesso das cooperativas de crédito no Brasil: Uma análise do risco. Revista Contabilidade & Finanças USP, 26, 7084. 10.1590/1808-057x201411390.

Coelli, T. J., Rao, D. S. P., O’Donnell, C. J., & Battese, G. E. (2005). An introduction to efficiency and productivity analysis. (2nd ed.) New York, NY: Springer.

Cook, W. D., & Bala, K. (2007). Performance measurement and classification data in DEA: Input-oriented model. Omega, 35, 3952. 10.1016/j.omega.2005.02.002.

Cook, W. D., & Seiford, L. M. (2009). Data envelopment analysis (DEA) – thirty years on. European Journal of Operational Research, 192, 117. 10.1016/j.ejor.2008.01.032.

Emmons, W. R., & Schmid, F. A. (2002). Pricing and dividend policies in open credit cooperatives. Journal of Institutional and Theoretical Economics, 158, 234255. 10.1628/0932456022975439.

Ferreira, M. A. M. (2005). Eficiência técnica e de escala de cooperativas e sociedades de Capital na indústria de laticínios no Brasil (tese de doutorado). Universidade Federal de Viçosa, MG. Retirado de: www.dominiopublico.gov.br/download/texto/cp088211.pdf

Ferreira, M. A. M., Gonçalves, R. M. L., & Braga, M. J. (2007). Investigação do desempenho das cooperativas de crédito de Minas Gerais por meio da análise envoltória de dados (DEA). Economia Aplicada, 11, 425445. 10.1590/S1413-80502007000300006.

Frame, W. S., Karels, G. V., & Mcclatchey, C. A. (2003). Do credit unions use their tax advantage to benefit members? Evidence from a cost function. Review of Financial Economics, 12, 3547. 10.1016/S1058-3300(03)00005-3.

Freaza, F. P., Guedes, L. E. M., & Gomes, L. F. A. M. (2006). Análise de Eficiência do Mercado Bancário Brasileiro Utilizando a Metodologia da Análise Envoltória de Dados. XXXVIII Simpósio Brasileiro de Pesquisa Operacional. Goiânia, Goiás.

Fried, H. O., Lovell, C. A. K., & Eeckaut, P. V. (1993). Evaluating the performance of U.S. credit unions. Journal of Banking & Finance, 17, 251265. 10.1016/0378-4266(93)90031-8.

Glass, J. C., Mckillop, D. G., & Rasaratnam, S. (2010). Irish credit unions: Investigating performance determinants and the opportunity cost of regulatory compliance. Journal of Banking & Finance, 34, 6776. 10.1016/j.jbankfin.2009.07.001.

Gonçalves, R. M. L., Braga, M. J. & Ferreira, M. A. M. (2012). Restrições financeiras em cooperativas agropecuárias brasileiras. Economia, 13, 647670.

Holod, D., & Lewis, H. F. (2011). Resolving the deposit dilemma: A new DEA bank efficiency model. Journal of Banking & Finance, 35, 28012810. 10.1016/j.jbankfin.2011.03.007.

Kassai, S. (2002). Utilização da análise por envoltória de dados (DEA) na análise de demonstrações contábeis (tese de doutorado, São Paulo: Faculdade de Economia, Administração e Contabilidade, Universidade de São Paulo.

Lee, Y. J., Joo, S.-J., & Park, H. G. (2017). An application of data envelopment analysis for Korean banks with negative data. Benchmarking: An International Journal, 24, 10521064. 10.1108/BIJ-02-2016-0023.

Lovell, C. A. K., & Pastor, J. T. (1995). Units invariant and translation invariant DEA models. Operations Research Letters, 18, 147151. 10.1016/0167-6377(95)00044-5.

Mckillop, D. G., Glass, J. C., & Fergunson, C. (2002). Investigating the cost performance of UK credit unions using radial and non-radial efficiency measures. Journal of Banking & Finance, 26, 15631591. 10.1016/S0378-4266(01)00164-9.

McKillop, D., & Wilson, J. O. S. (2011). Credit unions: A theoretical and empirical overview. Financial Markets, Institutions & Instruments, 20, 79123. 10.1111/j.1468-0416.2011.00166.x.

Matias, A. B., Quaglio, G. M., Lima, J. P. R., & Magnani, V. M. (2014). Bancos versus cooperativas de crédito: um estudo dos índices de eficiência e receitas da prestação de serviços entre 2002 e 2012. Ram. Revista de Administração Mackenzie, 15, 195223. 10.1590/1678-69712014/administracao.v15n5p195-223.

Milgrom, P., & Roberts, J. (1992). Economixcs, organization and management. Economic organization and efficiency. (pp. 1952) Englewood Cliffs, NJ: Prentice Hall. 07632, Chapter 2.

Pastor, J. T. (1996). Translation invariance in DEA: A generalization. Annals of Operations Research, 66, 93102. 10.1007/BF02187295.

Portela, M. C. A. S., Thanassoulis, E., & Simpson, G. (2004). Negative data in DEA: A directional distance approach applied to bank branches. Journal of the Operational Research Society, 55, 11111121. 10.1057/palgrave.jors.2601768.

Porter, P. K., & Scully, G. W. (1987). Economic efficiency in cooperatives. The Journal of Law and Economics, 30, 489512. 10.1086/467146.

Quiroga, M. P. S., Bressan, V. G. F., & Braga, M. J. (2005). Avaliação financeira das cooperativas de crédito mútuo do estado de Minas Gerais. Revista de Ciências Empresariais da Unipar, 6, 119.

Rubin, G. M., Overstreet, G. A., Jr, Beling, P., & Rajaratnam, K. (2013). A dynamic theory of the credit union. Annals of Operations Research, 205, 2953, 10.1007/s10479-012-1246-7.

Silva, T. P., Gollo, V., & Rodrigues, M. M. Jr (2013). Análise da eficiência na atividade de concessão de crédito em cooperativas brasileiras. ENANGRAD. FIN – Finanças. Florianópolis, Santa Catarina, Brasil, Vol. 24.

Silva, T. P., Leite, M., Guse, J. C., & Gollo, V. (2017). Financial and economic performance of major brazilian credit cooperatives. Contaduría y Administración, 62, 14421459. 10.1016/j.cya.2017.05.006.

Soares Mello, J. C. C. B., Ângulo Meza, L., Gomes, E. G., & Biondi Neto, L. (2005). Curso de análise envoltória de dados. Anais do simpósio brasileiro de pesquisa operacional (SBPO). (Vol. 37, pp. 25202547) Rio Grande do Sul, Brasil: Gramado. Disponível em: www.uff.br/decisao/sbpo2005_curso.pdf.

Taylor, R. A. (1971). The credit union as a cooperative institution. Review of Social Economy, 29, 208217. 10.1080/00346767100000033.

Vilela, D. L., Nagano, M. S., & Merlo, E. M. (2007). Aplicação da análise envoltória de dados em cooperativas de crédito rural. Revista de Administração Contemporânea, 11, 99120. 10.1590/S1415-65552007000600006.

Wheelock, D. C., & Wilson, P. W. (2013). The evolution of cost-productivity and efficiency among US credit unions. Journal of Banking & Finance, 37, 7588. 10.1016/j.jbankfin.2012.08.003.

Wooldridge, J. M. (2008). Introdução à econometria: Uma abordagem moderna, São Paulo: Cengage Learning.

Yamori, N., Harimaya, K., & Tomimura, K. (2017). The efficiency of Japanese financial cooperatives: An application of parametric distance functions. Journal of Economics and Business, 94, 4353. 10.1016/j.jeconbus.2017.09.001.

Acknowledgements

The authors contributed in the following ways to this paper: Barros, Manuela Gonçalves, Corresponding Author: conceptualization (equal), data curation (lead), formal analysis (equal), investigation (lead), methodology (equal), resources (equal), validation (equal) and writing-original draft (lead); Moraes, Marcelo Botelho da Costa: conceptualization (lead), data curation (equal), formal analysis (equal), investigation (equal), methodology (lead), resources (equal), validation (equal), writing-original draft (supporting) and writing-review and editing (equal); Salgado Junior, Alexandre Pereira: formal analysis (supporting), resources (equal), software (equal) and validation (equal); Souza Junior, Marco Antonio Alves de: formal analysis (supporting), methodology (supporting) and software (equal).

This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) – Finance Code 001.

Corresponding author

Manuela Gonçalves Barros can be contacted at: manuelagb@hotmail.com Associate Editor: Wesley Mendes-Da-Silva

Related articles