Event performance index: a holistic valuation tool

Monika Bandi Tanner (Center for Regional Economic Development, University of Bern, Bern, Switzerland)
Adrian Künzi (Center for Regional Economic Development, University of Bern, Bern, Switzerland)
Therese Lehmann Friedli (Center for Regional Economic Development, University of Bern, Bern, Switzerland)
Hansruedi Müller (Center for Regional Economic Development, University of Bern, Bern, Switzerland)

International Journal of Event and Festival Management

ISSN: 1785-2954

Publication date: 4 June 2018

Abstract

Purpose

The subsidization of events by public authorities at different administrative levels has become increasingly important in recent years. Event portfolios are an important supply component of tourism destinations. The development of a valuation tool with an event performance index (EPI) as the key output should enable public authorities to develop transparent, systematic and fair subsidization practices in the future. The paper aims discuss these issues.

Design/methodology/approach

Using a theoretical impact model and event evaluation practices, this work develops a new holistic valuation tool for events with key indicators along the dimensions of sustainable development. Basic cost-benefit analysis ideas enrich the approach conceptually. Indicator development was based on a process of elaboration that considered the scientific literature, event stakeholders and municipal representatives.

Findings

The EPI consists of seven core indicators: size, economic value, touristic value and image, innovative strength, value of networking, value of participation and social exchange and relative ecological burden. The application of this tool to a case study revealed that it generates comprehensive and robust indicators of multifaceted and destination-unspecific event values and supports the process of allocating event subsidies using different remuneration schemes. Straightforward and destination-unspecific indicators assure the transferability and adaptability of the valuation tool to different complex and multifaceted contexts of event subsidization.

Originality/value

The EPI seeks to reduce complexity and incentivize event organizers to meet future sustainable development goals. Additionally, this work contributes to future discussions of both the form and process of event subsidization.

Keywords

Citation

Bandi Tanner, M., Künzi, A., Lehmann Friedli, T. and Müller, H. (2018), "Event performance index: a holistic valuation tool", International Journal of Event and Festival Management, Vol. 9 No. 2, pp. 166-182. https://doi.org/10.1108/IJEFM-09-2017-0047

Download as .RIS

Publisher

:

Emerald Publishing Limited

Copyright © 2018, Emerald Publishing Limited


1. Introduction

The allocation of public resources to events represents a major challenge for governments and institutions, as a holistic assessment of their costs and benefits is subject to a wide range of uncertainties (Mules and Dwyer, 2005). Nevertheless, event policy has become important to both municipalities and destinations. The strategic use of events ideally furthers the goal of welfare enhancement by inducing persistent economic and social benefits to locations and their societies (Pugh and Wood, 2004). In the past, the scientific discourse regarding event impacts was dominated by assessing their economic benefits, which were measured in monetary terms (Fredline et al., 2005). Social, cultural and environmental dimensions were frequently ignored, leading to distortions in public perceptions of event impacts (Collins and Cooper, 2017). However, various stakeholder perspectives from the private and public sectors must be considered by policy-makers to evaluate event outcomes and sustainability (Timur and Getz, 2009), and clearly, the social, economic, political and environmental structure of a region adds complexity (Whitford et al., 2014).

The evaluation of event impacts has become a popular field of research for tourism destinations. The scientific literature focusing on a single dimension of event impacts has been criticized by more recent research, which stresses the importance of assessing the multitude of impacts of holding events (Brown et al., 2015). Consequently, a comprehensive evaluation of events is guided by the principles of sustainable development and considers the local economy, environment and local community (see Thomas and Wood, 2003; Dwyer, 2005; Wood, 2005; Collins et al., 2009). These complex and interconnected impacts call for the development of a holistic framework to assess event performance. Various studies have discussed the development and implementation of an impact evaluation framework. An examination of previous research indicates that frameworks consider the social (see Fredline et al., 2003; Small et al., 2005), ecological (see Collins and Cooper, 2017) and economic (see Dwyer et al., 2000; Tyrrell and Johnston, 2001; Whitford, 2004; Wood, 2008) dimensions of event impacts.

The development and empirical application of holistic event evaluation frameworks remain limited. Brown et al. (2015) report that only a few studies have been conducted to determine the worth of events. By including factors of sustainable development, the extension of evaluation systems calls for new approaches. Sadd et al. (2017) emphasize the importance of an instrument for event evaluation that is collectively proportioned in terms of perspectives and knowledge. Effective and long-term interaction and commitment at the destination level require integrative and collective strategic decisions. Additionally, there is no consensus about the choice and measurement of variables to operationalize event impacts, primarily because of the wide range of partially divergent effects and stakeholder interests at stake (Torres-Delgado and Palomeque, 2014). Regarding the practitioner level, tourism stakeholders encounter major challenges to developing systematic, transparent and fair subsidization policy. Moreover, as Davies et al. (2013) elaborate, event evaluation by practitioners involves both time and monetary constraints. Therefore, the development of indicators that are summarized in an index of event performance would be valuable (Torres-Delgado and Palomeque, 2014).

The benefits of events include characteristics of public goods that justify public support and investment. Therefore, it is important for public authorities and institutions to clearly explain to the population the substantial benefits and advantages of events (Getz, 2009). An important reference point when considering public good issues is social equity. Examples of the potential social benefits of various types of events include cultural education, social integration, leisure/recreation and health. Permanent support for an event can be achieved only when the event is consistent with the social, cultural, economic and environmental values of the community (Getz, 2009).

Several events on a national and international scale have been comprehensively assessed in the national literature of tourism science (see Müller et al., 1997; Bieger et al., 2003; Müller et al., 2010; Bandi Tanner et al., 2017). Consequently, the development of a handbook of event evaluation would transform the findings of the present study into a practice-oriented tool for event evaluation.

The purpose of this study is to develop a valuation tool that permits the evaluation of the performance of small- and medium-sized events (see Hall, 1989) based on principles of sustainable development by relying conceptually on the cost-benefit approach. A previous comprehensive analysis of the scientific literature and existing tools has shown that for a variety of municipalities and events, many tools are too complicated to implement. Therefore, the authors combine indicators derived from the scientific literature and weighting schemes to develop a practical event evaluation benchmark. A published, step-by-step manual and educational documents support this practical applicability. The establishment of the proposed framework is sufficiently flexible to allow for user- or destination-specific adaptations to all dimensions. In addition, the development of a remuneration scheme connected to the event performance index (EPI) should enable public authorities to determine the level of monetary subsidies allocated to different events in a more systematic manner. Furthermore, this study contributes to discussions of the appropriate form and process of event support to respond to future challenges that may be experienced by the tourism industry.

This paper proceeds by reviewing the existing literature on subsidization policies and cost-benefit analysis (CBA) in the context of events. Section 3 describes the methodological approach of this study. In Section 4, the EPI indicators are explained, and the process of monetary remuneration in an event portfolio is characterized. The results of the case study are then described and critically discussed. A sensitivity analysis complements this section. The last section concludes the paper.

2. Literature review

2.1 CBA as a decision-making framework for the public support of events

Notwithstanding the intricacy of event impacts and interactions in tourism destinations, the use of indicators helps in the evaluation of events in the context of sustainable development. A system of indicators that fulfills methodological and scientific conditions and has practical applicability reduces complexity, particularly at the municipal level (Torres-Delgado and Palomeque, 2014). In accordance with Wood (2008) and Whitford et al. (2014), event evaluation proceeds in specific and unique contexts, and indicators should be tested and iteratively examined. As Fernández and Rivero (2009) state, the sustainable development approach to tourism is characterized by both its multifaceted dimensions and its relative meaning. Event funding requires both measurement and systematic evaluation of the event’s objectives. In addition, it is necessary to regularly reassess events to adapt the joint focus of the subsidized events and the funding institutions (Pugh and Wood, 2004). Economic considerations related to this issue require the public authority to engage in entrepreneurial behavior (Whitford et al., 2014).

For tourism destinations, the assessment of events calls for a comprehensive framework that includes costs and benefits because the exclusive valuation of economic impacts is insufficient (Kesenne, 2005). Since various effects influence the decision-making process at different administrative levels, the application of a CBA could serve as a foundation for decision-making and justification of public support (Mules and Dwyer, 2005; Dwyer and Forsyth, 2009).

According to Kesenne (2005), the benefits of events consist of the expenditures and consumer surplus generated, whereas the costs of events include both financial and opportunity costs. When assessing the net benefit of an event, a multitude of cost-benefit factors should be reasonably approximated and quantified. A distinction in the quantification of these cost-and-benefit components is related to measurability and intangibility. The quantitative evaluation of measurable elements and the qualitative evaluation of intangible elements, e.g., social benefits or costs should complement each other and be incorporated with appropriate weights (Mules and Dwyer, 2005). Investments in events must be evaluated in terms of both the real costs imposed on society and the potential benefits that could be obtained through alternative allocations of public support. Social opportunity costs, i.e., outcomes that could be generated from an alternative investment, must be considered in the cost assessment. The cost-benefit framework enables public authorities both to implement social decision-making and reduce uncertainty in determining the appropriate amount of support for event performance (Vanhove, 2011).

2.2 Theoretical impact model

Below, we present the scientific literature that underlies the developed impact model; this literature was subsequently used to derive the important dimensions of the evaluation tool.

Impact models serve as instruments for incidence analysis. In this case, more detailed information about different incidences can be captured if an incidence is a monetary or non-monetary impact (Scherer, 2002). There are different incidences of an event and its stakeholders, i.e., private households, businesses and the municipality or city where the event occurs (see Figure 1). The impacts of incidences can be grouped according to the dimension of sustainable development, namely, economic, social or ecological impacts. Examples of relations between the event and the stakeholders and relations among various stakeholders are shown.

The primary focus of the model is the event. Thus, the size of the event plays a crucial role. Although bigger events have greater impacts via visitor frequency, higher numbers of visitors can have both positive and negative effects. The economic impacts are positive: more participants visit the event, and thus, the destination will receive higher returns. However, from an ecological and social perspective, smaller events tend to perform better. First, this tendency occurs because existing resources (in terms of infrastructure) are often sufficient to organize small events. Second, Hall (1989) states that the involvement of the local population decreases with the size of an event. Events are often used as strategic instruments to prolong a tourist season (Connell et al., 2015). Scholars have found that events that are held during the low season tend to have stronger-than-average positive impacts on overnight stays because crowding-out effects are weaker than during busy seasons (Fourie and Santana-Gallego, 2011; Andersson and Lundberg, 2013). Although additional participants visit the event and generate income for the destination, ordinary tourists could be crowded-out because of limited capacities. Consequently, event valuation needs to consider the “net benefit” that results from additional visitors and displaced tourists (Preuss, 2007, 2011). The attractiveness and image of a tourism destination can be positively affected by improving existing events or creating new events (Getz, 2008).

From an economic perspective, the regional economy of the destination holding the event benefits the most if there are cooperation contracts among the event organization committee, local suppliers and the public. It has been demonstrated that cooperation is more intense if the event is locally rooted (Walo et al., 1996).

The involvement of local society and the public is also important from a social perspective because events can be used to represent a society’s value system; this can strengthen traditions (Gursoy and Kendall, 2006). Furthermore, events that are consistent with local needs and that encourage visitors and locals to exchange experiences and attitudes are more socially valuable than other types of events (Fredline et al., 2003). This value can be achieved through public participation in the planning and execution of the event. Participation can lead to a higher sense of community and social capital (Walo et al., 1996). In terms of social impact, image is another important component. Richards and Wilson (2004) state that events influence the image of a destination when they are consistent with the strategic positioning of the destination.

Finally, the ecological dimension of events is crucial. The ecological burden of events is primarily caused by traffic, noise, waste and negative externalities affecting the landscape. Locals consider event traffic to be one of the most important ecological burdens imposed by an event (Robbins et al., 2007). Gössling et al. (2002) support these findings by measuring ecological footprints that are substantially influenced by tourists’ journeys to or from the destination. Furthermore, ecological measures should be reflected on a global scale (Hunter and Shaw, 2007).

3. Methodological approach

The methodological approach to developing the valuation tool included several steps of theoretical analysis and empirical testing. The theoretical impact model helped determine the relevant dimensions of economic, social and environmental event impacts. The indicators were derived in accordance with the principles of sustainable development. CBA informed the general design of the valuation tool and helped provide guidelines for its practical implementation.

The development of the evaluation tool started with an analysis of the characteristics of an existing event portfolio and subsidizing practices in a municipality. Despite this inductive approach, it can be assumed that many public authorities have limited budgets with which to support events, which they would like to distribute across individual events based on clear indicators. This inductive process provided the empirical foundation for the theoretical impact model. The initial derivation of indicators was based on the scientific literature. In a second phase of development, event organizers and public administration representatives were consulted via workshops to elaborate on and operationalize the indicators. Through an iterative process of dimension reduction, a feasible tool was created through a comprehensive analysis, like a CBA, supported by science and pragmatic evaluation of an event portfolio in practice allowing the identification of seven indicators.

Complexity reduction was crucial to guaranteeing a basis for discussion of the various stakeholders. To summarize, deductive derivation of the indicators was based on the scientific literature, whereas the operationalization of the indicators was derived inductively.

The case study was conducted in the Swiss municipality of Saanen, which lies in the alpine tourism destination of Gstaad-Saanenland. This municipality is considered relevant for the application and testing of the valuation tool owing to its multifaceted event portfolio and the economic importance of the tourism sector. The event portfolio contains six major events, including an international tennis tournament, the Safra Sarasin Swiss Open; the World Tour Beach Volleyball tournament; and the renowned Menuhin Festival, a classical music event. In addition to these major events, a variety of medium-sized and minor events are publicly subsidized. In the period from 2012 to 2015, the municipality spent approximately 5.7 million Swiss francs (4.6 mn British pounds) on event subsidization. This amount was split across major events (70 percent of the total expenses) and intermediate and minor events (30 percent of the total expenses).

The final step included the development of remuneration schemes to translate the evaluation outcome into monetary subsidization amounts. The robustness and instrument objectivity of the qualitative indicators were assessed through a sensitivity analysis. An application process for event organizers and a remuneration process for supporting authorities are proposed, as structured proceedings will help foster the credibility and acceptance of the EPI.

4. Design of the proposed valuation tool

4.1 Detailed descriptions of the indicators

The economic, ecological and social assessment of the indicators results in the creation of the EPI as the key outcome of the valuation process. This study illustrates the application of the EPI to a tourism destination. The scientifically substantiated key indicators, however, can be universally applied to the different event portfolios of municipalities, destinations or institutions. A detailed description of the indicators is presented in Table I.

The size and economic value of the event are measured quantitatively. The five remaining indicators are evaluated using qualitative Likert scales. The size of the event (SIZ) is operationalized using daily frequencies generated by all event participants. This measure is calculated by summing the length of stay (in days) of all participants (spectators, active participants, staff, volunteers, guests invited by sponsors and media professionals). The ordinal measurement scale for the size indicator was derived from a second-degree polynomial function (see Figure 2). The specific polynomial function was determined by referencing the size of the events hosted by the case study city. This determination is advantageous because the scale adequately accommodates the smaller and larger events in an event portfolio and leads to sufficient differentiation in event size. If an event portfolio contains large events, the size scale can be extended past the polynomial function.

The economic value (VAL) of the event is approximated by the average expenditures (tickets, catering, souvenirs, overnight stays and other expenditures) per participant as an indirect measure of the generated revenue. To estimate this key figure, the groups’ different spending behaviors (e.g. daytime visitors vs overnight visitors) and their relative importance to the mix of participants must be considered. In our case study, which is embedded in the Swiss tax system, municipal tax revenue plays a marginal role. Point values are represented by monetary values and are assessed on a scale from 1 to 5.

Five detailed dimensions assess the touristic value and image of an event (TOU). The proportion of tourist and non-local participants in the event, which is measured as the proportion of participants from outside a defined region, is integrated. Touristic attractiveness is operationalized by the proportion of participants from different places. The more participants are from farther away, the higher the touristic attractiveness of the event. The intensity and area of media coverage reflects the contribution of the event to the awareness of the destination. Potential positive effects on the image, the fit between the event and the targeted image of the destination and other aspects, such as the vision, mission and values of the destination, are assessed via an additional dimension. Furthermore, the flexibility of the date of execution is integrated to evaluate the event in terms of meaningful additions to the annual touristic planning of the destination, contributions to optimal bed occupancy and reduction of crowding-out effects.

The innovative strength of the event (INN) can be reflected by different levels of innovation. Differentiation is conducted among existing events with new features, existing events with noticeable innovations for the participants and the execution of a completely new event. New features can include an optimized event program, process or technical innovations or a clearer thematic focus and positioning. Noticeable innovations are understood as product innovations or the focused targeting of new groups of participants. This indicator is most positively assessed if the event is planned or executed for the first time and has a new and innovative concept compared to other destinations.

The value of networking (NET) is reflected in local cooperation and exchange. The proportion of locals on the executive board, the proportion of inputs purchased locally and the intensity and quality of cooperation with the local DMO indicate local cooperation. The presence of prominent persons from different areas and the existence of platforms and opportunities to establish valuable contacts reflect the extent of networking possibilities and hospitality at the event.

The social dimension of the event (SOC) is measured by the value of participation and social exchange. An event should be consistent with local needs and expectations to strengthen regional values and obtain broad local support. In addition, opportunities for locals to become involved in the event increase the value of this indicator. Volunteering and opportunities to be heard when organizing the event are detailed measures. Social exchange is enhanced if the event targets a broad group of participants in terms of age or social groups and develops social capital.

Four dimensions measure the relative ecological burden: waste, noise, on-site and journey traffic and town scenery and landscape. Positively assessed features are waste separation, formalized waste concepts and predominant use of reusable materials. The treatment of noise is negatively evaluated according to the following dimensions: evening programs occurring after 10 p.m., open-air events and the use of loudspeakers. On-site and journey traffic are assessed in terms of the proportion of daytime visitors, the participants’ choice of transport, the event being within walking distance from public transport and the presence of organized shuttle buses and parking guidance systems. The impairment of the town’s scenery and landscape are assessed through the exclusive use of existing infrastructure, the selective, temporal extension of the existing infrastructure and the fit of the constructed infrastructure with the scenery and landscape.

4.2 Aggregation of the EPI

To derive the EPI, the abovementioned indicators are summed using a formula. The objective of this formula is to obtain an EPI value that reflects the performance of each event in a portfolio that indicates the overall value of the event. The basic design of the formula, in accordance with a cost-benefit approach, adds positive indicators and subtracts negative ones. The indicators for economic value (VAL), touristic value (TOU) and relative ecological burden (ECO) are weighted by the size of the event (SIZ) because their positive or negative impacts are size dependent (Richards and Wilson, 2004; Robbins et al., 2007). The economic indicator (VAL) is assigned the highest weight because it is fully multiplied by the size of the event (SIZ). The assessment of touristic value (TOU) is weighted by one-half of the size indicator, and the relative ecological burden (ECO) is weighted by one-fourth of the size indicator. In addition to the estimation of their relevance to the case study, the degree of uncertainty about the assessment of the qualitative measures was considered. The points assigned for innovative strength (INN) as an engine of tourism development are doubled. Nevertheless, innovation must be considered a soft indicator because its definition is very vague and exhibits considerable assessment uncertainty. The indicators for the value of networking (NET) and participation and social exchange (SOC) enter the formula with weights of one. In sum, the EPI formula is as follows:

(1) EPI = ( SIZ × VAL ) + ( SIZ 2 × TOU ) + 2 × INN + NET + SOC ( SIZ 4 × ECO )

These indicators in the basic formula (see Equation (1)) must be considered to comprehensively evaluate events according to the dimensions of sustainable development. These key indicators are important for future event evaluation and the justification of event value.

Different weighting schemes for the indicators were tested and repetitively discussed. The choice of the key indicators was based on principles of sustainable development and therefore generally applicable. The weighting of the different EPI components, however, is subject to the normative decisions of the users and their strategic intentions.

4.2.1 Start-up financing

The basic EPI value can be adjusted upwards using an additional factor that considers the supplementary costs of organizing and holding new events. After the determination of the basic EPI value, it is multiplied by positive factor for start-up financing. During the process of assessing new events, the need for a financial push can be realized and implemented through this additional indicator (Table II).

4.3 Remuneration schemes

Two remuneration schemes that translate the basic EPI value into the amount of financial support are proposed. The main decision parameter is the EPI remuneration base, which specifies the monetary amount with which one EPI point is compensated financially. The case study showed that a linear remuneration scheme may not lead to fair and plausible outcomes due to the considerable size differences of events. With a size-dependent remuneration scheme, local authorities can establish a single remuneration base for all events, regardless of their size and thus the magnitude of the event impacts. Therefore, the basic EPI value is multiplied by specific size factors. The resulting EPI score is the decisive factor for monetary remuneration. To identify events with lower basic EPI values and remunerate them appropriately, a finer scale of categorization for the size factor is chosen for the lower half of the scale. The size categories are divided into linear steps of 12.5 EPI points (see Table III).

As a mathematical example, a hypothetical event with a basic EPI value of 52 points lies in the ninth category and is accordingly multiplied by a size factor of 5. This event has an EPI score of 260, which is then multiplied by the hypothetical EPI remuneration base, which is set to 350.00 CHF in this example, which leads to municipal financial support of approximately CHF91,000.00.

In alternative remuneration schemes, different bases can be applied for different event sizes. In our case study, the municipality representatives confirmed this scheme during the political process. This practitioner-oriented method enables local authorities to allocate financial support across events of different size categories. However, in terms of the overall portfolio of events, the fairness of the distribution must be questioned. This approach will not be further discussed in this paper.

5. Results

5.1 Application to Gstaad-Saanenland

Applying the EPI to the Gstaad-Saanenland case provided important insights into its practical implementation. Local authorities wanted to design a valuation framework for events of all sizes and types. Therefore, six major events and eight intermediate events were evaluated based on seven key indicators. Getz (2008) claims that policies or direct interventions consist of appropriate administrative expenses and should be accepted by stakeholders. Therefore, only events with a budget of more than CHF20,000.00 were evaluated using the EPI.

5.1.1 Indicator-specific results of the EPI valuation

When examining the valuation outcomes of major events, the considerable variation in terms of the size indicator is interesting. The size assessments range from 6 to 19 points. Based on high intra-event variation and multiplication by the economic value indicators, touristic value and ecological burden, the range of resulting EPI values is very broad. In terms of qualitative indicators, the value of participation and social exchange, which is measured on a five-point Likert-scale, varies the most (Figure 3).

The main differences in the results for major and intermediate events lie in variations in size and economic value. As expected, the size-independent indicators do not differ substantially between intermediate and major events (Figure 4).

5.1.2 Overall results based on the EPI valuation

The results in terms of the basic EPI values provide evidence that the valuation tool can create appropriate variation in event portfolio terms. The basic EPI values for the sample of intermediate events lie between 10 and 32 points, whereas the range for major events is from 43.5 to 113 points. A threshold of approximately 40 EPI points appears to separate intermediate events from major events (Figure 5).

5.2 Empirical validation of the instrument: a sensitivity analysis

Evaluators should assess qualitative indicators by relying on a comprehensive step-by-step manual. Therefore, the subjective aspects of the assessment during the valuation process must be considered. If the valuation tool produces slight differences because of subjective assessments, both robustness and instrument objectivity must be questioned. Therefore, shifts in the relative ranking of events within the event portfolio are critical for the sensitivity analysis. The sensitivity analysis was conducted by randomly changing the values of two major events in different groups of basic EPI values. These amendments do not alter the rank order in either sub-group or in the whole group of major events. This finding indicates that the major events in our case study are robustly assessed, and the instrument appears to fulfill the requirement of instrument objectivity. The same analysis was conducted for the sample of intermediate events. Positive misjudgments resulted in improvements in the ranks of events. Both affected events gained one position in the rank orders of the basic EPI value and the EPI score. These changes are caused by smaller absolute differences in basic EPI values between the intermediate events (see Tables IV and V).

The application of different remuneration schemes has consequences for the overall expenses and allocation of financial support among event size categories. The size-dependent remuneration scheme represented in Table III leads to a more even distribution of financial support among size categories. Consequently, the uniform remuneration base means that medium-sized events are appropriately supported. Furthermore, this remuneration scheme results in a greater variation in support for the events in the portfolio. A linear remuneration scheme, however, necessitates the definition of different remuneration bases for each size category to allow for fair remuneration. As previously mentioned, alternative remuneration schemes hamper fairness and comparability.

One additional relevant observation regarding the application of remuneration schemes is that total expenditures react proportionally to changes in the monetary remuneration base. This feature of EPI remuneration helps local authorities, which are subject to different budget constraints, control their total expenditures (Table VI).

6. Discussion

This study contributes to the discussions of event evaluation and the appropriate subsidization of events by public authorities or institutions based on evaluations. The EPI was developed to provide a framework for the evaluation of event performance. The index and its seven key indicators aim to reduce the complexity of the cost-benefit assessment of events in different administrative and institutional contexts (e.g. municipalities and tourism destinations). This index ensures the consideration of all dimensions of sustainable development and provides incentives for future advancements in this direction. To be applicable in practice, the valuation tool is designed such that relevant scientific foundations were transformed into a system of easily understandable indicators that guide the process of event subsidization and ensure the sustainable development of event portfolios. Although the EPI was applied to a tourist destination and its event-subsidizing municipality, this valuation tool addresses various audiences in different contexts (e.g. institutions, DMOs and associations). A step-by-step manual was developed to help users obtain standardized evaluations that are reliable and comparable.

The theoretical analysis of different event impacts and the CBA resulted in a new formula for the valuation of event performance, reflecting the dimensions of sustainable development. The destination-specific adaptation of the framework to the case study was achieved through workshops and discussions with relevant stakeholders (e.g. event organizers, tourism experts and municipal representatives). Our case study confirmed that the EPI allows for comprehensive and adequate evaluations of events. The index creates considerable variation across events and is robust to differences in the assessment of their qualitative dimensions. The proposed process of event remuneration is transparent and enables local authorities to implement different subsidization strategies while adhering to budget constraints.

A second case study that was applied to a sample of events in a Swiss city resulted in an appropriate assessment of the events and the associated monetary support, which indicates that the EPI is applicable to different administrative levels, regional contexts and areas, including those in which tourism may be of relatively minor importance.

Assessments of the indicators and the remuneration process help foster acceptance of public resource allocation by stakeholders. The decision-making processes of public authorities and institutions are facilitated, well founded and justifiable. Transparent, systematic and fair subsidization enhances the credibility of public authorities and adds to the consistency of their subsidization strategies. If resource allocation to specific events is to decrease, public authorities have a better basis for negotiation and an instrument with which to derive recommendations regarding dimensions for improvement. The local community is integrated through political participation. In this case study, decisions regarding the guiding principles of tourist destinations and the authorization of monetary support by the municipality are made via political processes. The principle is transferable to other case studies and stakeholders if approval of financial support in an association—similar to a community—is granted by an elected committee. The community, one important stakeholder, should be represented by the actions of municipal representatives.

This finding is consistent with the results of Torres-Delgado and Palomeque (2014), who confirm the effectiveness of indicators to foster sustainable development at the municipal level. The developed framework of indicators qualifies for general application to event portfolios. However, the complex relationships among actors and the multidimensional impacts of events require flexible adaptation to different contexts and their characteristics.

The process of subsidization is based on the ex ante applications of event organizers. The event organizers complete a specific application that includes the required information and enables the evaluator to determine the EPI indicators. If the event is staged for the first time, applicants are instructed to make assumptions for the requested measures. Applications for events that have been previously organized should rely on existing event details. Because the evaluation tool provides a benchmark, the evaluators gain knowledge by comparing the different indicators. The proposed controlling and reporting after execution of the event incentivizes organizers to provide reliable event details. Any cognitive biases of the evaluators could be limited, as the proposed process suggests separating the evaluation of the event and the decision about financial support by assigning different persons or boards with decision-making power responsible for them.

This ex ante evaluation of events and the resulting amount of remuneration leads to the unidirectional decision-making of supporting authorities. Nevertheless, this process of repeated evaluation enables events to progress in terms of the preferred indicators. In the case of recurring events, improvements, for example, in organization can result in additional points obtained with the next application, allowing the amount of financial support to increase. If some events have higher costs than benefits for the destination, the EPI will not indicate a negative value. In this case, there exist two possibilities to renounce support for such events: a preceding negative decision of the supporting authorities or an EPI threshold for support to ensure the desired cost-benefit characteristics.

The limitations of this study stem from the relative measurement of event performance and the multitude of factors influencing events. The EPI serves as a benchmark instrument that only allows the comparison of events within an event portfolio. Because of its pragmatic and destination-unspecific indicators, it is possible to transfer and adapt the valuation tool to the user’s context. The operationalization of the indicators requires some reduction in the complexity of event surroundings to guarantee applicability. Because the proposed evaluation tool has degrees of freedom, the evaluators are responsible for its appropriate application.

The cost-benefit EPI framework should enable bidirectional impacts in the social and ecological dimensions because these indicators potentially have positive and negative effects (Ap and Crompton, 1998; Collins et al., 2009). For the analyzed event sizes in the present study, these effects have limited significance.

The evaluation of events by the EPI is mainly restricted to local impacts. The practical applicability and the aspired restriction of administrative effort support some simplification of the indicators. From a more comprehensive point of view, the impacts should have been evaluated on a broader scale (i.e. global ecological impacts or economic leakages out of the municipality) (Long and Perdue, 1990; Collins et al., 2009; Davies et al., 2013).

The achievement of a balance between scientific foundation and practical applicability represented a major challenge in the development of the EPI. Scientific standards, such as comprehensive understanding, objectivity and appropriate specification of the indicators, must be weighed against practical demands, such as comprehensibility and implementation expense.

7. Conclusion

The EPI represents a comprehensive benchmarking instrument for assessing events and their value for destinations or municipalities. It is suitable for small and medium-sized events. However, mega-events follow their own rules and are not the focus of this tool. By including seven indicators and their sub-indicators, the EPI attempts to comprehensively cover the important dimensions of event performance. This evaluation framework satisfies the demands of various tourism and administration practitioners by guiding event subsidization policies and contributes to the sustainable development of events. Furthermore, event subsidization policies are practically supported through the connection of the EPI to a systematic remuneration scheme.

In the case study, a sample of events was assessed with the valuation tool, demonstrating that the practical implementation of this tool delivered results that are acceptable to the stakeholders of both the destination and the local authorities. The process of developing the EPI helped reduce complexity and enabled local stakeholders to discuss the worth of events. The indicators and resulting EPI are applicable to different tourism destinations or municipalities. These indicators create sufficient variation in evaluation outcomes and appropriately assess different types of events. The tool appears sufficiently robust to subjective variation in terms of quantitative and qualitative assessments of the indicators conducted by different assessors. For local authorities, different remuneration schemes create a systematic budgetary scope of action and an opportunity to shape their subsidization policy based on their strategic focus. The possibility that users of the valuation tool can adjust the remuneration base, depending on their budget constraints, ensures the general applicability of the tool to different public institutions.

The application of the valuation tool facilitates the event subsidization process for local authorities and their decision about the desirable allocation of financial resources to events in the portfolio. These findings should lead to subsidization policies that are more justifiable and could even result in improved event performance in terms of sustainable regional development. Event organizers can be confident that the subsidization decisions of local authorities are well founded, transparent and fair. In addition, event organizers are encouraged to sustainably improve their events.

Figures

Theoretical impact model

Figure 1

Theoretical impact model

Polynomial function and size categories

Figure 2

Polynomial function and size categories

Variation in indicators for major events

Figure 3

Variation in indicators for major events

Variation in indicators for intermediate events

Figure 4

Variation in indicators for intermediate events

EPI basic value of major and intermediate events

Figure 5

EPI basic value of major and intermediate events

Valuation indicators of the EPI

Indicator Assessment Shortcut Dimensions of evaluation Scale
Size of the event Quantitative (SIZ) Daily frequencies (length of stay of all event participants) 1–20 EPI points
Economic value of the event (VAL) Average monetary expenses per participant, weighted by the daily frequencies of groups with different spending behavior 1–5 EPI points
Touristic value and image of the event Qualitative (TOU) Proportion of tourist/non-local participants at the event
Touristic attractiveness of the event
Contribution of the event to the awareness level of the destination
Image effects of the event on the destination
Flexibility of the organizers regarding the date of execution
1–5 EPI points
Innovative strength of the event (INN) Existing event with new features
Transfer of the concept of an event established elsewhere to the destination
Existing event with noticeable innovations for the participants
Execution of a completely new event
1–5 EPI points
Value of networking (local cooperation and exchange) (NET) Origin of the organizing committee
Cooperation between event organizers local stakeholders
Cooperation of event organizers with the Destination Management Organization
Networking possibilities and hospitality at the event
1–5 EPI points
Value of participation and social exchange (SOC) Accordance of the event with local needs and expectations
Opportunities for locals to get involved in the event
Opportunities of social exchange for locals/ development of collective social capital
1–5 EPI points
Relative ecological burden (ECO) Dealing with waste
Dealing with noise
Dealing with traffic both on-site and at arrival/departure
Dealing with town scenery and landscape
1–5 EPI points

Start-up financing

Indicator Shortcut Multiplier Measurement
Start-up financing (NEW) 1 Financial support to the extent of the valuation through the EPI basic value or EPI score (no start-up financing)
1.25 Modest additional support with low support (once or several times)
1.5 Substantial additional start-up financing support (once or several times)

Size-dependent remuneration for Saanen

EPI basic value size category Size factor EPI remuneration base (in monetary terms)
 (1) EPI basic value⩽6.25 0.5
 (2) EPI basic value⩽9.5 0.75
 (3) EPI basic value⩽12.5 1 350
 (4) EPI basic value⩽18.75 1.5
 (5) EPI basic value⩽25 2
 (6) EPI basic value⩽31.25 2.5
 (7) EPI basic value⩽37.5 3
 (8) EPI basic value⩽50 4
 (9) EPI basic value⩽62.5 5
(10) EPI basic value⩽75 6
(11) EPI basic value⩽87.5 7
(12) EPI basic value>87.5 8

Sensitivity analysis—subjective indicators assessment of major events

Event Groups G W T I N P Ö EPI basic value Rank order EPI basic value EPI score Rank order EPI score
Major Event 4 Group 1 16 4 5 4 5 4 2 113 1 904 1
Major Event 1 19 3 4 2 5 3 4 88 2 704 2
Major Event 3 17 2 3 2 4 5 2 64 3 384 3
Major Event 6 Group 2 6 5 3 2 3 2 1 46.5 4 186 4
Major Event 2 7 5 3 1 2 1 3 45.25 5 181 5
Major Event 5 10 3 2 1 4 5 3 43.5 6 174 6
Major Event 4 Group 1 16 4 5 4 5 4 2 113 1 904 1
Major Event 1 19 3 5 2 5 3 4 97.5 2 780 2
Major Event 3 17 2 3 2 4 5 2 64 3 384 3
Major Event 6 Group 2 6 5 3 2 3 2 1 46.5 4 186 4
Major Event 2 7 5 3 1 3 1 3 46.25 5 185 5
Major Event 5 10 3 2 1 4 5 3 43.5 6 174 6

Sensitivity analysis—subjective indicators assessment of intermediate events

Event Groups G W T I N P Ö EPI basic value Rank order EPI basic value EPI score Rank order EPI score
Intermediate Event 8 Group 1 4 3 4 4 3 3 2 32 1 96 1
Intermediate Event 4 8 2 3 2 3 4 4 31 2 77.5 2
Intermediate Event 2 4 2 3 2 3 4 2 23 3 46 3
Intermediate Event 1 4 2 2 1 2 2 3 15 4 22.5 4
Intermediate Event 5 Group 2 3 1 2 2 3 2 1 14.25 5 21.5 5
Intermediate Event 6 4 1 2 1 2 2 1 13 6 19.5 6
Intermediate Event 3 3 1 2 1 3 3 2 12.5 7 12.5 7
Intermediate Event 7 3 1 1 1 3 2 2 10 8 10 8
Intermediate Event 8 Group 1 4 3 4 4 3 3 2 32 2 96 2
Intermediate Event 4 8 2 4 2 3 4 4 35 1 105 1
Intermediate Event 2 4 2 3 2 3 4 2 23 3 46 3
Intermediate Event 1 4 2 2 1 2 2 3 15 4 22.5 4
Intermediate Event 5 Group 2 3 1 2 2 3 2 1 14.25 5 21.5 5
Intermediate Event 6 4 1 2 1 2 2 1 13 7 19.5 7
Intermediate Event 3 3 1 2 1 4 3 2 13.5 6 20.5 6
Intermediate Event 7 3 1 1 1 3 2 2 10 8 10 8

Remuneration schemes

Event G W T I N P Ö EPI basic value EPI score Size-dependent remuneration in CHF Alternative remuneration in CHF
Major Event 4 15 5 4 3 3 4 1 114.25 914.00 251,350.00 205,650.00
Major Event 1 19 4 3 2 5 3 3 102.25 818.00 224,950.00 184,050.00
Major Event 3 16 2 3 4 4 5 2 65.00 390.00 107,250.00 117,000.00
Major Event 6 12 3 4 2 1 1 3 57.00 285.00 78,375.00 102,600.00
Major Event 2 10 3 4 1 2 1 5 42.50 170.00 46,750.00 76,500.00
Major Event 5 9 2 2 3 4 5 2 37.50 112.50 30,937.50 67,500.00
Total expenditures 739,612.50 753,300.00
Proportion of total expenditures of major events 90% 93%
Mean 184,903.24 188,325.12
Median 92,812.50 109,800.00
SD 93,162.34 56,945.18
Intermediate Event 8 4 3 4 4 3 3 2 32.00 96.00 26,400.00 12,800.00
Intermediate Event 4 8 2 3 2 3 4 4 31.00 77.50 21,312.50 12,400.00
Intermediate Event 2 4 2 3 2 3 4 2 23.00 46.00 12,650.00 9,200.00
Intermediate Event 1 4 2 2 1 2 2 3 15.00 22.50 6,187.50 6,000.00
Intermediate Event 5 3 1 2 2 3 2 1 14.25 21.38 5,878.13 5,700.00
Intermediate Event 6 3 1 2 1 2 2 1 11.25 11.25 3,093.75 4,500.00
Intermediate Event 3 2 1 2 1 2 2 2 9.00 6.75 1,856.25 3,600.00
Intermediate Event 7 1 1 1 1 1 2 2 6.00 3.00 825.00 2,400.00
Total expenditures 78,203.13 56,600.00
Proportion of total expenditures of intermediate events 10% 7%
Mean 15,640.63 11,320.01
Median 6,032.81 5,850.00
SD 9,511.45 3,950.68

References

Andersson, T.D. and Lundberg, E. (2013), “Commensurability and sustainability: triple impact assessments of a tourism event”, Tourism Management, Vol. 37 No. C, pp. 99-109.

Ap, J. and Crompton, J.L. (1998), “Developing and testing a tourism impact scale”, Journal of Travel Research, Vol. 37 No. 2, pp. 120-130.

Bandi Tanner, M., Lehmann Friedli, T., Künzi, A. and Müller, H. (2017), Event Performance Index (EPI). Bewertung von Events für eine transparente Unterstützungspraxis—Das “Saaner Modell”, Forschungsstelle Tourismus (CRED-T), Bern.

Bieger, T., Müller, H., Elsasser, H., Johnsen, J., Weber, F. and Wyss, U. (2003), “Nachhaltigkeit der Ski WM 2003 St Moritz, Engadin—Schlussbericht”, Universität Bern, Bern; Universität St Gallen, St Gallen; Universität Zürich, Zürich.

Brown, S., Getz, D., Pettersson, R. and Wallstam, M. (2015), “Event evaluation: definitions, concepts and a state of the art review”, International Journal of Event and Festival Management, Vol. 6 No. 2, pp. 135-157.

Collins, A. and Cooper, C. (2017), “Measuring and managing the environmental impact of festivals: the contribution of the ecological footprint”, Journal of Sustainable Tourism, Vol. 25 No. 1, pp. 148-162.

Collins, A., Jones, C. and Munday, M. (2009), “Assessing the environmental impacts of mega sporting events: two options?”, Tourism Management, Vol. 30 No. 6, pp. 828-837.

Connell, J., Page, S.J. and Meyer, D. (2015), “Visitor attractions and events: responding to seasonality”, Tourism Management, Vol. 46 No. C, pp. 283-298.

Davies, L., Coleman, R. and Ramchandani, G. (2013), “Evaluating event economic impact: rigour versus reality?”, International Journal of Event and Festival Management, Vol. 4 No. 1, pp. 31-42.

Dwyer, L. (2005), “Estimating the impacts of special events on an economy”, Journal of Travel Research, Vol. 43 No. 4, pp. 351-359.

Dwyer, L. and Forsyth, P. (2009), “Public sector support for special events”, Eastern Economic Journal, Vol. 35 No. 4, pp. 481-499.

Dwyer, L., Mellor, R., Mistilis, N. and Mules, T. (2000), “A framework for assessing ‘tangible’ and ‘intangible’ impacts of events and conventions”, Event Management, Vol. 6 No. 3, pp. 175-189.

Fernández, J.I.P. and Rivero, M.S. (2009), “Measuring tourism sustainability: proposal for a composite index”, Tourism Economics, Vol. 15 No. 2, pp. 277-296.

Fourie, J. and Santana-Gallego, M. (2011), “The impact of mega-sport events on tourist arrivals”, Tourism Management, Vol. 32 No. 6, pp. 1364-1370.

Fredline, L., Jago, L. and Deery, M. (2003), “The development of a generic scale to measure the social impacts of events”, Event Management, Vol. 8 No. 1, pp. 23-37.

Fredline, L., Raybould, M., Jago, L. and Deery, M. (2005), “Triple bottom line event evaluation: a proposed framework for holistic event evaluation”, The Impacts of Events: Proceedings of the International Event Research Conference held in Sydney, pp. 2-15.

Getz, D. (2008), Event Studies Theory, Research and Policy for Planned Events, Elsevier, Oxford.

Getz, D. (2009), “Policy for sustainable and responsible festivals and events: institutionalization of a new paradigm”, Journal of Policy Research in Tourism, Leisure and Events, Vol. 1 No. 1, pp. 61-78.

Gössling, S., Hansson, C.B., Hörstmeier, O. and Saggel, S. (2002), “Ecological footprint analysis as a tool to assess tourism sustainability”, Ecological Economics, Vol. 43 No. 2, pp. 199-211.

Gursoy, D. and Kendall, K.W. (2006), “Hosting mega events”, Annals of Tourism Research, Vol. 33 No. 3, pp. 603-623.

Hall, C.M. (1989), “The definition and analysis of hallmark tourist events”, GeoJournal, Vol. 19 No. 3, pp. 263-268.

Hunter, C. and Shaw, J. (2007), “The ecological footprint as a key indicator of sustainable tourism”, Tourism Management, Vol. 28 No. 1, pp. 46-57.

Kesenne, S. (2005), “Do we need an economic impact study or a cost-benefit analysis of a sports event?”, European Sport Management Quarterly, Vol. 5 No. 2, pp. 133-142.

Long, P.T. and Perdue, R.R. (1990), “The economic impact of rural festivals and special events: assessing the spatial distribution of expenditures”, Journal of Travel Research, Vol. 28 No. 4, pp. 10-14.

Mules, T. and Dwyer, L. (2005), “Public sector support for sport tourism events: the role of cost-benefit analysis”, Sport in Society, Vol. 8 No. 2, pp. 338-355.

Müller, H., Rütter, H. and Stettler, J. (2010), UEFA EURO 2008TM und Nachhaltigkeit—Erkenntnisse zu Auswirkungen und Einschätzungen in der Schweiz. Wissenschaftlicher Bericht, Forschungsinstitut für Freizeit und Tourismus der Universität Bern, Bern.

Müller, H., Bolliger, H.J., Mathys, H., Rindlisbacher, B. and Stettler, J. (1997), “Sportveranstaltungen und Umwelt—das Fallbeispiel Eidgenössisches Turnfest Bern 96”, OK ETF Bern No. 96, Bern.

Preuss, H. (2007), “The conceptualisation and measurement of mega sport event legacies”, Journal of Sport & Tourism, Vol. 12 Nos 3/4, pp. 207-228.

Preuss, H. (2011), “A method for calculating the crowding-out effect in sport mega-event impact studies: the 2010 FIFA World Cup”, Development Southern Africa, Vol. 28 No. 3, pp. 367-385.

Pugh, C. and Wood, E.H. (2004), “The strategic use of events within local government: a study of London Borough Councils”, Event Management, Vol. 9 Nos 1/2, pp. 61-71.

Richards, G. and Wilson, J. (2004), “The impact of cultural events on city image: Rotterdam, cultural capital of Europe 2001”, Urban Studies, Vol. 41 No. 10, pp. 1931-1951.

Robbins, D., Dickinson, J. and Calver, S. (2007), “Planning transport for special events: a conceptual framework and future agenda for research”, International Journal of Tourism Research, Vol. 9 No. 5, pp. 303-314.

Rütter+Partner and Hochschule Luzern (2009), “Volkswirtschaftliche Bedeutung der Eishockey-Weltmeisterschaft 2009 in der Schweiz”, Luzern/RÜschlikon.

Sadd, D., Fyall, A. and Wardrop, K. (2017), “Evaluative event frameworks: a learning destination perspective”, International Journal of Tourism Research, Vol. 19 No. 3, pp. 339-348.

Scherer, R. (2002), “Die räumliche Inzidenz des Lucerne Festivals”, Informationen zur Raumentwicklung, Vol. 7 Nos 4/5, pp. 271-275.

Small, K., Edwards, D. and Sheridan, L. (2005), “A flexible framework for evaluating the socio-cultural impacts of a (small) festival”, International Journal of Event Management Research, Vol. 1 No. 1, pp. 66-77.

Thomas, R. and Wood, E. (2003), “Events-based tourism: a survey of local authority strategies in the UK”, Local Governance, Vol. 29 No. 2, pp. 127-136.

Timur, S. and Getz, D. (2009), “Sustainable tourism development: how do destination stakeholders perceive sustainable urban tourism?”, Sustainable Development, Vol. 17 No. 4, pp. 220-232.

Torres-Delgado, A. and Palomeque, F.L. (2014), “Measuring sustainable tourism at the municipal level”, Annals of Tourism Research, Vol. 49 No. C, pp. 122-137.

Tyrrell, T.J. and Johnston, R.J. (2001), “A framework for assessing direct economic impacts of tourist events: distinguishing origins, destinations, and causes of expenditures”, Journal of Travel Research, Vol. 40 No. 1, pp. 94-100.

Vanhove, N. (2011), The Economics of Tourism Destinations, Routledge, Amsterdam.

Walo, M., Bull, A. and Breen, H. (1996), “Achieving economic benefits at local events: a case study of a local sports event”, Festival Management and Event Tourism, Vol. 4 Nos 3/1, pp. 95-106.

Whitford, M. (2004), “Regional development through domestic and tourist event policies: Gold Coast and Brisbane, 1974–2003”, UNLV Journal of Hospitality, Tourism and Leisure Science, Vol. 1, pp. 1-24.

Whitford, M., Phi, G.T. and Dredge, D. (2014), “Principles to practice: indicators for measuring event governance performance”, Event Management, Vol. 18 No. 3, pp. 387-403.

Wood, E.H. (2005), “Measuring the economic and social impacts of local authority events”, International Journal of Public Sector Management, Vol. 18 No. 1, pp. 37-53.

Wood, E.H. (2008), “An impact evaluation framework: local government community festivals”, Event Management, Vol. 12 Nos 3/4, pp. 171-185.

Corresponding author

Monika Bandi Tanner can be contacted at: monika.bandi@cred.unibe.ch