Using analytics to inform category management and strategic sourcing

Purpose – The purpose of this paper is to show how data analytics can be used to identify areas of potential cost savings for category managers of installation-level services. Using integrated solid waste management (ISWM) as a test case, the authors also examine the impact of small business set- asides on price and contractor performance. Design/methodology/approach – The authorsusedata analytics, speci ﬁ cally sequential regression, the Wilcoxon rank-sum test and ordered logistic regression to investigate the in ﬂ uence of service- and contracting-related variableson priceandcontractorperformance. Findings – The authors ﬁ nd that service- and contracting-related variables in ﬂ uence price. Speci ﬁ cally, they identify that a service-related variable, number of containers, signi ﬁ cantly affects price, and that two contracting-related variables, one type of small business set-aside and the number of offers received, also signi ﬁ cantly affect price. The authors quantify the price premiums paid for using various types of small businessset-asides. Research limitations/implications – Although the ﬁ ndings were signi ﬁ cant, the authors believe that the robustness of the conclusions could be enhanced if the Air Force captured more data. Additional observations would increasethegeneralizabilityof theresults. Practical implications – This empirical experiment demonstrates that detailed analyses are required to gain insights into services ’ price drivers to craft more appropriate category management strategies for installation-level services. Originality/value – This empirical study shows how historical data can be used to assess price drivers of installation-levelservices.Itisalsooneofthe ﬁ rsttoquantifytheimpactthatsmallbusinessset-asideshaveonprice.


Introduction
The transformation of the procurement function from a transaction-oriented perspective to a strategic-oriented focus was first discussed by Henderson (1975) when he predicted that there would be greater importance placed on the procurement function in corporate management. Kraljic (1983) purported that "purchasing must become supply management", and that organizations should develop specific sourcing strategies for products and services based on the strategic importance of the procured product or service to the organization and the complexity of the market for that product or service (Kraljic, 1983, p. 109). One aspect of the purchasing transformation to a strategic function is the use of category management and strategic sourcing for the procurement of product and services. Category management is concerned with the management of a specific category of product and services to ensure the related policies, sourcing and use of those products and services meets corporate-level strategic objectives. Category management is an ongoing process that is focused on value elements that go beyond simple price savings. Category management involves engaging stakeholders and fully understanding their product and service requirements, market intelligence on market trends, cost drivers and risks pertaining to those product and services and developing a sourcing strategy that aligns stakeholder requirements with the realities of the market (Monczka et al., 2015, pp. 199-201).

Importance of strategic sourcing
Strategic sourcing continues to grow in importance as an enabler of an organization's competitive advantage. Traditionally thought of as back-office operations, purchasing and supply management have moved to the corporate suite in terms of adding value to corporate strategy. Deciding which functions to outsource, selecting the outsource supplier and structuring the sourcing partnership have become a critical part of corporate strategic sourcing. Past research has identified tools and techniques which organizations can implement to elevate their sourcing function to a strategic level. Gottfredson et al. (2005) developed a three-step process that identifies which organizational functions should remain in-house and which functions should be considered for outsourcing and also provide increased value for the organization. Additionally, Anderson and Katz (1998) discuss that what a company procures and how the company procures has increased in importance and complexity. By developing a systematic framework of strategic sourcing, a company can leverage total spend, optimize the supply base and minimize total supply chain cost and maximize the value of procured supplies and services (Anderson and Katz, 1998, p. 1). Strategic sourcing also involves taking a strategic approach to the selection of suppliersan approach that is more aligned with the organization's strategic objectives and reflects the integration of sourcing with corporate strategy (Rendon, 2005, p. 9). Strategic sourcing is differentiated from category management, in that strategic sourcing is a tool of category management that is focused primarily on leveraging volume to drive down costs.

Federal government initiatives
The federal government has implemented both category management and strategic sourcing as part of its initiatives to reduce costs and increase efficiency and effectiveness. The federal government's initial focus was on strategic sourcing, which equated more to strategic contractingcreating large, indefinite delivery, indefinite quantity type contracts that could be used by any federal agency. As reported by the Office of Management and Budget (OMB), federal agencies have saved money by pooling their spending, either by centralizing the agency's contracting decisions or by using government-wide strategic sourcing vehicles, to lower prices and reduce duplication and administrative costs. Since fiscal year (FY) 2010, government-wide contracts for office supplies have saved over US $140m by offering lower prices than any single agency could negotiate on its own. Similar vehicles for domestic delivery services saved over US$31m in FY 2011 over what agencies were paying under previous agreements (OMB, 2012, p. 1). While these initiatives reaped savings in price and process by leveraging the spending power of the federal government and reducing time-to-contract, there were savings that could not be achieved through strategic sourcing (i.e. contracting) solutions. The government's form of strategic sourcing, which was more like strategic contracting, failed to address the total cost of ownership, to include proper demand management practicescontrolling consumption, standardizing brand names and/or performance standards, etc.and failed to align spend into common categories that match the way industry produces products and services.
The transition to category management as a more comprehensive sourcing process began in 2014 when the Obama administration launched its Category Management Initiative. Then-Office of Federal Procurement Policy (OFPP) administrator, Anne Rung, issued her memo, "Transforming the Marketplace: Simplifying Federal Procurement to Improve Performance, Drive Innovation, and Increase Savings". In it, she noted: There is a critical need for a new paradigm for purchasing that moves from managing purchases and price individually across thousands of procurement units to managing entire categories of common spend and total cost through category management (OMB, 2014, p. 2).
The memo encourages federal agencies to "buy as one", noting that category management techniques are used extensively in industry: This approach includes strategic sourcing, but also a broader set of strategies that drive performance, like developing common standards in practices and contracts, driving greater transparency in acquisition performance, improving data analysis, and more frequently using private sector (as well as government) best practices" (OMB, 2014, p. 2).
Specifically to the US military, the Department of Defense (DoD) has also been undergoing this same transformation of its procurement function from a transaction-oriented perspective to a strategic-oriented enterprise. The DoD procurement function is no longer seen as a tactical, clerical or administrative function, but more of a strategic function. This transformation can be attributed to the fact that the DoD has begun to understand and realize the importance of procurement in achieving its strategic objectives, as well as the impact of procurement on reducing costs. The DoD annually obligates billions of dollars for the procurement of supplies and services in support of the national military strategy. In FY 2016, the DoD obligated approximately US$239bn on contracts for defense-related supplies and services. Specific to the US Air Force (USAF), over US$50bn were obligated on contracts for supplies and services in FY2016 (USA Spending, 2016). Services typically account for over half of the DoD procurement budget compared to the acquisition of supplies. In the current environment of budget and manpower cuts, the DoD is transforming its acquisition process to ensure that critical supplies and services are sourced cost-effectively.

Air Force initiatives
Within the USAF, one aspect of this strategic transformation is the use of category management for the procurement of services at military installations. The USAF's category management initiative emphasizes a focus on "increased efficiency and effectiveness, lessening costs, and reducing redundancies" (Sharkey, 2015). Arguably, the USAF is leading the DoD in its category management initiative. The USAF recently appointed the first Category Management Accountable Official (CMAO) in the DoD, marking a giant step forward in standing up the authority structure required to implement such a large paradigm shift in the way the USAF thinks about and executes installation spend. Within the USAF, the Air Force Installation Contracting Agency (AFICA) has crafted the change strategy, which involves category planning, category execution and category performance management. Part of understanding a category of spend involves identifying specific cost drivers, so strategies can be made to increase efficiency and effectiveness and reduce costs.
The USAF is implementing its category management initiative by focusing on "leveraging buying power, improving efficiencies, and managing consumption" (Sharkey, 2015, p. 7). With the stand-up of the Air Force Installation and Mission Support Center (AFIMSC) and its six functionally based primary subordinate units (PSUs) (e.g. Air Force Civil Engineer Center, Air Force Security Forces Center, Air Force Installation Contracting Agency, etc.), the USAF is postured to implement category management and strategically source appropriate installation-level products and services. AFIMSC's mission is "to make the best use of limited resources in managing and operating its installations" and to serve as: The single intermediate-level headquarters responsible for providing installation and mission support capabilities to 77 USAF installations, nine major commands and two direct reporting units with an annual budget of approximately $10 billion (AFIMSC, 2017, np).
The USAF's category management operating model includes category planning, category execution and category performance management. Category planning involves conducting a spend analysis, requirement analysis, market analysis and risk analysis. This phase also includes analyzing the four major performance levers (demand management, supplier management, strategic sourcing and total cost management) to identify category improvement initiatives. Category improvement initiatives within total cost management includes the identification of specific price drivers in the acquisition that can result in increased efficiency and effectiveness and a reduction in costs. Price drivers can be either product-or service-related or contracting-related and impact savings associated with rate (getting more for less), process (getting more with less) and demand (getting less) (Sharkey, 2015, pp. 21-24). The product-or service-related price drivers impact rate savings, process savings and demand savings. Contracting-related price drivers impact rate savings and process savings. Category execution involves the execution of selected performance levers identified in the planning phase. This includes executing changes associated with the product-or service-related or contractingrelated price drivers (Sharkey, 2015, pp. 25-30). Category performance management includes the performance tracking, benchmarking and continuous improvement of the management of the specific category of product or service (Sharkey, 2015, pp. 31-33).

Purpose of research
The purpose of this research is to show how data analytics can be used to identify areas of potential cost savings for category managers. We analyze the price drivers for one of the DoD's most commonly procured installation-level services, integrated solid waste management (ISWM), as a test case. Specifically, we focus on the procurement of ISWM services within the USAF to identify the relationship between service-related price drivers, contracting-related price drivers, price and contractor performance. We test seven hypotheses to determine the effect that service-and contracting-related variables have on price and contractor performance. Based on our research findings, we provide recommendations to the USAF for managing ISWM services that will result in increased efficiency, effectiveness and a reduction in costs. Perhaps more importantly, we show how data analytics can be used to help category managers and category management teams identify areas ripe for cost reduction.
In this research, we use statistical analysis to explore ISWM price drivers to properly manage the service. While we focus on the impact of price drivers, we also use performance data to understand the correlation between price and contractor performance. The goal is to provide insight into pricing and performance so that category managers can establish a more effective sourcing strategy (i.e. acquisition plan). We use three statistical methods to determine how service-related and contracting-related independent variables (tonnage of waste, number of containers, wage rates, number of offers and type of competition) affect the dependent variables (total price and contractor performance).

Literature review
The literature reflects an abundance of research on the application of strategic sourcing to corporate strategy. Kraljic developed a systematic framework for incorporating environmental and other strategic factors into procurement strategy formulation for procured products and services. The use of the Kraljic framework results in a contingencybased model for formulating the appropriate sourcing strategy for products and services. The Kraljic framework has been widely applied throughout many industries. Rendon and Templin (1992) explored the application of the Kraljic framework to National Cash Register's (NCR's) supply management program. Montgomery et al. (2017) created a multiobjective decision analysis (MODA) model that uses an organization's spend data to quantify the location of various products and services on the Kraljic matrix. The use of the Kraljic framework enables organizations to determine the appropriate sourcing strategy for specific products and services. The market complexity and importance of the product or service to the organization may indicate that a strategic sourcing strategy is appropriate.
Few studies exist that analyze installation-level spending. Apte et al. (2011) developed an optimization model for selecting a set of proposals from among multiple offerors for services to be performed at multiple installations. The selection achieved the most favorable objective by balancing the confidence level in an offeror's past performance with the cost of services to the USAF. The research findings, based on a realistic scenario, demonstrate improvements over the traditional sourcing process in both overall performance and cost. The work performed by Apte et al. (2011) validated the importance of using data analytics to assess installation-level spending. Boehmke et al. (2017) use a data envelopment analysis (DEA) approach to measure efficiency in installation support services, thus improving senior-level decision-making. By focusing specifically on facility sustainment activities, their DEA approach supports decision-making by "quantifying cost savings and performance improvements, and systematically bench-marking to identify best-practice peers" (Boehmke et al., 2017, p. 39). In exploring how to assess the impact of USAF force reductions on support resources (i.e. tooth-to-tail impact), Boehmke et al. (2016aBoehmke et al. ( , 2016b used multivariate linear regression to identify relationships between the tooth and tail and Bayesian networks to model the toothto-tail cost consequences. They then used scenario analysis to illustrate how a Bayesian network can provide decision-makers with "the ability to model uncertainty in the decision environment, a visual illustration of cause-and-effect impacts, and the ability to perform multi-directional reasoning in light of new information" (Boehmke et al., 2016a(Boehmke et al., , 2016b. Another tooth-to-tail focused analysis includes Boehmke et al. (2016aBoehmke et al. ( , 2016b) research on analyzing force reductions impacts on indirect personnel costs. They use a multi-level modeling approach to assess the influence that each organization level has on indirect cost behavior and relationships. Their findings "identify the operational variables that influence indirect personnel costs in the USAF enterprise, providing Air Force decision-makers with evidence-based knowledge to inform decisions regarding budget reduction strategies" (Boehmke et al., 2016a(Boehmke et al., , 2016b).
In their research on analyzing cost growth and investigating approaches to reducing cost growth in the USAF, Boehmke et al. (2015) apply a growth curve clustering approach to identify underlying cost curve behavior. Their findings indicate micro-level growth curves vary greatly from the aggregate cost curves. They also found that their clustering approach can help decision-makers direct their focus and policies toward specific growth curves that must be "bent" (Boehmke et al., 2016a(Boehmke et al., , 2016b. As previously stated, the purpose of this research is to analyze the price drivers for the USAF's procurement of ISWM to identify the relationship between ISWM service-related price drivers, contracting-related price drivers, price and contractor performance. Past technical research regarding waste disposal has focused on solving waste management social and environmental problems (Achillas et al., 2013), cost-effective waste collection systems (Boskovic et al., 2016;Arribas et al., 2010) and alternative solid waste strategies that meet cost, energy and environmental emissions objectives (Solano et al., 2002). We focus on the procurement side of the service, specifically on category management and identifying price drivers to strategically source the service. The next section discusses our research methodology.

Data
We combined data from three different sources to create our comprehensive set. First, FY2014 ISWM service-related data for 63 USAF installations were collected by the Facilities and Construction Category Management Team, Facility Related Services subcategory. Second, contracting-related data from the Federal Procurement Data System-Next Generation (FPDS-NG) were matched to those observations. Finally, contractor performance data from the Past Performance Information Retrieval System-Report Card (PPIRS-RC), a retrieval system for the Contractor Performance Assessment Reporting System (CPARS), were matched to 32 of the observations[1]. Our goal in collecting the data was to better understand the ISWM needs across all pertinent installations to develop a sound strategy for the service. Specifically, the team was looking for potential rate (i.e. price), process (i.e. ordering and delivery of the service) and demand (i.e. ordering the right amount of the service to meet needs) savings.
In this study, we use the data to determine the relative importance of each of the ISWM price drivers. We find the effect of price drivers, if any, on contract price and contractor performance. Price of the contract is from the viewpoint of the customer, the USAF, whereas the cost of providing the service is from the viewpoint of the supplier vice "vendor". Further, we examine the effect of small business set-asides on the price of the service by comparing price premiums of contracts that used one of five different types of small business set-asides to the price paid for contracts that used full and open competition (i.e. no small business setaside).
The data pertinent to this study consist of 16 variables and 63 observations. In the course of testing our assumptions, we removed one outlier for unrealistic annual tonnage (likely a data input error), thus reducing our useful observations to n = 62. Variable descriptions and types (dependent variable, DV, or independent variable, IV) are given in Table I. For performance DVs, the buyer rates the contractor's performance on a 1 to 5 scale, where: 1 = unsatisfactory; 2 = marginal; 3 = satisfactory; 4 = very good; and 5 = exceptional.
Basic descriptive statistics are given in Table II, and correlations are provided in Table III. Of note, number of containers is strongly correlated with total price (r = 0.72), and the number of tons is moderately correlated with total price (r = 0.46). Total price is also moderately positively correlated with performance quality (r = 0.46), but counterintuitively has a moderate negative relationship with wage rate (r = À0.49). Finally, quality performance is strongly correlated with cost performance (r = 0.67), schedule performance (r = 0.71) and management performance (r = 0.91). Figure 1 shows the distribution of set-aside categories. Nearly a quarter of the contracts were solicited via full and open competition, while 64.5 per cent were solicited using some sort of small business set-aside. Note that 11 per cent of the observations did not list how the contract was solicited. These observations were dropped via listwise deletion during the analyses.

Hypotheses
Using these data, we test seven hypotheses. We begin with the price-related hypotheses, then move to the performance-related hypotheses. Notes: *Wage rate was determined using the Department of Labor rates required for all federal contracts; **8(a) (named after legislation that created the program) is for a special category of disadvantaged, small businesses that require significant development. The program assists those companies by offering special set-asides and even sole-source awards

Price-related hypotheses
The first hypothesis seeks to determine the relative importance of each of the price drivers of the ISWM service. Identifying the price drivers is important to determining how the organization can control, and if possible, reduce price. We are interested in understanding whether ISWM service-related variables or contracting-related variables contribute, and if so, identifying the largest price drivers. ISWM service-related price drivers may be able to be controlled or reduced by changing certain user-related activities. Similarly, identifying significant contracting-related price drivers can help the organization craft better acquisition strategies to control or reduce overall price. Naturally, we expect that aspects of the service itself would be more influential on price than aspects of the contracting process used to acquire the service because the service-related costs are substantive to the end product (i.e. service delivery), while the contracting-related costs are administrative, and should therefore be relatively less influential on price. We test that the ISWM service-related variables will have more effect on the price than the contracting-related variables. Specifically, we hypothesize that the tonnage of waste, number of containers and wage rate will influence price more than the number of offers received or the type of small business set-aside (if any): H1. ISWM service-related variables have a greater effect on price than contractingrelated variables.
The second hypothesis tests the relative effects the ISWM service-related variables have on price. While both tonnage of waste and the number of containers to be emptied logically contribute to the overall price of the contract, we speculate that tonnage of waste has a greater effect on price because more tonnage requires more contracted trucks to dispose of the waste, and it also increases landfill costs (assuming the landfills have either a "per truck" or "per ton" fee). Further, because federal contractors are required to use standard Department of Labor wage rates (in dollars per hour) when estimating their costs, we test that wage rates will have less effect on the overall price of the service (because the wage rates are pre-determined): H2a. Tonnage of waste has a greater effect on price than number of containers.
H2b. Number of containers has a greater effect on price than wage rate.
The third hypothesis tests the relative effects the contracting-related variables have on price. Again, while both small business set-asides and the number of offers received logically affect overall price of the contract, we posit that limiting competition through the use of set-asides has a greater effect on price because, unlike large businesses, small businesses typically do not have the volume of work required to offer deep discounts. Therefore, even if a small business set-aside contract were to receive the same (or more) offers than a full and open competition contract (i.e. a contract that allows any business to Strategic sourcing compete, regardless of size), the prices offered by small businesses are likely to be higher than prices offered by large businesses: H3. Small business set-asides have a greater effect on price than number of offers.
The fourth hypothesis tests the effect small business set-asides have on price. As described above, small businesses typically cannot match or beat the prices of larger businesses. We use two unit-price variables to examine the effect of small business set-asides on price: price per ton and price per container: H4a. Small business set-asides result in a higher price per ton than full and open competition.
H4b. Small business set-asides result in a higher price per container than full and open competition.
The fifth hypothesis tests the relative effects the types of small business set-asides have on price per ton. There are five different types of small business set-asides represented in the data: 8(a) sole source, 8(a) competed, historically underutilized business zone (HUBZone), Service-Disabled Veteran-Owned Small Business (SDVOSB) and total small business set-aside. Of these five categories, the first four are less inclusive than the fifth. Total small business allows for any small business to compete for the contractto include any businesses that are in the first four categorieshowever, the reverse is not true. For example, if the contracting officer were to specify that the contract is a total small business set-aside, any small business type is able to compete for the contract. However, if the contracting officer were to specify that the contract is a HUBZone setaside, only those small businesses that qualify for HUBZone status are eligible to compete. Thus, the less inclusive the small business set-aside type, the fewer number of contractors eligible to compete for the contract. We, therefore, hypothesize that restriction on competition is expected to increase the price per ton and price per container of waste removal: H5a. Less inclusive small business set-asides (i.e. 8(a) sole source, 8(a) competed, HUBZone and SDVOSB) result in a higher price per ton than the more inclusive small business set-aside (i.e. Total Small Business).
H5b. Less inclusive small business set-asides (i.e. 8(a) sole source, 8(a) competed, HUBZone and SDVOSB) result in a higher price per container than the more inclusive small business set-aside (i.e. Total Small Business).
Among the less inclusive small business set-aside types, one group is particularly exclusive: 8(a) sole source. In this situation, the contracting officer can choose not to compete the requirement at all; instead, he or she can simply award the contract to an eligible 8(a) contractor. Therefore, we hypothesize that without competition, the price per ton and price per container of waste removal are expected to increase: H6a. Among the less inclusive small business set-asides, the sole source set-aside (i.e. 8(a) sole source) results in a higher price per ton than the competed set-asides (i.e. 8(a) competed, HUBZone, and SDVOSB).
H6b. Among the less inclusive small business set-asides, the sole source set-aside (i.e. 8(a) sole source) results in a higher price per container than the competed setasides (i.e. 8(a) competed, HUBZone, and SDVOSB).

Performance-related hypotheses
Similar to our first hypothesis, in our seventh hypothesis, we seek to determine whether each of the ISWM service-and contracting-related variables affect contractor performance. Because the ISWM service-related variables (i.e. tonnage of waste, number of containers and wage rate) were provided to potential contractors early in the acquisition process (i.e. were understood prior to bidding) and tend to remain stable throughout the life of the contract, we do not expect to find that ISWM service-related variables significantly affect performance: H7a. ISWM service-related variables do not affect contractor performance.
On the other hand, because adequate competition is known to simultaneously decrease price and increase performance, we do expect to find a significant relationship between the contracting-related factors (i.e. small business set-asides and number of offers) and performance: H7b. Contracting-related variables affect contractor performance.

Methods
To test these hypotheses, we use three different statistical methods. We first describe the price-related methods and then move to the performance-related method.

Sequential multiple regression
For H1 to H3, we use sequential multiple regression to determine the amount of variance in price (i.e. increase in R 2 ) captured by each variable.
In sequential regression (sometimes called hierarchical regression), independent variables enter the equation in an order specified by the researcher. Each IV (or set of IVs) is assessed in terms of what it adds to the equation at its own point of entry [. . .] [t]he researcher normally assigns order of entry of variables according to logical or theoretical considerations. (Tabachnick and Fidell, 2007, p. 138) To test the amount of variance in price each IV captures, we entered them in sequence with the hypotheses. Specifically, we made five groups of predictors: Group k = 1 consists of v 1 = number of tons; Group k = 2 consists of v 2 = number of containers; Group k = 3 consists of v 3 = wage rate; Group k = 4 consists of v 4 , . . . , v 8 = small business set-aside categories; and Group k = 5 consists of v 9 = number of offers. Accordingly, we perform k = 1 [. . .] 5 linear regressions given by equation (1): where, at the k-th stage in the sequence: p k is dependent variable Price; a k is the intercept regression coefficient; b ik is the slope regression coefficient associated with independent variable i; v i is the value of the i-th variable; and, e k is the error term. Note that, in this sequential approach, the group order in which the new variable(s) are added to explain the DV matters. Given our knowledge of the problem, we posit that number of tons should have the leading role and so on. We later revise this assumption based on the results.
Also, like any regression analysis, certain assumptions about the data were met prior to performing the regressions. First, the types of small business set-asides are dummy variables. We exclude the full and open competition group to compare the set-asides to full competition. Normality was achieved through a series of data transformations. Specifically, total price, number of tons and number of containers received logarithmic transformations, and wage rate received an inverse transformation [2]. Linearity and homoscedasticity of the residuals were verified. Multicollinearity was ruled out and the errors were deemed to be independent (i.e. non-correlated).

Wilcoxon rank sum test
For H4 through H6, we use the Wilcoxon rank sum test to determine whether the difference in the median prices of the groups are statistically different. The Wilcoxon rank sum test is the non-parametric equivalent of the independent t-test, which is used to determine whether there is a statistically significant difference between the means of two unrelated groups. We use this non-parametric test because the price for each of the categories was not normally distributed; however, the general shapes of the distributions for each group were the same. The null hypothesis for this test is that there are no differences in price between the groups being comparedthat they have equal medians.

Ordered logistic regression
For H7a and H7b, we use ordered logistic regression to determine whether or not the ISWM service-and contracting-related variables affect contractor performance. Ordered logistic regression is appropriate, given the categorical (i.e. non-continuous) nature of the DVs. The categorical nature of the performance scale makes it inappropriate for multiple regression. Ordered logistical regression is like the more typical binary logistic regression in that it makes probabilistic predictions that an observation belongs in a given group; however, ordered logistic regression is appropriate for outcomes with multiple (vice the binary two) categories. Ordered logistic regression uses a series of equations to determine the probability that the observation is above the first group (i.e. above unsatisfactory), above the second group (i.e. above marginal), and so on. Equation (2) shows this multiple-group approach. The right-hand side of the equation represents the more common logistic regression (here, u represents a linear regression calculation involving any number of predictors). The equation predicts the probability that the actual outcome Y exceeds type j: (2) With the hypotheses specified and the methods described, we turn to the results and implications.

Price-related results
Sequential multiple regression results. The results of the sequential multiple regressions are provided in Table IV. When using price as the DV, we found that ISWM service-related variables account for 84 per cent of the variance in price, while contracting-related variables accounted for less than 1 per cent. To account for the fact that there is declining available DV variance the later a variable is input into the regression, we performed a second sequential multiple regression whereby the contracting-related variables were entered first.
In this analysis, we found that the ISWM service-related variables accounted for 27 per cent of the variance in price, and the contracting-related variables accounted for 57 per cent of the variance. In the final equation in the sequence (k = 5), only number of containers, 8(a) sole source set-aside, and number of offers significantly affected price. Further, the total h 2 for the ISWM service-related variables was 0.24, while the total h 2 for the contracting-related variables was 0.02. These results suggest that the ISWM service-related variables (tonnage of waste, number of containers and wage rate) influence price more than the contracting-related variables (small business set-asides and number of offers). Thus, H1 is supported. This may be welcome news for buying organizations, as most desire to make their processes as efficient as possible to have minimal negative effect (if any) on price.
Testing the relative effects of the ISWM service-related variables on price, we find that number of tons (h 2 = 0.00) does not have a greater effect on price than number of containers (h 2 = 0.24). Thus, H2a is not supported. However, H2b is supported, as number of containers (h 2 = 0.24) has a greater effect on price than wage rate (h 2 = 0.00). These results suggest that the largest ISWM service-related price driver is the number of containers, with tons of waste and wage rate having a minimal effect on price. Clearly, organizations receiving the ISWM service should examine the number of containers they are using, as reducing containers may significantly reduce price.
Testing the relative effects the contracting-related variables have on price, we find that the small business set-asides (total h 2 = 0.01) have the same effect on price as the number of offers received (h 2 = 0.01). Thus, H3 is not supported. This result is important, in the sense that buying organizations cannot simply reduce price by stirring up competition. Buying organizations should understand the price premium they can expect to pay for meeting certain socio-economic goals so they can make informed acquisition decisions. Notes: *p < 0.05; ***p < 0.001; ns stands for non-significant; Adjusted R 2 is the percent of variance in the DV (price) that is explained by the IVs, adjusted by the number of predictors in the model; h 2 is the percent of variance in the DV (price) that is explained by each IV

Wilcoxon rank sum test results
The results of the Wilcoxon rank sum test did not support H4a, H5 or H6. Table V illustrates the results.
H4a and H4b test whether there is a difference in median price per ton and median price per container, respectively, between small business set-asides and full and open competition contract awards. For H4a, contracts solicited as small business set-asides did not result in significantly higher median prices per ton than contracts that were solicited using full and open competition. In these data, the median price per ton for small business set-asides is US $142/ton, while the median price per ton for full and open competition is US$165/ton. Thus, H4a is not supported. These results are counterintuitive, with the small business set-asides garnering a lower price per ton, although the difference is not statistically significant. Examining the list of bases, it appears there are many reserve bases in the full and open competition group. The lower amount of tonnage produced by reserve bases is likely resulting in a higher price per ton. However, there is a significant difference in the median price per container between small business set-asides and full and open competitions. These results support H4b. The median price per container for small business set-asides is US$1,934/container, while the median price per container for full and open competitions is US$1,432/container. In these data, the buying organization appears to be paying approximately US$500 more per container (across the life of the contract) on small business set-asides. This result again calls for the organizations receiving the service to carefully examine the number of containers they are using, particularly given the fact that the difference in median price per ton was not significant. In other words, it is not the amount of waste disposed that affects the price difference between small business set-asides and full and open competition; rather, it is the number of containers being serviced.
H5a and H5b test the notion that less inclusive small business set-asides would result in higher median price per ton and median price per container, respectively, than a simple total small business set-aside. In these data, the median price per ton for less inclusive small business set-asides is US$142/ton, while the median price per ton for total small business set-asides is US$151/ton. Thus, H5a is not supported. These results are counterintuitive, with the more inclusive set-aside having a higher median price per ton of waste removal, although the difference is not statistically significant. We can find no reason why the more inclusive small business set-aside would cost morethe geographic mix in the two comparison groups is fairly equal (i.e. similar numbers of highand low-cost areas). Examining the price per ton within each group closely warrants a call for using USAF-wide historical data to determine price fair and reasonable. For example, within the more inclusive set-aside category, total small business set-aside, the price per ton at Eielson Air Base, AK is US$230.29, while the price per ton at Joint Base Elmendorf-Richardson, AK is US$81.95. This price difference appears quite large, given the relative proximity of the bases. More information is needed to determine whether the remoteness of Eielson Air Base (and therefore lack of competition) is driving the near tripling of the price.
For H5b, the less inclusive set-asides did result in a slightly higher median price per container (US$2,051/container) than the more inclusive set-asides (US$1,714/container); however, the difference was not statistically significant. Thus, H5b is not supported.
Finally, H6a and H6b test the notion that a sole source small business set-asides would result in higher median price per ton and median price per container, respectively, than the other less inclusive small business set-asides. In these data, the median price per ton for 8(a) sole source set-asides is US$103/ton, while the median price per ton for the other less inclusive small business set-asides is US$171/ton. Although not statistically significant, these results are also counterintuitivethe sole source median price per ton is less than the competed median price per ton amongst less inclusive small business set-asides. Thus, H6a is not supported. The results of H6b show that, although not statistically significantly different, the sole source median price per container (US$2,144/container) is slightly higher than the competed median price per container (US$1,856/container) among less inclusive small business set-asides. Thus, H6b is not supported.
Again, historical data should be used to pinpoint prices that warrant reexamination. In these data, the price per ton at Moody Air Force Base in Valdosta, GA (8(a) competed contract) is US$226.49, while the price per ton at Robins Air Force Base in Warner-Robins, GA (near Macon, 140 miles from Moody and a HUBZone contract) is US$69.43. Further, the cost per container at Moody Air Force Base is US$1,609.35, compared to Robins Air Force Base's US$1,169.90. Are these large price ranges truly fair and reasonable? More market research is needed to answer this question.
The results of H5 and H6 suggest that, once the buying organization has chosen to solicit the requirement using a small business set-aside, the type of set-aside does not affect price per ton nor price per container. This information is critical to the buying organization, as they often try to spread their budgets among the different types of set-asides in order to meet statutory small business administration (SBA) goals. Using less inclusive set-asides may help organizations meet their SBA goals faster, assuming the organization is able to meet the requirements for fair and reasonable pricing.

Performance-related results
Interestingly, we found that neither the ISWM service-nor contracting-related variables affected contractor performance (see Table VI). These results support H7a, but not H7b. The results suggest that there are no differences in quality, cost, schedule or management performance based on: the amount of the service required (i.e. tonnage and containers); the prevailing wage rate in a given area; whether the requirement was solicited and awarded using a small business set-aside or full and open competition; or the size of the competition (i.e. number of offers).
Given the few performance ratings available for small business subcontracting, the ordered logit could not converge; thus, small business subcontracting was removed from the individual DV analysis. We also tested the combined average performance score across the four performance categories (again, exclusive of small business subcontracting); however, the results are not different from those reported. A larger set of performance data is needed to confirm these results. A summarized version of our hypotheses and their related results are presented in Table VII.

Conclusion
We highly encourage the use of data analytics to help category management teams make data-driven decisions concerning the management of spending in their category. This study empirically tested the influence of ISWM service-and contracting-related variables on price. Our analysis yielded that 84 per cent of the variance in prices is accounted for by ISWM service-related variables, whereas contracting-related variables account for less than 1 per cent of the variance in price. These results suggest a change in demand management practices could greatly influence total price paid for ISWM services. When all ISWM service-and contracting-related variables are included in a regression, we find that the number of containers (a service-related variable) has the largest effect on price. Totally, 24 per cent of the influence on price can be credited to the number of containers, whereas the influence from tonnage of waste and wage rate influence combined is less than 1 per cent. This result is particularly important, as it suggests that the USAF may be able to significantly reduce the price of their ISWM contracts simply by managing the number of containers that must be serviced on each installation (i.e. demand management). These results bolster the findings of the category intelligence report research performed by the Facilities and Construction category management team. In their research of the industry, they found that waste removal trucks were a significant cost-driver, given their low gas mileage (2-5 miles per gallon of diesel fuel) (Brady et al., 2016, p. 37). Essentially, the more containers the inefficient truck has to service, the higher the price of ISWM. Importantly, Brady et al. (2016) found that industry best practices, like "container monitoring systems, anaerobic digestion systems, [and] waste dehydration systems" can reduce overall service demand. Other best practices might include adjusting container size to reduce the number of required pick-ups, partnering with local municipalities to further leverage volume discounts and/or performing a lease versus buy analysis on a fleet of fuelefficient trucks (another potential municipality partnering opportunity).
Two contracting-related variables, 8(a) sole source set-aside and number of offers, significantly affect price. Small business set-asides and number of offers both had a 1 per cent influence on price. That small business set-asides increase price is neither particularly Tonnage of waste has a greater effect on price than number of containers N H2b Number of containers has a greater effect on price than wage rate Y H3 Small business set-asides have a greater effect on price than number of offers N H4a Small business set-asides result in a higher price per ton than full and open competition N

H4b
Small business set-asides result in a higher price per container than full and open competition Y

H5a
Less inclusive small business set-asides (i.e. 8(a) sole source, 8(a) competed, HUBZone and SDVOSB) result in a higher price per ton than the more inclusive small business set-aside (i.e. Total Small Business) N

H5b
Less inclusive small business set-asides (i.e., 8(a) sole source, 8(a) competed, HUBZone, and SDVOSB) result in a higher price per container than the more inclusive small business set-aside (i.e., Total Small Business) N

H6a
Among the less inclusive small business set-asides, the sole source set-aside (i.e. 8(a) sole source) results in a higher price per ton than the competed setasides (i.e. 8(a) competed, HUBZone and SDVOSB) N

H6b
Among the less inclusive small business set-asides, the sole source set-aside (i.e. 8(a) sole source) results in a higher price per container than the competed set-asides (i.e. 8(a) competed, HUBZone and SDVOSB) N H7a ISWM service-related variables do not affect contractor performance Y H7b Contracting-related variables affect contractor performance N surprising nor interesting; however, the quantification of the effect on price is important. First, understanding the expected premiums associated with the different types of small business set-asides is important for category management teams for budgetary planning and for determining a proposal price to be fair and reasonable. Second, understanding why the premium exists is important for the category management execution strategy. For example, if the small business price premium for ISWM is simply due to diseconomies of scale at the landfill (i.e. higher landfill fees due to lower volume disposal by a small business), the category management team may choose to intervene to directly negotiate a long-term volume rate with the landfill management company. By truly understanding the price drivers of the service, the category management team can better shape their strategy.
Turning to the number of offers, ironically, the results suggest that as the number of offers increases, the total price also increases. These results are counterintuitive, as the ISWM requirement would typically be subject to the lowest cost technically acceptable source selection method, where price is the main determinant of award. Interestingly, we find no differences in median price per ton between: small business set-asides and full and open competition; less inclusive small business set-asides and the more inclusive total small business set-aside type; and the 8(a) sole source set-aside type and the less inclusive competed set-asides.
Using the same comparison categories, we find only one difference in median price per container: between small business set-asides and full and open competition. These results once again highlight the importance of number of containers as a price driver and suggest that buying organizations can choose to target their small business set-asides without significantly affecting price per ton. Knowing the median prices paid across USAF installations, as well as the difference in the median prices between comparison categories, may help acquisition teams craft their strategies and understand whether or not proposals (bids) represent a relatively good or a relatively bad deal, as compared to historical prices paid. Finally, based on 32 available CPARS observations, we found no difference in performance based on tonnage of waste, number of containers, prevailing wage rate, small business set-aside status and number of offers. More data are needed to confirm these results.
Turning back to Kraljic's matrix, ISWM services would likely be plotted in quadrant II, materials management. Purchases in this quadrant typically have multiple suppliers, most of whom are local to the delivery area. The key performance criteria are cost/price and delivery management, and the decision authority is decentralized. The procurement focus is on leveraging expenses to achieve efficiencies. To achieve those efficiencies, contracting officers, as business advisors to our customers, should use historical data to make datadriven decisions to negotiate and influence total price. Further, efficiencies can be gained by standardizing both the performance work statement and performance measures to enable expedited pre-award and award phases and to achieve common, USAF-wide performance expectations.

Limitations and further research
Like all research, there were limitations to our analyses. Data limitations do not allow us to account for other factors that may affect the price and performance of the ISWM service, such as distance from the USAF installation to the landfill, the cost to dispose of waste in a given geographical area and the size and capacity of the containers and trucks being used to hold, pick up and dispose of the waste. We suggest the USAF category management team capture those variables for future analyses.
Data limitations also limit the generalizability of the Wilcoxon rank sum test results. For adequate statistical power, each comparison group should contain at least 15 observations. That criterion was only met for six of the 12 groups. Additionally, more CPARS data are needed to reach more accurate conclusions concerning contractor performance.
Finally, future research should account for price differences based on relative geographic isolation of the Air Force base, which can affect competition levels. Research should also examine similarities/differences between the ISWM prices paid at Air Force bases, and the prices paid for ISWM in the local municipalities. Notes 1. The small business subcontracting measure in CPARS was not used because only five observations received scores for this measure.
2. Back-transforming the relationship between an inversely transformed IV and a logarithmically transformed DV can be problematic; however, in this research, we are only estimating effect size, so raw coefficient meanings are less important.
research articles have been published in prestigious journals, including Production and Operations Management, Decision Sciences Journal and Interfaces. She has received Defense-Focused Research Award and Excellence in Teaching Awards. Dr Rene G. Rendon is an Associate Professor of Acquisition Management at the Graduate School of Business and Public Policy, Naval Postgraduate School, Monterey, California. Dr Rendon teaches courses in contract management, supply management and program management. His research interests focus on assessing contract management process capability in DoD and industry organizations. Prior to joining academia, he completed a career as an Air Force Acquisition Contracting Officer in major acquisition systems and base operations support. Dr Rendon earned a BBA from Angelo State University, MBA from the University of North Dakota and DBA from Argosy University, Orange County, California.
Dr Javier Salmer on is an Associate Professor in the Operations Research Department at the Naval Postgraduate School where he teaches graduate courses in optimization and operations management. His research focuses in the area of mathematical optimization and its applications, where he has developed multiple applications in civilian and military problems, and published in peer-reviewed journals. He is also the recipient of several research and teaching awards. Before his time with NPS, Dr Salmer on worked for Spanish electric utility Iberdrola and was adjunct professor at the Statistics School of Complutense University of Madrid.
For instructions on how to order reprints of this article, please visit our website: www.emeraldgrouppublishing.com/licensing/reprints.htm Or contact us for further details: permissions@emeraldinsight.com