Measuring preconceived beliefs on the results of overall equipment effectiveness – A case study in the automotive manufacturing industry

Abstract

Purpose

The purpose of the study is to test if it, by the use of a survey methodology, is possible to measure managers' awareness on, and specifically if there exist preconceived beliefs on, overall equipment effectiveness (OEE) results. The paper presents the design of the survey methodology as well as a test of the survey in one case company.

Design/methodology/approach

Actual OEE logs from a case company are collected and a survey on the data is designed and managers at the same case company are asked to answer the survey. The survey results are followed-up by an interview study in order to get deeper insights to both the results of the survey as well as the OEE strategy at the case company.

Findings

The findings show that the managers at this particular case company, on a general level, does not suffer too much from preconceived beliefs. However, it is clear that the managers have a preconceived belief that lack of material is logged as a loss much more often than what it actually is.

Research limitations/implications

The test has only been performed with data from one case company within the automotive manufacturing industry and only the managers at that case company has been active in the test.

Practical implications

The survey methodology can be replicated and used by other companies to find out how aware their employees are on their OEE results and if possible preconceived beliefs exists.

Originality/value

To the authors' knowledge, this is the first attempt at measuring if preconceived beliefs on OEE results exist.

Citation

Bengtsson, M., Andersson, L.-G. and Ekström, P. (2022), "Measuring preconceived beliefs on the results of overall equipment effectiveness – A case study in the automotive manufacturing industry", Journal of Quality in Maintenance Engineering, Vol. 28 No. 2, pp. 391-410. https://doi.org/10.1108/JQME-03-2020-0016

Introduction

Overall equipment effectiveness (OEE) is a commonly used and accepted performance indicator to increase the effectiveness of equipment by reducing losses (Kumar et al., 2013). OEE combines three measures by multiplying the factors of availability, performance and quality. OEE has been pointed out as a key measure for both total productive maintenance (TPM) and lean maintenance (Anvari et al., 2010).

According to Sohal et al. (2010), most existing literature on OEE deals with technical aspects of the measurement and work method, whereas little has been published on the challenges of implementing OEE. Binti Aminuddin et al. (2016) categorize some OEE studies that have been published in four areas: (1) those that present the application of OEE through case studies, (2) those that expand the application scope of OEE, (3) those that explain the relationship between OEE and other performance measures and (4) those that review OEE and explore different approaches to loss classification. Binti Aminuddin et al. (2016) conclude that little empirical research has been conducted on the managerial implications when implementing OEE.

Managerial implications such as commitment and engagement are though often mentioned as important issues when implementing OEE (Ferko and Žnidaršič, 2007; Sohal et al., 2010; Binti Aminuddin et al., 2016). Other issues include misunderstanding of the measurements, not enough training and awareness, and lack of focus (Sohal et al., 2010; Binti Aminuddin et al., 2016). Hedman et al. (2016) indicate that there is a need for studies investigating the level of awareness among decision-makers regarding measurement of OEE and how it affects their long-term decisions.

The main purpose of the study is to test if it, by the use of a survey methodology, is possible to measure managers' awareness on, and specifically if there exist preconceived beliefs on, OEE results. The paper presents the design of the survey methodology as well as a test of the survey. The test will be performed through a case study approach. One case company, operating in automotive manufacturing industry, will serve as empirical evidence. The test will be performed on managers' awareness of their site's measurements of overall equipment effectiveness. The managers' make up of production managers and support function managers, such as maintenance, quality and manufacturing engineering. A secondary purpose of the study is to also investigate the status of the OEE program at the case company to give some general recommendations for improvement.

Theoretical background

This section is divided into four subsections. First, OEE as a measurement and improvement method will shortly be presented; secondly, OEE implementation will be presented followed by a presentation on preconceived beliefs on OEE. The section will end with a summation on the research gaps within the topic.

OEE as a measurement and improvement method

OEE was introduced by Nakajima (1988) as a performance indicator within the TPM concept. The objective of the indicator was to maximize equipment effectiveness by reducing six big losses: equipment failure, setup and adjustments, idling and minor stoppages, reduced speed, process defects and reduced yield (Nakajima, 1988). Several examples can be found visualizing the positive effects from working actively with OEE. For instance, Badiger et al. (2008) present an improvement of OEE from 51.35 to 73.77% of a shot peening machine using small group team activities. Further, Gupta and Vardhan (2016), present a case from manufacturing industry where OEE were increased from 63 to 88% through implementation of activities such as TPM, reduction of setup times and bin change over times, reduction on tool change loss, etc. Lastly, Saleem et al. (2017) present a case in the tyre curing press industry where OEE is increased from 45.36 to 60.38% using failure mode and effect analysis to find the root cause of failures. Soltanali et al. (2018) summarize additional examples on the use of OEE in automotive industries and beyond. Additional practical cases has been presented by, for example: Dal et al. (2000), Bamber et al. (2003), Zammori et al. (2011), Zuashkiani et al. (2011), Tsarouhas (2013a, 2013b, 2019), Benjamin et al. (2015).

The OEE indicator is calculated by multiplying the availability, performance and quality measures of a machine or equipment. Nakajima (1988) presents the following ideal conditions for OEE: availability is greater than 90%, performance is greater than 95% and quality is greater than 99%. This equals an OEE of (0.90 × 0.95 × 0.99) 85%, which is commonly referred to as the world-class level (Jonsson and Lesshammar, 1999). Empirical studies though, such as Ljungberg (1998), Hedman et al. (2016) and Ylipää et al. (2017), visualize that OEE levels in industry in general is a lot lower. Koch (2003), similar to the abovementioned studies, finds OEE levels much lower than the world-class level and explains that this is because of different ways of setting the OEE definitions. What is included in OEE losses and what is not? Ljungberg (1998), for instance, asserts that the original definition of OEE (Nakajima, 1988) does not consider all factors that reduce capacity utilization, which includes planned downtime, lack of material, lack of labor, etc. These, and others, are logged as losses by Koch (2003) and calculated in OEE. The initial six big losses defined by Nakajima (1988) have thus been multiplied in many practical applications.

The disturbances or losses captured by OEE logging and measurements can roughly be divided into two types of losses, namely, chronic and sporadic (Shirose, 1992; Jonsson and Lesshammar, 1999; Dal et al., 2000; Bamber et al., 2003). The chronic losses are usually more complicated, often hidden and considered small (Bamber et al., 2003). Due to the fact that they often have more than one cause, they tend to persist even after improvements have been implemented (Shirose, 1992). Sporadic loss tends to be more obvious since they often occur quickly and cause large deviations from normal state (Shirose, 1992; Bamber et al., 2003). Sporadic losses are often seen as a serious problem, but it is often the chronic losses that accounts for low utilization and large costs (Bamber et al., 2003). In OEE, sporadic losses can for instance be exemplified with lack of personnel, lack of material or larger breakdowns such as machine spindle crashes. Chronic losses in OEE can be exemplified with setup and changeover, tool change, cleaning and autonomous maintenance, quality control, etc.

Implications when implementing and running an OEE program

Sohal et al. (2010), through a case study including six companies, identify challenges to consider when implementing OEE as a measurement and work method: operator involvement, education and competence development, visibility and communication, simplicity in measurement processes, resistive cultures, and top management engagement and commitment. Binti Aminuddin et al. (2016), through an international survey study, also discuss challenges when implementing OEE as a measurement and work method. These include inadequate training and awareness, lack of resources, lack of support from management, resistance from personnel, lack of focus, lack of experienced personnel and lack of standardized systems for OEE calculations (Binti Aminuddin et al., 2016). Ferko and Žnidaršič (2007) present reasons as to why OEE implementation efforts fail, such as a management business strategy that does not drive OEE implementation, preimplementation activities that are poorly planned, untrustworthy data due to poor use of the system support, and lengthy implementation procedures that cause missed deadlines. All four references above present managerial issues that must be taken into consideration both when implementing OEE and when continuously running an OEE program. It is simply not enough to have a good standard definition and a smart way of logging the losses to be successful in implementing and running an OEE program. Managerial issues and understanding and awareness of OEE as a measure and work method is equally important to reduce the risk of, for instance, failing an implementation attempt.

When expanding the view of and considering theories on the implementation of performance measurement systems, it has been pointed out that many efforts for implementation of balanced scorecards fail (McCunn, 1998). Neely and Bourne (2000) argue that the reason for this is twofold: (1) measurement systems are often poorly designed and (2) they are difficult to implement. Bourne et al. (2003), in a literature review, discuss some difficulties when implementing performance measurement systems, such as lack of leadership and resistance to change, poorly defined metrics, significant time and expenses, goals that are negotiated rather than based on stakeholder requirements and unrealistic goals (Bourne et al., 2003).

Preconceived beliefs on OEE results: a hotbed for misunderstandings

There may of course be many causes to why there exist managerial issues, such as misunderstanding, focus and awareness issues on OEE measurements. This paper will focus on ignorance on OEE measurement results and specifically ignorance that may have its roots in the existence of preconceived beliefs that causes misunderstandings and awareness issues.

Something being preconceived has been defined by Merriam-Webster as, “to form (an opinion) prior to actual knowledge or experience” (www.merriam-webster.com). It has been pointed out that preconceived belief is one of the reasons as to why ignorance exist (Gapminder, n.d. Rosling, 2007; Rosling and Härgestam, 2017). Further, statistical facts do not come to people easily and most people understands the world by generalizing personal experience which tends to be biased (Gapminder, n.d.). In media, the news which focus on swift changes gets prioritized over slow and steady changes in major trends (Gapminder, n.d.). This can be compared to the discussion above on chronic and sporadic losses in manufacturing industries.

Further, Bamber et al. (1999), in a discussion on critical factors in implementing TPM, bring up the importance of measuring performance so that managers are given the possibility of basing their decisions on facts and not opinions. Similarly, Gopalakrishnan (2018) and Gopalakrishnan and Skoogh (2018) discuss the importance for, for instance, maintenance managers to base their decisions on facts and to do this through data-driven approaches. As a projection for future maintenance planning, fact-based decision is mentioned as one important area (Bokrantz et al., 2017). Gopalakrishnan (2018), though, states that (in general) current maintenance decisions are seldom fact-based. Similarly, Hedman et al. (2016) through a study with a dataset of OEE measures from 23 companies and 884 machines concludes that, for decision-makers, much of the OEE results are hidden in unclassified losses and that this has negative impact on improvement initiatives. Preconceived belief among managers on various measurements such as OEE is one threat to fact-based decisions. In a way, preconceived beliefs can be seen as the opposite of fact-based decisions.

Research gap found in theoretical background

Bourne et al. (2003) pointed out an existing gap in research concerning managerial issues within implementation of performance measurement systems in the beginning of the millennium. Some years later Sohal et al. (2010) point out a similar gap when implementing OEE in particular. Even later, Binti Aminuddin et al. (2016) state similar findings. Researchers, such as Badiger et al. (2008) and Cheah et al. (2020), have also at various points in time pointed out that there is a lack of systematic approaches/framework to support and guide implementation efforts of OEE. Sohal et al. (2010), though, mean that most existing literature on OEE deals with technical aspects of OEE. For example, many papers has been published on various KPIs (Key Performance Indicators) being developed out of OEE, see summations in for instance: Muchiri and Pintelon (2008), Lanza et al. (2013), Oliveira et al. (2019), Bengtsson et al. (2020). As management issues can be the root cause as to why implementation efforts fail, such as why there is lack of training or lack of focus, it is important that managers (and ultimately of course all employees) have an understanding and awareness of OEE as a measurement and work method and are aware of the results of the measurements being taken. However, Ljungberg (1998) means that much of the OEE data are never compiled or analyzed at all. Similarly, Gopalakrishnan (2018) points out that decisions are seldom fact-based, at least within maintenance. Further, Hedman et al. (2016) mean that there is a need for studies including investigations of awareness among decision-makers on the measurements of OEE. Combining these gaps with the reasoning on preconceived beliefs of Rosling (Gapminder, n.d.; Rosling, 2007; Rosling and Härgestam, 2017), the objective of this paper is to provide industrial companies with a methodology they may use to measure the awareness of OEE results and particularly if preconceived beliefs of the same exist.

Research methodology

This section is divided into two subsections: case company presentation and case study methodology.

Case company

Plant setup

The case company is a discrete item manufacturing company supplying internal customers with components for the automotive industry. During the first six months of 2018, production volumes had been high as seen in a historical perspective. From 2015 until 2018 the production volumes for the first six months had increased by 38%. During the first six months of 2018, the plant had roughly 750 employees. The plant's manufacturing processes include machining, curing, assembling, testing and painting of components such as axles and transmissions. Company functions such as logistics, production planning, manufacturing engineering, maintenance, quality, safety and environment, and human resources support the value stream – that is, production. The plant is divided into various production cells based on product families. The machining-related production cells are divided into two areas: axles and transmissions. Within the machining production cells, a line manager always manages two or more production cells. The manufacturing machines used in the machining-related production cells differ significantly in their type, setup, material being manufactured and age. Machine types include turning machines, machining centers, gear hobbing machines, gear shaving machines, gear grinding machines and induction hardening machines. In total, about 400 manufacturing machines exist within the plant. Some machines are stand-alone, whereas others are situated in a value stream with material handling systems, such as material handling robots, conveyors, and gantry cranes, and others are situated in a storage crane setup. Material comprises cast and forged materials as well as, to a smaller extent, aluminum.

OEE program at the case company

Since the case company has more than 10 years implemented lean manufacturing principles in a structured way. OEE has, in various methods and to various extent, been used since. The current definition of OEE and system being used had at the time of this study been in place for five years. During implementation of the current system being used, a training and education program was performed. All operators that were going to be affected by the OEE program participated in an OEE education and were trained in how they should think when logging losses. All managers were trained in a “train-the-trainer”-fashion so that they would be able to hold training session to new employees. OEE is measured on, at minimum, the bottleneck machine in all production cells that perform machining; however, in some production cells, additional machines are measured depending on the setup of the production cell. In total, OEE is logged on 50 machines at the case company. These 50 machines differs in the same way as all machines in the plant (as explained above) when it comes to type, setup, material being manufactured and age. For instance, the age of the machines differs. A few of the machines are as old as 40 years (these are however renovated), but most machines are installed in the year 2000 and sooner.

The loss categories logged are wider than the original definition proposed by Nakajima (1988) and are closer to the ones proposed by Koch (2003). This implies that for instance lack of orders, lack of material, preventive maintenance and manufacturing engineering-related work are included as losses as input to the OEE calculation.

The operators log the losses manually. The manual log is entered into a computer system that performs the OEE calculation and depicts the data as various graphs. It is possible to look at the OEE measures on an aggregated level (all machines and loss logs), as well as on detailed levels (machine levels). It is also possible to choose time intervals. The data are available for all employees through this computer system. Also, the OEE measures are discussed and analyzed with operators and support team on machine level in the respective production cell on weekly basis. The data are also discussed and analyzed on a comprehensive level with the managers of the production cells that performs machining and with the plant management team, also on a weekly basis.

Case study methodology

The case study consists of a document review, a survey and lastly interviews. The study started with document review of the case company's OEE measurements. From these, a survey, measuring preconceived beliefs on OEE results, was designed and performed. Based on the results of the survey, follow-up interviews were performed with some of the respondents that had answered the survey, this in order to gain a deeper understanding as well as to investigate the status of the OEE program at the case company. All empirical data were collected at the same manufacturing plant.

Document review

The case study data collection began by downloading data from the case company's OEE measurements. Data from the first six months of 2018 were downloaded for all machines on which OEE was measured, which totaled 50 manufacturing machines. The total trend per week and total loss charts were downloaded, as shown in Figures 1 and 2. The average OEE of the 50 machines was 60%.

Survey study

A quick methodology to measure if preconceived beliefs exits were unintentionally developed and introduced by Professor Hans Rosling of the Karolinska Institute of Sweden (at a TED talk in 2006) (Gapminder, n.d.; Rosling, 2007). In one of the questions of a pretest for university students within Global Health, Professor Rosling presented five pair of countries, each composed of one European country and one non-European country. Students were then asked to pick a country from the pair that they thought had the highest child mortality of the two. If the students had guessed randomly they would score an average of 2.5 (50%) correct answers. To Professor Rosling's surprise, his students performed worse than the random average – they only provided 36% correct answers on average. The answers could therefore not have resulted from guesses alone but must have been formed by preconceived beliefs that created and maintained ignorance (Gapminder, n.d.; Rosling and Härgestam, 2017).

In order to investigate whether preconceived beliefs within the employees of a manufacturing plant exist on the results of OEE measurements, it was decided that a similar survey study as used by Professor Rosling would be designed and tested. Instead of combining five pairs of countries, we combined five pairs of OEE losses using actual OEE measurements from the case company to test it on managers within that organization.

When Professor Rosling (Rosling and Härgestam, 2017) selected the five pairs of countries, he ensured that, in each pair, one country had twice as high child mortality as the other to ensure the difference would be much higher than the uncertainty of the data (Rosling, 2007). In our study, we made similar pairings as follows (the losses are marked with “c” for chronic loss and “s” for sporadic loss):

1. Manufacturing engineering-related work (s) vs preventive maintenance (s);

2. Breakdown in equipment. Long stop (s) vs lack of orders (s);

3. Setup (c) vs lack of personnel (s);

4. Meetings and education/training within the department (c) vs waiting on previous or succeeding operation (s);

5. Lack of materials (raw material) (s) vs tool change and quality control (c).

Not all pairs had twice amount of loss but there was still a big difference, see Figure 2. To some extent the losses were paired to include one loss being of chronic and one loss being of sporadic nature. The respondents were asked to answer which of the losses, within each pair, they believe to be the one with the most logged hours in the first six months of 2018. In addition to these five pairs, the respondents were asked to answer what they believed was the average OEE level in the plant during the first six months of 2018. To be used in the analysis, the respondents were also asked to indicate to which function they belonged as well as for how many years they had worked at this particular plant. To ensure anonymity, the respondents were asked to indicate if they belonged to the category of machining, assembly/process or support function (e.g. logistics and maintenance).

Selection of survey respondents. The survey was first performed in the Plant Management Team (PMT) on August 28, 2018, during one of their meetings. The PMT includes managers for functions such as production, manufacturing engineering, quality, logistics, maintenance, human resources, and safety and environment. Six out of a total of eight PMT members answered the survey, while two were absent. The survey was later conducted during a management meeting on September 20, 2018, when a total of 22 out of 28 line managers answered the survey. The line managers also belonged to one of the abovementioned functions. It was chosen to include respondents in the survey from all functions of the plant. The reason for this was to reflect the cross-functionality of OEE, see for instance Bamber et al. (2003), in that it, to a great extent, is the responsibility of all functions to support the value stream with activities to improve OEE it should also be of interest for all to have an idea of the results of the OEE measurements.

Analysis of survey. Answers from the survey were analyzed quantitatively through calculations of average scores, both on the question regarding the average OEE level and on the pair of losses, individually per pair as well as on the total result. In total, 28 respondents, including the PMT and line managers, answered the survey. At the time of the study, there were 36 managers employed within the case company. Three respondents, however, failed to fill out the survey completely, and their answers were disregarded, leaving a total of 25 responses. The 25 respondents represented 71.4% of the total management body at the case company at the time of the study. Eight of the respondents belonged to the machining category, four to the assembly/process category and 13 to the support function category. The analysis was divided into these three categories as well as into two groups based on whether the respondents had worked in the plant for less or more than five years. Seven respondents had worked in the plant for less than five years, and 18 had worked for more than five years.

Interview study

The purpose of the interview study was to gain a deeper understanding on the survey results, a secondary purpose was to investigate the status of the OEE program at the case company. A total of 10 respondents were interviewed. The 10 respondents represented 28.6% of the total management body at the case company at the time of the study. The interviews took, on average, 45 min to complete. Before the interviews began, results of the survey study were shown to the respondents using an MS PowerPoint presentation. During the interview, notes were taken and an immediate check back was performed with the respondent throughout the interview to ensure correct notes had been taken. The questions in the interview study may be considered semi-structured (Lantz, 1993) which gives the respondent room to develop their answers freely. The questions were written down and shown in a MS PowerPoint presentation to the respondents. The interview comprised a total of 11 questions and may be found in Appendix. Questions on data from the survey (question 2–6; see Appendix) were also shown in the presentation.

The questions were formulated after the survey results had been analyzed. The first and last questions were asked to receive general opinions on the survey methodology. Questions 2–6 were asked to gain deeper knowledge on why the respondents had answered as they had on the survey. Questions 7 and 8 were asked to penetrate an interesting finding of the survey regarding the results of the last pair of losses. Questions 9 and 10 were asked in general to further investigate the status of the OEE program at the case company.

Selection of interview respondents. Two area managers of the machining category were interviewed first. After the interview, the respondents were asked who they thought would be a good candidate for additional interviews. An additional four managers from machining, two from assembly/process and two from support functions were added to the list and interviewed. Most respondents belonged to the machining category was purposefully chosen as the OEE measurement are performed in these production cells and by that the respondents were assumed to have most input. According to Maxwell (2005), purposeful sampling is a strategy in which one deliberately selects settings, persons or activities to study in order to provide important information. The interviews were performed between October 22 and November 26, 2018. See Table 1 for more information on the respondents.

Analysis of interviews. Notes from the respondents were transcribed directly after each interview, as encouraged by Ejvegård (1996), and then carefully read through. Notes were taken on similarities and differences. Out of those, different categories of challenges in the case company's OEE program were mapped. Categorizing strategies is, according to Maxwell (2005), one analytic option in qualitative analysis. Further, Maxwell (2005) means that the main categorizing strategy is coding and that coding is not about counting things but to fracture data into different categories to facilitate comparison in order to develop theoretical concepts.

Empirical findings

This section is divided into two subsections. First, empirical findings of the survey are presented, followed by empirical findings of the interview.

Survey

On average, the survey respondents had 12.6 years of experience working at the case company. The seven respondents that had worked at the case company for less than five years had an average of 1.67 years of experience working at the case company. The 18 respondents that had worked at the case company for more than five years had an average of 16.83 years of experience working at the case company. As stated in the Document review section, the true average value of OEE during the first six months of 2018 was 60%. The estimated average value of OEE as answered by the respondents was 55.32%; see Figure 3. The estimated median value was 56%. The lowest estimated value was 40%, and the highest was 66%. Three respondents estimated the average value to be 60%, eight respondents estimated it to be over 60% and 14 respondents estimated it to be below 60%. Respondents from the machining, assembly/process and support function categories estimated the average to be 57.25%, 53.25% and 54.77%, respectively. Respondents with more than five years and less than five years' experience working at the case company estimated the value to be 56.55 and 52.14%, respectively.

Respondents' answers on which pair of losses they thought had the most logged hours were 68.8% correct on average. However, the first four pairs seemed easier for the respondents to answer correctly than the last pair; see Table 2 and the Discussion section. Regarding the pairs, respondents from the machining, assembly/process and support function categories provided 70, 60 and 70.88% correct answers, respectively. Interestingly, respondents with more than five years and less than five years of experience working in the plant answered correctly for 66.67 and 74.29% of the pairs, respectively, see Discussion section.

Interviews

On average, the interview respondents had 13.5 years' experience working in this particular plant. Findings of the interview study are presented question by question. Quotes from the respondents are used to prove the points, which as stated by Befring (1994) is a common approach in qualitative analysis. The quotes are coded by number of respondent corresponding to Table 1. The quotes have been translated from Swedish.

What did you think of the survey?

Some of the respondents stated they found the survey to be: good, interesting, important or eye opening. One respondent stated, “Good, the survey tested the organization's knowledge.” (R5). One respondent argued, “We probably guess too much; here we see the true result.” (R2). On a similar note, one respondent state, “We have facts but still run our operations based on feelings.” (R7). Another respondent stated, “It is quite easy that old truths turn into facts. We probably spend too little time on analyzing the OEE data.” (R10). Other respondents focused more on the results and their future use, as follows: “A lot of people answered correctly. But where are the action-plans to reduce the wastes?” (R9). Some respondents focused on how they had answered the survey: “I lack training and education in OEE. I answered based on hearsay… I used what is said during the daily morning meetings as input [to answer the survey].” (R4).

Do you have any thoughts on why the respondents answered the way they did on the first pair of losses?

For the first pair (manufacturing engineering-related work vs preventive maintenance) 76% of the respondents answered correctly. Although only 50% of the respondents answered correctly in the machining category, 75 and 92% answered correctly in the assembly/process and support function categories, respectively. Interview answers to this question are significantly related to the fact that preventive maintenance stops the machine completely, while manufacturing engineering, which also stops the machines sometimes, often uses them to run prototypes or other products. This, to some extent, might lead managers to believe, mistakenly, that the machine is running planned production when it is not. One respondent answered by stating, “The managers in the machining category probably see manufacturing engineering as production, while preventive maintenance is perceived as more of a disturbance; the machine is shutdown. I believe that people have answered based on their feelings.” (R1). Another respondent, however, stated, “… we probably analyze our losses to a too small extent. This increases the risk that we will prioritize the wrong losses.” (R10). Finally, a third respondent state, “Maybe the managers in the machining category has a preconceived idea that maintenance stops the machines while manufacturing engineering runs the machines….” (R6).

Do you have any thoughts on why the respondents answered the way they did on the second pair of losses?

For the second pair (breakdown in equipment, long stops vs lack of orders), 80% of the respondents answered correctly. Although, only 50% of the respondents in the assembly/process category answered correctly, 100 and 77% of the respondents answered correctly in the machining and support function categories, respectively. Through the interviews, no real explanation was obtained regarding why only 50% of the respondents in the assembly/process category answered correctly. However, the small size of the sample could explain the low percentage. One respondent stated, “Not good; it seems as we lack consensus. This can mean that they [assembly/process category] might start creating their own theories.” (R8).

Do you have any thoughts on why the respondents answered the way they did on the third pair of losses?

For the third pair (setup vs lack of personnel), 80% of the respondents answered correctly. In the machining, assembly/processes and support function categories, respectively, 75, 100 and 77% of the respondents answered correctly. One respondent stated, “… they [assembly/process category] hear that we have lack of personnel at every morning meeting. During the morning meetings, production managers never brings up that they have too much set-up time.” (R1). Another respondent stated, “It could possibly be that we have logged the wrong loss though; some hours might have ended up in lack of personnel by mistake.” (R4). A third respondent stated, “We have a large focus on personnel, number of shifts, and overtime. It was probably easy to guess correctly on this particular pair. We probably need to work smarter; the solution is not always to increase number of employees.” (R6). Finally, a fourth respondent stated, “Lack of personnel is something that is always mentioned during morning meetings, set-up is never mentioned. In meetings when we discuss losses and improvements, we almost exclusively talk about set-up, we never bring up lack of personnel.” (R7).

Do you have any thoughts on why the respondents answered the way they did on the fourth pair of losses?

For the fourth pair (meetings and education/training within the department vs waiting on previous or succeeding operation), 88% of the respondents answered correctly. In the machining, assembly/processes and support function categories, respectively, 88%, 75%, and 92% of the respondents answered correctly. One respondent stated, “I am surprised that not everyone answered correctly; as we have so much disturbances in our machines, it should be obvious that also machines before or after should have a similar amount of disturbance.” (R1). Another respondent stated, “Given our situation in production [i.e. backlog], we often cancel meetings and educations.” (R4).

Do you have any thoughts on why the respondents answered the way they did on the fifth pair of losses?

For the fifth pair (lack of material vs tool change and quality control), only 20% of the respondents answered correctly. In the machining, assembly/processes and support function categories, respectively, 38%, 0%, and 15% of the respondents answered correctly. This was the only pair of losses for which the survey respondents answered completely wrong. One respondent explained this phenomenon: “If we lack material for a week, it gets very expensive, and we might be the reason why our customers have to shut down production. Tool change and quality control is something that is ongoing constantly. If we have problems with lack of material, we often, if it is possible, produce something else or perform, for instance, preventive maintenance; maybe we also change the loss code in our reporting system and thereby reduce the number of hours on lack of material.” (R1). Another respondent stated, “We probably do not think about how much measurement we actually perform. […] We speak more loudly when lack of material happens.” (R9). A third respondent stated, “When we have lack of material, we usually make a big deal out of it; it is a loss that gets quite hyped. This might make us believe that it is a bigger loss than it actually is. This can lead to wrongful decisions.”(R10).

How is (daily/weekly) production planned in the machining groups?

Respondents within the machining category shared that they perform some detailed planning mostly on a daily basis; some even pointed out that they perform this planning shift by shift. They all get support for this planning from a main planning function on a weekly basis. This plan is made based on customer orders and matches what needs to be assembled and shipped to customers. On a detailed level, there are always some incidents, for instance due to lack of personnel or lack of material, which makes it necessary for the machining groups to perform some rescheduling on their own.

How do the employees in the machining groups act when they lack material?

Of the six respondents who belong to the machining category, four stated that they try to run some other article/product if they experience lack of material for a particular article/product. One of these respondents stated, “You manufacture something else so that not a bigger backlog will be built when the material actually arrives.” (R2). Two other respondents of the same category stated that they do not have that much problem with external suppliers of material but, instead, lack material from in-house. A respondent from the assembly/process category stated, “It once happened that our racks were filled with the wrong kind of material […] so that the correct kind of material that we at the time needed could not be manufactured.” (R4). A respondent from the support function category stated, “They manufacture something else. I fully understand that, for instance, when we run five shifts, we do not have the possibility to catch up.” (R10).

Do you see any pros and cons with OEE as it is used today?

None of the respondents challenged the fact that OEE is measured at the plant. However, there are several views on how the OEE program could be improved. One respondent stated, “We do not act enough on low OEE results; the solution is often to expand with an extra shift. Doing so removes the incentive to improve OEE. Increasing the number of shifts also decreases the needed OEE level calculated vis-à-vis production plan.” (R6). The same respondent also mentions, “… a lot of employees probably still believe that OEE is used to measure the operators and not the machines, which might cause wrongful logs.” (R6). Further, a respondent shared his view that “Requests and interest from top management could be better, we do measure OEE but could do it better. Too few in the plant really know the theory on OEE. We put too much trust in very few people.” (R10). Lastly, one respondent state, “Maybe we need to work to improve the understanding of OEE, to increase the knowledge; a lot of people find logging losses difficult.” (R8).

What is your opinion on the fact that OEE data are logged manually? Is there any unregistered time?

For this question, 9 out of 10 respondents focused some part of their answer on whether the measurements would be of better quality with a different system that has more automation. Seven of the respondents would like a system with higher automation in logging. Two respondents expressed their skepticism toward the belief that an automated system would be the solution to any problem from which the OEE program currently suffer. One respondent stated, “… it is important that management constantly show an interest in the measurements.” (R3).

Any last comments on the survey or OEE in general?

Although the respondents made some comments, they mostly summed up the interview. One respondent stated, “OEE must be implemented top-down. The management team must have the understanding and will to work with OEE.” (R1). Another respondent summed up the interview by stating, “The survey almost turned into an education, what we believe and what is fact.” (R4).

Discussion

This section is divided into two subsections. Firstly, the empirical findings regarding if preconceived beliefs at the case company exist are discussed; secondly, the empirical findings regarding the status of OEE at the case company is discussed.

Preconceived beliefs on the results of OEE at the case company

Of all five pairs, the respondents answered correctly on an average of 68.8%. Overall (or at least on these five pairs), respondents at this particular plant may, according to the reasoning of Rosling (Rosling and Härgestam, 2007; Gapminder, n.d.), not suffer too much from preconceived beliefs or ignorance on their OEE results. However, since they gave 31.2% incorrect answers, they do not have full awareness or knowledge about how they are logging the losses of their machines overall. Also, in the interview study, respondents raise issues on the OEE program and uses words such as, “preconceived ideas”, “hearsay”, “feelings”, etc. which point in the direction that respondents themselves believe there are preconceived beliefs on the OEE results. As previously mentioned, Bamber et al. (1999), Bokrantz et al. (2017), Gopalakrishnan (2018) and Gopalakrishnan and Skoogh (2018) stress that managers need to base their decision on facts and to do this through data-driven approaches. The result of the survey in this particular case company show that, even though they are not suffering from preconceived beliefs on their OEE results in general, they are lacking in awareness on the facts provided by the measurements which will obstruct fact-based decisions.

For the first four pairs, the respondents answered correctly on 81% of the pairs, and on the last pair (lack of material vs tool change and quality control), only 20% answered correctly. Thus, it is fair to say that a preconceived belief exists among the managers that lack of material is a bigger problem than it gets logged. The fact that lack of material is a sporadic loss and tool change and quality control is a chronic loss could possibly explain the respondents view on this particular pair of losses. As Bamber et al. (2003) state, sporadic losses are often seen as more serious than chronic. In the other two examples where chronic and sporadic losses had been paired, setup (c) vs lack of personnel (s) and meetings and education/training within the department (c) vs waiting on previous or succeeding operation (s), it was actually the sporadic losses that were most logged. There is thusly not possible to generalize that either category is bigger or smaller than the other.

A question that must be posed at this stage, however is, how wrong did the respondents really answer on the last pair of losses? In the interview study, four respondents, from the machining category, state that they try to run some other article/product when they are faced with lack of material for a particular article/product. This maintains the high (or at least retains the current) OEE level through high effective utilization of the equipment, which, of course, is positive. What is negative is that the loss gets hidden and not prioritized accordingly. This could be considered a shortcoming in OEE as a measure and work method. As long as a company does not stop the machines as soon as there is lack of material – OEE as a measure does not consider compliance with the production plan. Therefore, if the respondents answered the question on the last pair of losses based on what they thought were the biggest loss they might actually be correct, unfortunately there is no way of telling. If the respondents answered the question as intended (and as it was stipulated in the survey instruction) on which loss within the pair, they believed to be the one with most logged hours they have a preconceived belief. Further, as OEE levels are discussed on a weekly basis, both within individual teams as well as on a comprehensive level, it can be concluded that a preconceived belief exists.

It is not very difficult to understand that there is a difference in the number of correct answers from respondents of different categories. This difference can be explained by the fact that respondents in the assembly/process category do not measure OEE on their equipment and have difficulties in answering how operators in other areas of the plant log the OEE data. The fact that respondents in the support functions category answered on the same level (and, in fact, with a higher accuracy) than respondents in the machining category may be explained by the fact that the former have a better overall view of the plant than the latter. The fact that respondents in the machining category did not answer with higher accuracy can be explained by the fact that they were asked to answer on a plant level and not for their specific OEE results. The fact that respondents with less than five years of experience working in the case company answered with a higher accuracy than respondents with more than five years of experience working in the case company can possibly be explained with the fact that they have less preconceived beliefs or that they guess randomly to a higher degree. In the last pair of losses, the respondents with more than five years and less than five years of experience working at the case company answered correctly in 16.7 and 28.6%, respectively. Thus, also on this, what turned out to be the most difficult pair to answer, the respondents with less experience working at the case company answers correct to a higher degree.

Status of OEE program at the case company

Even though the results from the survey, with average correct answers in 68.8% of the pairs and even higher on the first four pairs of losses, should be considered not having too much preconceived beliefs, the respondents' statements hint that there are uncertainties within the plant's OEE program.

Analyzing the qualitative results from the interview study, statements such as the following imply that the plant lacks communication and visualization of the results of the measurements and that this may be a reason to preconceived beliefs: “Maybe there is a preconceived idea that …” (R6); “I believe that people have answered based on their feelings.” (R1); “We probably guess too much; here we see the results.” (R2) and “I answered based on hearsay …” (R4). The results of the survey displayed no preconceived beliefs, at least on the first four pairs of losses. Thus, communication in the plant seems to be at a high level, such as through daily morning meetings. However, based on the last pair of losses, where 80% of the answers were incorrect, and the statements above, results of the OEE measurement can be communicated more clearly. This factor is identified by Sohal et al. (2010) and Andersson and Bellgran (2011) as important in the implementation phase of OEE.

Further, statements such as the following imply that the plant lacks understanding of OEE as well as education and basic training in OEE: “I lack training and education in OEE.” (R4); “Too few in the plant really know the theory on OEE. We put too much trust in very few employees.” (R10) and “Maybe we need to work to improve the understanding of OEE, to increase the knowledge; a lot of employees find logging losses difficult.” (R8). This factor is identified by both Sohal et al. (2010) and Binti Aminuddin et al. (2016) as important for successful implementation of OEE. Bengtsson (2020) exemplifies how education and training of OEE can be performed through a game-based learning approach.

Moreover, statements such as the following imply that the plant lacks management engagement, commitment and requirement setting from the management positions: “Requests and interest from top management could be better…” (R10); “Too often we accept lack of personnel.” (R8); “We do not act enough on low OEE results; the solution is often to expand with an extra shift. Doing so removes the incentive to improve OEE.” (R6) and “OEE must be implemented top-down. The management team must have the understanding and will to work with OEE.” (R1). This factor is mentioned by Ferko and Žnidaršič (2007), Sohal et al. (2010) and Binti Aminuddin et al. (2016) as important for successful implementation of OEE, as well as by Bourne et al. (2003) for successful implementation of performance measurement systems.

Finally, statements such as the following imply that the plant is lacking in performing analysis of OEE measurement as well as performing actions based on the results: “We have facts but still run our operations based on feelings.” (R7); “We probably spend too little time on analyzing the OEE data …”(R10) and “We do not really know what our losses are.”(R2). Ljungberg (1998) states, generally, that much of the OEE data are never compiled or analyzed at all. This is not only a problem of wasted time in logging something that is not being used but it is also difficult to motivate operators to keep logging OEE with high accuracy so that it can be used when there is truly a need to compile and analyze the measurements. There seems to be a similar problem in implementing performance measurement systems. Neely and Bourne (2000) discuss that many companies set up well-developed infrastructures to support performance management, but managers end up not analyzing the data.

Muchiri and Pintelon (2008) conclude that OEE is a valuable measure, which, if used correctly, provides information on lost time and lost production. Further, they state that companies routinely consider providing overtime for existing workers, hiring workers for new shifts and increasing capacity through purchase of new production equipment as soon as a company hits capacity restraint. A similar comment was made in the interview study by one of the respondents. The use of OEE as a tool to reduce losses could very well be the solution to help companies find the capacity in their existing production system.

In this study, similar to what is stated in Shirose (1992), Jonsson and Lesshammar (1999), Dal et al. (2000), Bamber et al. (2003), losses that occur continuously – such as setup and tool change and quality control – are generally not the focus of daily meetings, such as morning meetings. Therefore, these losses might be valued less than their actual value. Contrarily, losses that do not occur as frequently but may give rise to lengthy stops – such as lack of material and personnel – seem to gain a lot of attention at daily meetings. Therefore, they might be valued higher than their actual value, at least in the case of lack of material. One respondent gives voice to this issue, but also states that during improvement meetings and when conducting the loss analysis, the view is the opposite. During these meetings, lengthy stops, such as due to lack of material, are discussed to a smaller degree in comparison with losses that occur frequently, such as due to setup.

Regarding the issue of manual or automated OEE measurement system, Jonsson and Lesshammar (1999) state that data collection should be detailed enough so that it fulfills its objective without being unnecessarily demanding of resources. Dal et al. (2000), however, stress that inaccurate data will lead to lack of credibility, and therefore, it is important to invest time into the data collection process. Muchiri and Pintelon (2008) state that manual data collection, while being low in cost, has low data accuracy; contrarily, automatic data collection is high in cost, but its data accuracy is higher and the data-collection process is simplified. A good point is made in Jones et al. (2002, p. 128), where it is stated that “…technology can assure the availability of data but not guarantee that the data is accurate.” Hedman et al. (2016) report on a study where a dataset of OEE measurements (23 companies and 884 machines over a six-month period) was analyzed. The companies used the same system for automatic measurement of OEE. Even though the system was semi-automated, 19% of the scheduled production time was categorized as unclassified losses. Hedman et al. (2016, p. 132) conclude their study by stating, “It has been found that when the measurement is automated, it is even more important that companies do not distance themselves from managing the detailed characteristics of their manufacturing process.” Our suggestion to this particular case company is to improve the usage of OEE first and then consider automation of data logging as a secondary improvement.

Conclusions

When it comes to the main purpose of this study, it looks as if the suggested survey methodology, built on Rosling's ideas (Rosling and Härgestam, 2017), in measuring if preconceived beliefs on OEE measurements exist, work or rather, it looks as if it cannot be determined that it does not work. The survey methodology and the following interview study did show awareness issues in the management team though and gave the case company insights that they may start to work on in order to improve their OEE program. It is possible that the OEE definition of the case company has influenced the awareness of the OEE results and by such also introduced preconceived beliefs. This is though, by us, not considered an issue as it is the purpose of the survey methodology to find shortcomings in awareness and preconceived beliefs and not to find what is right or wrong in OEE methodology.

Due to the fact that the survey methodology includes pairing of losses from a plants' actual OEE logs it is only possible to perform the survey in that particular plant. If one wishes to run the survey in other plants, OEE logs needs to be collected for those particular plants and the losses needs to be paired built on that particular data. The survey will, as such, (probably) never look the same for different plants. That is not sought after, what is sought after is how the respondents at particular plants answer on their particular pair of losses. This makes number of respondents less important, of course, there needs to be a relevant number of respondents within the particular plants (also to keep anonymity) but there is no need to have enough respondents to claim generalizability. To generalize in general is not the purpose of the survey, the purpose is to generalize within a plant and to investigate the awareness of the OEE measurement and results of that particular plant and if the employees at that plant might suffer from preconceived beliefs. In this study, in this particular case company, we set-out to test the managers' awareness and possible preconceived beliefs. The survey was answered by more than 70% of the managerial body (at the case company) and almost 30% of the managerial body were also interviewed on the results of the survey which reflect awareness of OEE results and potential preconceived beliefs of the same at the case company, at least within the management body. Hopefully, the study will encourage other researchers or company representatives to perform similar investigations if preconceived beliefs exist in other plants, industries or contexts. As an input to this, it is necessary to carefully pair the losses in the survey and consider eventual sources of errors beforehand. In this attempt, the loss category “lack of material” made the results of the survey, at least on the last pair of losses, somewhat questionable. If the pairing of losses would have been performed in another way in this survey the results of the survey would of course also be different. Again, what is interesting is that through the survey and interview study, we were able to find awareness issues and possible preconceived beliefs on the OEE results. This, we most likely also would have found if the pairings had been done differently.

The study revealed a shortcoming in OEE. Or at least a shortcoming in the way this particular case company has setup their interpretation of OEE in that they are considering lack of material as a loss in OEE. As in many instances, it is possible to reroute and reschedule production, particularly within manufacturing industries, as soon as there is lack of a particular material the loss is getting hidden. The solution, of rerouting and rescheduling, helps maintain the OEE level but could possibly cause bigger problem downstream in production and could in fact create one of the worst waste in production, that is overproduction (of a particular article/product). This shows that the use of OEE as a performance measure needs to be balanced with other measures in order for it not to become an imposition in production. Researchers such as Jonsson and Lesshammar, 1999, Dal et al. (2000) and Binti Aminuddin et al. (2016), has previously pointed out that OEE needs to be balanced with additional measures in improvement activities.

When it comes to the secondary purpose of the interview study, in investigating the status of the OEE program of this specific case company, the results are in line with previous studies, at least concerning implementation and running of an OEE program (Ferko and Žnidaršič, 2007; Sohal et al., 2010; Binti Aminuddin et al., 2016). As an improvement suggestion, the case company is encouraged to keep working on the practical application of OEE, both as a measure and work method to reduce losses. This can be done through the following:

1. Increasing communication and visualization;

2. Increasing the understanding of OEE and review training needs of OEE;

3. Working on management commitment and engagement as well as on the requirement setting; and

4. Improving data analysis and then acting upon it.

Future studies within this specific case company will include zooming into individual machines or at least machine groups and design similar surveys to see if operators and team leaders have preconceived beliefs. Another possible future study would be to replicate the study in one or several other case companies, using those companies' specific OEE data to further test the survey methodology.

Figures

Figure 1

Overall equipment effectiveness (OEE) trend per week for the 50 machines measured in the plant

Figure 2

Aggregated loss chart for the 50 manufacturing machines. The loss chart has been normalized in percentage as opposed to hours

Figure 3

Box and whiskers plot on estimations of average overall equipment effectiveness (OEE) levels in the plant, first six months of 2018

Table 1

Information on the respondents in the interview study

Respondent (R)	Function	Number of years of experience of working at the plant	Interview date
R1	Machining	7	22/10/2018
R2	Machining	16	29/10/2018
R3	Machining	19	16/11/2018
R4	Assembly/process	21	19/11/2018
R5	Assembly/process	10	19/11/2018
R6	Support function	15	23/11/2018
R7	Machining	24	23/11/2018
R8	Machining	10	26/11/2018
R9	Machining	0.5	26/11/2018
R10	Support function	12	26/11/2018

Table 2

Results from the survey study

Loss		vs		Loss
Manufacturing engineering-related work, manufacturing of spare parts, working with process fluids, etc.	3.6%	Loss in % of total loss	2.1%	Preventive maintenance
	19	Total no. of answers	6
	76%	Answers in %	24%
Breakdown in equipment. Long stop	17.9%	Loss in % of total loss	8.3%	Lack of orders
	20	Total no. of answers	5
	80%	Answers in %	20%
Set-up	6.8%	Loss in % of total loss	14.4%	Lack of personnel
	5	Total no. of answers	20
	20%	Answers in %	80%
Meetings and education/training within the department	4.7%	Loss in % of total loss	8.5%	Waiting on previous or succeeding operation
	3	Total no. of answers	22
	12%	Answers in %	88%
Lack of material (raw material)	2.5%	Loss in % of total loss	9.8%	Tool change and quality control
	20	Total no. of answers	5
	80%	Answers in %	20%

Appendix Interview questions

1. What did you think of the survey?

2. Do you have any thoughts on why the respondents answered the way they did on the first pair of losses?

3. Do you have any thoughts on why the respondents answered the way they did on the second pair of losses?

4. Do you have any thoughts on why the respondents answered the way they did on the third pair of losses?

5. Do you have any thoughts on why the respondents answered the way they did on the fourth pair of losses?

6. Do you have any thoughts on why the respondents answered the way they did on the fifth pair of losses?

7. How do the machining groups plan the production (daily/weekly)?

8. How do the machining groups act when they lack material?

9. Do you see any pros and cons with OEE as it is used today?

10. What is your opinion on the fact that OEE data are logged manually? Is there any unregistered time?

11. Any last comments on the survey or OEE in general?