The effectiveness of applied learning: an empirical evaluation using role playing in the classroom

Harneel Acharya (School of Engineering and Computing, National University, San Diego, California, USA)
Rakesh Reddy (School of Engineering and Computing, National University, San Diego, California, USA)
Ahmed Hussein (School of Engineering and Computing, National University, San Diego, California, USA)
Jaspreet Bagga (CGI Group, San Diego, California, USA)
Timothy Pettit (Department of Management and Marketing, National University, San Diego, California, USA)

Journal of Research in Innovative Teaching & Learning

ISSN: 2397-7604

Article publication date: 10 December 2018

Issue publication date: 3 December 2019

7329

Abstract

Purpose

The purpose of this paper is to evaluate the effectiveness of role playing as an applied learning technique for enhanced classroom experiences as compared to traditional lecture methods.

Design/methodology/approach

This study uses the pre-test/post-test design to conduct experiments with several control and experimental groups. Subjects are graduate students in an MBA program at a private, non-profit university in a traditional classroom setting.

Findings

Students in the experimental group gained significantly more knowledge (post-test minus pre-test scores) – 45 percent higher – through participation in the role playing exercise as compared to the control group.

Research limitations/implications

This study represents only a single educational discipline explored using a single role playing learning activity. Impacts on the long-term retention of the knowledge should be studied further.

Practical implications

Educators should enhance their classroom experience with more applied learning activities such as role playing in order to increase knowledge gain and potentially longer knowledge retention.

Originality/value

This study uses a customized role playing activity within a business curriculum as one of many applied learning techniques. The value to students was shown by significantly higher gain in knowledge while simultaneously enhancing their enjoyment of the classroom experience to potentially encourage further lifelong learning.

Keywords

Citation

Acharya, H., Reddy, R., Hussein, A., Bagga, J. and Pettit, T. (2019), "The effectiveness of applied learning: an empirical evaluation using role playing in the classroom", Journal of Research in Innovative Teaching & Learning, Vol. 12 No. 3, pp. 295-310. https://doi.org/10.1108/JRIT-06-2018-0013

Publisher

:

Emerald Publishing Limited

Copyright © 2018, Harneel Acharya, Rakesh Reddy, Ahmed Hussein, Jaspreet Bagga and Timothy Pettit

License

Published in Journal of Research in Innovative Teaching & Learning. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode


1. Introduction

In the era of globalization, the functions of business are dynamic, and their success depends on how efficiently companies manage their inter-firm relationships with different channel members within the supply chain. Be it a manufacturing or a service company, supply chain management (SCM) plays a vital role in determining the performance of the business. The current research is focused on preparing supply chain leaders to best establish and maintain proper business relationships through the use of a selected applied learning technique – role playing. This work will compare evolving thought on applied learning vs traditional lectures and recommend an alternative way to prepare future managers to serve their supply chain.

1.1 Background

With the rise in competition in the global market, SCM activities are becoming more dynamic, complex and diversified. Therefore, it has become imperative to improve and enhance the skills of managers to deal with the rise in challenges in the supply chain (Kovacs and Pato, 2014; Kotzab et al., 2018). Further, the pace of change, explosion of knowledge and technological revolution have had a great influence on the field of education, raising awareness of the need for new pedagogical and learning approaches (Bates, 2005; Zhang and Nunamaker, 2003). In order to prepare students as ready professionals, it is vital that instructors of business courses like SCM know how to best prepare their customers – the students.

Motivating students so that they can understand the importance of integrated-topic scenarios is another issue of concern (Chang et al., 2009). Therefore, the courses taught at educational institutions are changing their teaching patterns and utilize comprehensive learning solutions instead of focusing on subject knowledge only (Ten Eyck, 2011; Preziosi and Alexakis, 2011; Tran and Lewis, 2012; Westrup and Planander, 2013; Tabak and Lebron, 2017; Paul and Ponnam, 2018). For instance, in order to determine the quantities to produce, managers need to consider their decisions with regard to demand forecasting, allocation of resources and inventory policy as well as scheduling of manpower and machinery. Hence, it is important that future managers first study different aspects of a decision, then consider them in totality and not in isolation. Several opportunities do exist for advanced learning in applied settings, such as internships, residencies and practical learning courses such as capstone projects with industry clients (Brown et al., 2018; Silva et al., 2018; Bhattacharya and Neelam, 2018). However, this option of applied learning is also available for basic and intermediate courses to meet the changing demands of the student.

A vast body of literature evaluates the effectiveness of different practical and applied approaches to learning that can be embraced by teaching institutes so that they can contribute significantly in producing ready managers. The research by Alessi and Trollip (1985), Martocchio and Webster (1992), Quinn (1996) and Li et al. (2017) shows the use of applied gaming tools in teaching different subjects resulted in better knowledge retention and have acted as useful pedagogical tools supplementing the traditional classroom course material.

In light of the changing business environment and the complex dynamics of SCM, there is a need to alter the current traditional pedagogy and equip educators with modern, comprehensive, applied, scenario-based teaching techniques. Therefore, this project will test the effectiveness and benefits of the technique of role playing as just one technique that can be used to implement applied learning in the field of SCM.

1.2 Problem statement

The development of hands-on skills is challenging for many workers due to lack of coaching, mentoring and applied learning opportunities. In order to improve productivity and implement best practices, businesses require better results than standard classroom lectures. This study consolidates a thorough review of the benefits of applied learning and recommends ways in which traditional classroom sessions can be enhanced by using applied learning solutions:

H1.

An applied learning role playing workshop to teach a SCM concept will yield higher levels of knowledge gain as compared to the same aspect of SCM taught through traditional learning systems.

In the following sections, this paper will present a comprehensive review of literature regarding applied learning, a research methodology, the analysis of the data and final conclusions.

2. Literature review

The rise in globalization and need for innovation have marked significant changes in supply chains. The increasing role of SCM is associated with the rise in integration among different members in the supply chain (Christopher et al., 2011). In the case of a manufacturing chain, the manufacturer does not reach directly to the consumers in order to sell its products, and therefore needs channel partners such as wholesalers, distributors and retailers to reach consumers. Being an intermediary, the retailer transfers information about customer tastes and preferences and feedback about the products to the manufacturer. Hence, resellers become a vital part of a SCM. Therefore, the success of the manufacturer is highly dependent upon the relationship between the manufacturer their channel partners in SCM. This project will explore best practices in educating future supply chain managers on the specific process of establishing and maintaining successful business relationships.

2.1 Effective learning and pedagogy

Effective learning and education is characterized by rigorous coursework and effective pedagogical approaches that aim at enhancing the success level for all students (Rhodes, 2007). Hmelo-Silver and Barrows (2006) assume that teaching is a complex task and the instructor should make use of a repertoire of strategies in order to make the learning effective. Effective education is not just about teaching but learning. To enhance learning, students should be equipped with the optimal level of information and assignments (Preziosi and Alexakis, 2011). This means that the information imparted should be justified and not overly demanding. In other words, if it is too demanding, the students will be discouraged and if it is non-challenging, the interest level of the learners will tend to be shattered.

Instructors should focus on enhanced learning that is possible by being flexible and assimilating new subject content while adopting a variety of pedagogical methodologies (Douglas et al., 2008). It is also vital that the teaching style of the instructor should be effective enough that they can be well understood, and their lessons can be applied by the learners (Gooden et al., 2009). As provided by Niemi et al. (2010), effective education can be developed only when the students are involved in the decision-making process of learning and their experiences and stories as learners are heard. Ferguson et al. (2011) also supported this argument and focused on involving the students’ opinions in generating meaningful dialogue and informed decision making. Instructors should be flexible to adopt changes in their teaching practices after understanding the experiences of students in order to develop an effective learning forum (Robinson and Taylor, 2007).

Another tool toward developing effective learning sessions and pedagogy is the ability to scaffold the learning of the students (Husbands and Pearce, 2012). The scaffolding for learning can be provided by offering intellectual, social and emotional support (James and Pollard, 2011). This move will motivate the students to develop interest in their studies. Kim and Hannafin (2011) have posited that the students who do not develop an interest in the studies and learning process are not able to progress. Real-world scenarios are forums for creating interest, motivation and scaffolding. These techniques culminate into a new teaching method called applied learning.

2.2 Applied learning

Learning is known to be effective when the learner is not a passive recipient of knowledge but is a proactive personality in the entire learning process. Active learning is possible when the learner perceives the existence of a relationship between prior knowledge and new knowledge (Christmas, 2014). Bringing the real world to the classroom is viewed as one of the crucial ways to promote effective learning. Therefore, there is an absolute need to transcend from the traditional approaches and pedagogy to new techniques such that the learners can use their learning to solve existential problems (Hui and Koplin, 2011).

Learning-by-doing is considered an apt approach in today’s technology-driven world, wherein continuous learning should be the premise for overall development of the students (Serbessa, 2006). The pedagogical activity in the case of applied learning should be active and involve concrete processes by focusing on the motto of learning by doing (Schwartzman and Henry, 2009). The focus of effective education should be centered toward the learners wherein the role of the instructor is to guide and facilitate the learner and the learning process and not assert control or discharge lecturers toward the targeted learning goals (Silcock and Brundrett, 2001). With the rise in globalization and changing demands of the business world, the learning approaches of the students may need to be altered to provide the best possible fit. In the light of this dynamism, the student-centered approach has become a necessity while deciding upon different learning styles to apply in the classroom (Hudson, 2009). Moreover, the minds of the learners are complex and have a heterogeneous nature and therefore, an effective classroom should consider the constructivist perspective of learning. In order to develop a constructivist perspective of learning, it is vital to implement a student-centered model of instruction which should blend differentiated curriculum and assessment paradigms (Klein, 2003). The learning theory of constructivism aims at making learning meaningful. As per this perspective, learning is a self-directed sociocultural process in which the instructor assumes the role of a facilitator (Tobin and Tippins, 1993).

2.3 Tools of applied learning in SCM

SCM education techniques are similar to those of all other fields. As suggested by Phillips (2005), applied learning can be achieved through chats, role plays and case studies for application-based education (Miller et al., 2016). Further, additional tools of applied learning adopted by Ruey (2010) include submitted artifacts, interviews, surveys and observations. Simulation is another technique used for teaching useful learning concepts using applied learning (Ten Eyck, 2011). Instructors can adopt different gaming tools to facilitate real-time applied learning in teaching SCM, logistics management and business relationship management (Lewis and Maylor, 2007; Quinn, 1996). These gaming tools are effective in motivating active participation in the course as they foster competition and add excitement in the learning process (Elgood, 1997). Further, Dempsey et al. (1996) established that games lead to a playful learning environment that is goal-oriented, challenging and promoting competitiveness.

Using simulation tools and models in teaching SCM is more insightful and helps to evaluate alternatives and outcomes of supply chains which further enhances performance (Persson and Araldi, 2009). Moreover, it becomes possible to use quantifying measures for problem solving, which is a more reliable and objective approach in teaching (Katsaliakia et al., 2014). Simulation-based models should be used with caution, however, as the learners may find it difficult to understand the use and importance of these pedagogies (Chwif et al., 2000). Therefore, it is imperative that the learners become familiar with the use of these applied learning tools and have the competence for successfully implementing these models (Katsaliakia et al., 2014). Hence, the role of the instructor becomes important in explaining the need and the correct way to implement these applied learning tools so that the learning benefits can be maximized. The most popular simulation game used while teaching SCM is the MIT Beer Game, in which production and distribution in a multi-stage distribution channel is facilitated (Chang et al., 2009).

2.4 Use of role playing as an effective tool for applied learning

Role playing as an active learning technique involves a high level of participation from students (Howell, 1991; Westrup and Planander, 2013; Barnabe, 2016; Tabak and Lebron, 2017). Role playing is a group activity involving more than one person who assume different roles in a given situation (Rao and Stupans, 2012) with the aim to acquire learning experiences (Sogunro, 2004). As per Yardley-Matwiejczuk (1997), role playing puts the people in “as if” situations through simulations and actions depending on specific events and circumstances such that different behaviors, roles and arguments directly influence and deepen the learning experience. The preparatory part of role playing is crucial as it involves establishing the descriptions of roles along with the pre-requisites of the involved participants (Westrup and Planander, 2013).

Learning in role playing is facilitated by observing as well as acting out the series of events happening in the respective situation. Role playing helps students to understand the dilemmas in situations and highlight the values of interpretation, which are not possible to study in the traditional lecture mode of teaching (Bryant and Darwin, 2004). As per Rao and Stupans (2012), role playing helps to provide a better understanding of situations and augments that are critical to the subject matter. Yang and Quadir (2018) found significant evidence to support language learning during the role playing game. Further, McCarthy and Anderson (2000) undertook a study to measure the effectiveness of role playing as compared with the traditional methods of teaching and concluded that the active learning tool of role playing leads to better absorption and retention of information. Not only is it more effective, role play provides a variation from the usual lecture-based pedagogy and thus provides an applied and practical aspect to the theory taught in the classroom (Sogunro, 2004). Role playing is also recognized as a pedagogical tool that helps to depict the manner in which one acts, reacts and communicates in a social interaction (Daly et al., 2009). Thereby along with critical thinking, it facilitates enhancement of communication and inter-personal skills. Further, as per Rao and Stupans (2012), role playing provides deeper insight about the business world and the students who take part in it can learn about the situations, issues, alternatives and methods to tackle later in their working life. This means of pedagogy hones the skill sets of students that are required in future employment (Ruhanen, 2005). Sogunro (2004) posited that role playing helps students view a specific situation from different perspectives and not just the single perspective of the instructor or any individual. Therefore, it can be said that role playing acts as an important active learning tool. However, there are drawbacks in regard to student learning as related to the different knowledge background of each student (Qiu et al., 2018). Another study explored role playing for learning in the marketing field to help students create favorable customer experience without direct supervision from the instructor (Paul and Ponnam, 2018). Business ethics is another well-suited educational realm for role playing learning (Sauser and Sims, 2018). Role playing also helps students to understand the feelings, values, attitudes and body language that provide the context of the business situation at hand. This is very valuable in business education as role playing supports cross-sectoral integration and help students experience the complexity in decision-making (Ferrero et al., 2018). Moreover, as per Alkin and Christie (2002), role playing is a useful tool in teaching conflict resolution and preparing students with the necessary skills required for effective conflict management.

2.5 Knowledge retention

According to the Accreditation Council for Pharmacy Education (nd), schools and colleges should make use of active learning methods so that critical thinking, problem-solving skills and better retention of knowledge can take place. There have been many researchers who concluded that there is a positive relationship between the collaborative and applied learning approaches and higher cognitive and affective outcomes (Tran and Lewis, 2012; Chang et al., 2009; Li et al., 2017; Ferrero et al., 2018). The research has been supported by Johnson and Johnson (2009) who posit that applied learning tools augment the achievement rates of the students and increase their knowledge retention. It has also been shown that cooperative simulations like role playing lead to better academic, social and psychological advantages (Johnson and Johnson, 2005). As per Beck and Chizhik (2008) and Zain et al. (2009), academic performance improves drastically when the pedagogy contains applied learning tools.

The pre-test and post-test experimental method is the best method to evaluate the performance of students to adjust for previous knowledge of the learners while directly measuring the gain of knowledge for each student (Kilic, 2008; Doymus et al., 2010). The growing literature continues to support applied learning techniques, and this study explores one such option in the realm of SCM education.

3. Methodology

The experiment for this project was conducted with 91 graduate students at a private, non-profit university over the period of 1.5 years with seven different sections of the same course, each course lasting four weeks. These seven classes were pre-selected to approximately divide the sample into control and experimental groups. The control group was comprised of 59 students who were first given a pre-test on the subject matter, followed by the traditional mode of face-to-face lecture by the instructor using a PowerPoint presentation, and finally each student was given a post-test to capture their knowledge gained. The experimental group had 32 students, who were also given a pre-test and then the learning was facilitated using the applied learning technique of role playing, and again a post–test was conducted to assess the knowledge gain. The Appendix shows the questions used for the evaluations, with post-test questions re-sorted.

3.1 Data collection

Traditional teaching for the Control Group used a 30-min lecture on the subject of business relationships in the context of SCM, focusing on the partnership model (see Lambert et al., 2004, 2010). In contrast, the applied learning was facilitated using a role playing exercise for the experimental group. The exercise required a real-world business relationship consisting of a seller and a buyer, as selected by the students. For example, one student group self-selected the Coca-Cola Company selling soft drinks to the McDonald’s Corporation – guidelines were that both companies be familiar to all students in the class. The role playing exercise took 45 min, in which the first 10 min explained the research study and the partnership model. Thereafter, the experimental classes were randomly divided into two groups, whereby the first group played various roles in the Coca-Cola Company (e.g. the supplier), such as the President/Chief Executive Officer, Vice President for Supply Management, Vice President for Operations and Vice President for Distribution, while the other group performed similar roles for the McDonald’s Corporation (e.g. the buyer). Both the groups were then given time to find background data on the company, such as identifying the actual president or CEO, location of the headquarters, number of employees, number of stores, revenues and the mission statement of the company. After this, based on the partnership model, the major drivers – “compelling reasons for partnering with the other firm” (Lambert et al., 2004, p. 23) – were identified and scored by each team. Once the drivers were identified by the teams separately, a facilitator session was conducted, whereby both the teams were asked to rate the facilitators jointly. They then selected the proper level of business relationship based on the partnership model and discussed the appropriate actions that these companies should undertake toward increasing performance otherwise not obtainable without a partnership. With such a process, students played an active role in gathering the data and reaching their own conclusions based on their active participation.

3.2 Hypotheses testing

In order to meet the research objectives, the research hypotheses were framed as:

H2(a/b).

There is a gain in knowledge during the learning activities (lecture and workshop).

H3.

There is a greater gain in knowledge for the workshop as compared to the lecture learning activities.

Figure 1 shows the knowledge connections before and after the learning experience for both groups. It should be noted that scoring was evaluated on a scale of 0 to 6 (see Appendix for sample instrument) with partial credit given as appropriate in increments of 1/2, 1/3 and 1/6 points based on the question, so continuous variables were assumed. However, results were also confirmed with the non-parametric Mann-Whitney U-tests used when the data are ordinal or when the assumptions of the t-test are not met (Boslaugh and Watters, 2008).

4. Analysis

This section analyzes the knowledge acquired during the learning sessions, followed by a comparison of knowledge gain between the control and experimental groups. After all experiments were conducted, data were reviewed to test assumptions of normalcy and continuality. Figures 2 and 3 show slight skewness on the lower end for the control group and slight skewness on the upper end for the experimental group, with no outliers except for a single perfect score in the experimental group (role playing).

For a statistical test of the normality assumption, the Shapiro-Wilk Normality Test (Shapiro and Wilk, 1965) was performed, with all four data sets as acceptable at the p<0.05 level (see Table I), which along with group sample size over 30, the normality assumption holds.

It was next appropriate to verify the equivalency of the groups before making any comparisons. This evaluation of the experimental group as compared to the control group must be done prior to the implementation of the treatment. Therefore, the pre-test scores were used as a direct measure of the level of knowledge of subjects prior to the experimental treatment (e.g. students may have had previous experience with the material or may have reviewed course material prior to the class). The below student’s t-test for two-samples assuming unequal variances verifies this assumption in that the there is no statistically significant difference in means between the two group’s pre-test average scores (see Table II).

4.1 Evaluation of learning

The next step was to compute the gain in knowledge from both the traditional and applied learning techniques, using each students’ pre- and post-test scores. To verify that learning occurred in each instructional format, scores were tested again using a student’s t-test but this time for paired samples as each subject’s pre- and post-test scores are not independent. The results obtained for the lecture and workshop are given in Tables III and IV; using each group’s average pre-test score as baseline, the control group scored 53 percent higher after the traditional lecture while the experimental group scored 76 percent higher after their role playing activity. A graphical interpretation can be seen in Figures 4 and 5. With these results (Tables II–IV), the experiment is considered valid and H2 can be validated for both groups so further analysis can proceed to evaluate H3, which is to compare between the groups.

4.2 Comparison of learning methods

To directly evaluate H3 as to the comparison between learning methods, a two-sample comparison assuming unequal variances was utilized. Since the previous analyses confirmed that the control group and experimental group were equivalent prior to their, respectively, learning activities and that learning did occur in both groups, Table V details the analysis showing a statistically significant higher level of learning by the experimental group (p<0.10).

Using this role playing applied learning activity, students in the experimental group reported a 45 percent gain in learning in comparison to those attending traditional lectures. To further evaluate this result based on the relatively small sample and moderate variation as expected in any human-subjects experiment, the effect size should also be considered. A standard Cohen’s d can be computed if the standard deviations of the two samples are roughly the same and therefore assumed they are estimating a common population standard deviation. The computed effect size for this experiment using Cohen’s d was 0.360, yielding a relatively small effect size as d<0.50 (Cohen, 1988):

Cohen s d = μ 1 μ 2 σ pooled ,
where σ pooled = ( σ 1 2 + σ 2 2 ) / 2 .

However, the standard deviations of the two groups did differ slightly (1.094 and 1.398, respectively) so the homogeneity of variance assumption is questionable and pooling the standard deviations may not be appropriate. One solution is to insert the standard deviation of the control group into the equation and calculate the Glass’ Δ (Glass et al., 1981), assuming that the standard deviation of the control group is untainted by the effects of the experiment and will therefore reflect more closely the true population variance. “The strength of this assumption is directly proportional to the size of the control group. The larger the control group, the more it is likely to resemble the population from which it was drawn” (Ellis, 2009). With the Control Group containing n=51, the Glass Δ was 0.413, yielding a similar small effect size as the Cohen’s d for this relatively small experiment:

Glass Δ = μ 1 μ 2 σ control .

And finally, it should also be considered that the experimental group (n=30) was smaller than the control group (n=51). Taking this into account, the Hedges’ g will account for sample size differences by weighting the variances by their respective sample size (Hedges, 1981). Hedges’ g was 0.372, re-confirming again the small effect size:

Hedges g = μ 1 μ 2 σ pooled * ,
where σ pooled * = ( ( n 1 1 ) σ 1 2 + ( n 2 1 ) σ 2 2 ) / ( n 1 + n 2 2 ) .

5. Recommendations and conclusion

From the analysis of this experiment comparing the learning benefits of traditional lectures vs applied learning using role playing, it was first verified that there was a gain in immediate knowledge in both the modes of learning, confirming H2(a/b). The goals of this study were met as the analysis showed significantly greater gain in knowledge (post-test verses pre-test for each student) for the role playing groups (1.463 average gain in score) as compared to those exposed only to the traditional lecture (1.012 average gain in score). This represents a valuable 45 percent improvement in learning scores. Therefore, it is determined that the time investment in preparing and executing the role playing activity was of significant value to the customer – the students.

5.1 Limitations

This research was conducted using a single type of Applied Learning – role playing – and using a single discipline within a business school. In addition, the application was tested in the graduate-level educational environment, so findings are not generalizable to other student levels or fields of study. However, findings are in-line with various other studies that reported similar benefits of applied learning techniques (Brown et al., 2018; Silva et al., 2018; Bhattacharya and Neelam, 2018). Larger samples using a variety of applications should continue to be explored.

5.2 Scope for future research

Applied learning has a vast potential in the fields of training and education. The scope for this research has been limited to one topic area (SCM) and one tool of applied learning (role playing). The learning assessment measures in this study (i.e. pre- and post-test) were comprised of only a few multiple-choice questions, but real learning may be better captured using a more extensive test including essay questions that are application-based. It is also possible to conduct a qualitative assessment on the participants to study their subjective experience of applied learning in comparison with traditional learning. Further, in the current research, the independent variable of gain in knowledge was considered, but other variables like the learning experience, quality of learning, time efficiency of learning and the capability to implement the concepts in business and managerial situations should also be considered. And finally, long-term retention of knowledge gained in the classroom may be of even greater importance to meeting the goals of education; longitudinal studies could be conducted at the one-month, six-month and one-year milestones as done with many alumni surveys that attempt to assess the value of the education as assessed after graduation. Because the scope of research in the field of applied learning is vast, continued study and implementation of applied learning techniques is highly recommended.

Figures

Visualization of hypotheses

Figure 1

Visualization of hypotheses

Data visualization from control group (Traditional lecture)

Figure 2

Data visualization from control group (Traditional lecture)

Data visualization from experimental group (role playing)

Figure 3

Data visualization from experimental group (role playing)

Histogram of student learning for the control group (traditional lecture)

Figure 4

Histogram of student learning for the control group (traditional lecture)

Histogram of student learning for the Experimental Group (Role Playing)

Figure 5

Histogram of student learning for the Experimental Group (Role Playing)

Test for normality – Shapiro-Wilk normality test (W)

Control group Experimental group
Sample size 51 31
Pre-test 0.9461** 0.8679***
Post-test 0.9381** 0.9186**

Notes: N reduced due to subjects with an incomplete Pre- or Post-test. *p<0.10; **p<0.05; ***p<0.01

Comparison of groups – pre-test scores

Control group Experimental group
Mean 1.911 1.933
Variance 1.408 0.711
Observations 51 30
Hypothesized mean difference 0
df 76
t-Stat 0.0995
P(Tt) two-tail 0.9210
t critical two-tail 1.9917

Verification that learning occurred – control group (traditional lecture)

Pre-test Post-test
Mean 1.911 2.923
Variance 1.408 1.459
Observations 51 51
Pearson correlation 0.582
Hypothesized mean difference 0
df 50
t Stat −6.603
P(Tt) one-tail 0.000
t critical one-tail 1.676

Verification that learning occurred – experimental group (role playing)

Pre-test Post-test
Mean 1.933 3.397
Variance 0.711 1.600
Observations 30 30
Pearson correlation 0.167
Hypothesized mean difference 0
df 29
t Stat −5.732
P(Tt) one-tail 0.000
t critical one-tail 1.699

Comparison of groups – gain in knowledge (post- minus pre-test scores)

Control group (traditional lecture) Experimental group (role playing)
Mean 1.012 1.463
Variance 1.198 1.956
Observations 51 30
Hypothesized mean difference 0
df 50
t Stat −1.516
P(Tt) one-tail 0.068
t critical one-tail 1.676

References

Accreditation Council for Pharmacy Education (n.d.), “Accreditation standards and guidelines for the professional program in pharmacy leading to the doctor of pharmacy degree”, Version 2.0., available at: www.acpe-accredit.org/pdf/FinalS2007Guidelines2.0.pdf (accessed March 5, 2016).

Alessi, S. and Trollip, S. (1985), Computer–Based Instruction: Methods and development, Prentice-Hall, Upper Saddle River, NJ.

Alkin, M. and Christie, C. (2002), “The use of role-play in teaching evaluation”, American Journal of Evaluation, Vol. 23 No. 2, pp. 209-218.

Barnabe, F. (2016), “Policy development and learning in complex business domains: the potentials of role playing”, International Journal of Business and Management, Vol. 11 No. 12, pp. 15-29.

Bates, T. (2005), Teachnology, E-learning and Distance Education, Routledge, New York, NY.

Beck, L. and Chizhik, A. (2008), “An experimental study of cooperative learning in CS1”, Proceedings of the 39th SIGCSE Technical Symposium on Computer Science Education, New York, NY, pp. 205-209.

Bhattacharya, S. and Neelam, N. (2018), “Perceived value of internship experience: a try before you leap”, Higher Education, Skills and Work-Based Learning, available at: www.emeraldinsight.com/doi/full/10.1108/HESWBL-03-2018-0025

Boslaugh, S. and Watters, P.A. (2008), Statistics in a Nutshell: A Desktop Quick Reference, OReilly Media, Sebastopol, CA.

Brown, C., Willett, J., Goldfine, R. and Goldfine, B. (2018), “Sport management internships: recommendations for improving upon experiential learning”, Journal of Hospitality, Leisure, Sport & Tourism Education, Vol. 22 No. 22, pp. 75-81.

Bryant, J. and Darwin, J. (2004), “Exploring inter-organisational relationships in the health service: an immersive drama approach”, European Journal of Operational Research, Vol. 152 No. 3, pp. 655-666.

Chang, Y., Chen, W., Yang, Y. and Chao, H. (2009), “A flexible web-based simulation game for production and logistics management courses”, Simulation Modelling Practice and Theory, Vol. 17 No. 7, pp. 1241-1253.

Christmas, D. (2014), “Authentic pedagogy: Implications for education”, European Journal of Research and Reflection in Educational Sciences, Vol. 2 No. 4, pp. 51-57.

Christopher, M., Mena, C., Khan, O. and Yurt, O. (2011), “Approaches to managing global sourcing risk”, International Journal of Supply Chain Management, Vol. 16 No. 2, pp. 67-81.

Chwif, L., Barretto, M. and Paul, R. (2000), “On simulation model complexity”, Proceedings Winter Simulation Conference, IEEE, December, Orlando, FL, pp. 449-455.

Cohen, J. (1988), Statistical Power Analysis for the Behavioral Sciences, Routledge Academic, New York, NY.

Daly, A., Grove, S., Dorsch, M. and Fisk, R. (2009), “The impact of improvisation training on service employees in a European airline: a case study”, European journal of marketing, Vol. 43 No. 3, pp. 459-472.

Dempsey, J., Lucassen, B., Haynes, L. and Casey, M. (1996), “Instructional applications of computer games”, Technical Report No. ED394500, University of South Alabama, Mobile, AL.

Douglas, A., Miller, B., Kwansa, F. and Cummings, P. (2008), “Students’ perceptions of the usefulness of a virtual simulation in post- secondary hospitality education”, Journal of Teaching in Travel and Tourism, Vol. 7 No. 3, pp. 1-19.

Doymus, K., Karacop, A. and Simsek, U. (2010), “Effects of jigsaw and animation techniques on students’ understanding of concepts and subjects in electrochemistry”, Educational Technology Research and Development, Vol. 58 No. 6, pp. 671-691.

Elgood, C. (1997), Handbook of Management Games and Simulation, 6th ed., Gower Publishing, Aldershot.

Ellis, P. (2009), “Effect size equations”, available at: www.polyu.edu.hk/mm/effectsizefaqs/effect_size_equations2.html (accessed December 5, 2018).

Ferguson, D., Hanreddy, A. and Draxton, S. (2011), “Giving students voice as a strategy for improving teacher practice”, London Review of Education, Vol. 9 No. 1, pp. 55-70.

Ferrero, G., Bichai, F. and Rusca, M. (2018), “Experiential learning through role-playing: enhancing stakeholder collaboration in water safety plans”, Water, Vol. 10 No. 2, pp. 227-237.

Glass, G., Smith, M. and McGaw, B. (1981), Meta-Analysis in Social Research, Sage Publications, Beverly Hills, CA.

Gooden, D., Preziosi, R. and Barnes, F. (2009), “An examination of Kolb’s learning style inventory”, American Journal of Business Education, Vol. 2 No. 3, pp. 57-62.

Hedges, L. (1981), “Distribution theory for glass’s estimator of effect size and related estimators”, Journal of Educational Statistics, Vol. 6 No. 2, pp. 107-128.

Hmelo-Silver, C. and Barrows, H. (2006), “Goals and strategies of a problem-based learning facilitator”, The Interdisciplinary Journal of Problem-Based Learning, Vol. 1 No. 1, pp. 21-39.

Howell, J. (1991), “Using role-play as a teaching method”, Teaching Public Administration, Vol. 12 No. 1, pp. 69-75.

Hudson, J. (2009), Pathways Between Eastern and Western Education, Information Age Publishing Inc, Charlotte, NC.

Hui, F. and Koplin, M. (2011), “The implementation of authentic activities for learning; a case study in finance education”, E-Journal of Business Education and Scholarship of Teaching, Vol. 5 No. 1, pp. 59-72.

Husbands, C. and Pearce, J. (2012), What Makes Great Pedagogy? Nine Claims from Research. Research and Development Network National Themes: Theme One, National College for School Leadership, Nottingham.

James, M. and Pollard, A. (2011), “TLRP’s ten principles for effective pedagogy: rationale, development, evidence, argument and impact”, Research Papers in Education, Vol. 26 No. 3, pp. 275-328.

Johnson, D. and Johnson, R. (2005), “New developments in social interdependence theory”, Genetic, Social, & General Psychology Monographs, Vol. 131 No. 4, pp. 285-358.

Johnson, D. and Johnson, R. (2009), “An educational psychology success story: Social interdependence theory and cooperative learning”, Educational Researcher, Vol. 38 No. 5, pp. 365-379.

Katsaliakia, K., Mustafeeb, N. and Kumar, S. (2014), “A game-based approach towards facilitating decision making for perishable products: an example of blood supply chain”, Expert Systems with Applications, Vol. 41 No. 9, pp. 4043-4059.

Kilic, D. (2008), “The effects of the jigsaw technique on learning the concepts of the principles and methods of teaching”, World Applied Sciences Journal, Vol. 4 No. 1, pp. 109-114.

Kim, M. and Hannafin, M. (2011), “Scaffolding 6th graders’ problem-solving in technology-enhanced science classrooms: a qualitative case study”, Instructional Science: An International Journal of the Learning Sciences, Vol. 39 No. 3, pp. 255-282.

Klein, P. (2003), “Rethinking the multiplicity of cognitive resources and curricular representations: alternatives to ‘learning styles’ and ‘multiple intelligences”, Journal of Curriculum Studies, Vol. 35 No. 1, pp. 45-81.

Kotzab, H., Teller, C., Bourlakis, M. and Wünsche, S. (2018), “Key competences of logistics and SCM professionals – the lifelong learning perspective”, Supply Chain Management: An International Journal, Vol. 23 No. 1, pp. 50-64.

Kovacs, Z. and Pato, B. (2014), “Jobs and competency requirements in supply chains”, Procedia – Social and Behavioral Sciences, Vol. 109 No. 8, pp. 83-91.

Lambert, D., Knemeyer, A. and Gardner, J. (2004), “Supply chain partnerships: model validation and implementation”, Journal of Business Logistics, Vol. 25 No. 2, pp. 21-24.

Lambert, D., Knemeyer, A.M. and Gardner, J. (2010), Building High Performance Business Relationships, Supply Chain Management Institute, Sarasota, FL.

Lewis, M. and Maylor, H. (2007), “Game playing and operations management education”, International Journal of Production Economics, Vol. 105 No. 1, pp. 134-149.

Li, K., Hall, M. and Bermell-Garcia, P. (2017), “Measuring the learning effectiveness of serious gaming for training of complex manufacturing tasks”, Simulation & Gaming, Vol. 48 No. 6, pp. 770-790.

McCarthy, J. and Anderson, L. (2000), “Active learning techniques versus traditional teaching styles: two experiments from history and political science”, Innovative Higher Education, Vol. 24 No. 4, pp. 279-294.

Martocchio, J. and Webster, J. (1992), “Effect of feedback and cognitive playfulness on performance in microcomputer software training”, Personnel Psychology, Vol. 45 No. 3, pp. 553-578.

Miller, K., Hill, C. and Miller, A. (2016), “Bringing lean six sigma to the supply chain classroom: a problem-based learning case”, Decision Sciences Journal of Innovative Education, Vol. 14 No. 4, pp. 382-411.

Niemi, R., Heikkinen, H. and Kannas, L. (2010), “Polyphony in the classroom: reporting narrative action research reflexively”, Educational Action Research, Vol. 18 No. 2, pp. 137-149.

Paul, R. and Ponnam, A. (2018), “Teaching customer experience quality and its significance in retail management: a role playing game using Chinese Puzzle ‘Tangram’”, Decision Sciences Journal of Innovative Education, Vol. 16 No. 2, pp. 126-139.

Persson, F. and Araldi, M. (2009), “The development of a dynamic supply chain analysis tool – integration of SCOR and discrete event simulation”, International Journal of Production Economics, Vol. 121 No. 2, pp. 574-583.

Phillips, J. (2005), “Strategies for active learning in online continuing education”, Journal of Continuing Education in Nursing, Vol. 36 No. 2, pp. 77-83.

Preziosi, R. and Alexakis, G. (2011), “A comparison of traditional instructional methods and accelerated learning methods in leadership education”, International Leadership Journal, Vol. 3 No. 1, pp. 79-89.

Qiu, T., Liu, H. and Yin, E. (2018), “Learners’ experiences on role-playing collaborative learning supported by ELS: a case study of virtual company program”, International Conference on Blended Learning, Springer, Cham, pp. 275-286.

Quinn, C. (1996), “Designing an instructional game: reflections on quest for independence”, Education and Information Technologies, Vol. 1 Nos 3-4, pp. 251-269.

Rao, D. and Stupans, I. (2012), “Exploring the potential of role-play in higher education: development of a typology and teacher guidelines”, Innovations in Education and Teaching International, Vol. 49 No. 4, pp. 427-436.

Rhodes, T. (2007), “Accelerated learning for what?”, Peer Review, Vol. 9 No. 1, pp. 9-12.

Robinson, C. and Taylor, C. (2007), “Theorizing student voice: values and perspectives”, Improving Schools, Vol. 10 No. 5, pp. 5-17.

Ruey, S. (2010), “A case study of constructivist instructional strategies for adult online learning”, British Journal of Educational Technology, Vol. 41 No. 5, pp. 706-720.

Ruhanen, L. (2005), “Bridging the divide between theory and practice: experential learning approaches for tourism and hospitality management education”, Journal of Teaching in Travel and Tourism, Vol. 5 No. 4, pp. 33-51.

Sauser, W.I. Jr and Sims, R.R. (2018), “Showing business students how to contribute to organizational cultures grounded in moral character”, in Khosrow-Pou, M., Clarke, S., Jennex, M., Becker, A. and Anttiroiko, A. (Eds), Business Education and Ethics: Concepts, Methodologies, Tools, and Applications, IGI Global, Hershey, PA, pp. 485-507, available at: www.igi-global.com/pdf.aspx?tid=186560&ptid=179832&ctid=15&t=Editorial%20Advisory%20Board

Schwartzman, R. and Henry, K. (2009), “From celebration to critical reflection: charting the course of scholarship in applied learning”, Journal of Applied Learning in Higher Education, Vol. 1, pp. 23-25, available at: https://files.eric.ed.gov/fulltext/EJ1188538.pdf

Serbessa, D. (2006), “Tension between traditional and modern teaching: learning approaches in Ethiopian primary schools”, Journal of International Cooperation in Education, Vol. 9 No. 1, pp. 123-140.

Shapiro, S. and Wilk, M. (1965), “An analysis of variance test for normality (complete samples)”, Biometrika, Vol. 52 Nos 3/4, pp. 591-611.

Silcock, P. and Brundrett, M. (2001), “The management consequences of different models of teaching and learning”, in Middlewood, D. and Burton, M. (Eds), Chapter 3, Managing the Curriculum, Paul Chapman, London, pp. 35-54.

Silva, P., Lopes, B., Costa, M., Melo, A.I., Dias, G.P., Brito, E. and Seabra, D. (2018), “The million-dollar question: can internships boost employment?”, Studies in Higher Education, Vol. 43 No. 1, pp. 2-21.

Sogunro, O. (2004), “Efficacy of role-playing pedagogy in training leaders: some reflections”, Journal of Management Development, Vol. 23 No. 4, pp. 355-371.

Tabak, F. and Lebron, M. (2017), “Learning by doing in leadership education: experiencing followership and effective leadership communication through role-play”, Journal of Leadership Education, Vol. 16 No. 2, pp. 199-212.

Ten Eyck, R. (2011), “Simulation in emergency medicine training”, Pediatric Emergency Care, Vol. 27 No. 4, pp. 333-341.

Tobin, K. and Tippins, D. (1993), “Constructivism as a referent for teaching and learning”, in Tobin, K. (Ed.), The Practice of Constructivism in Science Education, American Association for the Advancement of Science, Washington, DC, pp. 3-21.

Tran, V. and Lewis, R. (2012), “Effects of cooperative learning on students at Giang university in Vietnam”, International Education Studies, Vol. 5 No. 1, pp. 86-99.

Westrup, U. and Planander, A. (2013), “Role-play as a pedagogical method to prepare students for practice: the students’ voice”, Högre Utbildning, Vol. 3 No. 3, pp. 199-210.

Yang, J.C. and Quadir, B. (2018), “Effects of prior knowledge on learning performance and anxiety in an English learning online role-playing game”, Journal of Educational Technology & Society, Vol. 21 No. 3, pp. 174-185.

Yardley-Matwiejczuk, K. (1997), Role–Play: Theory & Practice, Sage, London.

Zain, Z., Subramaniam, G., Rashid, A. and Ghani, E. (2009), “Teaching students’ performance and attitude”, Canadian Social Science, Vol. 5 No. 6, pp. 92-102.

Zhang, D. and Nunamaker, J. (2003), “Powering e-learning in the new millennium: an overview of e-learning and enabling technology”, Information Systems Frontiers, Vol. 5 No. 2, pp. 207-218.

Corresponding author

Timothy Pettit can be contacted at: tpettit@nu.edu

Related articles