Teaching a while measuring b: cultural bias in assessing student performance

Sandra Seno-Alday (The University of Sydney Business School, Sydney, Australia)
Amanda Budde-Sung (Department of Management, U.S. Air Force Academy, Colorado, USA)

Journal of International Education in Business

ISSN: 2046-469X

Article publication date: 7 June 2021

Issue publication date: 23 September 2022

169

Abstract

Purpose

This paper aims to explore the impact of differences in educational traditions on conventions of teaching and learning, and on the measurement of learning outcomes. These are critical issues within the context of business schools that are steeped in one dominant tradition but have a large population of international students previously educated in other traditions. The paper argues that international students face the challenge of satisfactorily demonstrating learning according to foreign conventions that are different from what they would have been accustomed to within the framework of their home educational tradition.

Design/methodology/approach

This study draws on a bilingual literature review to capture differences in educational traditions between Australia and China. It then uses logistic regression to analyze the performance of 800 domestic and international Chinese students across a range of different assessment formats at a large Australian business school.

Findings

The study finds statistically significant differences in the performance of these two student groups on different assessment types. It concludes that the conventions on approaches to the assessment of learning shaped by a specific educational tradition can hamper the effective demonstration of learning among students from other educational traditions.

Originality/value

The paper focuses on issues related to the assessment of learning in multicultural higher education contexts, which has received less attention in the literature compared to issues on teaching approaches in multicultural contexts. The paper also highlights important implications on the validity of the measurement of learning outcomes and on the subsequent impact on graduate recruitment.

Keywords

Citation

Seno-Alday, S. and Budde-Sung, A. (2022), "Teaching a while measuring b: cultural bias in assessing student performance", Journal of International Education in Business, Vol. 15 No. 2, pp. 273-289. https://doi.org/10.1108/JIEB-01-2021-0005

Publisher

:

Emerald Publishing Limited

Copyright © 2021, Emerald Publishing Limited


Introduction

Management and business education scholars have underscored the importance of examining the effects of culture on learning styles (Haugh, 2016; Tran, 2008). Interest in this space has been driven by increased international student mobility that has led to greater multiculturalism in business schools (Hardy and Tolhurst, 2014). Existing literature has found evidence that cultural values do make a significant impact on learning styles (Holtbrügge and Mohr, 2010; Manikutty et al., 2007; Yamazaki et al., 2018). Cultural values have been found to shape not only learning styles but also teaching approaches (Czuchry and Yasin, 2008). If students from different cultures have been enculturated to learn using different learning styles, then it must be acknowledged that certain teaching approaches could foster learning in some students better than it does in others, within the context of a multicultural classroom.

Teaching approaches are concerned with instructional design and delivery but they also incorporate strategies to measure student learning outcomes. This is in recognition of the fact that good instructional design may not necessarily result in the acquisition of intended knowledge or skills (William, 2011). The measurement of student learning outcomes or assessment for learning involves some form of testing or evaluation of what a student knows and/or can do. It serves to give feedback to the student on their learning progress (in the case of formative assessment) or results in a grade that is recognized as a record of educational achievement (in the case of summative assessment) (Sambell et al., 2012).

Given that teachers themselves have culturally influenced preferred instructional styles, it thus follows that they would also have preferred approaches to designing assessments for learning. Assessments measure the level and quality of students’ learning against desired learning outcomes (Ramsden, 2003), and take the form of examinations, essays, presentations and others. Approaches to designing assessments are influenced by historical and deeply rooted paradigms on learning (Shepard, 2000), which are themselves shaped by well-entrenched educational traditions and systems (Checchi and van de Werfhorst, 2018). Problems can arise when a university views teaching, learning and assessments of learning from one tradition, but its students come from other traditions that view the learning and teaching process very differently.

In particular, significant issues can result in situations in which international students who have been culturally conditioned to demonstrate learning in one particular way are called upon to demonstrate their learning in a very different, possibly unfamiliar way. This can lead to poor performance on the part of students who may in fact have achieved specified learning outcomes, but who may not have been able to effectively demonstrate their learning because of a lack of familiarity with the assessment format. Therefore, the students with the highest grades may not necessary be the most capable: rather, they could simply be the most adept at conforming to traditionally-based expectations of demonstrating learning.

This paper explores the impact of culturally influenced learning assessment approaches on the performance of students from various cultural backgrounds. Investigating this is significant not only because educators need to ensure valid and reliable assessments to effectively close the loop on learning and teaching but also because grades have important implications beyond serving as indicators of academic performance. Grades are used as sources of information used to evaluate an individual’s fitness for further undertakings, such as further education or employment (McCracken et al., 2016).

Indeed, there have been parallel concerns in the management education literature that university graduates do not meet the skill and knowledge requirements of employers (Kilgo et al., 2017; US Department of Education, 2006). This study argues that issues of assessment performance and graduate employability may in fact be linked: that is, if culturally-rooted issues weaken the validity of student performance in assessments for learning, then grades are a poor indicator of student capabilities. Employers may have thus been drawing on a sub-optimal source of information for graduate employability.

The paper begins with a brief description of the increasing cultural diversity in business classrooms, followed by a review of different approaches to assessment for learning in both English and Chinese language education literature. The study focuses specifically on China because the country accounts for the largest population of international students in major Western business schools (UNESCO, 2020). Finding differences in the two bodies of literature, hypotheses on the relationship between the background of students and their performance on a range of assessment types are developed. These hypotheses are tested on a sample of domestic Australian students and international Chinese students in a large Australian business school. The study finds that compared to their Chinese peers, domestic Australian students tend to perform better on individual written assessment tasks, but not necessarily on other assessment formats, such as multiple choice exams. This offers empirical evidence that conventional assessment formats from a particular educational tradition can confound the measurement of learning outcomes among students from other educational traditions. The paper concludes with a discussion on the validity and reliability of learning outcome measurement, which affects employer perceptions and prospects of graduate employability.

Literature review

Diversity in the business classroom

The number of students enrolled in tertiary study outside of their country of origin grew by around 5% per year from 1998–2018, and the worldwide international student population was estimated to be around 6 million in 2018 (The Organisation for Economic Co-operation and Development [OECD], 2020). Universities in Western countries (Gupta et al., 2002) are the primary destinations, with the USA, the UK, Australia and Canada hosting the bulk of international students (UNESCO, 2020). By contrast, Asian countries accounted for 57% of international students studying in OECD countries in 2018, with China and India alone accounting for more than 30% (OECD, 2020), which are countries from a range of Asian cultures (Gupta et al., 2002). The most popular area of study for these students is business (Institute of International Education, 2010), effectively making the tertiary business classroom one of the most diverse educational environments today. This presents a unique teaching and learning context with a constantly evolving set of challenges.

This increasing global student mobility has led to the development of an extensive body of literature on teaching and learning in multicultural higher education environments, particularly in business (Kuzhabekova et al., 2015). This includes investigations into cultural differences in learning styles in business classrooms (Apfelthaler et al., 2007; Bordia et al., 2015; Budde-Sung, 2011; Rodrigues et al., 2000); the role of language in business education (Yoon and Portman, 2004); and the exploration of the effectiveness of various teaching approaches (Eisenberg et al., 2013; Kolb and Kolb, 2005; Ng et al., 2009). While much has been done to explore the pedagogical challenges in a multicultural business classroom, there has been little focus on issues related to student learning outcome measurement among international students (Morrison et al., 2005). Given the increasing number of international students undertaking business degrees and the continuing pressure from employers for universities to produce workforce-ready graduates, this gap represents an important area of inquiry.

Traditions in teaching and learning

In the education literature, East-West distinctions are often drawn when exploring approaches to teaching and learning. Hardré et al. (2006) noted that “Western” cultures have a tradition rooted in training students in the art of discourse (Boghossian, 2006): that is arriving at knowledge through questioning, forming a position and then arguing in defense of that position. By contrast, many Asian (“Eastern”) cultures emphasize a tradition in which the goal is the transfer of knowledge from the teacher to the student (Eisenberg et al., 2013). For example, Shaw (1999) noted that in the Chinese educational tradition, it is assumed that a correct answer exists for every problem, and that correct answers come from books or authority figures. In this tradition, learning from authority figures stimulates and reinforces thinking.

These differences in educational traditions lead to different approaches between how teachers teach and how students learn. This is because prior experience in deeply rooted educational traditions consciously and unconsciously influence students’ study habits (Entwistle, 1998). Knowing what material to focus on and how to study may differ according to students’ previous experiences and the expectations of their former teachers. The same biases can likewise apply to teaching, where teachers who have been previously enculturated in a particular educational tradition have well-entrenched ideas on how (and how not to) teach. For example, the student enculturated in the Chinese tradition described above will tend to focus on committing a theory to memory and searching for that one unambiguously “correct” answer to a problem while the teacher enculturated in traditions more deeply rooted in discourse will instead tend to focus on highlighting ambiguity and dichotomy in the search for alternative solutions to a problem.

These differences likewise lead to different approaches between how teachers prefer to assess student learning outcomes, and how students are wired to demonstrate learning outcomes. Previous studies have indicated that international students studying in the USA regularly encounter unfamiliar assessment types and performance expectations, which significantly contribute to academic stress (Wan et al., 1992). For example, an international student commented in a US News and World Report article (Hopkins, 2012) that professors in the USA expect students to write research papers and justify positions with well-argued commentary and analysis, but that he had never done that type of assessment prior to studying in the USA. In comparing writing expectations for Chinese and American (USA) students, DeVries (2002) argued that both traditions demand clarity in student writing, but that the two traditions have very different ideas of what comprises clarity. Gao and Ting-Toomey (1998) observed that an argument that a Chinese scholar might consider to be clear and well-defined might be interpreted as being obscure by an American scholar.

Unfamiliarity with particular types and formats of assessments on the part of international students adds a layer of challenge and complexity to the assessment process: not only must students demonstrate their learning of the subject matter but they must also do so in a manner that is traditionally accepted to be a “good” indicator of learning. This introduces bias in the definition of what constitutes “good” learning, immediately putting foreign students (from a different educational tradition) at a disadvantage compared to native or domestic students who may have spent many years developing the skills required to demonstrate learning success in their own country, and who thereby have the benefit of relevant prior experience and practice.

The issue of assessments in business education has been considered, albeit not extensively, within the English language education literature (Seno-Alday and Budde-Sung, 2016), and a review of Chinese language education literature reveals a very different perspective on what constitutes the appropriate assessment of learning and what the demonstration of that learning should resemble. These differences in perspectives are discussed in the bilingual literature review of assessment approaches in the following sections, with a particular focus on open-ended assessment tasks such as essay-type examinations versus closed-ended assessment tasks such as multiple choice examinations.

A review of assessment approaches: English-language literature

The English language literature on education is dominated by perspectives from the Western traditions, where quality teaching is typically associated with an emphasis on “deep” learning over “surface” learning (Biggs, 1973). According to this tradition, deep learning involves formulating hypotheses, understanding concepts and tasks as part of a whole, putting forth theories, and integrating meaning between and across subjects. The Western conception of surface learning, by contrast, involves a focus on and reproduction of essential information, rote memorization, and a separation of parts of a task (Phillips and Trainor, 2014; Prosser and Trigwell, 1999). Consequently, attitudes toward the appropriateness and effectiveness of various types of assessment tools tend to reflect these learning categories.

Assessments that are typically considered appropriate to encourage students to pursue deep learning include open-ended formats such as case studies or discussions in which there is no single correct answer. In these formats, students must gather and analyze information, propose a solution or a course of action, and justify that choice (Reinhardt, 2010). Other examples of preferred assessment formats in this tradition include essays, written papers and class discussions, which are considered to “involve analytical, critical thinking and communication skills, which are suggestive of a deep learning approach” (Scouller, 1998, p. 455). Here, there is a clear preference for assessment formats where students are not viewed to be simply memorizing and regurgitating knowledge (Phillips and Trainor, 2014).

There is also a significant premium attached to formats that require students to exercise their communication skills, as demonstrated in essays and reports (Welsh and Saunders, 1998) and oral presentations (Conrad and Newberry, 2012; Kerby and Romine, 2009). Consequently, students who have spent long years being educated in this tradition tend to be comfortable with these formats, and are much more likely to be convinced that these are important and valid opportunities to demonstrate their knowledge and abilities compared to students educated in non-Western traditions (Selvarajah, 2006).

Much has also been written in the English language literature about the use of group assessments in business classrooms. In the Western tradition, collaborative learning in groups is encouraged as it is seen to enhance critical thinking, and thus, deep learning (Gokhale, 1995; Cajiao and Burke, 2016). Successful ability to work in groups is considered to be a key quality in future career success (Loignon et al., 2017). Indeed, the pedagogical benefits of group work in Western universities are so important that the US-based accrediting body, the Association to Advance Collegiate Schools of Business (AACSB), requires that its accredited business schools offer group-based learning opportunities or opportunities for learners to engage with each other (AACSB, 2020).

By contrast, other types of assessments such as the multiple choice examination, with its closed-ended structured format requiring memorization and retention of knowledge, has been subject to substantial criticism in the English language education literature. Criticisms include concerns that multiple choice exams lack the ability to measure important skills, and have less relevance compared to open-ended essay-style exams (Katz et al., 2000). They are seen to fail to help students develop critical thinking skills, with requirements that students merely regurgitate information (Sánchez, 2013). Furthermore, they are argued to be less rigorous and less effective in developing higher-level cognitive thought or stimulating higher-level thinking compared to essay-type exams (Stanger-Hall, 2012; Welsh and Saunders, 1998). These exams are said to reflect only surface-level learning (Biggs, 1973; Entwistle and Entwistle, 1992; Scouller, 1998), with a perception that “deep level learning strategies are not required to satisfy” the requirements of multiple choice (and similar) examinations (Watkins and Hattie, 1985). While there are a few points in the English-language literature that acknowledge some benefits of closed-ended assessments (Welsh and Saunders, 1998; Brook and Milner, 2014; Morris and Maxey, 2014), the majority of the English literature argues in favor of the discourse-based approach to education, which is best served by more open-ended assessments such as written papers and oral presentations.

A review of assessment approaches: Chinese-language literature

In sharp contrast to the content of the English language education literature, multiple choice (and similar) exams enjoy a high level of credibility in the Chinese language literature. Studies in China have compared multiple choice examinations to other types of assessments and found the multiple choice format to be either a superior or equivalent method of evaluating learning alongside other types of assessments, including essays and short answer examinations (Ren, 2010; Li and Fu, 2009; Wang, 2003).

The first justification for this is the ability of the format to objectively evaluate several learning outcomes at varying levels of difficulty across several questions on a wide range of topics. The format allows teachers to design knowledge questions, problem-solving questions and critical thinking questions. While a student may be able to correctly guess answers on some questions, their ability to do that across multiple questions on different outcomes and degrees of difficulty is significantly reduced, thus allowing for a comprehensive and accurate picture of student learning. Xiao (2002) notes that multiple choice styles of exams offer the opportunity to test a wider variety of topics in one exam, as the student need not spend a lot of time writing an answer to only one question, but can demonstrate outcomes on several different topics in a short amount of time. The flexibility afforded by the multiple choice format in designing question difficulty and in evaluating a range of learning outcomes in a broad range of topics was also noted by Li and Xu (1999), who conclude that the multiple choice format allows for the most definitive determination of student learning.

The second argument for the superiority of these types of closed-ended assessments in the Chinese language literature is that they allow for anonymity in grading, and provide an unbiased platform on which to thoroughly evaluate learning (Feng, 2007; Xiao, 2002). Multiple choice exams allow a teacher to directly test learning without the confounding effects of linguistic ability, writing ability or strength of expression (Wen, 2009). Exam papers can be anonymous, with no identifying handwriting or unique idiomatic expressions. Student papers that are graded against pre-established answers introduce no marker bias in the grading process: wrong answers are definitively wrong while correct ones are unambiguously correct (Wang, 2003). Particularly in highly ascribed status societies such as China (Trompenaars and Hampden-Turner, 2011), in which family name or other status affiliation is intertwined with many aspects of society, the ability for students to feel that they are being graded fairly is important. Grader anonymity allows instructors to not feel any pressure to give students from more influential families higher grades.

The third justification is related to effectiveness in supporting student learning. Formal, closed-ended assessments, such as multiple choice exams, can be graded and returned quickly, giving students more immediate feedback while the tested material remains at the forefront of their minds, strongly reinforcing learning. By contrast, individual written papers take much longer to grade, and the delayed feedback may provide less learning reinforcement to students, leading many scholars to argue in favor of the superiority of multiple choice exams (Li and Fu, 2009).

The strengths of the closed assessment format as described in the Chinese language literature explain why the multiple choice exam is a primary means of assessing learning in China. By contrast, open-ended group projects, used as a classroom assessment, are unusual in Chinese universities, as learning is viewed as a competitive, rather than a cooperative, endeavor (Li, 2014; Clarke et al., 2007; Nguyen et al., 2006). Li (2014) found that in traditional Chinese pedagogy, learning happens when a teacher transfers knowledge to a student; thus, participation in group work has little benefit and is superfluous to learning. Clarke et al. (2007) stated that “Chinese students (in Western universities) are positive about the social benefits of working in groups, but often do not see its educational value […] they bring with them their prior learning experiences from the Chinese educational system, with its emphasis on respect for the teacher, knowledge, competition and high grades” (Clarke et al., 2007, p. 7). Indeed, Li (2014) included a statement from a university instructor that Chinese students will feel lost if an instructor goes beyond the textbook in their instructional style, and may not understand how to effectively contribute to – or learn from – a group project.

Hypothesis development

The differences in assessment traditions and approaches between China and Western countries described above highlight significant issues faced by Chinese students who have had to sit predominantly closed-ended assessments in their home countries, and who find themselves having to sit predominantly open-ended assessments in a Western university.

Educational traditions clearly influence both teachers and students in all aspects of the teaching and learning process. Critical challenges arise when very different traditions meet within the context of the diverse higher education environment. The precise impact of this divide on education outcomes needs to be better understood, as this may have far-reaching implications well beyond the classroom. This paper investigates the question: how do students’ prior educational tradition impact their performance on different types of assessment formats?

This question was previously explored by DeVita (2002), who looked at performance differences between domestic students and international students in one class in the UK, and found that international students exhibited lower levels of performance on essays and written exams in that class. This paper extends that study by specifically comparing differences in the performance of Australian and Chinese students in a range of assessment formats across different courses, academic levels and time periods.

The different perspectives of the English language literature versus the Chinese language literature reveal a gap in the traditionally rooted assumptions on the purpose and appropriate formats of assessments to evaluate learning outcomes. In light of this, it is proposed that open-ended types of assessments give an inherent advantage to domestic Australian students studying in Australian higher educational institutions because they would have received prior conditioning in the Western educational tradition. Students who have received prior conditioning in the Chinese educational tradition, however, will have less experience with open-ended assessment types, and are thus, expected to exhibit lower levels of performance. These propositions are articulated in H1 and H2 below:

H1.

Domestic Australian students will exhibit higher scores on individual written assessments.

H2.

International Chinese students will exhibit lower scores on individual written assessments.

The literature further suggests that closed-ended assessments such as multiple choice exams do not disadvantage students previously educated in China, as this format is an existing convention in the Chinese educational tradition. Previous research further suggests that the performance of students educated in the Western tradition on both closed- and open-ended assessments tends to be correlated. Therefore, multiple choice exams likewise do not disadvantage Australian students. These expectations lead to H3:

H3.

Scores on multiple choice exams will be unrelated to students’ country of origin (that is domestic Australian or international Chinese).

Finally, prior literature suggests that students from China would have negative views and little experience with group projects, whereas team-based assessments are a standard and important part of Western pedagogy. Students may thus arrive at a group project with different understandings of how to do well in this assessment format. This leads to H4:

H4.

Domestic Australian students will exhibit higher scores in group assessments compared to international Chinese students.

Study

This study uses data from seven consecutive years in two undergraduate business courses and one postgraduate business course. The setting was the business school of a large Australian business school with a significant international Chinese student population. The time frame spans the years 2010-2016, and the entire dataset included results from 1,672 students. The random choice function in Excel was used to draw a sample of 400 Australian students and 400 Chinese students from the dataset, resulting in a balanced random sample of 800 students. All students were fluent in English, the language of instruction of the Australian business school. The international student admission requirements specified a minimum test of English as a foreign language (TOEFL) score of 96 or an overall international language test system (IELTS) band of 7.0, with a minimum band of 6.0 on each of the individual tested components: writing, speaking and reading. These tests are widely used in the admissions process for international students in universities around the world and have been updated to specifically capture academic linguistic requirements (Ingram and Bayliss, 2007; Jamieson et al., 2000). While these English language tests alone cannot predict academic performance at university (Hill et al., 1999; Ingram and Bayliss, 2007; Uysal, 2010), test scores in this study control for the minimum English language proficiency required to perform the academic reading, writing and speaking tasks required in higher education.

Data on assessment results from individual written assessments (e.g. research papers, essays, case analyses), multiple choice exams, and group presentations were obtained and standardized. To standardize results across different classes and different courses, the results for different types of assessments were entered as percentages. For example, if in one course an essay was graded out of 20 and if a student received a score of 15, then the grade was standardized to 75% (or 15 out of 20). If in another course, an essay was graded out of 50 and if a student received a score of 37, then the grade was standardized to 74% (or 37 out of 50). The results across all assessments included in the sample were thus, standardized on a scale of 100. The courses were delivered in one university with clearly defined rubrics for assessment performance. Teachers and teaching assistants were typically required to undergo training before grading; therefore, biases were kept to a minimum. Assessments that were used in only one course (and not in any of the other courses) were dropped from the dataset.

Logistic regressions were used to determine if the student’s country of origin (dummy coded into Australia and China), could be predicted by the student’s performance on various assessment types. The descriptive statistics are shown in Table 1 and the mean performance of each student group on each assessment type is shown in Table 2.

Results

H1 and H2 predicted that domestic Australian students would exhibit higher scores on individual open-ended assessments such as essays and individual written work, whereas Chinese students would demonstrate lower scores on these formats. The descriptive statistics in Table 2 confirm these predicted trends and the logistic regression results in Table 3 indicate that these differences are statistically significant. The findings show that a student’s country of origin can be predicted by their performance on individual written assessments: that is higher scores on written assessments will most likely point to the work of a domestic Australian student. Both H1 and H2 receive full support.

H3 predicted that the multiple choice exam format would favor no particular student group (domestic Australian or international Chinese), and receives empirical support, as a student’s country of origin cannot be significantly predicted based upon their performance on a multiple choice exam. H3 thus, receives full support, as reflected in Table 3.

Finally, H4 predicted that the performance of domestic Australian students would outperform international Chinese students in group projects. Interestingly, H4 does not receive support; in fact, the opposite appears to be true. The results show that international Chinese students significantly perform better in group projects. This unexpected finding may be related to collectivism within Chinese culture. Previous studies have found that China’s cultural values of collectivism influence performance in team settings, and Chinese people who identify their group or team as important to them put forth more effort and engage in more productive teamwork than they might otherwise, to elevate the entire group (Earley, 1989; Tjosvold et al., 2003). The finding that Chinese students perform better in group projects may be reflective of the Chinese students’ desire to be seen as contributing in a valuable way to the group, in other words, to not lose face with the other group members.

The findings on H4 could also be affected by team composition. Student behavior in homogeneous teams composed of individuals with prior enculturation in the same educational tradition could be different from behavior in heterogeneous teams with students enculturated in different educational traditions. In diverse teams, the dynamic could further be affected by the balance in composition where one educational tradition dominates over the others or where no single tradition dominates. The differences in team dynamics resulting from team composition could further lead to differences in performance on a team task or assessment.

There is clearly much that remains to be understood about how culture, educational traditions, and team composition affect performance on team assessments. Controlling for composition and evaluating team assessment performance will help uncover further insights on the effects of educational traditions. Additionally, evaluating the performance of students from different educational traditions on assessments completed individually versus the assessment performance of teams composed of students from different educational traditions presents a clear avenue for future research.

Discussion

The results of this study show that educational traditions and paradigms impact students’ performance on different types of assessments. In particular, the study highlights how written assessments that heavily rely on argumentation and exposition – and that are generally favored in universities with Western educational traditions – can put some students from non-Western educational traditions at a disadvantage. When students have received prior enculturation in educational traditions that do not heavily rely on written or verbal presentation of arguments, they encounter significant challenges in effectively demonstrating learning. These challenges are further compounded by language issues. Non-native English-speaking students may exhibit proficiency in reading, writing and speaking English; however, they may not necessarily have the high level of language mastery required for academic writing and comprehension.

This raises questions and significant issues of assessment validity, equity and fairness – among others – in an environment of increased international student mobility, where significant numbers of students from China and other Asian countries pursue higher degrees in Western universities with different educational traditions than their own.

Implications on educational practice and policy

In light of these findings, the challenge is for educators to re-examine the validity and reliability of assessment formats given the mix of educational traditions among students in a cohort. If current assessment approaches cannot readily be changed, then educators must find ways to help international students achieve clarity on expectations regarding the structure and format of written academic work and other assessment forms that may be new or unfamiliar to them. One clear strategy would be to embed academic writing training in induction or orientation programs for international students. While new international students may have demonstrated English language proficiency in general academic reading, writing, and listening in their IELTS or TOEFL tests, further training is required for them to structure arguments and analytical pieces to the academic standards required by Western universities.

Another strategy would be for courses to give students opportunities to undertake and receive feedback for non-graded written work before undertaking graded work. Formative assessments and feedback allow students to learn how to demonstrate learning, preparing them to effectively showcase what they have learned in subsequent summative assessments that have an impact on their grades.

A third strategy would be to redesign assessments such that essay questions are rephrased and scaffolding questions are incorporated. This better supports the thinking process and writing exposition of students. Doing so will help students from non-Western educational traditions adequately demonstrate the learning that they have actually acquired in the educational process. As mentioned above, what is considered to be clear and concise writing looks very different from a Chinese perspective than from a Western perspective. Thus, additional instruction on how to demonstrate learning on various types of assessments might also prove to be helpful in the multicultural business classroom, rather than assuming that all students automatically understand what is meant by clarity, or how to effectively assert a position and support an argument.

A fourth strategy is for business management educators to move away from traditional individual written assessments and design new assessment formats that allow students to effectively demonstrate learning regardless of deep-seated educational traditions. These “educational tradition-fair” assessments acknowledge that while students may have acquired the same levels of learning, their ability to demonstrate this learning may differ depending on prior enculturation. Therefore, assessment design must either correct for biases (for example, by giving students several opportunities to demonstrate the same learning across different assessment formats) or must be structured differently. Alternative strategies may include designing assessments around simulations and work-integrated learning methods that rely less on language or written expression but require students to demonstrate learning by doing.

The findings of this study open an important area for future research on group work in multicultural classrooms. The unexpected results indicate that Chinese students exhibit superior performance on group projects, in spite of receiving prior training in an educational tradition that does not offer significant opportunities for team-based assessments, and that in fact views this format negatively. There are several possible explanations for this, which require further investigation. The first possibility is the nature of the group composition: the mix of Chinese and Australian students in teams could have contributed to improved performance among Chinese students. In this scenario, the Australian students would have transferred some of their learning on how to approach group projects, which would have been learned quickly by their Chinese group members. If, however, Chinese students tended to form homogeneous teams with other Chinese students, then in this scenario, one possible explanation could be that the students – mindful of their “foreignness” − are driven to achieve superior levels of performance. This drive to perform within the context of a foreign environment could further aided by collectivist cultural values. In either scenario, further important research is required.

Finally, educators must also ensure that all students are given equal opportunities to perform well, regardless of their prior exposure to a different educational tradition. This may involve providing instruction or training on how to approach a range of different assessments. This will also involve designing an assessment portfolio that is composed of different assessment formats, which effectively allows students multiple opportunities to demonstrate the same learning outcomes in different ways. The portfolio approach gives educators an opportunity to minimize the performance biases that arise from previous training in different educational traditions.

Implications for graduate recruitment

Issues on teaching, learning and the assessment of student performance are clearly not contained exclusively within the domain of the classroom. Recent studies suggest that employers do not believe that university graduates have the necessary skills to succeed in the business world (Kilgo et al., 2017; US Department of Education, 2006). It may be that employers are hiring new graduates with the best grades, but not the best level of knowledge and skills, as grades may misrepresent graduates’ true capabilities due to systematic biases in assessment practices. While business educators need to more explicitly consider educational tradition influences and design more tradition-neutral assessments, employers may wish to use the results of this study to reflect upon approaches to graduate screening and selection.

Contribution to theory

The study has found that students’ prior conditioning in particular educational traditions makes an impact on their performance in different assessment formats. Theorizing in the context of an increasingly diverse higher education environment must therefore be advanced to account for this significant variable. The challenge is for scholars to unpack the dimensions that constitute the diversity of student backgrounds and explore the effects of these complex dimensions not just on the learning process but also on the outcomes of assessments for learning. This is particular scope to investigate the dynamics of diverse student teams and explore the impact on learning and group assessment performance.

On the matter of learning outcome evaluation, the literature reviewed above in two languages indicates a disagreement between these two bodies of literature. The literature is written in English largely argues that open-ended types of assessments (reflective of the Western educational tradition) are the best way for students to demonstrate learning. The literature written in Chinese argues instead that more closed-ended assessments are most effective in objectively evaluating student learning. There is clearly much that needs to be understood about the diversity in educational traditions and approaches across countries and cultures, and the ways in which these change the dynamics in a multicultural higher education setting. There is also a critical need for future research to survey literature in multiple languages for a comprehensive picture of the state of the field from a range of perspectives and traditions.

It is important to note that diversity characterizes not only students but educators, given that international mobility has also increased in higher education careers. This further increases the diversity in the higher education setting and likewise needs to be incorporated in educational theorizing.

Conclusions, limitations and future directions

The study finds that international students’ prior conditioning in their home educational traditions makes a significant impact on their performance in assessments designed in the foreign university’s educational tradition. This highlights the need for educators to cultivate a heightened sense of awareness and sensitivity to educational traditions that both the teacher and the student bring to the higher education process. It is absolutely essential for business educators to revisit conventional teaching and learning approaches and reassess their continued relevance within the context of an increasingly mobile and culturally diverse higher education sector.

As with all studies, this current research is limited by its sample, involving domestic Australian students and only one specific group of international students. Future studies may wish to investigate the issue among international students from multiple countries in other Western universities. There is also a need to explore the performance of international students from Western countries who opt to undertake higher degrees in non-Western universities. While Western universities host the majority of the world’s international students, China, Japan, Taiwan, South Korea and Malaysia are looking to rapidly grow their international student numbers (QS Intelligence Unit, 2017). These countries may encounter similar concerns as their own business classrooms become increasingly multicultural and multilingual.

Additionally, the sample included student results from different courses, in different departments and while all courses had multiple choice exams, group projects, and individual written assessments, the exact mix of assessments in each class differed. While these different combinations allowed the study to focus on the main types of assessments such as exams and papers, they do not allow the consideration of other assessment formats. Future studies may wish to explore student performance on other types of assessments within a multicultural classroom. It is also an outside possibility that the Chinese students simply learned the material for the individual written assessments less well than did their Australian peers; however, if that were the case, those results would likely have been exhibited consistently across all assessment types, rather than in just some.

Finally, the study does not mean to suggest that invalid or unreliable assessment practices are the main reason why firms find graduates to lack employable skills. This study opens a possible explanation for why employers have encountered challenges in graduate recruitment. Future studies may want to continue this line of inquiry with longitudinal studies investigating student performance and subsequent career performance.

Business school instructors are being challenged to teach to some of the most culturally diverse classrooms in higher education. It is important for them to ensure that they are measuring the extent to which their students have actually learned the subject matter, rather than measuring the extent to which their students can demonstrate learning in a culturally expected manner. To do this, they must first recognize that differences in educational traditions exist and that those differences may impact the demonstration of learning, rather than the actual learning itself. Armed with this knowledge, business schools can better meet their goals of educating international students alongside domestic students.

Descriptive statistics

Assessment type N Range Minimum Maximum Mean SD
Multiple choice exam 800 92.50 0 92.50 60.36 18.54
Group project 800 95.00 0 95.00 65.07 16.80
Individual written 800 95.00 0 95.00 61.41 16.06

Sample mean scores

Student’s home country Multiple choice exam Group Individual written
China 58.85 66.34 58.60
Australia 61.87 63.80 64.22

Regression results

DEP VAR: Country code^ B SIG
Multiple choice exam 0.000 0.951
Individual written −0.066 0.000***
Group 0.054 0.000***
Constant 0.606 0.059
Nagelkerke R2 0.144 0.000***
Notes:

^0 = Domestic Australian students; 1 = International Chinese students.

***ρ < 0.01

**ρ < 0.05

*ρ < 0.10

References

AACSB (2020), 2020 Guiding Principles and Standards for Business Accreditation, Effective July 28, 2020. Tampa, FL, AACSB International – The Association to Advance Collegiate Schools of Business.

Apfelthaler, G., Hansen, K., Keuchel, S., Mueller, C., Neubauer, M., Ong, S.H., et al. (2007), “Cross-cultural differences in learning and education: stereotypes, myths, and realities”, in Palfreyman, D. and McBride, D. (Eds), Learning and Teaching across Cultures in Higher Education, Basingstoke, Palgrave MacMillan, pp. 15-35.

Biggs, J. (1973), “Study behaviour and performance in objective and essay formats”, Australian Journal of Education, Vol. 17 No. 2, pp. 157-167.

Boghossian, P. (2006), “Behaviorism, constructivism, and socratic pedagogy”, Educational Philosophy and Theory, Vol. 38 No. 6, pp. 713-722.

Bordia, S., Bordia, P. and Restubog, S. (2015), “Promises from Afar: a model of international student psychological contracts in management education”, Studies in Higher Education, Vol. 40 No. 2, pp. 212-232.

Brook, C. and Milner, C. (2014), “I have no English friends’: some observations on the practice of action learning with international business students”, Action Learning: Research and Practice, Vol. 11 No. 2, pp. 225-232.

Budde-Sung, A.E. (2011), “The increasing internationalization of the international business classroom: cultural and generational considerations”, Business Horizons, Vol. 54 No. 4, pp. 365-373.

Cajiao, J. and Burke, M. (2016), “How instructional methods influence skill development in management education”, Academy of Management Learning and Education, Vol. 15 No. 3, pp. 508-524.

Checchi, D. and van de Werfhorst, H.G. (2018), “Policies, skills and earnings: how educational inequality affects earnings inequality”, Socio-Economic Review, Vol. 16 No. 1, pp. 137-160.

Clarke, J., Baker, T. and Li, M. (2007), “Student success: bridging the gap for Chinese students in collaborative learning”, ISANA International Conference, 27-30 November, Adelaide, Australia.

Conrad, D. and Newberry, R. (2012), “Identification and instruction of important business communication skills for graduate business education”, Journal of Education for Business, Vol. 87 No. 2, pp. 112-120.

Czuchry, A.J. and Yasin, M.M. (2008), “International entrepreneurship: the influence of culture on teaching and learning styles”, Journal of Entrepreneurship Education, Vol. 11, p. 1.

DeVita, G. (2002), “Cultural equivalence in the assessment of home and international business management students: a UK exploratory study”, Studies in Higher Education, Vol. 27 No. 2, pp. 221-231.

DeVries, K. (2002), “Writing ‘clearly’: differing perceptions of clarity in Chinese and American text”, Proceedings of the International Symposium on Contrastive and Translation Studies between Chinese and English, Shanghai, China.

Earley, P.C. (1989), “Social loafing and collectivism: a comparison of the United States and the People’s Republic of China”, Administrative Science Quarterly, Vol. 34 No. 4, pp. 565-581.

Eisenberg, J., Hyun-Jung, L.E.E., Bruck, F., Brenner, B., Claes, M.-T., Mironski, J. and Bell, R. (2013), “Can business schools make students culturally competent? Effects of cross-cultural management courses on cultural intelligence”, Academy of Management Learning and Education, Vol. 12 No. 4, pp. 603-621.

Entwistle, N. (1998), “Approaches to learning and forms of understanding. Teaching and learning in higher education”, Vol. 72, p. 98.

Entwistle, A. and Entwistle, N. (1992), “Experiences of understanding in revising for degree examinations”, Learning and Instruction, Vol. 2 No. 1, pp. 1-22.

Feng, J. (2007), “The study and exploration of compilation for mathematic subjects in test in Middle school under the new school curriculum”, 633.6, China Academic Journal Electronic Publishing House, Beijing, viewed 25 Jun 2015. 冯精华. 新课程理念下中学数学试题编制的研究与探索[M]. 北京:中国学术文献网络出版总库 2007.2.

Gao, G. and Ting-Toomey, S. (1998), Communicating Effectively with the Chinese, Sage Publications, Thousand Oaks, CA.

Gokhale, A.A. (1995), “Collaborative learning enhances critical thinking”, Journal of Technology Education, Vol. 7 No. 1.

Gupta, V., Hanges, P.J. and Dorfman, P. (2002), “Cultural clusters: methodology and findings”, Journal of World Business, Vol. 37 No. 1, pp. 11-15.

Hardré, P. L., Chen, C. H., Huang, S. H., Chiang, C. T., Jen, F. L. and Warden, L. (2006), “Factors affecting high school students’ academic motivation in Taiwan”, Asia Pacific Journal of Education, Vol. 26 No. 2, pp. 189-207.

Hardy, C. and Tolhurst, D. (2014), “Epistemological beliefs and cultural diversity matters in management education and learning: a critical review and future directions”, Academy of Management Learning and Education, Vol. 13 No. 2, pp. 265-289.

Haugh, M. (2016), “Complaints and troubles talk about the English language skills of international students in Australian universities”, Higher Education Research and Development, Vol. 35 No. 4, pp. 727-740.

Hill, K., Storch, N. and Lynch, B. (1999), “A comparison of IELTS and TOEFL as predictors of academic success”, in Tulloh, R. (Ed.), International English Language System (IELTS) Research Reports (1999), No. 2, pp. 63-73.

Holtbrügge, D. and Mohr, A.T. (2010), “Cultural determinants of learning style preferences”, Academy of Management Learning and Education, Vol. 9 No. 4, pp. 622-637.

Hopkins, K. (2012), “Six challenges for international students in college”, US News and World Report, August 28, available at: www.usnews.com (accessed 3 June 2015).

Ingram, D. and Bayliss, A. (2007), “IELTS as a predictor of academic language performance, part 1”, International English Language System (IELTS) Research Reports, No. 7.

Institute of International Education (2010), “Open doors 2010”, Sponsored by the US Department of State Bureau of Education and Cultural Affairs.

Jamieson, J., Jones, S., Kirsch, I., Mosenthal, P. and Taylor, C. (2000), “TOEFL 2000 framework: a working paper”, TOEFL Monograph Series, No. 16.

Katz, I.R., Bennett, R.E. and Berger, A.E. (2000), “Effects of response format on difficulty of SAT‐mathematics items: it's not the strategy”, Journal of Educational Measurement, Vol. 37 No. 1, pp. 39-57.

Kerby, D. and Romine, J. (2009), “Develop oral presentation skills through accounting curriculum design and course-embedded assessment”, Journal of Education for Business, Vol. 85 No. 3, pp. 172-179.

Kilgo, C.A., Culver, K.C., Young, R.L. and Paulsen, M.B. (2017), “The relationship between students’ perceptions of “good practices for undergraduate education” and the paradigmatic development of disciplines in course-taking behavior”, Research in Higher Education, Vol. 58 No. 4, pp. 430-445.

Kolb, A.Y. and Kolb, D.A. (2005), “Learning styles and learning spaces: enhancing experiential learning in higher education”, Academy of Management Learning and Education, Vol. 4 No. 2, pp. 193-212.

Kuzhabekova, A., Hendel, D.D. and Chapman, D.W. (2015), “Mapping global research on international higher education”, Research in Higher Education, Vol. 56 No. 8, pp. 861-882.

Li, D. (2014), “Changing conceptions of Chinese groupwork: collaboration, culture, and context”, PhD Dissertation, University of Melbourne, Melbourne, Australia.

Li, K. and R. Fu. (2009), “Discussion on validity of translation multiple-choice test”, Journal of Yangling Vocational and Technical College, Vol. 8. No. 4, pp. 95-97. 李凯. 付瑞阳. 论翻译多项选择题测试的有效性[M], 陕西 杨凌 2009.

Li, N. and Xu, L. (1999), “Multiple-choice questions: benefits and shortfalls”, Journal of Liaoning Teachers College (Social Science Edition), Vol. 1 No. 2, pp. 111-111. 李娜. 徐莉.多项选择题利弊谈[J], 辽宁 鞍山 1999.

Loignon, A., Woehr, D., Thomas, Ja. Loughry, M., Ohland, M. and Ferguson, D. (2017), “Facilitating peer evaluation in team contexts: the impact of frame-of-reference rater training”, Academy of Management Learning and Education, Vol. 16 No. 4, pp. 562-578.

McCracken, M., Currie, D. and Harrison, J. (2016), “Understanding graduate recruitment, development and retention for the enhancement of talent management: sharpening ‘the edge’ of graduate talent”, The International Journal of Human Resource Management, Vol. 27 No. 22, pp. 2727-2752.

Manikutty, S., Anuradha, N.S. and Hansen, K. (2007), “Does culture influence learning styles in higher education?”, International Journal of Learning and Change, Vol. 2 No. 1, pp. 70-87.

Morris, M. and Maxey, S. (2014), “The importance of English language competency in the academic success of international accounting students”, Journal of Education for Business, Vol. 89 No. 4, pp. 178-185.

Morrison, J., Merrick, B., Higgs, S. and Le Métais, J. (2005), “Researching the performance of international students in the UK”, Studies in Higher Education, Vol. 30 No. 3, pp. 327-337.

Ng, K.-Y., Van Dyne, L. and Ang, S. (2009), “From experience to experiential learning: cultural intelligence as a learning capability for global leader development”, Academy of Management Learning and Education, Vol. 8 No. 4, pp. 511-526.

Nguyen, P.M., Terlouw, C. and Pilot, A. (2006), “Culturally appropriate pedagogy: the case of group learning in a Confucian heritage culture context”, Intercultural Education, Vol. 17 No. 1, pp. 1-19.

OECD (2020), Education at a Glance 2020: OECD Indicators, OECD Publishing, Paris, doi: 10.1787/69096873-en.

Phillips, C.R. and Trainor, J.E. (2014), “Millennial students and the flipped classroom”, Journal of Business and Educational Leadership, Vol. 5 No. 1, p. 102.

Prosser, M. and Trigwell, K. (1999), Understanding Learning and Teaching: The Experience in Higher Education, Berkshire, SRHE and Open University Press.

QS Intelligence Unit (2017), “Student mobility and demographic changes: QS reimagine education”.

Ramsden, P. (2003), Learning to Teach in Higher Education (Second), New York, NY, RoutledgeFalmer.

Reinhardt, M.M. (2010), “The use of deep learning strategies in online business courses to impact student retention”, American Journal of Business Education, Vol. 3 No. 12, p. 49.

Ren, B. (2010), “The study on advantages and disadvantages of multiple-choice questions in language tests”, China Academic Journal Electronic Publishing House, Beijing, viewed 4 Jul 2015. 任蓓蓓. 论英语多项选择题的利与弊及其命题原则[J]. 北京:中国学术文献网络出版总库 2010.

Rodrigues, C., Bu, N. and Min, B. (2000), “Learners training approach preference: national culture as a determinant cross cultural management”, An International Journal, Vol. 7 No. 1, pp. 23-32.

Sambell, K., McDowell, L. and Montgomery, C. (2012), Assessment for Learning in Higher Education, Routledge.

Sánchez, E. (2013), “America’s dumbest idea: creating a multiple-choice test generation”, The Guardian, November, Vol. 13.

Scouller, K. (1998), “The influence of assessment method on students' learning approaches: multiple choice question examination versus assessment essay”, Higher Education, Vol. 35 No. 4, pp. 453-472.

Selvarajah, C. (2006), “Cross-cultural study of Asian and European student perception: the need to understand the changing educational environment in New Zealand. Cross cultural management”, An International Journal, Vol. 13 No. 2, pp. 142-155.

Seno-Alday, S. and Budde-Sung, A.E.K. (2016), “Closing the learning loop: a review of assessments in international business education”, Journal of Teaching in International Business, Vol. 27 Nos 2/3, pp. 68-87.

Shaw, S.M. (1999), “It’s true”, Asians Can’t Think. Time, 31 May, p. 23.

Shepard, L.A. (2000), “The role of assessment in a learning culture”, Educational Researcher, Vol. 29 No. 7, p. 4.

Stanger-Hall, K.F. (2012), “Multiple-choice exams: an obstacle for higher-level thinking in introductory science classes”, CBE – Life Sciences Education, Vol. 11 No. 3, pp. 294-306.

Tjosvold, D., Law, K. and Sun, H. (2003), “Collectivistic and individualistic values: their effects on group dynamics and productivity in China”, Group Decision and Negotiation, Vol. 12 No. 3, pp. 243-263.

Tran, L.T. (2008), “Unpacking academic requirements: international students in management and education disciplines”, Higher Education Research and Development, Vol. 27 No. 3.

Trompenaars, F. and Hampden-Turner, C. (2011), Riding the Waves of Culture: Understanding Diversity in Global Business, Nicholas Brealey Publishing.

UNESCO (2020), Global Flow of Tertiary-Level Students, United Nations Institute for Statistics, available at: www.uis.unesco.org (accessed 11 October 2020).

US Department of Education (2006), “A test of leadership: charting the future of US higher education”, Washington, DC, available at: www2.ed.gov/about/bdscomm/list/hiedfuture/reports/pre-pub-report.pdf (accessed 1 August 2017).

Uysal, H.H. (2010), “A critical review of the IELTS writing test”, ELT Journal, Vol. 64 No. 3, pp. 314-320.

Wan, T.Y., Chapman, D.W. and Biggs, D.A. (1992), “Academic stress of international students attending US universities”, Research in Higher Education, Vol. 33 No. 5, pp. 607-623.

Wang, Z. (2003), “How to scientifically assess the quality of test designed with multiple-choice questions”, Journal of Teachers Forum, Vol. 9, pp. 60-61, 王志玲. 如何通过多项选择题科学分析命题质量 [J], 天津 2003.

Watkins, D. and Hattie, J. (1985), “A longitudinal study of the approaches to learning of Australian tertiary students”, Human Learning: Journal of Practical Research & Applications.

Welsh, A.L. and Saunders, P. (1998), “Essay questions and tests”, in Teaching Undergraduate Economics: A Handbook for Instructors, edited by William B. Walstad and Phillip Saunders. New York, NY, McGraw-Hill, pp. 305-318.

Wen, M. (2009), “A comparative study on item validity of multiple choice questions and short answer questions”, China Academic Journal Electronic Publishing House, Beijing, Vol. 1. 文妙章. 多项选择题和简答题的题目效度对比研究 [J]. 北京:中国学术文献网络出版总库 2009.

Xiao, H. (2002), “The advantage and disadvantage of multiple-choice questions in tests”, Journal of Suihua Teachers College, Vol. 01.22 No. 2, p. 7172. 肖红. 探析 多项选择测试题型 的利与弊[M], 浙江宁波 2002.6.

Yamazaki, Y., Toyama, M. and Attrapreyangkul, T. (2018), “Cross-cultural differences in learning style and learning skills: a comparison of Japan, Thailand, and the USA”, In Handbook of Research on Cross-Cultural Business Education, IGI Global, pp. 160-182.

Yoon, E. and Portman, T.A.A. (2004), “Critical issues of literature on counseling international students”, Journal of Multicultural Counseling and Development, Vol. 32 No. 1, p. 33.

Further reading

Economist (2011), “2-17-2011. Exams in South Korea: the one-shot society”, available at: www.economist.com

Egri, C., Khilji, S., Ralston, D., Palmer, I., Girson, I., Milton, L., Richards, M., Ramburuth, P. and Mockaitis, A. (2012), “Do anglo countries still form a values cluster? Evidence of the complexity of value change”, Journal of World Business, Vol. 47 No. 2, pp. 267-276.

Marton, F., Wen, Q. and Wong, K.C. (2005), “Read a hundred times and the meaning will appear. Changes in Chinese university students’ views of the temporal structure of learning”, Higher Education, Vol. 49 No. 3, pp. 291-318.

U.S. Department of Education (2006), A Test of Leadership: Charting the Future of US Higher Education, Washington, DC, available at: www2.ed.gov/about/bdscomm/list/hiedfuture/reports/pre-pub-report.pdf (accessed 1 August 2017).

Wiliam, D. (2011), “What is assessment for learning?”, Studies in Educational Evaluation, Vol. 37 No. 1, pp. 3-14.

Acknowledgements

The views expressed in this paper are those of the authors and do not necessarily reflect the official policy or position of the US Air Force Academy, the Air Force, the Department of Defense or the US Government.

Corresponding author

Sandra Seno-Alday can be contacted at: sandra.seno-alday@sydney.edu.au

Related articles