Search results

Utilising Item Response Theory (IRT) methodologies, the Armed Services Vocational Aptitude Battery (ASVAB) was examined for differential item functioning (DIF) on the basis of crossed gender and ethnicity variables. Both the Mantel‐Haenszel procedure and an IRT area‐based technique were utilised to assess the degree of uniform and non‐uniform DIF in a sample of ASVAB takers. Findings were mixed. At the item level, DIF fluctuated greatly. Numerous instances of DIF favouring the reference as well as the focal group were found. At the scale level, inconsistencies existed across the forms and versions. Tests varied in their tendency to be potentially biased against the focal group of interest and at times, performed contrary to expectations. Implications for the ASVAB as well as other g‐loaded selection instruments are considered.

The purpose of this paper is to investigate the measurement invariance of the Chinese version NEO PI‐R conscientiousness scale.

Based on Hofstede's cultural dimensions, it was predicted that certain items might exhibit culture‐related differential item functioning (DIF). The partial credit Rasch model was used to analyze the item responses. The authors also examined the impact of DIF on the measurement invariance of the overall conscientiousness scale using differential test functioning statistics.

Most of the predicted culture‐related DIF were supported. Although the results suggested a substantial proportion of items showing DIF, the conscientiousness scale functioned consistently across the two cultures under study, suggesting that observed group mean scores can be compared directly.

The authors demonstrate that an understanding of the culture differences may help when translating instrument across cultures to anticipate potential threats to measurement invariance. The current study employed student samples. Results of the study need to be replicated using diverse populations.

Assessment and selection instruments have been increasingly used across nations for HRM purposes. Organizations intending to establish global talent management systems need to evaluate and ensure the cross‐cultural equivalence of the assessment. Findings from the current study support the adoption of the translated conscientiousness scale in China.

This paper is one of the few in the literature that examines the measurement invariance using a confirmatory approach.

Large-Scale Assessments in Germany have shown that language-minority students as well as students with special educational needs (SEN) perform significantly less well than language-majority students or students without SEN. This performance gap may be related to a limited accessibility of the tests. One way to test whether assessments allow all students to demonstrate their knowledge in a comparable way is the analysis of differential item functioning (DIF). In this chapter, we evaluate DIF coefficients in order to examine group-specific difficulties in reading comprehension for language-minority students and students with SEN in the German National Educational Assessment.

In the first study, we investigate the assessment of reading literacy of language-minority learners and German monolinguals from low-SES families. We found only a few items with moderate DIF and no items with large DIF. This indicates that the reading assessment was equally valid for second-language learners and German monolingual students.

In our second study, we report about the psychometrically successful development of easy and more accessible reading tasks for students with SEN. Further analyses showed that DIF predominantly occurred in items that captured contents that are not necessarily covered in literacy instruction targeted at students with SEN.

The purpose of this study is to assess the psychometric properties of the Flourishing Scale (FS) by employing samples of Malaysian and Indonesian undergraduates. In addition, the present study seeks to identify aspects of the social-psychological functioning that were perceived differently by both samples.

The present study employs the descriptive design to provide descriptions about psychometric properties of the FS. In total, 351 first-year undergraduates were employed as samples of the study. The authors employed the Rasch measurement model framework using the quantitative approach to analyze the data.

Generally, the results showed that the FS demonstrated acceptable psychometric properties for both samples. However, the Malaysian and Indonesian samples differ significantly in half of the social-psychological functioning aspects.

One limitation of the study was the relatively small sample size students in one higher education institution in each country. In addition, in both countries, the undergraduates demonstrated obligatory roles toward their lecturers. Therefore, their response might not be truthful and only respond as requested.

Whilst social psychological functioning has been identified as important aspects of well-being, the construct has yet to be studied in the eastern culture in Malaysia and Indonesia.

Job satisfaction and leadership behavior are recognized by the organizational world as fundamental elements that influence the overall effectiveness of a company. However, as the first step for an adequate intervention on any of these variables, it is the evaluation. The purpose of this paper is to develop and validate two brief measures on job satisfaction and leadership behavior.

The sample was made up of 246 workers located in Bogota, Colombia. The study was an instrumental research. To collect validity evidence, the internal structure and the relationship with other variables were used. For the evaluation of equity, the differential item functioning was analyzed according to the sex of the participants. Reliability was estimated through the ordinal omega coefficient.

Both brief measures presented a unifactorial structure, where job satisfaction was measured by five items and leadership behavior by four items. On the other hand, only one item of leadership behavior showed differential item functioning; however, its magnitude was trivial. Also, convergent and discriminant evidence was provided for both measures, and the reliability levels were adequate.

The measures developed represents an effort to briefly measure job satisfaction and leadership behavior. Likewise, it constitutes two of the few instruments to measure job satisfaction and leadership behavior in Latin American, representing a good alternative for the measurement of the referred constructs in an organizational context.

This study aims to evaluate the suitability of emergency remote education (ERE) during the COVID-19 pandemic and subsequently during the return to in-person education (IPE), investigating aspects of perceived quality differences in these transitions that may have impacted the students’ learning process.

This study applied a 41-item instrument to undergraduate students in higher education institutions (HEIs) in Brazil. For the suitability group, 493 responses were collected, and for the return to IPE group, 187 responses were collected. Three calibration approaches were performed using item response theory, with the first two adjusting data independently, and the third one involving a multigroup estimation.

During ERE, students perceived an increase in course offerings, better organization of classes and more study materials. However, the larger volume of content posed challenges in class monitoring, content comprehension and concentration. Upon returning to IPE, students noticed more organized assessments and activities. They found favorable test environments and collaborative tasks with peers, but they experienced difficulties in accessing professors and with class/content organization.

By identifying the most suitable items for both teaching formats, HEIs can enhance teaching practices that are more satisfactory from the students’ perspective.

This study provides an item ranking procedure inspired by differential item functioning and differential test functioning statistics. In an atypical situation, the item ranking procedure demonstrated its ability to generate useful information for identifying and quantifying the longitudinal effects of perceived quality on students.

This study examined the psychometric properties of intention to drop out and study satisfaction measures for first-year South African students. The factorial validity, item bias, measurement invariance and reliability were tested.

A cross-sectional design was used. For the study on intention to drop out, 1,820 first-year students participated, whilst 780 first-year students participated in the study on satisfaction with studies. Confirmatory factor analysis (CFA), differential item functioning (DIF), measurement invariance and internal consistency were used to test the scales.

A one-factor structure was confirmed for both scales. For the intention to drop out scale, Items 3 and 4 were identified with statistically significant item bias; however, these differences had no practical impact. Except for scalar invariance for language, sufficient measurement invariance was established. No problematic items were identified for the study satisfaction scale.

In essence, this study provides evidence of two short measures that are culturally sensitive that could be used as short and valid measures across contextual boundaries as practically valuable tools to measure intention to drop out and study satisfaction in diverse and multicultural contexts.

This study contributes to limited research on bias and invariance analyses for scales that can be used in interventions to identify students at risk of leaving the university and utilising psychometric analyses to ensure the applicability of these two scales in diverse and multicultural settings.

The purpose of this chapter is to provide researchers a summary of some of the latest developments in item response theory (IRT), and to help these groups realize that psychometric tools can now be used for theory testing in addition to the traditional role of improving construct measurement. The author first reviews some of the fundamental tenets of classical test theory to contrast with IRT. He then describes recent advances in goodness-of-fit tests that have helped turn IRT into a model-testing tool. Finally, the author reviews several new test models that provide new flexibilities, summarizing summarize several examples of research that has used these new models in organizational research. At the end of this review, the author provides suggestions to help researchers better use these new IRT tools. Although there have been significant advances in IRT in the past decade, there has not been a systematic review of these developments. This review places those developments in context to provide readers a real appreciation of these breakthroughs.

Universities have shifted from face-to-face learning environments to e-learning in response to the COVID-19 pandemic. However, the sudden change to online teaching has raised concerns among lecturers about students' readiness for e-learning. This study investigates students' readiness for e-learning during the COVID-19 pandemic and specifically assesses any significant differences between students' gender, age, ethnicity, level of education, field of study and their readiness for an e-learning environment.

The study employed a non-experimental quantitative research design. Data were gathered from a sample of 298 undergraduate and 101 postgraduate students. WINSTEPS Rasch model measurement software was used to determine the reliability and validity of the research instrument. Descriptive, inferential statistics and differential item functioning (DIF) test were used to assess students' readiness for an e-learning mode of instruction with the latter specifically analysing students' demographic factors and their readiness for an e-learning environment.

Findings identified that most students are ready for an e-learning mode of instruction. Further analysis indicated that there were differences in students' readiness for e-learning based on their demographic profiles.

This study provides insights on students' readiness towards e-learning, discusses implications for e-learning practices in higher education institutions during the COVID-19 pandemic and offers recommendations for future research.

This study provides evidences of students' readiness for e-learning in respect to their gender, age, ethnicity, level of education and field of study. This information could help lecturers to reflect on their own teaching practices, adjust their teaching approaches and subsequently, develop appropriate e-learning methods that best suit the student diversity in their classrooms.

The purpose of the current study is to develop a generic and short questionnaire to measure work performance at the individual level – the Individual Work Performance Questionnaire (IWPQ). The IWPQ was based on a four‐dimensional conceptual framework, in which individual work performance consisted of task performance, contextual performance, adaptive performance, and counterproductive work behavior.

After pilot‐testing, the 47‐item IWPQ was field‐tested amongst a representative sample of 1,181 Dutch blue, pink, and white collar workers. Factor analysis was used to examine whether the four‐dimensional conceptual framework could be confirmed. Rasch analysis was used to examine the functioning of the items in more detail. Finally, it was examined whether generic scales could be constructed.

A generic, three‐dimensional conceptual framework was identified, in which individual work performance consisted of task performance, contextual performance, and counterproductive work behavior. Generic, short scales could be constructed that fitted the Rasch model.

A generic, short questionnaire can be used to measure individual work performance across occupational sectors. In future versions of the IWPQ, more difficult items should be added to improve discriminative ability at the high ranges of the scale.

This study shows that, using Rasch analysis, a generic and short questionnaire can be used to measure individual work performance.