Search results

The purpose of this paper is to find appropriate forms of analysis of multiple-choice questions (MCQ) to obtain an assessment method, as fair as possible, for the students. The authors intend to ascertain if it is possible to control the quality of the MCQ contained in a bank of questions, implemented in Moodle, presenting some evidence with Item Response Theory (IRT) and Classical Test Theory (CTT). The used techniques can be considered a type of Descriptive Learning Analytics since they allow the measurement, collection, analysis and reporting of data generated from students’ assessment.

A representative data set of students’ grades from tests, randomly generated with a bank of questions implemented in Moodle, was used for analysis. The data were extracted from a Moodle database using MySQL with an ODBC connector, and collected in MS Excel^TM worksheets, and appropriate macros programmed with VBA were used. The analysis with the CTT was done through appropriate MS Excel^TM formulas, and the analysis with the IRT was approached with an MS Excel^TM add-in.

The Difficulty and Discrimination Indexes were calculated for all the questions having enough answers. It was found that the majority of the questions presented values for these indexes, which leads to a conclusion that they have quality. The analysis also showed that the bank of questions presents some internal consistency and, consequently, some reliability. Groups of questions with similar features were obtained, which is very important for the teacher to develop tests as fair as possible.

The main contribution and originality that can be found in this research is the definition of groups of questions with similar features, regarding their difficulty and discrimination properties. These groups allow the identification of difficulty levels in the questions on the bank of questions, thus allowing teachers to build tests, randomly generated with Moodle, that include questions with several difficulty levels in the tests, as it should be done. As far as the authors’ knowledge, there are no similar results in the literature.

The purpose of this paper is to address the issue of objective knowledge operationalisation with specific focus on varying levels of scale items’ difficulty. The ultimate goal of the study was to develop a scale to measure objective wine knowledge, which would address the domain of wine knowledge and differentiate varying levels of consumer wine knowledge.

The process of items’ development was guided by recommendations suggested by DeVellis (2003) in his influential work on theory and application of scale development. Examination of items’ performance was conducted through a series of field tests with consumer samples (N = 756) in a US wine region. Item response theory (IRT) approach was applied for items’ testing. The developed items were analysed using the two-parameter logistic model in Mplus Version 5.

The study offers a 44-item test suitable for assessing wine knowledge across a broad spectrum of expertise. For example, if the goal is to assess wine knowledge differences among relatively knowledgeable respondents, a subset of more difficult items could be chosen. Alternatively, a test for novices could be constructed from easier scale’s items.

For researchers, the study offers conceptualisation of the wine knowledge domain, suggests a parsimonious instrument to measure the construct, offers a valid and reliable measure for use in testing theories of consumer knowledge and provides empirical evidence of the value and usefulness of the developed scale.

For professionals, the proposed test may be used to test consumer knowledge and to help assess a prospective employee’s general knowledge of wine. The test can also be given at hospitality programs, outreach and continuing education programs.

The current paper takes an alternative approach to classical test theory and offers an objective wine knowledge scale tested through IRT. This approach avoids shortcomings associated with classical measurement and offers an original scale that can discriminate among respondents with different levels of wine knowledge.

The paper investigates the suitability of the Rasch model for establishing data equivalence. The results based on a real data set are contrasted with findings from standard procedures based on CFA methods.

Sinkovics et al.'s data on technophobia was used and re‐evaluated using both classical test theory (CTT) (multiple‐group structural equations modelling) and Rasch measurement theory.

Data equivalence in particular and measurement in general cannot be addressed without reference to theory. While both procedures can be considered best practice approaches within their respective theoretical foundation of measurement, the Rasch model provides some theoretical virtues. Measurement derived from data that fit the Rasch model seems to be approximated by classical procedures reasonably well. However, the reverse is not necessarily true.

The more widespread application of Rasch models would lead to a stronger justification of measurement, in particular, in cross‐cultural studies but also whenever measures of individual respondents are of interest.

Measurement models outside the framework of CTT are still scarce exceptions in marketing research.

With a call for increased accountability for student learning across higher education, it is becoming more important for academic libraries to show their value to the greater university community with the use of quantitative data. This paper seeks to describe the development of an information literacy test at the University of Arizona to measure student learning in an online credit course. In order to measure the impact of an online course, a test that was statistically valid, and reliable was created by local librarians.

The methodology involved administering test items to undergraduate students enrolled in an online information literacy course and applying both classical test theory and item response theory models to evaluate the validity and reliability of test items. This study included the longitudinal and cross‐sectional development of test items for pre and post‐testing across different student groups. Over the course of two semesters, 125 items were developed and administered to over 1,400 students.

The creation of test of items and the process of making test items reliable and valid is discussed in detail. Items were checked for construct validity with the use of a national standardized test of information literacy (SAILS). Locally developed items were found to have a higher than average reliability rating.

The process described here offers a method for librarians without a background in assessment to develop their own statistically valid and reliable instrument.

One of the unique features of this research design was the correlation of SAILS items with local items to test for validity. Although SAILS items have been used by many libraries in the past, they have not been used to create new test items. The use of the original SAILS test items is a valuable resource for instruction librarians developing items locally.

Structural equation modelling (SEM) is a method that is very frequently applied by marketing and business researchers to assess empirically new theoretical proposals articulated by means of complex models. It is, therefore, a logical thought that the quality of the new advances in marketing and business theory depends, in part, on how well SEM is applied. This study aims to conduct an extensive review and empirical analysis of a broad variety of classic and recent controversies and issues related with the use of SEM, in order to identify problematic questions and prescribe a compendium of solutions for its suitable application.

The main analyses were conducted on a sample of 191 SEM‐based papers and 472 applications, i.e. all the SEM‐based studies published in four leading marketing journals during the period 1995‐2007.

Despite the maturity of SEM, its application in marketing research still has notable room for improvement. This is a general conclusion based on numerous problems detected and discussed here.

The study provides plausible solutions to the problems identified, a useful guide that is easy to follow and to apply adequately to present SEM issues in marketing and business studies.

The sample of SEM‐based papers and applications is limited to four publication outlets. A wider set or/and other journals different to those analyzed here may be preferred.

This is a valuable and timely study of the application of SEM in marketing and business research, and is also useful as a guiding framework for good practice. Likewise, as the problems discussed here presumably occur in other areas of social science, this paper should be welcome beyond the borders of the business disciplines.

This paper aims to present the results of a systematic review of the evidence on psychometric properties of information literacy (IL) tests.

A two-stage search strategy was used to find relevant studies in two subject and three general databases. A descriptive review of test characteristics and psychometric properties was presented. The review included 29 studies describing psychometric properties of 18 IL tests.

It was found that the classical test theory was applied for all tests. However, the item response theory was also applied in three cases. Most of the psychometric tests were developed in the USA using ACRL IL competency standards. The most commonly used psychometric analyses include content validity, discriminant validity and internal consistency reliability.

Only studies in English language are included in this review.

The study recommends that standards should be developed for the use and reporting of psychometric measures in designing IL tests. Librarians need to be trained in psychometric analysis of tests.

It is the first study that systematically reviewed psychometric properties of IL tests. The findings are useful for librarians who are teaching IL courses.

The research practice in management research is dominantly based on structural equation modeling (SEM), but almost exclusively, and often misguidedly, on covariance-based SEM. The purpose of this paper is to question the current research myopia in management research, because the paper adumbrates theoretical foundations and guidance for the two SEM streams: covariance-based and variance-based SEM; and improves the conceptual knowledge by comparing the most important procedures and elements in the SEM study, using different theoretical criteria.

The study thoroughly analyzes, reviews and presents two streams using common methodological background. The conceptual framework discusses the two streams by analysis of theory, measurement model specification, sample and goodness-of-fit.

The paper identifies and discusses the use and misuse of covariance-based and variance-based SEM utilizing common topics such as: first, theory (theory background, relation to theory and research orientation); second, measurement model specification (type of latent construct, type of study, reliability measures, etc.); third, sample (sample size and data distribution assumption); and fourth, goodness-of-fit (measurement of the model fit and residual co/variance).

The paper questions the usefulness of Cronbach's α research paradigm and discusses alternatives that are well established in social science, but not well known in the management research community. The author presents short research illustration in which analyzes the four recently published papers using common methodological background. The paper concludes with discussion of some open questions in management research practice that remain under-investigated and unutilized.

In this paper, the neoclassical theory of labour supply and a neo‐Keynesian theory of labour supply are investigated and the equilibrium prediction of the two theories are tested using cointegration methods. The Granger and Engle two step procedure is applied to quarterly Canadian labour supply data. This approach suggests that the existence of an equilibrium relationship in the data should coincide with the existence of a cointegrating vector for the individually non‐stationary time series. No such vector can be found for the neoclassical model but such a vector can be found for the neo‐Keynesian model.

The extent to which emerging multinational enterprises (EMNEs) challenge extant international management (IM) theories is a question under current discussion. The purpose of this paper is to confront two classic theories (internationalization process model (IPM) and post-acquisition integration types) to several EMNEs strategies within their initial conceptual frameworks, exploring how their practices lead to extend and update existing models.

This paper compares the classic IPM with the development stages of iconic EMNEs. This reveals how EMNEs’ strategies can be analyzed within extant conceptual frameworks, extending their theoretical content. This approach is then applied to an empirical study of post-acquisition integrations conducted in France by EMNEs from 11 countries of origin.

Two theories are discussed, with suggested updates, within their extant frameworks, taking into account EMNEs’ strategies. First, the initial IPM is re-explored to reveal concentrated paths, from all quadrants of the framework. Similarly, post-integration typologies are updated, including the partnering approach frequently implemented by EMNEs, and the holding approach empirically identified with some specific features (subjection).

This paper combines an analysis of some iconic EMNEs’ internationalization processes and empirical data on up-market acquisitions by EMNEs from diverse countries of origin. It provides suggestions to update two IM theories.

The purpose of this paper is to measure the degree of item difficulty in translation multiple-choice items in terms of 1-parameter logistic (1-PL) model of the item response theory (IRT). Also, the paper proposes a hypothesis in which a participant who answers a translation test possesses some amount of translation competence which affects the end-result.

In total, 150 translation students from the Bachelor of Arts in Translation Studies from the three Iranian universities participated in this research paper. The translation participants were requested to answer the questions. The items were formulated in such a way that the question was stated in English and the four choices were written in Farsi. To interpret the obtained results, this research paper employed 1-PL and 2-parameter logistic (2-PL) models using Stata (2016). In addition, to demonstrate results in terms of 1-PL, item characteristic curves (a graphical representation showing the degree of difficulty of each item) was used.

Using Stata platform, the findings of this research paper showed that through the application of IRT, evaluators were able to calculate the difficulty degree of each items (1-PL) and correspondingly the translation competence (2-PL) of each participant.

One of the limitations is the proportionately small number of translation participants at the Bachelor of Arts.

Although a few number of studies concentered on the role of translation competence, there did not exist any research focusing on translation competence empirically in higher education.