Measuring the degree of difficulty and translation competence: The two intertwining variables in modern translation multiple-choice items testing

Abstract

Purpose

The purpose of this paper is to measure the degree of item difficulty in translation multiple-choice items in terms of 1-parameter logistic (1-PL) model of the item response theory (IRT). Also, the paper proposes a hypothesis in which a participant who answers a translation test possesses some amount of translation competence which affects the end-result.

Design/methodology/approach

In total, 150 translation students from the Bachelor of Arts in Translation Studies from the three Iranian universities participated in this research paper. The translation participants were requested to answer the questions. The items were formulated in such a way that the question was stated in English and the four choices were written in Farsi. To interpret the obtained results, this research paper employed 1-PL and 2-parameter logistic (2-PL) models using Stata (2016). In addition, to demonstrate results in terms of 1-PL, item characteristic curves (a graphical representation showing the degree of difficulty of each item) was used.

Findings

Using Stata platform, the findings of this research paper showed that through the application of IRT, evaluators were able to calculate the difficulty degree of each items (1-PL) and correspondingly the translation competence (2-PL) of each participant.

Research limitations/implications

One of the limitations is the proportionately small number of translation participants at the Bachelor of Arts.

Originality/value

Although a few number of studies concentered on the role of translation competence, there did not exist any research focusing on translation competence empirically in higher education.

Citation

Akbari, A. (2020), "Measuring the degree of difficulty and translation competence: The two intertwining variables in modern translation multiple-choice items testing", Journal of Applied Research in Higher Education, Vol. 12 No. 3, pp. 475-494. https://doi.org/10.1108/JARHE-11-2018-0248