Search results

In industrial‐organizational psychology, research and practice has focused on the use of situational judgment tests to predict managerial job performance. Although there is considerable controversy over what these tests actually measure, many have argued that they at least partially measure practical intelligence or “common sense.” Therefore the purpose of this article is to discuss situational judgment tests as assessments of common sense.

Reviewing the relevant industrial‐organizational psychology literature, this article considers the history and development of situational judgment tests as well as how the field has defined situational judgment. It also reviews the empirical and theoretical literature on their effectiveness for assessing managerial decision‐making competence.

Situational judgment tests have been found to be effective predictors of many organizational outcomes in a variety of job settings and positions. In particular, they have been found to incrementally predict future performance on‐the‐job beyond other typical predictors. Situational judgment tests are also relatively easy to administer and score and are often well received by managers and job applicants.

This article suggests situational judgment tests offer an efficient method of assessing common sense that can easily be adapted to many employment settings.

This chapter examines EI, presents a history of EI including the various models, and a discussion of the three streams approach to classifying EI literature. The author advocates for the efficacy of the Stream One Ability Model (SOAM) of EI citing previous authors and literature. The commonly used SOAM instruments are discussed in light of recent studies. The discussion turns to alternate tests of the SOAM of EI including Situational Judgment Tests (SJTs). Recommendations include an analysis of SOAM instruments, a new approach to measurement, and increased use of SJTs to capture the four-branch ability model of EI.

Assessment gamification, which refers to the addition of game elements to existing assessments, is commonly implemented to improved applicant reactions to existing psychometric measures. This study aims to understand the effects of gamification on applicant reactions to and measurement quality of situational judgment tests.

In a 2 × 4 between-subjects experiment, this study randomly assigned 315 people to experience different versions of a gamified situational judgment test, crossing immersive game elements (text, audio, still pictures, video) with control game elements (high and low), measuring applicant reactions and assessing differences in convergent validity between conditions.

The use of immersive game elements improved perceptions of organizational technological sophistication, but no other reactions outcomes (test attitudes, procedural justice, organizational attractiveness). Convergent validity with cognitive ability was not affected by gamification.

This is the first study to experimentally examine applicant reactions and measurement quality to SJTs based upon the implementation of specific game elements. It demonstrates that small-scale efforts to gamify assessments are likely to lead to only small-scale gains. However, it also demonstrates that such modifications can be done without harming the measurement qualities of the test, making gamification a potentially useful marketing tool for assessment specialists. Thus, this study concludes that utility should be considered carefully and explicitly for any attempt to gamify assessment.

Branched situational judgment tests (BSJTs) are an increasingly common employee selection method, yet there is no theory and very little empirical work explaining the designs and impacts of branching. To encourage additional research on BSJTs, and to provide practitioners with a common language to describe their current and future practices, we sought to develop a theory of BSTJs.

Given the absence of theory on branching, we utilized a ground theory qualitative research design, conducting interviews with 25 BSJT practitioner subject matter experts.

Our final theory consists of three components: (1) a taxonomy of BSJT branching features (contingency, parallelism, convergence, and looping) and options within those features (which vary), (2) a causal theoretical model describing impacts of branching in general on applicant reactions via proximal effects on face validity, and (3) a causal theoretical model describing impacts on applicant reactions among branching designs via proximal effects on consistency of administration and opportunity to perform.

Our work provides the first theoretical foundation on which future confirmatory research in the BSJT domain can be built. It also gives both researchers and practitioners a common language for describing branching features and their options. Finally, it reveals BSJTs as the results of a complex set of interrelated design features, discouraging the oversimplified contrasting of “branching” vs “not branching.”

The purpose of this paper is to give an empirically‐based review of the strengths and weaknesses of situational judgment tests (SJTs).

The features, history, and development of SJTs are discussed. Next, a computerized search (from 1990‐2007) is conducted to retrieve empirical studies related to SJTs. The effectiveness of SJTs is discussed in terms of reliability, criterion‐related validity, incremental validity, construct‐related validity, utility, adverse impact, applicant perceptions, fakability, and susceptibility to practice and coaching effects.

Strengths of SJTs are that they show criterion‐related validity and incremental validity above cognitive ability and personality tests. SJTs have also less adverse impact towards minorities (especially if the cognitive loading of the SJT is low). Furthermore, applicant reactions towards SJTs are positive and SJTs enable to test large applicant groups at once (through the Internet). In terms of weaknesses, SJTs might be prone to faking, practice, and coaching effects. There is also debate about what constructs are measured by SJTs.

Five avenues for future research are discussed: construct‐related validity of SJTs, utility of SJTs vis‐à‐vis other predictors, impact of SJT features on validity and adverse impact, examination of alternative stimulus and response formats, and cross‐cultural transportability of SJTs.

Practitioners receive evidence‐based information about the features, development, and strengths and weaknesses of SJTs.

Apart from the USA, SJTs have not made strong inroads in selection practice in Europe and other parts of the world. This evidence‐based paper might highlight the value of SJTs.

We examined a large multi-year undergraduate leadership development program (LDP) across seven universities and used an integrated framework of transformational leadership and situational judgment tests (SJTs) during a critical and formative period of leadership development. This study was the first to show a significant relationship between experience and transformational leadership style in students in an undergraduate LDP using SJTs and the multifactor leadership questionnaire (MLQ). The results showed that greater experience was positively related to increased transformational leadership style and that high overall decision scores were indicated in all groups of students with varying leadership styles and varying experience levels and decision abilities. The study findings and implications are discussed, along with recommendations for leadership educators to develop decision quality in LDPs.

This chapter provides a contemporary view of state-of-the science research and thinking done in the areas of selection and classification. It takes as a starting point the observation that the world of work is undergoing important changes that are likely to result in different occupational and organizational structures. In this context, we review recent research on criteria, especially models of job performance, followed by sections on predictors, including ability, personality, vocational interests, biodata, and situational judgment tests. The paper also discusses person-organization fit models, as alternatives or complements to the traditional person-job fit paradigm.

This chapter reviews key research on the similarities and differences in leadership and management across different regions of the world. It also looks at similarities and differences on other relevant aspects, that is, commitment, work values, personality and emotional intelligence. Research has tended to focus on drawing out the differences as that appears to be worthy of news and attracts interest. We also report on the types of errors in research which might actually make real differences appear much larger. The reality is that what we find is a great deal of similarity in leadership and management behaviour across the different regions of the world. Given these similarities, can we develop a management level Situational Judgment Test (SJT) that can be used effectively across different world regions? We believe this can be achieved by identifying SJT items that work consistently across world regions and then assembling a bias-free test with robust psychometric properties.