Psychological assessment in human resource management: discrepancies between theory and practice and two examples of integration

Riccardo Sartori (Department of Human Sciences, Università degli Studi di Verona, Verona, Italy)
Arianna Costantini (Department of Human Sciences, Università degli Studi di Verona, Verona, Italy)
Andrea Ceschi (Department of Human Sciences, Università degli Studi di Verona, Verona, Italy)

Personnel Review

ISSN: 0048-3486

Article publication date: 2 November 2020

Issue publication date: 11 March 2022

13625

Abstract

Purpose

Psychological assessment refers to the process whereby different methods and techniques are used to test hypotheses about people and their psychological characteristics. Understanding employees' psychological makeup is key to allow effective human resource management, from hiring to retirement. However, the gap between scientific evidence and organizational practices dealing with psychological assessment is still great.

Design/methodology/approach

General review along with case study

Findings

This paper shows the differences between research and practice, i.e. between what scientific evidence suggests to assess people from a psychological point of view reliably and what practitioners do when they want to reach the same goal.

Originality/value

At the end of the article, two examples of integration between research and practice are presented. We discuss how methods and techniques of psychological assessment can be developed to both respect scientific criteria and meet specific organizational needs.

Keywords

Citation

Sartori, R., Costantini, A. and Ceschi, A. (2022), "Psychological assessment in human resource management: discrepancies between theory and practice and two examples of integration", Personnel Review, Vol. 51 No. 1, pp. 284-298. https://doi.org/10.1108/PR-05-2019-0281

Publisher

:

Emerald Publishing Limited

Copyright © 2020, Riccardo Sartori, Arianna Costantini and Andrea Ceschi

License

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode


Introduction

Assessment and psychological assessment

Assessment refers to a process aimed to deliver judgment and make an evaluation or decision (McDermott, 2012; Ceschi et al., 2017a, b, c). People can be assessed for several reasons, e.g. to monitor learning, to make a diagnosis, to decide who to hire (Sartori and Ceschi, 2013). When specifically aimed at investigating psychological characteristics, psychological assessments are carried out by using a combination of methods and techniques (Sartori and Pasini, 2007). These can either refer to the idiographic or clinical approach, aiming at a global evaluation of people, for example by using interviews, or to the nomothetic or psychometric approach, which focuses on a targeted assessment of specific features and mainly makes use of standardized instruments such as psychological tests (Luthans and Davis, 1981; Sartori, 2010).

Research has shown that the integration of different methods and techniques can increase the validity and reliability of psychological assessment and improve its predictive value (Sartori and Pasini, 2007; Sartori, 2010). Moreover, while assessment situations vary, the use of standardized tools is largely encouraged to avoid biases in evaluations (Sartori and Pasini, 2007). Besides, psychological assessment is often referred to as psychological testing, through which people can be described and differentiated based on a set of unidimensional psychological characteristics (Sartori, 2006), such as intelligence and personality traits (Sartori, 2006; Sartori and Pasini, 2007). These can be measured by a variety of different instruments, which have to meet criteria regarding validity, i.e. the extent to which they measure a specific construct, and reliability, i.e. the extent to which their results are consistent and stable over time (Sartori and Pasini, 2007; Sartori, 2010).

Psychological assessment in human resource management (HRM)

In the organizational context, psychological assessment is key to inform human resource management (HRM) including personnel selection (Dunlop et al., 2011; Lievens and De Soete, 2011; Vecchione et al., 2012; Sartori et al., 2016a, b), career development (Sartori et al., 2013a, b; Ceschi et al., 2017a, b, c; Bocciardi et al., 2017), talent management (Bray and Grant, 1966; Shore et al., 1998; Sartori and Rolandi, 2013, and training and coaching (Sartori et al., 2015a, b; Costantini et al., 2017; Costantini and Sartori, 2018). That is, psychological assessment represents a key component of HRM that can be used to assist employee-related decision-making, from employees' hiring to retirement (Sartori et al., 2013a, b, 2018).

Yet, despite the centrality of psychological assessment for HRM, a gap exists between evidence-based recommendations and organizational practices (Highhouse et al., 2016), where psychological instruments are rarely, if ever, employed (Ones et al., 2007). For example, according to a survey conducted among 1627 HR managers representing large organizations in the US, while 68% of employers engage in various forms of job skill testing, only 29% of them use one or more forms of psychological measurements (SIOP, http://www.siop.org/workplace/employment%20testing/usingoftests.aspx). These data are consistent with other findings showing that less than 20% of US companies currently use personality tests and that 82% of organizations do not use personality tests in the hiring or employee promotion process (Dattner, 2013).

Likely, research shows that when it comes to personnel selection, unstructured interviews are still the most common tool used to make hiring decisions despite abounding evidence on their lower validity and reliability compared to structured and standardized instruments (Sartori and Pasini, 2007; Sartori, 2010; Cubico et al., 2010). In a similar vein, a study conducted in the Netherlands showed that HR managers hold stronger intentions toward unstructured interviewing compared to structured interviewing (van der Zee et al., 2002). Similarly, in a study conducted in Italy among 21 HR managers and recruiters, participants perceived individual interviews as unavoidable to assess candidates' psychological characteristics (Sartori et al., 2017). Moreover, results from this study showed that psychological tests were perceived as lacking a fit with specific organizational needs or too time-consuming with regard to administration and analysis of results.

Overall, while generally larger organizations make more extensive use of psychological tests compared to small enterprises, evidence shows that interviews are nevertheless considered a final essential step of the selection process to allow real understanding of applicants' psychological characteristics (van der Zee et al., 2002). Such phenomena can be explained as the result of at least two cognitive biases, i.e. the illusion of control (Langer, 1975), which refers to the tendency to overestimate one's ability to control events, and the overconfidence effect, which occurs when subjective confidence in one's judgments is greater than one's objective accuracy (Sartori and Ceschi, 2013; Ceschi et al., 2019). For example, research shows that while HR managers are aware of the availability of standardized tests and instruments their beliefs regarding the validity of such tools are mixed and include perceptions of being skilled enough to reliably assess psychological traits through unstructured interviews (Sartori et al., 2017).

Personality assessment in HRM

In the literature, the importance and limited use of personality tests in organizations have been subject to considerable discussion. According to Morgeson et al. (2007), personality measures in HRM are useless because of their low validity and perceived problems with response distortion. On the other side, others argue that personality constructs have been shown to explain and predict attitudes, behaviors, performance and organizational outcomes (Ones et al., 2007), with hundreds of primary studies and dozens of meta-analyses indicating strong support for the use of personality measures in staffing decisions (Ones et al., 2007). Also, research has shown that employing different methods and techniques can improve predictive validity (Furnham et al., 2008; Gaugler et al., 1987; Goldstein et al., 1998; Hardison, 2006; Krause et al., 2006). Yet, studies reveal that the use of multiple methods is often considered expensive and time-consuming (Sartori and Ceschi, 2013; Krause et al., 2006), which lead HR managers to use intuitive and unstructured interviews (van der Zee et al., 2002). Moreover, from a methodological point of view, while unstructured interviews can pose problems in terms of reliability and validity (Ceschi et al., 2017a, b, c), the use of different methods and techniques can result in too much information that may be contradictory and difficult to manage, eventually leading to biased evaluations (Ceschi et al., 2019; Sartori and Ceschi, 2013).

Against this background, research is needed to shed light on how to develop assessment tools that can meet organizational needs while being at the same time reliable and valid. To reach this aim, researchers are called to be aware of the needs from applied contexts, i.e. to tailor and develop solutions that meet scientific criteria and are perceived as useful from practitioners. We present two such examples below, which highlight the efforts and ways to integrate the different needs from scholars and practitioners. In doing so, we aim to show how the two worlds of research and practice can be brought together to give life to tools of psychological assessment that are both valid and reliable from a scientific point of view, useful and useable from a professional point of view.

Case study 1: Development of a psychological test for an Italian health association

The first case study is about an Italian health association offering emergency first aid assistance in accidents, disasters and calamities (Sartori et al., 2014; Sartori and Ceschi, 2015). The association is composed of 12 branches with about 70 employees and 1500 volunteer rescuers who work in the ambulance. The occasion for the development of the specific psychological test presented here is the assessment and selection of the numerous candidate volunteer rescuers. Every year, 100 people are admitted for the training courses (two courses per year) at the end of which, if they pass the final test, they can access the association and operate as volunteer rescuers in the ambulance.

The organization, through its board of six directors, expressed the need for a psychological test with the following characteristics:

  1. Tailored on the population of volunteer rescuers of the association.

  2. Short and easy to administer, as well as valid and reliable.

  3. Not too selective from the personnel selection point of view since the organization needs volunteers to provide its services and the number of dropouts is generally high.

To reach these aims, the authors adopted an approach combining both qualitative and quantitative techniques (for further details, see Sartori et al., 2014, pp. 3039-3042).

The qualitative part comprised four focus groups with 45 volunteer rescuers divided into groups of 10–12 people each and two two-hour group discussions with the siz directors of the association, to define the characteristics of the test in terms of length, agility and psychological constructs to be measured. The two group discussions were carried out at the beginning and the end of the development process of the test. The authors decided to carry out fouor focus groups rather than only one or twoto access the highest number of ideas on the test to be developed, without, however, excessively prolonging this phase of data collection. Qualitative data were interpreted in the light of previous literature showing that volunteers are characterized by specific attitudes (Lammers, 1991; Sundeen, 1992; Chacon et al., 2011) and reasoning style (Haan et al., 1968; Briggs et al., 2010; Stolinski et al., 2004). Accordingly, it was established that the test should have measured two such constructs, i.e. one referring to the attitude candidate should have and the other referring to their reasoning. At the end of the qualitative phase, including data collection and interpretation in the light of previous evidence, the newly developed test was composed of 20 items, nine belonging to the dimension of attitude and 11 belonging to the dimension of reasoning (for further details, see Sartori et al., 2014, Tables 1 and 2).

The items measuring attitude and reasoning were either newly developed or drawn from previous research, even though adapted to the target. For attitude, the response scale ranged from 1 = completely false, to 6 = completely true, with an even number of options to avoid responses on the central point. Also, items measuring attitude were all reversed, meaning that higher scores indicated a less desirable attitude aligned with the values of the association. This methodological choice was thoroughly discussed during focus groups and group discussions and made according to the concept of face validity in personnel selection contexts (see: Sartori, 2010). The 11 items measuring reasoning comprised tasks or problems with only one correct answer. Each item had six alternatives, following the 6-point rating scale of the items measuring attitude (Burro et al., 2011). Moreover, the items measuring attitude and reasoning were mixed, a methodological choice thoroughly discussed in the light of literature (Nunnally, 1978; Kline, 1998).

Before going on with the administration of the test, 11 items collecting personal data were developed. The 20-item test resulting from theoretical considerations and the qualitative phase previously described was first administered to a pilot sample of 54 volunteers who did not participate in the focus groups and then to 481 participants to assess the properties of the newly developed test (see Tables 3 and 4).

After reversing all the answers to the items measuring attitude and transforming the answers to the 11 reasoning items in 0–1 score (where 0 = wrong answer, 1 = right answer), descriptive statistics (i.e. frequencies, percentages; minimum, maximum, mode, median, mean and SD skewness and kurtosis) were computed to test whether the variables fit a normal distribution, an assumption to be respected before proceeding with inferential statistics (Sartori, 2006). Then, two separate sets of principal component analyses were computed, one for the items measuring attitude and one for the items measuring reasoning (see Tables 5 and 6).

Finally, item-total correlations and Cronbach's alpha coefficients were computed as reliability measures (for further details, see Sartori et al., 2014, pp. 3043-3049). Results were in line with the theoretical considerations and showed, for example, that the correlation between attitude and reasoning was not statistically significant suggesting that the measured attitude and reasoning are different psychological constructs, as hypothesized.

Since its validation (Sartori et al., 2014), the test has been used to assess and select the candidate volunteer rescuers who want to enter the association. So far, about 3000 people (without considering the 481 participants belonging to the test sample) have been tested. The association expressed full satisfaction with the test and its results. Validity, reliability and statistical norms are computed after every administration with satisfactory results. Interestingly, the statistical norms resulting from the 54 people of the pilot sample overlap with those calculated on the overall sample of people tested so far. This finding suggests that the development of the test in conjunction with the target population led to a highly reliable tool. Finally, so far, a percentage varying from 5 to 11% of candidates have been found “not adequate” according to the statistical norms, meaning that the newly developed test, as requested, is not too selective.

Case study 2: Development of a personality test for the assessment of candidates and employees

FLORA (Sartori, 2014; Sartori et al., 2016a, b) is the name of an Italian personality test developed for the assessment of specific professional profiles in organizations and based on the five-factor model (FFM, also referred to as the Big Five model; Goldberg, 1981, 1990; McCrae and Costa, 1999). This test was commissioned by a consulting firm dealing with personnel selection, assessment and development. The consulting firm expressed the need for developing an evidence-based personality test able to identify the most relevant dimensions during assessment for different professional profiles. Hence, while the final version of the test is composed of many items referring to several personality dimensions, each dimension is weighted and has different validity and reliability indexes based on the specific professional profile to be assessed.

From a theoretical point of view, the FFM was chosen because it allows for identifying a number of basic dimensions describing individual differences in personality and professional profiles in organizations (Holland, 1966; Rothmann and Coetzer, 2003; van der Linden et al., 2010; Soto et al., 2011). According to the FFM, five personality traits, i.e. agreeableness, conscientiousness, emotional stability, extraversion and openness, can explain and predict individual differences over a wide range of settings, including job performance (Ones et al., 2007; Barrick and Mount, 1991; Barrick et al., 2001; Rothmann and Coetzer, 2003). Moreover, evidence from research shows that it is possible to detect lower-order traits, which contribute to describing different facets of the five personality traits. Findings showed that these facets can range from 12 (Mount et al., 1999) to 45 (Hofstee et al., 1992), passing through 18 (Saucier and Ostendorf, 1999), 30 (Costa and McCrae, 1992), 32 (Schmit et al., 2000) and 44 (Hogan and Hogan, 1992). Also, previous studies have shown that the five factors are relevant to different cultures (McCrae and Costa, 1997; McCrae et al., 2005; De Fruyt et al., 2004) and have been found consistently in factor analyses of peer- and self-ratings of trait descriptors involving diverse conditions, samples and factor extraction and rotation methods (Costa and McCrae, 1988; Grucza and Goldberg, 2007). Yet, in the Italian context, while many personality tests exist, also based on the FFM, none of them was specifically designed to be used in organizations and detect different facets of the five main factors. For example, the Big Five Questionnaire (BFQ – Caprara et al., 1993) and the Big Five Questionnaire 2 (BFQ 2 – Caprara et al., 2007) were developed to measure only the five main factors but not their facets and were not focused to the organizational context.

Against this background, FLORA, an Italian psychometric test developed based on the FFM, expressly aims at assessing personality in specific professional profiles described by numerous facets. Given the specific characteristics that the test was supposed to have, the process of its development and validation was split into two phases:

  1. A qualitative phase (i.e. test development), consisting of interviews to employees to detect the personal characteristics involved in successful performance, literature review to organize the characteristics previously detected according to the FFM, theoretical construction and development of the first version of the test;

  2. A quantitative phase (i.e. validation process), consisting of the administration of the first version of the test to a validation sample and, after changes due to exploratory statistical analyses, to a confirmation sample for confirmatory statistical analyses, monitoring of concurrent validity and calculation of the correlations between the test and job performance.

In the qualitative phase, 32 interviews with 16 different job profiles were carried out (for further details, see Sartori et al., 2016a, b, p. 2057). Two organizational psychologists were involved for each interview, one as a primary interviewer, the other one as an assistant taking notes. Each interview was audio-registered. Audio registrations and notes were given to five organizational psychologists who worked together for the extrapolation of the personal characteristics emerged in interviews and the categorization of the personal characteristics according to the Big Five (for further details on the procedure, see Barrick and Mount, 1991, pp. 8-9). Characteristics such as abilities, capabilities, skills, competencies, aptitudes and attitudes were eliminated to keep personality traits only (78% out of all the characteristics emerged). As for personality traits, synonyms and antonyms referring to the same characteristic were unified under one label. The personality traits not related to the Big Five, such as the ones referring to the honesty–humility dimension of the HEXACO model (Ashton and Lee, 2007), were eliminated. Content analyses of interviews led to the identification of 28 different personality traits involved in successful performance.

Based on the FFM, these 28 personality traits were organized into the following five categories:

  1. Extraversion, comprising eight dimensions (i.e. activism; autonomy; influence; initiative; interactivity; leadership; multitasking; velocity);

  2. Sociability, comprising six dimensions (i.e. care; collaboration; communicativeness; interpersonal sensitivity; positive affectivity; supportiveness);

  3. Conscientiousness, comprising five dimensions (i.e. accomplishment; constancy; deliberateness; precision; reliability);

  4. Openness, comprising five dimensions (i.e. curiosity; deepening; flexibility; inventiveness; learning);

  5. Emotionality, comprising five dimensions (i.e. emergency management; frustration tolerance; self-control; stress tolerance).

Each dimension was labeled and operationally defined according to the literature and the organizational aims of the test (Sartori, 2014). For each of the 28 dimensions, six items were generated, three positively and three negatively worded. In the end, 168 items were developed. Another eight items, drawn from the literature and aimed at measuring social desirability (Crowne and Marlowe, 1960; Manganelli Rattazzi et al., 2000), were added to form a Lie Scale. All the 176 items were randomized and accompanied by a 7-point rating scale, ranging from 1 = totally disagree; to 7 = totally agree.

The quantitative part involved a validation sample composed of 407 employees and a confirmation sample composed of 418 employees (for further details, see Sartori et al., 2016a, b, pp. 2058–2069) (see Tables 7 and 8).

As for the exploratory analyses, principal factor analyses (PFA) and principal component analyses (PCA) with the criterion of eigenvalue > 1 and different rotation methods (oblique and orthogonal) were carried out to explore the latent structure underlying the items and to monitor construct validity (factor loading cut-off = 0.30;, cf. Cronbach and Meehl, 1955; Kline, 1993, 1998). Based on the factor solutions obtained through exploratory analyses, confirmatory analyses were carried out using structural equation models with maximum likelihood estimator to test the robustness of the factor models previously identified. Analyses were carried out for each trait separately (extraversion, sociability, conscientiousness, openness and emotionality) and, within each trait, for each dimension of FLORA. Besides, the items belonging to the Lie Scale were analyzed, and correlation indexes (r) and coefficients of determination (r2) were computed between each dimension of FLORA and the Lie Scale total score to test whether and how each dimension was affected by social desirability. Second-order factor analyses (PFA and PCA) were carried out to test whether FLORA's dimensions overlapped with the original FFM. Also, Cronbach's alphas were calculated as reliability measures in terms of internal consistency between items.

Concurrent validity was also tested by administering FLORA together with the test presented in the first case study to 1028 subjects. Moreover, in line with research by Rothmann and Coetzer (2003) and van der Linden et al. (2010), FLORA was administered to 220 trade agents to test whether and how different facets were associated to job performance expressed in terms of sales figures (Sartori et al., 2016a, b). Overall, results from the different analyses conducted showed that the different dimensions of FLORA are sufficiently uncorrelated to each other and with the Lie Scale measuring social desirability, suggesting that the test was appropriately developed according to both the theoretical model and the needs of the consulting firm and is now a valid and reliable tool for personality assessment. Also, the correlations between attitude and reasoning measured by the test presented above and the different facets of FLORA were aligned with previous literature. Based on these results, FLORA is currently an Italian personality test based on the FFM and measuring 24 personality dimensions and a Lie Scale. It is composed of 149 items, 78 of which positively worded, 71 negatively worded. Moreover, three new dimensions have been added lately, based on emerging needs from different organizations, i.e. impulsivity (belonging to emotionality), openness to diversity and openness to change (both belonging to openness).

The characteristics of FLORA, which was developed starting from interviews to employees, seem to meet the criteria to make it useable for the assessment of specific professional profiles in organizations. Hence, it has reached the goal to be both a scientific instrument and a professional tool.

Final considerations and implications

While psychological assessment in organizations can contribute to better decision-making related to HR functions, often psychological tests may sound cumbersome to practitioners and employees (Hogan et al., 1996; Sartori et al., 2015a, b). As a result, personality profiles and other outputs from psychological assessment may sound meaningless or even abstruse to managers and decision-makers. Think about the Minnesota Multiphasic Personality Inventory (MMPI), for example, which is a standardized psychometric test of adult personality and psychopathology measuring 10 clinical dimensions which is also used in employment (Zapata-Sola et al., 2009). In this paper, we aimed to present findings from research attempting to fill the research-practice gap regarding psychological assessment in organizations.

The cases presented above show examples of how it is possible to develop instruments that are accessible and understandable to practitioners based on specific assessment needs. In doing so, the studies reviewed different processes that can be used to create assessment tools based on validated theoretical models, which are valid and reliable and meet organizational needs. Accordingly, a main implication of this contribution is the recognition of the possibilities deriving from the integration of research and practice. The studies show that the different needs from research and practice can be not only acknowledged but also integrated, leading to a process where research and practice enrich each other and result in assessment tools that are valid to both researchers and practitioners.

The first instrument presented in this article solved the problem of assessing a large number of candidate volunteer rescuers when they want to access an Italian health association. The collaboration between the association and academia has resulted in an instrument which is, as expected, dedicated to the population of candidate volunteer rescuers, valid and reliable, short and easy to administer, not too selective. The second instrument presented in the article, FLORA, has filled a gap in the context of Italian personality tests. It is a test composed of 24 (now 27) dimensions grouped according to the FFM and useable to assess different professional profiles. Feedback from the consulting firm, which has continually proposed the use of FLORA to its clients since the publication of the test (Sartori, 2014), is positive. Specifically, clients report being satisfied with the language used in the test and its outputs, perceived as accessible.

In conclusion, this article stems from the desire to show how the world of research and the world of practice can meet to develop psychological assessment tools that are both valid and reliable, actually useable in the HRM perspective. The main implication is that the combination of practice and research can give birth to valid and reliable psychological assessment tools that companies and organizations can trust and use for their psychological assessment activities included in HRM.

Position in the test, wording, origin and psychological constructs measured by the nine items belonging to the dimension named Attitude

Position in the testWordingOriginPsychological construct
Item 1I've always wanted to be a volunteer in the (name of the health association), since I was a babyAdaptedSocial desirability
Item 3I do not feel sorry for the people who are directly responsible for their own miseryBEES (Balanced Emotional Empathy Scale by Mehrabian)Empathy
Item 6I prefer to let things take their course rather than trying to understand why they go in a certain wayTAS-20 (Toronto Alexithymia Scale by Taylor, Bagby and Parker)Alexithymia
Item 8If my income were superior to my needs, I'd surely give all the extra money to charity16PF (Sixteen Personality Factor Test by Cattell)Social desirability
Item 10I have no sympathy for people who cause their illnesses with their lifestyle (diabetes, cardiovascular disease, lung cancer, etc.)BEES (Balanced Emotional Empathy Scale by Mehrabian)Empathy
Item 12When I am with others, I prefer that we talk about everyday things rather than making deep speeches about our emotions and what we feelTAS-20 (Toronto Alexithymia Scale by Taylor, Bagby and Parker)Alexithymia
Item 14Criticism from others have no effect on me16PF (Sixteen Personality Factor Test by Cattell)Social desirability
Item 15When I am asked something I do not know, I always say “I do not know” rather than pretending I know it16PF (Sixteen Personality Factor Test by Cattell)Social desirability
Item 17I have more mood swings than most people I know16PF (Sixteen Personality Factor Test by Cattell)Emotional stability

Position in the test, wording, origin and psychological constructs measured by the 11 items belonging to the dimension named Reasoning

Position in the testWordingOriginPsychological construct
Item 2The opposite of the opposite of “Opposite” is…16PF (Sixteen Personality Factor Test by Cattell)Logic (verbal reasoning)
Item 4“I did it!”, says the liar who sometimes speaks the truth. Based on the information in your possession, you can conclude that…DevelopedAttribution
Item 5“Frozen” is to “Cold” as “Mountain” is to…16PF (Sixteen Personality Factor Test by Cattell)Logic (verbal reasoning)
Item 7If I say that fields are up and clouds are down, then I can also say that…16PF (Sixteen Personality Factor Test by Cattell)Logic (abstract reasoning)
Item 9“You did it!”, says the wife who wants to blame the husband. Based on the information in your possession, you can conclude that…DevelopedAttribution
Item 11“I did it!”, says the culprit who committed the crime. Based on the information in your possession, you can conclude that…DevelopedAttribution
Item 13“You did it!”, says the wife who saw the husband commit the crime. Based on the information in your possession, you can conclude that…DevelopedAttribution
Item 16“I did it!”, says the innocent who did not commit the crime. Based on the information in your possession, you can conclude that…DevelopedAttribution
Item 18“I did it!”, says the sincere who sometimes lies. Based on the information in your possession, you can conclude that…DevelopedAttribution
Item 19If the brother of Andrea's father is Bruno's father, what is the degree of relationship that exists between Andrea's father and Bruno?16PF (Sixteen Personality Factor Test by Cattell)Logic
Item 20Four containers are marked with the numbers 1, 2, 3 and 4. Each container contains as twice quantity of liquid as the container marked with the number immediately preceding, so the container number 4 contains as twice quantity of liquid as the container number 3, which in turn contains as twice quantity of liquid as the container number 2, and so on. If we fill up the container number 3, which is empty, pouring from the container number 4, which is filled, how many containers number 1 can still be half filled with the liquid remaining in the container number 4?16PF (Sixteen Personality Factor Test by Cattell)Logic (numerical reasoning)

Characteristics of the pilot sample

Age
F%MinMaxModeMedianMeanSD
Males2648.1%19633637.540.310.7
Females2851.9%2153243535.211.3
Total sample54100%1963243737.711.1

Characteristics of the test sample

Age
F%MinMaxModeMedianMeanSD
Males23148.0%1865203334.213.0
Females25052.0%1866192931.111.9
Total sample481100%1866193032.612.5

Principal component analysis (varimax rotation with Keiser normalization, extraction of 2 dimensions, factor loading cut-off = 0.30) run on the 9 items belonging to the dimension called attitude

ConstructComponent
12
Item 10Empathy0.705
Item 6Alexithymia0.595
Item 12Alexithymia0.572
Item 3Empathy0.542
Item 17Emotional stability0.488
Item 14Social desirability 0.470
Item 8Social desirability 0.755
Item 1Social desirability 0.693
Item 15Social desirability 0.566
Variance explained 22.2%13.3%

Principal component analysis (varimax rotation with Keiser normalization, extraction of 2 dimensions, factor loading cut-off = 0.30) run on the 11 items belonging to the dimension called reasoning

ConstructComponent
12
Item 11Attribution0.755
Item 16Attribution0.680
Item 13Attribution0.677
Item 2Logic (verbal reasoning)0.4630.340
Item 4Attribution0.462
Item 18Attribution0.400
Item 9Attribution0.398
Item 7Logic (abstract reasoning) 0.682
Item 19Logic 0.370
Item 5Logic (verbal reasoning) 0.319
Item 20Logic (numerical reasoning) 0.310
Variance explained 18.2%17.0%

Characteristics of the validation sample

Age
F%MinMaxModeMedianMeanStandard deviation
Males17542.9%1761363740.1911.41
Females23257.1%1961323537.3813.03
Total sample407100%1761363738.5812.43

Characteristics of the confirmation sample

Age
F%MinMaxModeMedianMeanStandard deviation
Males15837.8%1761363639.3211.20
Females26062.2%1861333638.2012.10
Total sample418100%1761363638.6211.76

References

Ashton, M.C. and Lee, K. (2007), “Empirical, theoretical, and practical advantages of the HEXACO model of personality structure”, Personality and Social Psychology Review, Vol. 11, pp. 150-166.

Barrick, M.R. and Mount, M.K. (1991), “The Big Five personality dimensions and job performance: a meta-analysis”, Personnel Psychology, Vol. 44, pp. 1-26.

Barrick, M.R., Mount, M.K. and Judge, T.A. (2001), “Personality and performance at the beginning of the new millennium: what do we know and where do we go next?”, International Journal of Selection and Assessment, Vol. 9, pp. 9-30.

Bocciardi, F., Caputo, A., Fregonese, C., Langher, V. and Sartori, R. (2017), “Career adaptability as a strategic competence for career development: an exploratory study of its key predictors”, European Journal of Training and Development, Vol. 41, pp. 67-82.

Bray, D.W. and Grant, D.L. (1966), “The assessment center in the measurement of potential for business management”, Psychological Monographs: General and Applied, Vol. 80, pp. 1-27.

Briggs, E., Peterson, M. and Gregory, G. (2010), “Toward a better understanding of volunteering for nonprofit organizations: explaining volunteers' pro-social attitudes”, Journal of Macromarketing, Vol. 30, pp. 61-76.

Burro, R., Sartori, R. and Vidotto, G. (2011), “The method of constant stimuli with three rating categories and the use of Rasch models”, Quality and Quantity, Vol. 45, pp. 43-58.

Caprara, G.V., Barbaranelli, C. and Borgogni, L. (1993), BFQ. Big Five Questionnaire, Organizzazioni Speciali, Firenze.

Caprara, G.V., Barbaranelli, C., Borgogni, L. and Vecchione, M. (2007), BFQ 2. Big Five Questionnaire 2, Organizzazioni Speciali, Firenze.

Ceschi, A., Costantini, A., Dickert, S. and Sartori, R. (2017a), “The impact of occupational rewards on risk taking among managers”, Journal of Personnel Psychology, Vol. 16, pp. 104-111.

Ceschi, A., Demerouti, E., Sartori, R. and Weller, J. (2017b), “Decision-making processes in the workplace: how exhaustion, lack of resources and job demands impair them and affect performance”, Frontiers in Psychology, Vol. 8, p. 313.

Ceschi, A., Costantini, A., Phillips, S.D. and Sartori, R. (2017c), “The career decision-making competence: a new construct for the career realm”, European Journal of Training and Development, Vol. 41, pp. 8-27.

Ceschi, A., Costantini, A., Sartori, R., Weller, J. and Di Fabio, A. (2019), “Dimensions of decision-making: an evidence-based classification of heuristics and biases”, Personality and Individual Differences, Personality and Individual Differences, Vol. 146, pp. 188-200.

Chacon, F., Perez, T., Flores, J. and Vecina, M.L. (2011), “Motives for volunteering: categorization of volunteers' motivations using open-ended questions”, Psychology in Spain, Vol. 15, pp. 48-56.

Costa, P.T. and McCrae, R.R. (1988), “Personality in adulthood: a six-year longitudinal study of self-reports and spouse ratings on the NEO personality inventory”, Journal of Personality and Social Psychology, Vol. 54, pp. 853-863.

Costa, P.T. and McCrae, R.R. (1992), Revised NEO Personality Inventory (NEO-PI-R) and NEO Five Factor Model (NEO-FFI) Professional Manual, Psychological Assessment Resources, Odessa.

Costantini, A. and Sartori, R. (2018), “The intertwined relationship between job crafting, work-related positive emotions, and work engagement. Evidence from a positive psychology intervention study”, The Open Psychology Journal, Vol. 112018 No. 11, pp. 210-221, doi: 10.2174/1874350101811010210.

Costantini, A., De Paola, F., Ceschi, A., Sartori, R., Meneghini, A.M. and Di Fabio, A. (2017), “Work engagement and psychological capital in the italian public administration: a new resource-based intervention programme”, SA Journal of Industrial Psychology, Vol. 43, p. 1413.

Cronbach, L.J. and Meehl, P.E. (1955), “Construct validity in psychological tests”, Psychological Bullettin, Vol. 52, pp. 281-302.

Crowne, D.P. and Marlow, D. (1960), “A new scale of social desirability independent of psychopathology”, Journal of Consulting Psychology, Vol. 24, pp. 349-354.

Cubico, S., Bortolani, E., Favretto, G. and Sartori, R. (2010), “Describing the entrepreneurial profile: the entrepreneurial aptitude test (TAI)”, International Journal of Entrepreneurship and Small Business, Vol. 11, pp. 424-435.

Dattner, B. (2013), “How to use psychometric testing in hiring”, Harvard Business Review, available at: https://hbr.org/2013/09/how-to-use-psychometric-testin/.

De Fruyt, F., McCrae, R.R., Szirmák, Z. and Nagy, J. (2004), “The five-factor personality inventory as a measure of the five-factor model: Belgian, American, and Hungarian comparisons with the NEO-PI-R”, Assessment, Vol. 11, pp. 207-215.

Dunlop, P.D., Morrison, D.L. and Cordery, J.L. (2011), “Investigating retesting effects in a personnel selection context”, International Journal of Selection and Assessment, Vol. 19, pp. 217-221.

Furnham, A., Jensen, T. and Crump, J. (2008), “Personality, intelligence and assessment center experts ratings”, International Journal of Selection and Assessment, Vol. 16, pp. 356-365.

Gaugler, B.B., Rosenthal, D.B., Thornton, G.C. and Bentson, C. (1987), “Meta-analysis of assessment center validity”, Journal of Applied Psychology, Vol. 72, pp. 493-511.

Goldberg, L.R. (1981), “Language and individual differences: the search for universals in personality lexicon”, Journal of Personality and Social Psychology, Vol. 59, pp. 1216-1229.

Goldberg, L.R. (1990), “An alternative ‘description of personality’: the Big Five factor structure”, Journal of Personality and Social Psychology, Vol. 59, pp. 1216-1229.

Goldstein, H.W., Yusko, K.P., Braverman, E.P., Smith, D.B. and Chung, B. (1998), “The role of cognitive ability in the subgroup differences and incremental validity of assessment center exercises”, Personnel Psychology, Vol. 51, pp. 357-374.

Grucza, R.A. and Goldberg, L.R. (2007), “The comparative validity of 11 modern personality inventories: predictions of behavioral acts, informant reports, and clinical indicators”, Journal of Personality Assessment, Vol. 89, pp. 167-187.

Haan, N., Smith, M.B. and Block, J. (1968), “Moral reasoning of young adults: political-social behavior, family background, and personality correlates”, Journal of Personality and Social Psychology, Vol. 10, pp. 183-201.

Hardison, C.M. (2006), “Construct validity of assessment center overall ratings: an investigation of relationships with and incremental criterion validity over big 5 personality traits and cognitive ability”, Dissertation Abstracts International: Section BL: The Sciences and Engineering, Vol. 66, p. 6959.

Highhouse, S., Doverspike, D. and Guion, R.M. (2016), Essentials of Personnel Assessment & Selection, Routledge, London.

Hofstee, W.K.B., de Rand, B. and Goldberg, L.R. (1992), “Integration of the Big Five and circumplex approaches to trait structure”, Journal of Personality and Social Psychology, Vol. 63, pp. 146-163.

Hogan, R.T. and Hogan, J. (1992), Hogan Personality Inventory Manual, Hogan Assessment Systems, Tulsa.

Hogan, R., Hogan, J. and Roberts, B.W. (1996), “Personality measurement and employment decisions: questions and answers”, American Psychologist, Vol. 51, pp. 469-477.

Holland, J.L. (1966), The Psychology Of Vocational Choice: A Theory of Personality Types and Model Environments, Blaisdel, Walthman.

Kline, P. (1993), The Handbook of Psychological Testing, Routledge, London.

Kline, P. (1998), The New Psychometrics, Routledge, London.

Krause, D., Kersting, M., Heggestad, E. and Thornton, G. (2006), “Incremental validity of assessment center ratings over cognitive ability tests”, International Journal of Selection and Assessment, Vol. 14, pp. 360-371.

Lammers, J.C. (1991), “Attitudes, motives, and demographic predictors of volunteer commitment and service duration”, Journal of Social Service Research, Vol. 14, pp. 125-140.

Langer, E.J. (1975), “The illusion of control”, Journal of Personality and Social Psychology, Vol. 32, pp. 311-328, doi: 10.1037/0022-3514.32.2.311.

Lievens, F. and De Soete, B. (2011), “Instruments for personnel selection in the 21st century: research and practice”, Gedrag en Organisatie, Vol. 24, pp. 18-42.

Luthans, F. and Davis, T.R.V. (1981), Idiographic Versus Nomothetic Approaches to Research in Organizations, Defense Technical Information Center, Fort Belvoir.

Manganelli Rattazzi, A.M., Canova, L. and Marcorin, R. (2000), “La desiderabilità sociale. Un'analisi di forme brevi della scala di Marlowe e Crowne”, TPM – Testing Psicometria Metodologia, Vol. 7, pp. 5-16.

McCrae, R.R. and Costa, P.T. (1997), “Personality trait structure as human universal”, American Psychologist, Vol. 52, pp. 509-516.

McCrae, R.R. and Costa, P.T.. (1999), “A Five-factor theory of personality”, in Pervin, L.A. and John, O.P. (Eds.), Handbook of Personality: Theory and Research, 2nd ed., Guilford Press, New York, NY, pp. 139-153.

McCrae, R.R., Terracciano, A. and 79 Members of the Personality Profiles of Cultures Project (2005), “Personality profiles of cultures: aggregate personality traits”, Journal of Personality and Social Psychology, Vol. 89, pp. 407-425.

McDermott, B.E. (2012), “Psychological testing and the assessment of malingering”, Psychiatric Clinics of North America, Vol. 35, pp. 855-876.

Morgeson, F.P., Campion, M.A., Dipboye, R.L., Hollenbeck, J.R., Murphy, K. and Smith, N. (2007), “Reconsidering the use of personality tests in personnel selection contexts”, Personnel Psychology, Vol. 60, pp. 683-729.

Mount, M.K., Barrick, M.R., Laffitte, L.J. and Callans, M.C.. (1999), Personal Characteristics Inventory User's Manual, Wonderlic Consulting, Libertyville.

Nunnally, J.C. (1978), Psychometric Theory, McGraw-Hill, New York, NY.

Ones, D.S., Dilchert, S., Viswesvaran, C. and Judge, T.A. (2007), “In support of personality assessment in organizational settings”, Personnel Psychology, Vol. 60, pp. 995-1027.

Rothmann, S. and Coetzer, E.P. (2003), “The Big Five personality dimensions and job performance”, Journal of Industrial Psychology, Vol. 29, pp. 68-74.

Sartori, R. (2006), “The bell curve in psychological research and practice: mith or reality?”, Quality and Quantity, International Journal of Methodology, Vol. 40, pp. 407-418.

Sartori, R. (2010), “Face validity in personality tests: psychometric instruments and projective techniques in comparison”, Quality and Quantity, International Journal of Methodology, Vol. 44, pp. 749-759, doi: 10.1007/s11135-009-9224-0.

Sartori, R. (2014), FLORA. Role Related Personal Profile. Manuale d'uso, Edizioni UTilia, Verona.

Sartori, R. and Ceschi, A. (2013), “Assessment and development centers: judgment biases and risks of using idiographic and nomothetic approaches to collecting information on people to be evaluated and trained in organizations”, Quality and Quantity, International Journal of Methodology, Vol. 47, pp. 3277-3288, doi: 10.1007/s11135-012-9718-z.

Sartori, R. and Ceschi, A. (2015), “Motivazioni al volontariato e rischi psicosociali”, Psicologia di Comunità, Vol. 1, pp. 97-110.

Sartori, R. and Pasini, M. (2007), “Quality and quantity in test validity: how can we be sure that psychological tests measure what they have to?”, Quality and Quantity. International Journal of Methodology, Vol. 41, pp. 359-374, doi: 10.1007/s11135-006-9006-x.

Sartori, R. and Rolandi, A. (2013), “Validation study of a model for the assessment of potential in Italian young professionals working in medium-sized to large companies in different business sectors”, Quality and Quantity, International Journal of Methodology, Vol. 47, pp. 2729-2737, doi: 10.1007/s11135-012-9683-6.

Sartori, R., Favretto, G. and Ceschi, A. (2013a), “The relationships between innovation and human and psychological capital in organizations: a review”, Innovation Journal, Vol. 18 No. 3.

Sartori, R., Ceschi, A., Dorofeeva, K. and Bellotto, M. (2013b), “Psychological assessment in human resources management: characteristics, biases and solutions”, European Journal of Management, Vol. 13, pp. 43-50.

Sartori, R., Ceschi, A., Cubico, S. and Favretto, G. (2014), “Quality and quantity in the construction and validation of a psychological test for the assessment and selection of candidate volunteer rescuers: the action-research in an Italian health association”, Quality and Quantity. International Journal of Methodology, Vol. 48, pp. 3037-3051, doi: 10.1007/s11135-013-9939-9.

Sartori, R., Ceschi, A. and Costantini, A. (2015a), “On decision processes in businesses, companies and organizations computed through a generative approach: the case of the agent-based modeling”, International Journal of Business Research, Vol. 15, pp. 25-38.

Sartori, R., Tacconi, G. and Caputo, B. (2015b), “Competence-based analysis of needs in VET teachers and trainers: an Italian experience”, European Journal of Training and Development, Vol. 39, pp. 22-42.

Sartori, R., Costantini, A. and Ceschi, A. (2016a), “The indirect relationship between neuroticism and job performance in Italian trade workers: a cross-sectional study”, in Di Fabio, A. (Ed.), Neuroticism: Characteristics, Impact on Job Performance and Health Outcomes, Nova Science Publishers.

Sartori, R., Ceschi, A., Costantini, A. and Scalco, A. (2016b), “Big five for work and organizations: FLORA (role related personal profile), an Italian personality test based on the five-factor model and developed for the assessment of candidates and employees”, Quality and Quantity. International Journal of Methodology, Vol. 50, pp. 2055-2071, doi: 10.1007/s11135-015-0250-9.

Sartori, R., Costantini, A., Ceschi, A. and Scalco, A. (2017), “Not only correlations: a different approach for investigating the relationship between the Big Five personality traits and job performance based on workers and employees' perception”, Quality and Quantity. International Journal of Methodology, Vol. 51, pp. 2507-2519, doi: 10.1007/s11135-016-0406-2.

Sartori, R., Costantini, A., Ceschi, A. and Tommasi, F. (2018), “How do you manage change in organizations? Training, development, innovation, and their relationships”, Frontiers in Psychology, Vol. 9, p. 313.

Saucier, G. and Ostendorf, F. (1999), “Hierarchical subcomponents of the Big Five personality factors: a cross-language replication”, Journal of Personality and Social Psychology, Vol. 76, pp. 613-627.

Schmit, M.J., Kihm, J.A. and Robie, C. (2000), “The development of a global measure of personality”, Personnel Psychology, Vol. 1, pp. 153-193.

Shore, L.M., Tetrick, L.E. and Shore, T.H. (1998), “A comparison of self-, peer, and assessor evaluations of managerial potential”, Journal of Social Behavior and Personality, Vol. 13, pp. 85-101.

Soto, C.J., John, O.P., Gosling, S.D. and Potter, J. (2011), “Age differences in personality traits from 10 to 65: Big Five domains and facets in a large cross-sectional sample”, Journal of Personality and Social Psychology, Vol. 100, pp. 300-348.

Stolinski, A.M., Ryan, C.S., Hausmann, L.R.M. and Wernli, M.A. (2004), “Empathy, guilt, volunteer experiences, and intentions to continue volunteering among buddy volunteers in an AIDS organization”, Journal of Applied Biobehavioral Research, Vol. 9, pp. 1-22.

Sundeen, R.A. (1992), “Differences in personal goals and attitudes among volunteers”, Nonprofit and Voluntary Sector Quarterly, Vol. 21, pp. 271-291.

van der Linden, D., te Nijenhuis, J. and Bakker, A.B. (2010), “The General Factor of Personality: a meta-analysis of Big Five intercorrelations and a criterion-related validity study”, Journal of Research in Personality, Vol. 44, pp. 315-327.

van der Zee, K.I., Bakker, A.B. and Bakker, P. (2002), “Why are structured interviews so rarely used in personnel selection?”, Journal of Applied Psychology, Vol. 87, pp. 176-184.

Vecchione, M., Alessandri, G. and Barbaranelli, C. (2012), “The five factor model in personnel selection: measurement equivalence between applicant and non-applicant groups”, Personality and Individual Differences, Vol. 52 No. 4, pp. 503-508, doi: 10.1016/j.paid.2011.11.014.

Zapata-Sola, A., Kreuch, T., Landers, R.N., Hoyt, T. and Butcher, J.N. (2009), “Personality assessment in personnel selection using the MMPI-2: a cross-cultural comparison”, International Journal of Clinical and Health Psychology, Vol. 9, pp. 287-298.

Corresponding author

Riccardo Sartori can be contacted at: riccardo.sartori@univr.it

Related articles