Heuristics versus statistics in discriminant validity testing: a comparison of four procedures

Abstract

Purpose

The purpose of this paper is to review and extend recent simulation studies on discriminant validity measures, contrasting the use of cutoff values (i.e. heuristics) with inferential tests.

Design/methodology/approach

Based on a simulation study, which considers different construct correlations, sample sizes, numbers of indicators and loading patterns, the authors assess each criterion’s sensitivity to type I and type II errors.

Findings

The findings of the simulation study provide further evidence for the robustness of the heterotrait–monotrait (HTMT) ratio of correlations criterion as an estimator of disattenuated (perfectly reliable) correlations between constructs, whose performance parallels that of the standard constrained PHI approach. Furthermore, the authors identify situations in which both methods fail and suggest an alternative criterion.

Originality/value

Addressing the limitations of prior simulation studies, the authors use both directional comparisons (i.e. heuristics) and inferential tests to facilitate the comparison of the HTMT and PHI methods. Furthermore, the simulation considers criteria that have not been assessed in prior research.

Citation

Franke, G. and Sarstedt, M. (2019), "Heuristics versus statistics in discriminant validity testing: a comparison of four procedures", Internet Research, Vol. 29 No. 3, pp. 430-447. https://doi.org/10.1108/IntR-12-2017-0515