Abstract
Purpose
The purpose of this article is to show the gains that can be made if researchers were to use gain-probability (G-P) diagrams.
Design/methodology/approach
The authors present relevant mathematical equations, invented examples and real data examples.
Findings
G-P diagrams provide a more nuanced understanding of the data than typical summary statistics, effect sizes or significance tests.
Practical implications
Gain-probability diagrams provided a much better basis for making decisions than typical summary statistics, effect sizes or significance tests.
Originality/value
G-P diagrams provide a completely new way to traverse the distance from data to decision-making implications.
Keywords
Citation
Trafimow, D., Wang, Z., Tong, T. and Wang, T. (2023), "Gain-probability diagrams as an alternative to significance testing in economics and finance", Asian Journal of Economics and Banking, Vol. 7 No. 3, pp. 333-357. https://doi.org/10.1108/AJEB-05-2023-0045
Publisher
:Emerald Publishing Limited
Copyright © 2023, David Trafimow, Ziyuan Wang, Tingting Tong and Tonghui Wang
License
Published in Asian Journal of Economics and Banking. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
1. Introduction
Consider a medical scenario implying a government financing decision. Suppose researchers invent a medicine to treat a disease. Should the government invest to render the medicine more widely available to its citizens? At present, a widely accepted procedure would be to perform a significance test to determine if the medicine is effective at a statistically significant level. If so, that would constitute a reason to make the investment; if not, the government would not invest. However, significance testing is a poor way to evaluate the effectiveness of the medicine. It is quite possible for a medicine not to be effective, but nevertheless achieve statistical significance, if the sample of cases is sufficiently large. This is because statistical significance depends on two items: the size of the effect and the sample size. It is possible for a sufficiently large sample size to compensate for even an extremely small effect. The p-values upon which significance testing depends confound the effect sizes researchers obtain with the sample sizes they collect. In fact, McQuitty (2004, 2018) famously argued against too large sample sizes on the grounds that they ensure statistical significance even when there is not much of an effect. Of course, a problem with this argument is that statistical significance is not the only matter of concern. There is the much more important issue of the extent to which sample sizes are sufficiently large to engender confidence that the sample statistics provide good estimates of population parameters. The larger the sample size, the better the estimate. Thus, from the point of view of estimation, McQuitty's famous advice is contraindicated. In addition, the fact that the advice was so well received by business and economics researchers and cited over 1,000 times and received an accolade by Journal of Global Scholars of Marketing Science, points to one of many problems with significance testing thinking (Trafimow et al., 2021). There have been many criticisms of significance testing, including several reviews (Hubbard, 2016; Ziliak and McCloskey, 2016; Trafimow, 2019) and 43 articles in the 2019 special issue of The American Statistician, the highly respected journal of The American Statistical Association. Because the disadvantages of significance testing have been covered so extensively, there is no point in rehashing them here. Rather, the present goal is to present a more useful alternative.
In the medical scenario, there are many factors to consider. These include the seriousness of the disease, the cost of the medicine and many others. In addition, and crucially, there is the issue of how well the medicine works. Put in the form of a question, what is the probability that a randomly selected person who takes the medicine will be better off, or worse off, and by how much, with the medicine than without it?
Nor must we restrict ourselves to economics issues pertaining to medicine. If an economic intervention is proposed to increase people's living space, what is the probability that a randomly selected person will be better off, or worse off, and by how much living space, with the intervention than without it? Or if a government wishes to institute a policy to increase income in a poor area, what is the probability that a person will be better off, or worse off, and by how much income, with the new policy than without it? The issue of probabilities of being better off, or worse off, by varying degrees, is relevant to many potential economics applications.
To estimate the probability of being better off, or worse off, by varying degrees, with the intervention or policy change than without it, it is necessary to look carefully at the data to determine the distribution. We will consider two families of distributions: skew normal distributions, which include normal distributions, and delta log-skew-normal (LSN) distributions, which include lognormal distributions. The subsequent section includes crucial equations, with subsections for skew normal and delta LSN distributions. Following that, we provide examples of how to use the equations to construct gain-probability (G-P) diagrams to draw conclusions far beyond that which significance tests allow (Tong et al., 2022; Trafimow et al., 2022; Wang et al., 2022).
2. Probability of being better off or worse off by how much? Skew normal distributions
Whereas normal distributions have two parameters, mean μ and standard deviation σ; skew normal distributions have three parameters. The location ξ replaces the mean, the scale ω replaces the standard deviation, and there is a shape (or skewness) parameter α. When the shape parameter equals 0 the distribution is normal, and the location equals the mean and the scale equals the standard deviation. But when the shape parameter does not equal 0, the location does not equal the mean and the scale does not equal the standard deviation. In symbols, when α = 0, ξ = μ and ω = σ; but when α ≠ 0, ξ ≠ μ and ω ≠ σ. The family of normal distributions is a subset of the family of skew normal distributions. Thus, the family of skew normal distributions is more generally applicable than the family of normal distributions.
Let us now derive the probability that a randomly chosen person from one skew normal population has a higher score on the dependent variable than a randomly chosen person from another skew normal population. We will assume dependent populations at any level of correlation between −1 and +1, with independent populations (correlation = 0) as a special case.
Azzalini and Valle (1996) A random vector
2.1 Calculation of δab = P(Z > 0) in independent distributions
As we mentioned above, Let X ∼ SN(ξ1, ω1, α1), Y ∼ SN(ξ2, ω2, α2) and we assume that X and Y are independent. We want to obtain the distribution of the linear combination, Z = X + aY + b, of X and Y for specified
There are two advantages of using Z = X + aY + b. Both advantages stem from the possibility that a researcher might be interested in more than simply a probabilistic advantage for one group over another. A researcher might be interested in the probabilities of being better off, or worse off, to varying degrees. The equation provides two ways to assess this. One way is by setting a = −1 and varying b, to obtain the probability that a randomly selected person from one condition will score higher than a randomly selected person from another condition, by the amount the researcher specifies by setting b to that value. This is the most straightforward advantage.
A second advantage is that the equation provides a way to assess the probability of being better off, or worse off, by varying degrees, in terms of multiples. For example, what is the probability that a randomly selected person from one group will score twice as much, thrice as much and so on, as a randomly selected person from the other group? The equation facilitates such an assessment. The researcher merely sets b = 0 and lets a vary at multiples of interest.
Let
Here ϕ(z; ν, τ2) is the pdf of the normal distribution N(ν, τ2) with mean ν and variance τ2, and Φ2(u; 02, Δ) is the cumulative distribution function (cdf) of the bivariate normal distribution with mean vector 02 and covariance Δ.
The proof of Theorem 2.1 is given in Appendix. Note that the pdf of Z given in Equation (2.2) is the special case of the closed skew normal density. For details of closed skew normal distributions, see Gupta et al. and Zhu et al.
Now we can compute the probability P(Z > 0), which is given by
Density curves of Z for different parameters and their corresponding P(Z > 0) are given in Figures 1 and 2, respectively. These values can be instantiated into the equations presented earlier, but there is an easier way too. We provide a freely available online calculator at https://probab.shinyapps.io/inde_prob/
2.2 Calculation of δρab = P(U > 0) in dependent distributions
Now we consider the bivariate skew normal random vector
Then we have the following result and its proof.
Let X ∼ SN2(ξ, Σ, α) given above and consider U = X1 + aX2 + b with
with
The proof of Theorem 2.2 is given in Appendix.
The probability δρab = P(U > 0) is given by
Density curves of U for different parameters and their corresponding probability P(U > 0) are given in Figures 3 and 4, respectively. For example, in Figure 3, we can see that the density curves are affected by the location parameter ξ2 = 0, 2, 4, 2 with other parameters specified and ρ = 0.25, together with their corresponding probabilities P(U > 0) = 0.433816, 0.566184, 0.226627, 0.10565, respectively. We provide a different online calculator, still freely available online at https://probab.shinyapps.io/ProbU/
2.3 Estimation of δab = P(Z > 0) and δρab = P(U > 0)
The above two theorems imply that if the location, scale and shape parameters are known in two populations, one can calculate the probabilities δab = P(Z > 0) and δρab = P(U > 0). But in most situations in real life the population parameters are not known. Thus estimating δab and δρab is necessary to be studied. The parametric estimation of P(X > Y) for normal distributions in the context of probabilistic environmental risk assignment was discussed by Jacobs et al. Here, we consider the method of moment estimator (MME) and maximum likelihood estimator (MLE). The estimators obtained in this two ways are denoted by
Method of moment estimator (MME) Equation (2.6), below, relates location to mean and scale to standard deviation.
Rewriting Equation (2.6) in terms of sample estimates, as opposed to population parameters, renders Equation (2.9).
Then we can obtain the
Maximum likelihood estimator (MLE) The log-likelihood function for the skew normal distribution is given by
To derive the MLEs of the parameters in skew normal distribution, we take the partial derivatives of the lnL functions with respect to parameters of interest and equal them to zero. Then the corresponding MLEs are obtained by solving the following equations:
By the invariance property of MLEs, we obtain
To derive
by using the same methods as in the independent setting. Then by substituting the MMEs and MLEs of ξ∗, σ2 and α∗ in Equation (2.5), we can get
3. Probability of being better off or worse off by how much? Delta log-skew-normal distributions
The log-normal distributions is very popular in both theory and applications of probability and statistics. However, when modeling complex random phenomena in many applied areas, there are a real need of more flexible models which extend the log-normal. Lin and Stoyanov introduced the LSN distribution which is an extension for modeling positive data using log-normal models. A positive random variable X is LSNly distributed (i.e. X ∼ LSN(μ, σ, α)) if the logarithm of X is skew normally distributed with location parameter μ, scale parameter σ and slant parameter α. Let φ(.) and Φ(.) be respectively probability density function (pdf) and cumulative distribution function (cdf) of the standard normal distribution, then we have that
Density curves, fX(x), of X for parameters μ = 0, σ = 0.5 with different values of α = −1, 0, 1 are given in Figure 5. From Figure 5, we can see that the slant parameter α plays an important role in LSN family. The mean and variance of X are respectively given by
The following is the definition of the multivariate LSN distribution.
The random vector X = (X1, …, Xp)′ is said to have a p-dimensional LSN distribution if log X ∼ SNp(μ, Σ, α), the multivariate skew-normal distribution with location parameter μ, scale matrix Σ, and the skewness parameter α, denoted by X ∼ LSNp(μ, Σ, α). The pdf of X is
Note that, in this paper, we only consider the family of the bivariate distributions, p = 2, for both dependent and independent cases.
In life sciences it is common for data to contain a relatively large number of zeros. In many real situations, it is appropriate to model such data using a LSN distribution for the positive values, together with an additional probability mass at all zeros. This type of distribution is commonly referred to as the delta LSN, defined below.
The random variable X is said to have a delta-LSN distribution with parameters δ, μ, σ, and α, denoted as X ∼ DLSN(δ, μ, σ2, α), if the pdf of X is given by
Note that the requirement of
with δ0 = δ*1δ*2, δ1 = (1 − δ*1)δ*2, δ2 = δ*1(1 − δ*2) and H(x1, x2) = F(x1)G(x2) for all x1, x2 > 0.
3.1 Linear combination of independent delta log-skew-normal distributions
Let
Let
(i) for a < 0,
(ii) for a > 0,
The proof of Theorem 3.1 is given in Appendix. Note that if a = 0, then Z = X + b so that we only have the univariate case.
Now we can compute the probability P(Z > 0), which is given by.
(i) for a < 0,
(ii) for a > 0,
Here is a link of shiny apps and its corresponding R code is available upon the request: https://dlnprobability.shinyapps.io/dlsn_independent/.
3.2 Bivariate delta log-skew-normal distribution
Now we consider the bivariate delta LSN random vector
Then we have the following result and its proof is given in Appendix.
Let X ∼ DLSN2(δ, μ, Σ, α) given above and consider U = X1 + aX2 + b with constants a ≠ 0 and
(i) for a < 0
(ii) for a > 0
Here is a link of shiny apps and its corresponding code is available upon the request: https://dlnprobability.shinyapps.io/dlsn_dependent/.
4. Applications: invented examples
To explain the possibilities, we invented examples. First, we examine skew normal examples and then lognormal examples.
4.1 Skew normal examples
Suppose an economist is interested in comparing salaries of males versus females in a particular geographical area. In this geographical area, based on large samples of randomly selected males and females, suppose the mean salaries are $45,000 for both males and females, the standard deviations are $1,000 for each, with skews of 0.5 and −0.5 for males and females, respectively. Because the mean salary is the same for males and females, there seems no reason to promote a policy change to redress inequality in salaries.
However, let us convert the foregoing statistics to estimates of skew normal parameters: location, scale and shape. After making the conversion, the estimated locations are $43,947.79 and $46,052.21 for males and females, respectively; the estimated scales are $1,451.601 for both males and females (the squared value is $2107144.694); and the estimated distribution shapes are 2.173758 and − 2.173758 for males and females, respectively. We saw earlier that, going by means, there is no relative advantage for males or females. However, we now see that going by locations, there is a relative disadvantage for males relative to females. The difference in means implies a neutral policy whereas the difference in locations implies a policy to increase male salaries to bring them to the level of female salaries. However, neither difference answers the real question, which is: What is the probability of males being advantaged or disadvantaged, relative to females and by how much? Fortunately, the foregoing skew normal mathematical equations allow us to answer this question, especially with the aid of the program that can be accessed via the following link: https://probab.shinyapps.io/inde_prob/.
Figure 6 contains a G-P diagram that provides the probabilities that males will be advantaged (black bars) or disadvantaged (gray bars) relative to females. As will be seen presently, G-P diagrams provide the advantage that researchers can assess probabilistic advantages or disadvantages, at varying degrees of extremity. Note that each gray bar is higher than its corresponding black bar, demonstrating a consistent advantage for females over males. For example, the probability that a randomly selected male would have an advantage of 0–500 is 0.134 whereas the probability that a randomly selected female would have a similar advantage is 0.143. In general, there is asymmetry favoring females over males. Thus, using skew normal statistics to construct a G-P diagram can lead to different conclusions than remaining with normal statistics. Note, too, that if we perform a significance test on the means, the typical t-test would come out consistent with the null hypothesis of no difference whereas Figure 1 shows a very clear difference.
Or consider another example to illustrate the potential importance of seeming trivial skewness. Consider the deductible for doctor visits based on two insurance companies labeled INSURE-A and INSURE-B. For both insurance companies, the location is $60 and the scale is $10 (squared scale is $100). The difference is that the skewness for INSURE-A is 0.10 whereas it is −0.10 for INSURE-B. The small skewness magnitude is so small for both companies that the distributions are very near normal, and typical statisticians and data analysts would treat the distributions as normal. The obvious conclusion is that the two insurance companies are equivalent. However, constructing a G-P diagram suggests a very different conclusion.
Figure 7 shows that despite the two insurance companies having equivalent locations, scales and skews seemingly only trivially different from 0, the G-P diagram shows that a randomly selected doctor's visit is much more likely to be varying degrees more expensive with INSURE-A than with INSURE-B. Each black bar, representing INSURE-A resulting in more expensive payments, is substantially higher than its corresponding gray bar, representing INSURE-B resulting in more expensive payments. The G-P diagram is quite asymmetric; the black bar (INSURE-A) probabilities are higher than corresponding gray bar (INSURE-B) probabilities. Thus, if the goal is to reduce the expected payment amount, the G-P diagram indicates INSURE-B is a substantially statistically better bet than is INSURE-A, despite the seeming equivalence based on similar locations. More generally, Figure 7 shows that seeming trivial skewness differences can have extreme consequences.
4.2 Lognormal examples
Consider time wasted per hour at work in two different industries. Suppose that time wasted is lognormally distributed in companies in Business A and Business B. In Business A, the mean time wasted is 11.373, the standard deviation is 2.888 and the variance is 8.342. In Business B, the mean is 11.078, the standard deviation is 12.376 and the variance is 153.158. The effect size is 0.04, which is trivial, so the obvious conclusion is that type of business has little to do with time wasted per hour. Too, with such a small effect size, a significance test likely would come out insignificant with typical sample sizes.
However, let us now consider lognormal statistics. The log transformed mean and standard deviation for Business A is 2.4 and 0.25 (variance is 0.0625), respectively. These values are 2.0 and 0.9 (variance is 0.81) for Business B. Thus, the probability that a randomly selected person in Business A will waste more time than a randomly selected person in Business B is approximately 0.67, despite the miniscule conventional effect size in the previous paragraph. In addition, it is possible to construct a G-P diagram with more nuanced information. One capability of G-P diagrams is that effects can be reported as multiples. For example, it is possible to concern ourselves with the probability that time-wasting is twice, thrice, etc. that in one business relative to the other. The G-P diagram renders clear that which would be obscured otherwise; time-wasting is much more of a problem for Business A than for Business B (see Figure 8).
Imagine two large cities, City A and City B and an economist is concerned with comparing commuting times. Suppose commuting times are lognormally distributed and the mean commuting time for City A is 41.264 min, the standard deviation is 74.054 and the variance is 5484.041. For City B, these values are 30.114, 33.641 and 1131.691, respectively. There seems an obvious advantage for City B over City A in that the mean commute is shorter for City B than for City A by 11.15 min.
The mean of the logarithmically transformed distribution for City A is 3.0, the standard deviation is 1.2 and the variance is 1.44. For City B, these values are 3.0, 0.90 and 0.81, respectively. The probability that a randomly selected person from City A would have a shorter commuting time than a randomly selected person from City B is 0.50; despite appearances based on the previous paragraph, there is no probabilistic advantage for City A over City B. Moreover, the G-P diagram is symmetric; as each black bar in Figure 9 is the same size as its corresponding gray bar, there is no probabilistic advantage for either city, no matter the multiple under consideration.
Consider another example, using the same log normal parameter values as the previous example, but with the addition of considering shape. Suppose that the shape parameter for City A is 0.10 and the shape parameter for City B is −0.10. In that case, the probability that a randomly selected person in City A would have a longer commuting tie than a randomly selected person in City B is 0.54, not 0.50. And the probability that a randomly selected person in City B would have a longer commuting time than a randomly selected person in City A is 0.46. Thus, in contrast to the previous example, the slight difference in shape parameters leads to a clear probabilistic disadvantage for City A relative to City B. In addition, in contrast to Figure 9 that is symmetrical, Figure 10 that illustrates the present example is asymmetric.
5. Application: real data
In the previous section, we examined invented examples to demonstrate the lessons that can be learned by constructing G-P diagrams. In this section, we present analyses of real data.
5.1 NBA guards (2022–2023 regular season)
We downloaded data from the following website: https://www.espn.com/nba/stats/player/_/season/2023/seasontype/2/table/defensive/sort/avgSteals/dir/desc. Among other types of information, it contains data for minutes played by point guards and shooting guards. Let us compare them using traditional normal statistics versus skew normal statistics.
The means for point guards and shooting guards are 24.0 and 19.8, respectively. The standard deviations are 8.8 and 9.4, respectively. In addition, the skews are −0.3 and 0.2, respectively. Thus, point guards average 4.2 more minutes per game than shooting guards.
We might ask about how likely a randomly chosen point guard would average more minutes than a randomly chosen shooting guard. Based on the difference in averages, we might guess that the probability that randomly chosen point guard would average more minutes than a randomly chosen shooting guard would be greater than 0.50, but perhaps not too much more than that. But rather than guess, let us run out the calculations.
As usual, the first step is to find the skew normal values for location, scale and shape. For point guards these are 31.4, 11.5 (scale squared is 131) and −1.4, respectively. For shooting guards, these are 13.1, 11.6 (scale squared is 134) and 1.1, respectively. Based on these values, the probability that a randomly selected point guard would play more minutes per game than a randomly selected shooting guard is 0.63. Figure 11 provides a G-P diagram. An interesting aspect of the figure is that the middle two bars are not very far apart, thereby indicating that at small amounts of difference in minutes played, there is only a slight probabilistic advantage for point guards over shooting guards. However, comparing black against gray bars at greater extremes indicates a strong probabilistic advantage for point guards over shooting guards. The G-P diagram suggests a more subtle substantive story than does a mere difference in means or difference in locations.
5.2 Living space for males versus females
Hyman et al. (2002) obtained data comparing living space for their male and female participants, in square feet. The mean amount of living space for men is 2300 and the standard deviation is 705. The mean amount of living space for women is 2186 and the standard deviation is 671. The effect size is miniscule and not statistically significant: Cohen's d = 0.04, t(564) < 1. However, the data are lognormally distributed and a G-P diagram suggests a very different conclusion. To commence, let us find the means and standard deviations after a logarithmic transformation. The transformed mean and standard deviation for males is 7.69 and 0.31 (variance = 0.0961), respectively. The transformed mean and standard deviation for females is 7.64 and 0.32 (variance = 0.1024), respectively. Thus, the probability that a randomly selected male will have more living space than a randomly selected female is 0.54 whereas the probability that a randomly selected female will have more living space than a randomly selected male is only 0.46, a more impressive difference than is suggested by the Cohen's d or the nonsignificant p-value.
For a more fine-grained analysis, Figure 12 provides the G-P diagram. Because it is convenient to express gender effects with respect to living space in multiples, Figure 12 is in terms of multiples. That is, the black bars provide the probabilities that a randomly selected male will have 1.0 to 1.5 times as much living space as a randomly selected female, 1.5 to 2.0 times as much living space and so on. The gray bars provide analogous probabilistic advantages for females. Each black bar is larger than its corresponding gray bar thereby indicating that although the probabilistic advantage for males over females is not large, it is not trivial either. Again, we see that a G-P diagram provides both more valid and more nuanced conclusions than typical differences between means, Cohen's d, or significance tests.
Consider precipitation in Buffalo for 306 days starting in June in 2019 versus 2020 [https://www.ncdc.noaa.gov/cdo-web/]. Because there are days when there is no precipitation, the data fall under the category of delta skew lognormal distributions. Thus, there are the following bullet-listed summary parameter estimates for each year:
locations are 2.8712 for 2019 and 2.8327 for 2020,
squared scales are 3.884841 for 2019 and 3.806401 for 2020,
shapes are −2.929 for 2019 and -3.019 for 2020 and
probabilities of zeroes (lack of precipitation for a day) are 0.49346 for 2019 and 0.59150 for 2020.
This is an especially interesting example because (a) it engages all the parameters of delta skew lognormal distributions and (b) ties are possible. A tie can occur if there is no precipitation on a randomly selected day in 2019 or 2020. The probability of more precipitation on a randomly selected day in 2019 than on a randomly selected day in 2020 is 0.404. The probability of more precipitation on a randomly selected day in 2020 than on a randomly selected day in 2019 is 0.304 and the probability of a tie is 0.292. Thus, there is a general probabilistic advantage for 2019 over 2020, assuming precipitation is positive. (If precipitation is negative, then there is a probabilistic disadvantage for 2019 relative to 2020. However, we will assume that precipitation is positive.)
Figure 13 shows a G-P diagram. However, the diagram differs from the others because ties are possible. The black bar in the middle shows the probability of ties. The dark gray bars show the probability of more precipitation in 2019 than in 2020, by varying millimeters. And the light gray bars show the probability of more precipitation in 2020 than in 2019, by varying millimeters. The figure shows that the probabilistic advantage for 2019 over 2020 is clear both at small amounts and at large ones.
6. Discussion
The G-P diagrams based on both invented and real data indicate important lessons for economics researchers. One lesson is that both summary statistics and significance tests can be very misleading. It is possible for a difference in means to hide that there is no probabilistic advantage for either group over the other. However, it is possible, too, for a lack of difference in means to hide a very sizable probabilistic advantage for one group over the other. One solution is to use more appropriate summary statistics. For example, if the data are obtained from a skew-normal distribution, locations, scales and shapes can be better than means and standard deviations. However, even distribution-appropriate sample statistics can be misleading. For instance, we have seen an example where the locations and scales of two groups are the same and the difference in skews very small. Nevertheless, the G-P diagram (Figure 7) indicated a very large probabilistic difference.
Although it is dramatic to show that traditional statistics and G-P diagrams can come to opposing conclusions, G-P diagrams have another advantage. Specifically, G-P diagrams can support subtle conclusions that cannot be addressed by either summary statistics or significance tests. Figure 11 provides a nice case in point. It shows that at relatively small differences in minutes played, there is only a miniscule probabilistic advantage for point guards over shooting guards. But at more extreme differences in minutes played, the probabilistic advantage for point guards increases substantially.
In conclusion, there is no need for economics researchers to constrain themselves by depending solely on summary statistics and null hypothesis significance tests. As we have seen, G-P diagrams provide opportunities to contradict the seeming implications of summary statistics or null hypothesis significance tests. In addition, even in cases where there is no contradiction, such as the data pertaining to minutes played by National Basketball Association (NBA) point guards and shooting guards, G-P diagrams provide much more nuanced information than do summary statistics and null hypothesis significance tests. We hope and expect that future economics researchers will avail themselves of the potential advantages to be enjoyed by exploiting the capabilities of G-P diagrams for providing subtle probabilistic information about comparisons of interest.
Figures
Source(s): Appendix by authors
In this section, we provide detailed proofs of Theorem 2.1, 2.2, 3.1, Theorem 3.2.
A.1 . Proof of Theorem 2.1
The joint pdf of X ∼ SN(ξ1, ω1, α1) and Y ∼ SN(ξ2, ω2, α2) is
Let Z = X + aY + b, Y = Y, then X = Z − aY − b, Y = Y and it is easy to see that the Jacobian J of this transformation is 1. Thus, we obtain the pdf of Z as
which is
Let
with T ∼ N(0, 1). If we denote
Therefore, the joint distribution of
A.2 . Proof of Theorem 2.2
Let c = (1,a)′, we are trying to find the distribution of U = X1 + aX2 + b = c′X + b. First we derive the moment generating function (mgf) of U:
It is easy to obtain that c′ξ + b = ξ1 + aξ2 + b and
Thus, δ∗ = δ′Σ1/2c(c′Σc)−1/2. By equation (2.1), we obtain that U ∼ SN(ξ1 + aξ2 + b, σ2, α∗) with
A.3 . Proof of Theorem 3.1
The joint pdf of
For x > 0, y > 0, let Z = X + aY + b, Y = Y so that X = Z − aY − b, Y = Y. It is easy to see that the Jacobian J of this transformation is 1. Thus, the pdf of Z as
For x = 0, y > 0, we have x = 0, Z = aY + b and the pdf of Z is
Similarly, for x > 0, y = 0, the pdf of Z is
Note that the sign of a determines the range of Z. If a < 0, then the pdf of Z = X + aY + b is
Also if a > 0, the pdf of Z is given by
so that desired result follows.
A.4 . Proof of Theorem 3.2
Note that from Equation (3.4), we know that the joint pdf of
For x1 > 0, x2 > 0, let U = X1 + aX2 + b, X2 = X2, then X1 = U − aX2 − b, X2 = X2. It is easy to see the Jacobian J of this transformation is 1. Thus, we obtain the marginal pdf of U as
For x1 = 0, x2 > 0, let x1 = 0, U = aX2 + b and the Jacobian of this transformation is 1/a so that the pdf of U is
Similarly for x1 > 0, x2 = 0, the pdf of U is
Therefore the pdf of U = X1 + aX2 + b is, for a < 0,
and for a > 0, the pdf of U is
Thus the proof of Theorem 3.2 is completed.
References
Azzalini, A. and Valle, A.D. (1996), “The multivariate skew normal distribution”, Biometrika, Vol. 83 No. 4, pp. 715-726, doi: 10.1093/biomet/83.4.715.
Hubbard, R. (2016), Corrupt Research: the Case for Reconceptualizing Empirical Management and Social Science, Sage Publications, Los Angeles, CA.
Hyman, M.R., Ganesh, G. and McQuitty, S. (2002), “Augmenting the household affluence construct”, Journal of Marketing Theory and Practice, Vol. 10 No. 3, pp. 13-31, doi: 10.1080/10696679.2002.11501917.
McQuitty, S. (2004), “Statistical power and structural equation models in business research”, Journal of Business Research, Vol. 57 No. 2, pp. 175-183, doi: 10.1016/S0148-2963(01)00301-0.
McQuitty, S. (2018), “Reflections on ‘Statistical power and structural equation models in business research’”, Journal of Global Scholars of Marketing Science, Vol. 28 No. 3, pp. 272-277, doi: 10.1080/21639159.2018.1434806.
Tong, T., Wang, T., Trafimow, D. and Wang, C. (2022), “The probability of being better or worse off, and by how much, depending on experimental conditions with skew normal populations”, in SriboonchittaKreinovich, W.Y.E.S.V. (Ed.), Credible Asset allocation, Optimal Transport Methods, and Related Topics (141-149), Springer-Verlag.
Trafimow, D. (2019), “A frequentist alternative to significance testing, p-values, and confidence intervals”, Econometrics, Vol. 7 No. 2, pp. 1-14, available at: https://www.mdpi.com/2225-1146/7/2/26
Trafimow, D., Hyman, M.R., Kostyk, A., Wang, C. and Wang, T. (2021), “The harmful effect of null hypothesis significance testing on marketing research: an example”, Journal of Business Research, Vol. 125, pp. 39-44, doi: 10.1016/j.jbusres.2020.11.069.
Trafimow, D., Hyman, M.R., Kostyk, A., Wang, Z., Tong, T., Wang, T. and Wang, C. (2022), “Gain-probability diagrams in consumer research”, International Journal of Market Research, Vol. 64 No. 4, pp. 470-483, doi: 10.1177/14707853221085509.
Wang, Z., Wang, T., Trafimow, D. and Xu, Z. (2022), “A different kind of effect size based on samples from two populations with delta log-skew-normal distributions”, in Ngoc Thach, N., Ha, D.T., Trung, N.D. and Kreinovich, V. (Eds), Prediction and Causality in Econometrics and Related Topics. ECONVN 2021. Studies in Computational Intelligence, Springer, Cham, Vol. 983, doi: 10.1007/978-3-030-77094-5_10.
Ziliak, S.T. and McCloskey, D.N. (2016), The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives, The University of Michigan Press, Ann Arbor, MI.
Acknowledgements
It has come to the attention of the publisher that the article, Trafimow, D., Wang, Z., Tong, T. and Wang, T. “Gain-probability diagrams as an alternative to significance testing in economics and finance” published in Asian Journal of Economics and Banking https://doi.org/10.1108/AJEB-05-2023-0045 gave an incorrect location for Trafimow’s affiliation. This error was introduced in the editorial process and has now been corrected in the online version. The publisher sincerely apologises for this error.