Search results

This paper covers different types of research designs, like, longitudinal, cross-section and sequential design, experimental design (including factorial experimental design), and correlational design, with illustrative examples. This paper will help a scholar to know how choice of research design depends on a number of parameters, highlighted by author.

Education researchers have been urged to utilize causal inference methods to estimate the policy effect more rigorously. While randomized controlled trials (RCTs) are the gold standard for assessing causality, RCTs are infeasible in some educational settings, particularly when ethical concerns or high cost are involved. Quasi-experimental research designs are the best alternative approach to study educational topics not amenable to RCTs, as they mimic experimental conditions and use statistical techniques to reduce bias from variables omitted in the empirical models. In this chapter, we introduce and discuss the core concepts, applicability, and limitations of three quasi-experimental methods in higher education research (i.e., difference-in-differences, instrumental variables, and regression discontinuity). By introducing each of these techniques, we aim to expand the higher education researcher's toolbox and encourage the use of these quasi-experimental methods to evaluate educational interventions.

Our ongoing program of research works with teachers to help them become more autonomy supportive during instruction and hence more able to promote students’ classroom motivation and engagement.

We have published five experimentally based, longitudinally designed, teacher-focused intervention studies that have tested the effectiveness and educational benefits of an autonomy-supportive intervention program (ASIP).

Findings show that (1) teachers can learn how to become more autonomy supportive and less controlling toward students, (2) students of the teachers who participate in ASIP report greater psychological need satisfaction and lesser need frustration, (3) these same students report and behaviorally display a wide range of important educational benefits, such as greater classroom engagement, (4) teachers benefit as much from giving autonomy support as their students do from receiving it as teachers show large postintervention gains in outcomes such as teaching efficacy and job satisfaction, and (5) these ASIP-induced benefits are long lasting as teachers use the ASIP experience as a professional developmental opportunity to upgrade the quality of their motivating style.

Our ASIP helps teachers learn how to better support their students’ autonomy during instruction. The value of this teaching skill can be seen in teachers’ and students’ enhanced classroom experience and functioning.

Companies are starting to capitalize on the potential of experimental economics as a decision-making tool. Hewlett-Packard (HP) is one of such pioneering companies. Experiments, conducted at HP Labs, were used to test retailer contract policies in three areas: return, minimum advertised-price (MAP), and market development funds. The experimental design models the multifaceted contemporary market of consumer computer products. While the model is quite complex, participants were found to be effective decisions-makers and that their behavior is sensitive to variations in policies. Based on the experimental results, HP changed its policies; for example, it made the consequences for minimum advertisement price violations forward-looking as well as backward-looking. This line of research appears promising for complex industrial environments. In addition, methodological issues are discussed in the context of differences between business and academic economics experiments. Finally, the author speculates about potential future business applications.

Leadership training has led to a large amount of research due to the belief that such training can lead to (or more precisely cause) positive changes in followers’ behavior and work performance. This chapter describes some of the conditions necessary for research to show a causal relationship between leadership training and outcomes. It then describes different research designs, employed in leadership training research, and considers the types of problems that can affect inferences about causality. The chapter focuses on the role of randomization of leaders (e.g., into training vs. non-training conditions) as a key methodological procedure and alludes to problems of achieving this in field settings.

This study’s purpose is to examine whether providing prior consulting services influences internal auditors’ subsequent assessments when providing assurance services to assist management in its assessment of internal control over financial reporting. A behavioral experiment is used, with internal auditors as participants. We provide some evidence that internal auditors who perform prior consulting services are less likely than others to conclude that an identified control deficiency is a material weakness, but only when the deficiency is directly related to the prior consulting services performed. Limitations include relatively small sample sizes and manipulation check failure rates that, although consistent with several prior studies, are somewhat high. If internal auditors have provided consulting services, they may want to consider limiting the assurance services provided to management that are more directly related to their consulting services. While prior studies have examined the effects of internal auditors’ role in designing internal controls on subsequent services, this is the first study to focus on the impact of providing internal audit consulting services on subsequent assurance services.

This chapter considers whether and how race/ethnicity can be examined using experimental methods.

We begin by discussing the highly contextual nature of race/ethnicity and reviewing the properties of experiments. After examining existing experimental literature that focuses on race/ethnicity, we turn to our current study that uses the incompatible complexity condition to examine the multilevel interactions of diverse racial/ethnic groups composed of Mexican American and White participants in Texas and Black and White participants in Ohio.

We argue that experiments, when guided by formal theoretical approaches that allow for general inquiries of theoretical principles, are especially suitable for studying interactional characteristics such as race/ethnicity.

We suggest a particular approach that emphasizes interactional aspects of race/ethnicity and how these aspects can be utilized to diminish inequality in group processes.

The authors content analyze 76 empirical Strategic Management Journal articles to determine how studies control for threats to internal validity, a common source of flaws in research designs. Results indicate that most studies fail to control for one or more threats to internal validity. In particular, selection effects were the most frequently appearing threat, followed by history effects, ambiguity about the direction of causal inference, changing data sources and subject mortality. In general, the results suggest that strategy researchers need to more carefully account for threats to the internal validity of their research designs. Suggestions for addressing these problems are provided.

We evaluated the best-available evidence for the effects of receiving business-as-usual or naturally delivered special education services in K-12 US schools. Our best-evidence synthesis of 44 empirical studies evaluated which outcome domains and disability types have been investigated and whether findings varied by the rigor of the study design and methods. Regression-based studies comparing students with educational disabilities (SWED) to students without disabilities (SWOD) yielded mostly negative associations of receiving special education with academic achievement, behavior, and long-term or other outcomes. In contrast, regression-based studies that contrasted SWED receiving special education to other SWED not receiving special education produced a pattern of estimates similar to quasi-experimental designs that contrast SWED to SWOD. The most rigorous designs utilized quasi-experimental methods that compared SWED receiving special education services with SWED not receiving special education services, and generally reported more positive than negative evidence of receiving special education services across most outcome domains. Future research that utilizes rigorous quasi-experimental methodology and appropriate comparison groups to investigate the effectiveness of special education is needed, particularly for nonachievement outcome domains.

Stated choice experiments can be used to estimate the parameters in discrete choice models by showing hypothetical choice situations to respondents. These attribute levels in each choice situation are determined by an underlying experimental design. Often, an orthogonal design is used, although recent studies have shown that better experimental designs exist, such as efficient designs. These designs provide more reliable parameter estimates. However, they require prior information about the parameter values, which is often not readily available. Serial efficient designs are proposed in this paper in which the design is updated during the survey. In contrast to adaptive conjoint, serial conjoint only changes the design across respondents, not within-respondent thereby avoiding endogeneity bias as much as possible. After each respondent, new parameters are estimated and used as priors for generating a new efficient design. Results using the multinomial logit model show that using such a serial design, using zero initial prior values, provides the same reliability of the parameter estimates as the best efficient design (based on the true parameters). Any possible bias can be avoided by using an orthogonal design for the first few respondents. Serial designs do not suffer from misspecification of the priors as they are continuously updated. The disadvantage is the extra implementation cost of an automated parameter estimation and design generation procedure in the survey. Also, the respondents have to be surveyed in mostly serial fashion instead of all parallel.