Search results

Researchers in economics and other disciplines are often interested in the causal effect of a binary treatment on outcomes. Econometric methods used to estimate such effects are divided into one of two strands depending on whether they require unconfoundedness (i.e., independence of potential outcomes and treatment assignment conditional on a set of observable covariates). When this assumption holds, researchers now have a wide array of estimation techniques from which to choose. However, very little is known about their performance – both in absolute and relative terms – when measurement error is present. In this study, the performance of several estimators that require unconfoundedness, as well as some that do not, are evaluated in a Monte Carlo study. In all cases, the data-generating process is such that unconfoundedness holds with the ‘real’ data. However, measurement error is then introduced. Specifically, three types of measurement error are considered: (i) errors in treatment assignment, (ii) errors in the outcome, and (iii) errors in the vector of covariates. Recommendations for researchers are provided.

We discuss the two most popular frameworks for identification, estimation and inference in regression discontinuity (RD) designs: the continuity-based framework, where the conditional expectations of the potential outcomes are assumed to be continuous functions of the score at the cutoff, and the local randomization framework, where the treatment assignment is assumed to be as good as randomized in a neighborhood around the cutoff. Using various examples, we show that (i) assuming random assignment of the RD running variable in a neighborhood of the cutoff implies neither that the potential outcomes and the treatment are statistically independent, nor that the potential outcomes are unrelated to the running variable in this neighborhood; and (ii) assuming local independence between the potential outcomes and the treatment does not imply the exclusion restriction that the score affects the outcomes only through the treatment indicator. Our discussion highlights key distinctions between “locally randomized” RD designs and real experiments, including that statistical independence and random assignment are conceptually different in RD contexts, and that the RD treatment assignment rule places no restrictions on how the score and potential outcomes are related. Our findings imply that the methods for RD estimation, inference, and falsification used in practice will necessarily be different (both in formal properties and in interpretation) according to which of the two frameworks is invoked.

We study research designs where a binary treatment changes discontinuously at the border between administrative units such as states, counties, or municipalities, creating a treated and a control area. This type of geographically discontinuous treatment assignment can be analyzed in a standard regression discontinuity (RD) framework if the exact geographic location of each unit in the dataset is known. Such data, however, is often unavailable due to privacy considerations or measurement limitations. In the absence of geo-referenced individual-level data, two scenarios can arise depending on what kind of geographic information is available. If researchers have information about each observation’s location within aggregate but small geographic units, a modified RD framework can be applied, where the running variable is treated as discrete instead of continuous. If researchers lack this type of information and instead only have access to the location of units within coarse aggregate geographic units that are too large to be considered in an RD framework, the available coarse geographic information can be used to create a band or buffer around the border, only including in the analysis observations that fall within this band. We characterize each scenario, and also discuss several methodological challenges that are common to all research designs based on geographically discontinuous treatment assignments. We illustrate these issues with an original geographic application that studies the effect of introducing copayments for the use of the Children’s Health Insurance Program in the United States, focusing on the border between Illinois and Wisconsin.

Without controlling for selection bias and the potential endogeneity of the treatment by using proper methods, the estimation of treatment effect could lead to biased or incorrect conclusions. However, these issues are not addressed adequately and properly in higher education research. This study reviews the essence of self-selection bias, treatment assignment endogeneity, and treatment effect estimation. We introduce three treatment effect estimators – propensity score matching analysis, doubly robust estimation (augmented inverse probability weighted approach), and endogenous treatment estimator (control-function approach) – and examine literature that applies these methods to research in higher education. We then use the three methods in a case study that estimates the effects of transfer student pre-enrollment debt on persistence and first year grades. The final discussion provides guidelines and recommendations for causal inference research studies that use such quasi-experimental methods.

This is the second in a series of papers focused on alcohol and substance abuse rehabilitation centers. Centers face the ongoing challenge of validating outcomes to meet the burden of evidence for insurance companies. In the first paper, data mining was used to establish baseline patterns in treatment success rates, for the Futures: Palm Beach Rehabilitation Center, that have a direct impact on a client’s ability to receive insurance coverage for treatment programs. In this paper, we examine 2016 outcomes and report on facility efficacy, alumni progression and sobriety, and forecast treatment success rates (short and long term) in support of client insurability. Data collection has been standardized and includes admissions data, electronic medical records data, satisfaction survey data, post-discharge survey data, Centers for Disease Control (CDC) data, and demographic data. Clustering, partitioning, ANOVA, stepwise regression and stepwise Logistic regression are applied to the data to determine statistically significant drivers of treatment success.

We describe a new Bayesian estimation algorithm for fitting a binary treatment, ordered outcome selection model in a potential outcomes framework. We show how recent advances in simulation methods, namely data augmentation, the Gibbs sampler and the Metropolis-Hastings algorithm can be used to fit this model efficiently, and also introduce a reparameterization to help accelerate the convergence of our posterior simulator. Conventional “treatment effects” such as the Average Treatment Effect (ATE), the effect of treatment on the treated (TT) and the Local Average Treatment Effect (LATE) are adapted for this specific model, and Bayesian strategies for calculating these treatment effects are introduced. Finally, we review how one can potentially learn (or at least bound) the non-identified cross-regime correlation parameter and use this learning to calculate (or bound) parameters of interest beyond mean treatment effects.

Lechner and Miquel (2001) approached the causal analysis of sequences of interventions from a potential outcome perspective based on selection-on-observables-type assumptions (sequential conditional independence assumptions). Lechner (2004) proposed matching estimators for this framework. However, many practical issues that might have substantial consequences for the interpretation of the results have not been thoroughly investigated so far. This chapter discusses some of these practical issues. The discussion is related to estimates based on an artificial data set for which the true values of the parameters are known and that shares many features of data that could be used for an empirical dynamic matching analysis.

In the previous chapter, the reader will have become familiar with the idea of screening for traumatic experiences within organisations as a way to identify those who may benefit most from interventions and support. In this chapter, I present an overview of the trauma therapy literature in the first instance and then explore some of the debates regarding specific trauma-informed treatments versus general therapeutic approaches. The multicultural competency literature is discussed, and the multicultural orientation approach of cultural humility, cultural opportunity and cultural comfort is highlighted in a practice context. This chapter concludes with a case study vignette that brings it all together with a clinical example of what trauma-informed therapy through a multicultural lens might look like. As such I operationalise choice, collaboration , trust and transparency, and cultural principles from the trauma-informed care literature. Although applied here to specific trauma-informed organisations, some of the methods and processes that I unpack can be used in non-specific organisations where social/case managers are employed and wish to operationalise choice and collaboration in a structured way.

Education researchers have been urged to utilize causal inference methods to estimate the policy effect more rigorously. While randomized controlled trials (RCTs) are the gold standard for assessing causality, RCTs are infeasible in some educational settings, particularly when ethical concerns or high cost are involved. Quasi-experimental research designs are the best alternative approach to study educational topics not amenable to RCTs, as they mimic experimental conditions and use statistical techniques to reduce bias from variables omitted in the empirical models. In this chapter, we introduce and discuss the core concepts, applicability, and limitations of three quasi-experimental methods in higher education research (i.e., difference-in-differences, instrumental variables, and regression discontinuity). By introducing each of these techniques, we aim to expand the higher education researcher's toolbox and encourage the use of these quasi-experimental methods to evaluate educational interventions.

In the previous chapter, I introduced the reader to the ideas and research of the common factors. The common factors are varied and have demonstrated to have small to large effect sizes depending on what variable is being examined. In this chapter, I categorise four more evidence based relationship variables which tend to be more task orientated and aligned to the therapeutic alliance. Indeed, the therapeutic alliance, goals and collaboration, alliance rupture–repair, and feedback-informed care are four trans-theoretical factors that can contribute greatly to outcomes. At the same time, when poorly established they can and do impact negatively on client outcomes. This is not an exhaustive overview of the literature, rather each variable is briefly discussed, the evidence supporting the effectiveness is highlighted, and Top Tips are provided to assist the development of the practitioner.