Causal Inference in Econometrics and Statistics: Theory and Applications

How should researchers adjust for covariates? We show that if the propensity score is estimated using a specific covariate balancing approach, inverse probability weighting (IPW), augmented inverse probability weighting (AIPW), and inverse probability weighted regression adjustment (IPWRA) estimators are numerically equivalent for the average treatment effect (ATE), and likewise for the average treatment effect on the treated (ATT). The resulting weights are inherently normalized, making normalized and unnormalized IPW and AIPW identical. We discuss implications for instrumental variables, difference-in-differences, and distributional and quantile treatment effect estimators, and illustrate with two applications how these numerical equivalences simplify analysis and interpretation.

For many functionals, DML estimators are the state-of-the-art, incorporating the good predictive performance of black-box machine learning algorithms; the decreased bias of doubly robust estimators; and the analytic tractability and bias reduction of sample splitting with cross fitting. Recently Balakrishnan, Wasserman and Kennedy (BWK) introduced a novel assumption-lean model that formalizes the problem of functional estimation when no complexity reducing assumptions (such as smoothness or sparsity) are imposed on the nuisance functions occurring in the functional’s first order influence function (IF1). Then, for the integrated squared density and the expected conditional variance functionals, they showed that first-order estimators, which includes DML estimators, based on IF1 are rate minimax under squared error loss. However, earlier Liu, Mukherjee, and Robins (2020) had shown that, for these functionals, second-order estimators (i.e., estimators that add a debiasing second-order U-statistic IF22 to a first-order estimator) could have smaller risk (mean squared error) than the first-order estimator. In this talk, I resolve this apparent paradox by showing that, although minimax, DML estimators are asymptotically inadmissible under the BWK model because the risk of any first-order estimator is never less than that of the corresponding second-order estimator and, under many laws, may be much greater.

Coauthors: Lin Liu and Rajarshi Mukherjee

Randomized experiments have long been the gold standard for scientists seeking to learn about cause and effect. When randomized experiments are infeasible, scientists often resort to observational studies, which are widely available and often large but rely on untestable assumptions that, when violated, may result in biased estimates. Uncertainty about bias leads to a phenomenon known as the illusion of learning from observational research (Gerber, Green and Kaplan, 2004a): absent prior information about bias, observational results cannot meaningfully contribute to the estimation of a causal parameter. To shatter the illusion, we take an empirical Bayes perspective. We show that the distribution of observational biases can be learned from calibration studies-experiments that target a causal effect that is known a priori to be zero. Calibration identifies the distribution of observational bias and allows observational studies to inform the estimation of causal parameters via empirical Bayes shrinkage. We formalize the illusion phenomenon in an empirical Bayes setting and show that, with an increasing number of calibration and observation studies, both the bias distribution and the causal effect can be consistently recovered. We illustrate our method through a simulation study and a semi-synthetic application based on Ferraro and Miranda (2013)'s water-usage experiment.

Coauthors: Bohan Wu, Sebastian Salazar, and David M. Blei

Principal stratification (PS) is a commonly used approach for understanding the mechanisms through which a treatment affects an outcome. The goal of this work is to extend the PS framework to studies with continuous treatments, which introduces a number of both challenges and opportunities in terms of defining causal effects and performing inference. This manuscript provides multiple key methodological contributions: 1) we introduce principal causal estimands for continuous treatments that provide insights into different causal mechanisms, 2) we show that nonparametric identification is possible under a principal ignorability assumption, but only under a restrictive assumption on the joint distribution of potential mediators, which can be dropped under mild parametric assumptions, 3) we utilize nonparametric Bayesian models for the joint distribution of the potential mediating variables to ensure our approach is robust to model misspecification, and 4) we provide theoretical justification for utilizing an outcome model to identify the joint distribution of the potential mediating variables, and show that this is only possible if a principal ignorability assumption is violated. Lastly, we apply our methodology to a novel study of the relationship between the economy and arrest rates, and how this is potentially mediated by police capacity.

Coauthors: Joseph Antonelli, Minxuan Wu, Brenden Beck, and Alessandra Mattei

This paper considers identifying and estimating causal effect parameters in a staggered treatment adoption setting -- that is, where a researcher has access to panel data and treatment timing varies across units. We consider the case where untreated potential outcomes may follow non-parallel trends over time across groups. This implies that the identifying assumptions of leading approaches such as difference-in-differences do not hold. We mainly focus on the case where untreated potential outcomes are generated by an interactive fixed effects model and show that variation in treatment timing provides additional moment conditions that can be used to recover a large class of target causal effect parameters. Our approach exploits the variation in treatment timing without requiring either (i) a large number of time periods or (ii) requiring any extra exclusion restrictions. This is in contrast to essentially all of the literature on interactive fixed effects models which requires at least one of these extra conditions. Rather, our approach directly applies in settings where there is variation in treatment timing. Although our main focus is on a model with interactive fixed effects, our idea of using variation in treatment timing to recover causal effect parameters is quite general and could be adapted to other settings with non-parallel trends across groups such as dynamic panel data models.

In many randomized controlled trials, there may be imperfect compliance with randomization due to factors such as complexity of treatment, lack of tolerance of side effects, or, less commonly, subjects assigned to control being able to access treatment. While methods to develop causal estimands for the trial sample in the presence of non-compliance are reasonably well-developed, much less work has been done to consider transporting these results to general populations. We develop Bayesian methods that considering doubly-robust estimators with respect to sample selection and explore their behavior in simulation settings. We also consider application to the UK REFLUX Trial that assessed surgery or standard medical management provided better relief from gastro-oesophageal reflux disease: there was substantial non-compliance in this trial, and data available from a larger sample where subjects were allowed to choose surgery or medical management.

Coauthors: Yueying Hu and Yajuan Si

Generative modeling builds on and substantially extends the classical idea of generating synthetic data from observed samples. In this talk, I will show that this principle provides a natural and theoretically well-founded foundation for bootstrap inference. The resulting method, which we call the "generative bootstrap", yields statistically valid confidence intervals for both regular and irregular estimators, including settings in which Efron’s bootstrap fails. From this perspective, the generative bootstrap could be viewed as a modern extension of the smoothed bootstrap: it has the potential to mitigate the curse of dimensionality and remain effective in challenging regimes where estimators may not be root-n consistent or admit a Gaussian limit.

Non-causal "nonsense" associations can arise when an exposure and an outcome of interest exhibit similar patterns of dependence. For example, philosopher and economist Kevin Hoover, responding to Elliott Sober, attributes the positive association between British bread prices and Venetian sea levels to temporal dependence. This talk explores whether or not variables like time in the Sober example can be thought of as confounders, bringing to bear pragmatic observations about statistical and causal methodology in addition to philosophical considerations.

Ongoing work with: Frederick Eberhardt, Reuben Stern, and Naftali Weinberger

Coauthors: Rocio Titiunik and Rae Yu

Covariates are widely used in randomized experiments to improve precision. However, in the presence of interference, where outcomes may depend on the treatment assignments of other units, standard covariate adjustment methods may fail to preserve desirable properties such as the no-harm property, meaning that incorporating covariates does not worsen estimator performance. Existing approaches that incorporate covariates under interference typically rely on specifying a particular interference structure, and their guarantees can be sensitive to misspecification. In this paper, we study how to incorporate covariate information in a way that preserves a no-harm property while remaining largely agnostic to the underlying interference structure. We focus on the estimation of the expected average treatment effect (EATE) using the Hájek estimator and study rerandomization under interference, a design-stage procedure that restricts the assignment space to allocations with sufficiently small covariate imbalance. Our framework also allows the covariates used for rerandomization to depend on the treatment assignment itself, which naturally arises in settings with interference. We show that, even under interference, rerandomization can improve precision asymptotically relative to unrestricted Bernoulli randomization, relying only on mild conditions on the dependence structure across units. When a conservative dependence graph is available, we further develop an optimization-based conservative variance estimator for inference under rerandomization.

Coauthors: Ziang Yuan and Xinran Li

Experiments deliver credible treatment-effect estimates but, because they are costly, are often restricted to specific sites, small populations, or particular mechanisms. A common practice across several fields is therefore to combine experimental estimates with reduced-form or structural external (observational) evidence to answer broader policy questions such as those involving general equilibrium effects or external validity. We develop a unified framework for the design of experiments when combined with external evidence, i.e., choosing which experiment(s) to run and how to allocate sample size under arbitrary budget constraints. Because observational evidence may suffer bias unknown ex-ante, we evaluate designs using a minimax proportional-regret criterion that compares any candidate design to an oracle that knows the observational study bias and jointly chooses the design and estimator. This yields a transparent biasvariance trade-off that does not require the researcher to specify a bias bound and relies only on information already needed for conventional power calculations. We illustrate the framework by (i) designing cash-transfer experiments aimed at estimating general equilibrium effects and (ii) optimizing site selection for microfinance interventions.

Coauthor: Aristotelis Epanomeritakis

Persuasive communication is ubiquitous in economics, business, and public policy, yet measuring its causal impact raises distinctive identification challenges. This talk presents a framework for defining, identifying, and estimating persuasion effects: the causal effect of a message on the behavior of individuals who would not take the advocated action in its absence. This parameter is closely related to Pearl's probabilities of causation. The analysis establishes how persuasion effects can be point or partially identified under alternative sources of variation in exposure and different assumptions on potential outcomes. The framework accommodates experimental and instrumental-variable settings, difference-in-differences, and regression discontinuity designs, while highlighting the role of monotonicity restrictions in identification. Taken together, these results provide a causal-inference toolkit for studying persuasion and clarify how persuasion parameters relate to conventional treatment-effect estimands.

In this talk, we propose a novel application of space-filling designs within a finite-population inferential framework based on potential outcomes to increase the power to detect treatment effects in text-based randomized trials. An existing framework consists of four steps: (1) take a sample of documents to be hand-coded; (2) use the coded subset to train a machine learning model that predicts human-coded scores from text features that can be automatically extracted from all documents; (3) use the trained model to predict scores for all documents and estimate treatment effects based on these predictions; and (4) de-bias the effect estimates using the average residual difference between the human-coded and machine-predicted scores. Our focus is whether sampling methods other than simple random sampling (SRS) can be used in Step (1) to further increase power. While SRS is simple to implement and yields design-unbiased estimates, it may produce samples that do not adequately explore the auxiliary covariate space. We therefore investigate stratified and space-filling sampling methods and use standard test functions from the literature to understand when these designs offer advantages over SRS, with the greatest gains occurring when the human-coding budget is limited. A key challenge arising from this approach is estimating the finite-population variance of the de-biased treatment-effect estimator under space-filling sampling. Spatially balanced designs induce dependence among sampled units, making standard design-unbiased variance estimators difficult to implement because second-order inclusion probabilities may be unavailable, computationally expensive to estimate, or nearly zero. We develop Monte Carlo and model-assisted approaches to variance estimation that exploit the local dependence structure induced by spatially balanced designs. These methods provide reliable variance estimates while avoiding the computational burden of exhaustive simulation. Together, the results provide a unified framework for using space-filling designs to improve both the efficiency and inferential validity of text-based randomized trials.

Coauthors: Justin Greene, Ying Hung, Reagan Mozer, and Luke Miratrix

We develop a new approach for quantifying uncertainty in finite populations, by using design distributions to calibrate sensitivity parameters in finite population identified sets. This yields uncertainty intervals that can be interpreted as identified sets, robust Bayesian credible sets, or uniform frequentist design-based confidence sets. We focus on quantifying uncertainty about the average treatment effect, where our approach (1) yields design-based confidence intervals which allow for heterogeneous treatment effects without using asymptotics, (2) provides a new motivation for examining covariate balance, and (3) gives a new formal analysis of the role of randomization. We illustrate our approach in three empirical applications.

Coauthor: Brendan Kline

Crossover designs randomly assign each unit to receive a sequence of treatments. By comparing outcomes within the same unit, these designs can effectively eliminate between-unit variation and facilitate the identification of both instantaneous effects of current treatments and carryover effects from past treatments. They are widely used in traditional biomedical studies and are increasingly adopted in modern digital platforms. However, standard analyses of crossover designs often rely on strong parametric models, making inference vulnerable to model misspecification. This paper adopts a design-based framework to analyze general crossover designs. We make two main contributions. First, we use potential outcomes to formally define the causal estimands and assumptions on the data-generating process. For any given type of crossover design and assumptions on potential outcomes, we outline a procedure for identification and estimation, emphasizing the central role of the treatment assignment mechanism in design-based inference. Second, we unify the analysis of crossover designs using least squares, with restrictions on the coefficients and weights on the units. Based on the theory, we recommend the specification of the regression function, weighting scheme, and coefficient restrictions to assess identifiability, construct efficient estimators, and estimate variances in a unified fashion. Crucially, the least squares procedure is simple to implement, and yields not only consistent and efficient point estimates but also valid variance estimates even when the working regression model is misspecified.

Coauthor: Zhichao Jiang

Determining which organizations are more effective in implementing an intervention program is essential for theoretically and empirically characterizing exemplary practices and for intervening to enhance the capacity of ineffective ones. Yet sites differ in their local ecological conditions including client composition, alternative programs, and community context. Applying the causal inference framework, this study proposes a mathematical definition for the relative effectiveness of an organization. Capitalizing on multisite randomized trials, the identification leverages observed control group outcomes that capture the confounding impacts of some otherwise unmeasured between-site differences. We propose a two-step mixed-effects modeling (2SME) procedure, reveal through simulations its outperformance in comparison with unconditional or conditional site-specific intention-to-treat (ITT) effect estimates that adjust for observed baseline covariates only. Illustrated with an evaluation of Job Corps centers nationwide serving disadvantaged youths, the new framework promises to alleviate consequential misclassifications of some of the most effective Job Corps centers as least effective and vice versa.

Coauthors: Jonah Deutsch, Peter Kress, Jose Eos Trinidad, and Zhengyan Xu

Causal inference is vital for informed decision-making across fields such as biomedical research and the social sciences. Randomized controlled trials (RCTs) are often considered the gold standard for internal validity, whereas observational studies (OSs) may offer greater external validity. However, both data sources have limitations that can hinder broadly valid statistical inference: RCTs may lack generalizability because of selective eligibility criteria, and OSs are vulnerable to unobserved confounding. This paper proposes a new approach to integrating RCT and OS data that leverages the strengths of each study type to address the limitations of the other. The method uses a novel triplet matching algorithm to align RCT and OS samples and a new two-parameter sensitivity analysis framework to quantify internal and external validity biases. This combined approach yields causal estimates that are more robust to hidden bias than those based on OSs alone and provides more reliable inferences about treatment effects in the general population. We apply the method to study the effects of lactation on maternal health using a small RCT together with a long-term observational health records dataset from the California National Primate Research Center. This application illustrates the practical utility of our approach for generating scientifically grounded and actionable causal estimates.

We study randomization inference for treatment effects in observational data under unconfoundedness. The main testing procedure is design-based in that it holds outcomes and covariates fixed and repeatedly re-randomizes treatment assignment according to estimated propensity scores. Under the sharp null hypothesis of no treatment effect, we derive non-asymptotic bounds on excess Type I error for tests based on an arbitrary test statistic in terms of in-sample and out-of-sample estimation error. Under the weak null hypothesis of no average treatment effect, we show that our tests are asymptotically valid when based on suitable studentized estimators of the average treatment effect. In contrast to earlier work on studentization in related settings, a key challenge in our analysis is adequate control of the error stemming from the estimation of the propensity score for each individual observation. For this reason, naive studentization may fail to be asymptotically valid for the weak null hypothesis, but we show that validity can be achieved by studentizing a score-adjusted inverse propensity-weighted estimator of the average treatment effect in the parametric case, or a Neyman-orthogonal estimator in the nonparametric case. We then compare the randomization test with the corresponding test using a standard normal critical value. Under the weak null hypothesis, we find that they are first-order equivalent and that neither test dominates the other in a higher-order comparison based on Edgeworth expansions. The randomization test is, however, more accurate to higher-order in regimes "near the sharp null," i.e., in which treatment effects are sufficiently small or sparse. While randomization inference is common in experimental settings, our results support its use for testing weak null hypotheses in observational studies as well, especially when treatment effects are believed to lie in the near-sharp regime.

Randomization inference is a widely used and appealing approach for analyzing treatment effects in randomized experiments, as it is finite-sample valid and does not require any distributional assumptions. However, naive application of randomization inference may suffer from severe size distortion in the presence of sample attrition, where outcome data are missing for some units. In this paper, we propose new, computationally efficient methods for randomization inference that remain valid under a broad class of potentially informative missingness mechanisms, allowing a unit’s missingness to depend on its unobserved potential outcomes. Specifically, we construct valid p-values for testing both sharp and bounded null hypotheses on treatment effects via a worst-case consideration of the classical Fisher randomization test. Leveraging distribution-free test statistics, these worst-case p-values admit closed-form solutions. Importantly, by incorporating both potential outcomes and potential missingness indicators into the test statistic, our methods can exploit structural assumptions such as monotone missingness, which are commonly adopted in applications due to their plausibility and ability to substantially improve inferential power. Moreover, our approach connects to a range of partial identification bounds in the literature, which in some sense suggests the sharpness of our tests. We illustrate the proposed methods through both simulation studies and an empirical application. An R package implementing the proposed methods is publicly available.

Coauthors: Xinran Li and Peizan Sheng

We study the optimal design and analysis of experiments for estimating spillover effects. Assuming a known exposure mapping, we characterize the treatment-assignment distribution and regression-based estimator that minimize worst-case asymptotic variance against a broad class of distributions of unobservables. The design problem yields an intuitive solution in which the planner trades off spillover signal strength against diffusion of spillover variation. The analysis problem yields a simple recentered instrumental variable estimator to best leverage this variation. This framework produces natural solutions in several benchmark cases, such as clustered exposure, and suggests computationally tractable approximations for general networks, including bipartite settings. We illustrate these new tools in semi-synthetic experiments based on two applications from development economics. Our approach yields large standard error reductions in both experiments, increasing effective sample sizes by 50-100% or more.

Coauthors: Kirill Borusyak and Peter Hull