Causal inference, experimental design, and decision-making under uncertainty

A General Approach to Causal Mediation Analysis

Kosuke Imai, Luke Keele, Dustin Tingley · 2010 · Psychological Methods · 3,587 citations

Provides a canonical overview or reference point for the relevant DoOperator research area.

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests

Stefan Wager, Susan Athey · 2017 · Journal of the American Statistical Association · 2,737 citations

Provides a canonical overview or reference point for the relevant DoOperator research area.

Causal inference in statistics: An overview

Judea Pearl · 2009 · Statistics Surveys · 2,309 citations

Provides a canonical overview or reference point for the relevant DoOperator research area.

Metalearners for Estimating Heterogeneous Treatment Effects using Machine Learning

Soren R. Kunzel, Jasjeet S. Sekhon, Peter J. Bickel +1 more · 2019 · Proceedings of the National Academy of Sciences · 1,243 citations

Provides a canonical overview or reference point for the relevant DoOperator research area.

On Causal Inference in the Presence of Interference

Eric J. Tchetgen Tchetgen, Tyler J. VanderWeele · 2012 · Statistical Methods in Medical Research · 500 citations

Provides a canonical overview or reference point for the relevant DoOperator research area.

Online controlled experiments at large scale

Ron Kohavi, Alex Deng, Brian Frasca +3 more · 2013 · Knowledge Discovery and Data Mining · 424 citations

This paper addresses the challenge of scaling online controlled experiments (A/B tests) from isolated, one-off studies to an organization-wide decision-making engine that can run hundreds of concurrent experiments across millions of users. The core problem is that as organizations grow and adopt agile development, they need to evaluate many product ideas simultaneously — often hundreds per quarter — but traditional experimental designs and statistical methods break down under the combinatorial complexity of overlapping experiments, massive data volumes, and the need for rapid, trustworthy decisions. Existing approaches to product evaluation (focus groups, expert reviews, observational data analysis) fail to establish causal relationships reliably, while single-experiment A/B testing frameworks cannot handle the operational demands of concurrent experimentation at web scale. The paper synthesizes lessons from Microsoft's Bing, where over 200 concurrent experiments run daily across ~100 million monthly active users, and provides a framework for building the cultural, engineering, and statistical infrastructure needed to make experimentation a core organizational capability rather than a niche analytical tool.

Statistical Challenges in Online Controlled Experiments: A Review of A/B Testing Methodology

Nicholas Larsen, Alex Deng, Jiheng Zhang +2 more · 2024 · The American Statistician · 70 citations

Provides a canonical overview or reference point for the relevant DoOperator research area.

Designing Optimal Dynamic Treatment Regimes: A Causal Reinforcement Learning Approach

Junzhe Zhang · 2020 · International Conference on Machine Learning · 16 citations

Policy optimization in treatment settings can be invalid if it ignores how treatments were assigned and how confounders evolve.

Controlled experiments on the web: survey and practical guide

Ron Kohavi, Roger Longbotham, Dan Sommerfield +1 more · 2009 · Data Mining and Knowledge Discovery

Provides a canonical overview or reference point for the relevant DoOperator research area.

Causal Inference: What If

Miguel A. Hernan, James M. Robins · 2020 · Chapman & Hall/CRC

Provides a canonical overview or reference point for the relevant DoOperator research area.

Double/Debiased Machine Learning for Treatment and Causal Parameters

Victor Chernozhukov, Denis Chetverikov, Mert Demirer +4 more · 2016

This paper addresses the fundamental challenge of performing valid inference on a low-dimensional causal or structural parameter (e.g., an average treatment effect, a regression coefficient in a partially linear model, or a policy parameter) when the nuisance functions—such as outcome regressions, propensity scores, or instrument propensity models—must be estimated using high-dimensional or flexible machine learning methods. Classical semiparametric theory assumes that nuisance parameters can be estimated at sufficiently fast rates (typically root-n) and that the parameter space has low complexity (e.g., Donsker properties hold). In modern settings with many covariates, ML methods like lasso, random forests, boosting, and neural nets are natural choices for nuisance estimation, but they introduce two sources of bias that destroy root-n consistency and valid inference when naively plugged into estimating equations: (1) **regularization bias** from shrinkage or penalization that prevents the nuisance estimator from converging at root-n rates, and (2) **overfitting bias** from using the same data to estimate both the nuisance functions and the parameter of interest. The paper shows that combining Neyman-orthogonal moment conditions with cross-fitting (sample splitting) eliminates both biases, yielding estimators that are root-n consistent, asymptotically normal, and admit valid confidence intervals under remarkably weak conditions on the nuisance estimators.

Estimation and Inference of Heterogeneous Treatment Effects using Random Forests

Stefan Wager, Susan Athey · 2015

This paper addresses the challenge of estimating heterogeneous treatment effects (HTE) — how the causal effect of a treatment varies across individuals with different observed characteristics — in settings with many covariates and complex interactions. Classical nonparametric methods for HTE estimation, such as nearest-neighbor matching, kernel methods, and series estimation, perform well with few covariates but break down as the dimensionality increases due to the curse of dimensionality. Meanwhile, machine learning methods like random forests excel at high-dimensional prediction but lack the inferential guarantees needed for causal inference: researchers need confidence intervals and hypothesis tests for treatment effects, not just point predictions. The paper bridges this gap by developing causal forests — a modification of Breiman's random forests — that provide consistent, asymptotically normal estimates of heterogeneous treatment effects under unconfoundedness, along with valid confidence intervals. This enables researchers to explore treatment effect heterogeneity without pre-specifying subgroups, while avoiding the pitfalls of data dredging and false discovery.

Correlation Was Never the Problem

"Correlation is not causation" is one of the most-repeated phrases in empirical research. It is also, as usually understood, a dramatic understatement of the actual difficulty. The real challenge is not distinguishing correlation from causation — it is identifying which causal story is correct when several are consistent with the same data.

May 29, 2026

The Illusion of Control: Why Most A/B Tests Mislead More Than They Inform

Organizations run thousands of A/B tests every year and congratulate themselves on being data-driven. Most of those tests are statistically invalid. Here is why — and what rigorous experimentation actually requires.

May 27, 2026

What N-of-1 Trials Get Right That Population Studies Get Wrong

Randomized trials on populations measure average effects in heterogeneous groups. N-of-1 trials measure what actually happens to one specific person. For individual decision-making, the latter is usually more relevant.

May 26, 2026