We’ve established that causal inference requires a counterfactual, and that every analytical strategy is a way of constructing a credible stand-in for what we can’t observe. But we haven’t addressed a more basic question: how do you know whether your specific analysis is actually estimating a causal effect?
A different kind of tool
This is where Judea Pearl’s framework comes in. Pearl is a computer scientist and philosopher. His central insight was that causal relationships can be represented as graphs: specifically, directed acyclic graphs, or DAGs.
A DAG is simple. Variables are nodes. If one variable causes another, you draw an arrow from cause to effect. “Directed” means the arrows have direction: causes point to effects, not the other way around. “Acyclic” means no loops: you can’t follow the arrows in a circle back to where you started.
That’s it. No equations, no Greek letters. Just boxes and arrows representing your beliefs about what causes what.
The power isn’t in the diagram itself, it’s in what the diagram reveals about your analysis. Pearl showed that once you commit to a DAG, you can determine mathematically whether a given set of control variables will identify the causal effect you’re after, or whether your specification is doomed regardless of sample size. Rubin’s framework tells you what you’re trying to estimate. Pearl’s tells you whether your estimation strategy can get you there.
These are complementary tools. And right now, analysts are making the decisions that Pearl’s framework disciplines (which variables to control for, which to leave alone) every time they build a model. They’re just making them implicitly, based on intuition or convention. A DAG forces you to make them explicitly, on paper, before you touch the data.
The education example, drawn out
Let’s return to the returns to education. Here’s a simple DAG:
This says: education affects earnings, ability affects both education and earnings. Ability is a confounder: it creates a non-causal path between education and earnings. If you don’t control for it, your estimate of the effect of education is biased. So far, this is what you already know. It’s the omitted variable bias story from before.
The DAG just made it visual. But here’s where it earns its keep.
The control variable that makes things worse
Suppose you add occupation to the model as a control, reasoning that you want to compare people in similar jobs. Seems sensible. But draw the DAG:
Occupation is on the causal path between education and earnings. It’s one of the mechanisms through which education affects earnings: more education opens doors to higher-paying occupations. Controlling for it blocks part of the causal effect you’re trying to measure. Your estimate shrinks, and you conclude education doesn’t matter much. But you’ve just controlled away part of the very thing you were looking for.
This is called conditioning on a mediator, and your analysts are probably doing it right now without realising it’s a problem.
Now consider something worse. Suppose you control for a variable that is caused by both education and earnings: say, neighbourhood quality. Wealthier, better-educated people live in certain neighbourhoods. A variable like this, one that is caused by two or more other variables in the diagram, where the arrows “collide” into it, is called a collider. The DAG:
A collider is harmless on its own. But the moment you condition on it, you create a statistical association between its causes that doesn’t reflect any real causal relationship. You open a non-causal path between education and earnings. You create bias that wasn’t there before. Adding a control variable made your estimate worse.
This is the insight most analysts have never encountered: more controls can introduce bias, not just reduce it. A DAG tells you which controls help, which are irrelevant, and which are actively damaging.
The test for your own work
Before your next analysis, try this: draw the causal relationships between your treatment, your outcome, and every variable you’re thinking of controlling for. Use arrows. Be specific about direction.
Then ask: is each control variable a confounder (needs to be controlled), a mediator (must not be controlled), or a collider (must not be controlled)?
Pearl’s framework includes formal tools for this. The backdoor criterion tells you exactly which variables you need to control for to identify a causal effect. Do-calculus is a set of mathematical rules for determining when causal effects can be estimated from observational data. You don’t need to master either to benefit from DAGs. Just drawing the diagram and having the conversation with your team: “do we all agree this is how these variables relate?” That will surface disagreements and assumptions that would otherwise stay buried in a model specification nobody questions.
Pushback and disagreements welcome. That’s the point.
Further reading:
- Pearl, J. & Mackenzie, D. (2018). The Book of Why: The New Science of Cause and Effect. Basic Books. First fre chapters available https://bayes.cs.ucla.edu/WHY/. Start here, covers DAGs and colliders for a general audience.
- Cinelli, C., Forney, A. & Pearl, J. (2024). A Crash Course in Good and Bad Controls. Sociological Methods & Research, 53(3), 1071-1104. https://doi.org/10.1177/00491241221099552. Very good summary of common research design graphs.
- Huntington-Klein, N. (2021). The Effect: An Introduction to Research Design and Causality. Chapman & Hall/CRC. Excellent on causal diagrams and identification. Also free at theeffectbook.net.
- Pearl, J. (2009). Causality: Models, Reasoning, and Inference. 2nd edition. Cambridge University Press.
