Three econometrics experiments
Identification and inference
Experiments on confounding, confidence-interval coverage, and sensitivity to parallel trends.
Identification / Chapter 1
Different causal effects, identical observations
Try to make the causal effect negative. Now look for something in the observed data that changed.
Changing the causal coefficient leaves the observed joint distribution of training and output unchanged.
Read the argument and proof →Observational equivalence
160 synthetic observations · seed 2026 · standardized units
A one-unit intervention raises the mean outcome by 0.50. The observational distribution has not changed.
Sample OLS slope: 0.843. Sampling noise explains its distance from 1; it cannot select the causal world.
Inspect the model and its limits
Let U and V be independent standard normal variables. Set D = U and Y = bD + (1 − b)U + V. Observationally, Y = D + V for every b. Under do(D = d), U is unchanged and the mean is bd. The orange line is a model implication, never an estimate from these dots.
The slider range [−1, 2] is chosen for display. The observed distribution does not restrict b to that range. U is an unobserved cause; its equality with observed D in this deliberately stark construction makes conditional causal overlap fail.
Each unit's observed values remain fixed as b changes. No animation, fitting algorithm, or larger sample can identify b from this observational law alone.
Uncertainty / Chapter 4
Sample size and bias
First leave bias at zero and increase sample size. Then add a little bias and try again.
The first change buys precision. The second breaks the connection between the sampled population and the target. Notice what increasing sample size can and cannot repair.
Work through coverage and bias →Precision can make an error convincing.
100 nominal 95% intervals; target mean = 0.
Exact simulation, deliberately narrow assumptions
Each study samples n independent normal observations with known variance 1 and observed mean bias. We draw the sample mean directly from Normal(bias, 1/n), exactly its sampling distribution. The interval is mean ± 1.959964/√n. The policy target remains zero; positive bias represents a mismatch between the sampled population and that target.
Coverage is random across batches. Under zero bias the procedure has 95% coverage; 95 of 100 is not guaranteed. Unknown variance, dependent data, nonnormality, and estimated selection adjustments require different inference. This is a known-sigma illustration, not an OLS standard-error calculator.
Identification to action / Chapters 8 & 12
Sensitivity to the trend assumption
A rollout appears to raise output. Allow the two groups to have different untreated trends. How far can they differ before buying the program becomes a judgment call?
At M = 3, the worst-case effect just covers the program cost, so no allowed effect loses money and training weakly dominates skipping. Any larger bound admits a loss. Defending a bound of 3 or less requires evidence about the untreated trends.
Work through the identifying restriction →What trend difference would change the decision?
A hypothetical nonrandom training rollout.
Each outcome unit is worth $100 in this toy decision; training costs $300 per person. These fixed values are assumptions. No spillovers, heterogeneous costs, or equilibrium effects are modeled.
Derivation and what this band leaves out
Observed DiD = (62 − 50) − (46 − 40) = 6. Write δ for the treated group's untreated change minus the comparison group's untreated change. Then the treated-group average effect τ = 6 − δ, assuming no anticipation and stable composition. Restricting |δ| ≤ M gives τ ∈ [6 − M, 6 + M].
We treat the four means as known. There is no sampling uncertainty here. A defensible empirical interval must incorporate uncertainty in the estimated means and justify the bound. This algebraic sensitivity exercise is not an implementation of HonestDiD. The model also assumes a consistent treatment and no interference.