For classrooms, independent study, and analytical teams
Teaching guide
Schedules, exercises, and a capstone assignment for teaching from this course.
Adjust the schedules to students’ mathematical preparation and time for computation.
Prerequisites
- Beginning with evidence
Fractions, percentages, graphs, and basic algebra are enough to enter chapters 1 and 2. Read the worked examples aloud, redraw the comparison, and use the expandable solutions after a written attempt. Expect to pause for probability and algebra practice.
- Ready for applied estimation
Bring introductory probability, regression experience, and a working language or spreadsheet. Complete the numerical exercises, label the estimand, and reproduce each result from the stated inputs. Some later arguments require derivatives and matrix notation.
- Working at graduate depth
Bring linear algebra, multivariable calculus, and probability with limits. Derive the formal results, construct counterexamples, and follow the original papers. These compact chapters provide a common argument to investigate; a graduate course will need additional proofs, readings, and problem sets.
Pair learners across strengths when possible. One person can interrogate the design while another checks the calculation, then exchange roles. Both must be able to explain the final claim without depending on the other person’s specialty.
Teach one investigation at three depths
The measurement investigation follows one service-design choice through a narrative essay, an undergraduate calculation, and a doctoral treatment. Use the 75-minute field guide for a class, or assign the proofs and research problems for a seminar. Keep the empirical assumptions and the value judgments visible in every version.
A twelve-week course
Plan for one discussion and one working session each week. Assign the chapter before the discussion. Use the working session to produce the deliverable and expose a failure case. The capstone begins in week 1 and is revised as the tools become available.
| Week | Focus | Read | Bring back |
|---|---|---|---|
| 1 | Frame the argument | Ch. 1 | Write a target in one sentence. Construct two mechanisms that fit the same observations and imply different intervention effects. |
| 2 | Condition and average | Ch. 2 | Rebuild a joint probability table. Explain how selection changes an average and verify the law of iterated expectations. |
| 3 | Work through projection | Ch. 3 | Derive the normal equations, check residual orthogonality, and reproduce a partial-regression calculation. |
| 4 | Describe uncertainty | Ch. 4 | Run a repeated-sampling experiment. Identify the randomness, interval guarantee, and assumption that would invalidate it. |
| 5 | Design the comparison | Ch. 5 | Specify a randomized offer, a target population, and an attrition plan. Work a small randomization calculation by hand. |
| 6 | Audit the measurement | Ch. 6 | Build a variable dictionary and missingness log. Derive attenuation under stated conditions and supply a case where it fails. |
| 7 | Use an instrument carefully | Ch. 7 | Calculate a Wald ratio. State the LATE assumptions and quantify how a direct instrument effect changes the result. |
| 8 | Compare changes | Ch. 8 | Calculate a two-by-two DiD. Name the untreated comparison and find the trend deviation that reverses a decision. |
| 9 | Defend a local design | Ch. 9 | Write an RD or synthetic-control design note. Separate fit diagnostics from the assumptions supporting a causal interpretation. |
| 10 | Learn nuisance functions | Ch. 10 | Derive an orthogonal score and sketch a cross-fitting split. Keep prediction loss separate from the target of inference. |
| 11 | Move across time and populations | Ch. 11 | Build a time-ordered validation plan. State what must remain stable for a forecast or target-population claim. |
| 12 | Make and defend a decision | Ch. 12 | Submit the capstone memo, a reproducible arithmetic check, and a sensitivity analysis. Review a peer’s identifying argument. |
For a short introductory course, concentrate on weeks 1 through 6 and the decision memo. For an advanced group, shorten the prerequisite review and add full-paper discussions to weeks 7 through 12. Keep the deliverable requirement in both versions.
A ninety-minute company workshop
Question: should the firm expand an optional training offer? Analysts, managers, and domain specialists can work from the same brief. No coding is required for the session; participants can use the browser experiments and a calculator.
| Time | Work | Output |
|---|---|---|
| 0–10 min | Name the decision | Specify who receives the offer, the six-month outcome, and the break-even effect. |
| 10–25 min | Inspect rival explanations | Use the causal-worlds experiment. Record what can remain identical while an intervention effect changes. |
| 25–40 min | Build the comparison | Calculate the training example below. Write the parallel-trends assumption in the firm’s language. |
| 40–55 min | Stress-test the conclusion | Vary the allowed untreated-trend gap. Locate the decision-reversal threshold. |
| 55–70 min | Request the missing evidence | List the unit-level data, implementation details, and dependence information needed for inference. |
| 70–90 min | Write and challenge the memo | Exchange recommendations. Each reviewer must identify one assumption carrying a material part of the decision. |
Participants write a one-page decision memo, show the calculation, and specify the additional evidence they would request. A follow-up technical session can implement a design once the real data and business constraints have been established.
Capstone: the training offer
The firm and data in this exercise are fictional.
A firm offers optional training to all eligible workers in one operational group. A second group receives no offer. Assignment of the offer was not randomized. Management wants to know whether the offer raised six-month output enough to cover its cost, and whether a wider rollout is justified.
| Group | Before | After | Change |
|---|---|---|---|
| Offer group | 50 | 62 | 12 |
| Comparison group | 40 | 46 | 6 |
For this exercise, one additional output unit contributes $100. The average program cost is $300 per eligible worker in the offer group, including nonparticipants in that denominator. Treat those two amounts as known and assume no spillovers, capacity constraints, or change in unit contribution. The effect concerns offering the program to that group. A per-participant effect would require a separate argument.
Your assignment
- Target. Define the six-month average causal effect of the offer for eligible workers in the offer group. Separate this target from the effect of a future rollout to other workers.
- Design. Calculate DiD. State the untreated-trend assumption, the composition assumptions, and how anticipation or spillovers could undermine the comparison.
- Threat. Describe one concrete reason the groups’ untreated output changes might differ. Turn it into a parameter with the same units as the outcome.
- Sensitivity. Let the untreated offer-group trend exceed the comparison trend by an unknown amount δ, with |δ| ≤ M. Calculate effect and net-value bounds for M = 0, 3, and 4. Check the result in the design experiment.
- Inference. Explain why this table cannot supply a defensible standard error. Request the records, assignment details, and dependence information needed to proceed.
- Decision. Write a recommendation conditional on M. Name the additional evidence that would most change it, including information needed for a wider rollout.
Check the arithmetic after writing your argument
The observed difference in changes is (62 − 50) − (46 − 40) = 6 output units per eligible worker. If the untreated trend gap is δ, the causal effect consistent with this decomposition is 6 − δ. Restricting |δ| ≤ M therefore gives [6 − M, 6 + M].
The net contribution per eligible offered worker is $100 × effect − $300. The following ranges reflect the specified trend restriction alone. They contain no allowance for sampling uncertainty.
| Allowed gap M | Effect range | Net-value range |
|---|---|---|
| 0 | [6, 6] | [$300, $300] |
| 3 | [3, 9] | [$0, $600] |
| 4 | [2, 10] | [−$100, $700] |
Break-even requires 3 output units. A gap of δ = 3 erases the estimated surplus. Strictly positive net value for every allowed trend gap requires M < 3 under the exercise’s other assumptions. At M = 4, the available restriction permits both a loss and a gain.
None of this establishes a wider-rollout effect. Differences in workers, training delivery, capacity, or spillovers can change it. There is also no worker-level variation, group count, or assignment distribution here from which to justify a sampling interval.
What to submit
A memo of at most 700 words, one figure or table, and a runnable arithmetic check in a language or spreadsheet of your choice. Include a short appendix that defines the target, identifies every substantive assumption, and distinguishes synthetic inputs from quantities you derived. Keep an evidence-request list separate from the conclusions supported by the current brief.
Assessment rubric
Grade the reasoning and calculations; more than one recommendation may be justified.
| Criterion | Weight | Strong work makes this visible |
|---|---|---|
| Target and units | 15% | Population, intervention, comparison, horizon, and economic units are consistent. |
| Identification and threats | 30% | The proposed comparison has explicit assumptions and a plausible rival explanation. |
| Calculation and reproducibility | 20% | Inputs, transformations, and outputs can be checked without guessing. |
| Uncertainty and sensitivity | 20% | Sampling and identification uncertainty are distinguished; the reversal threshold is correct. |
| Decision and communication | 15% | The recommendation follows from the stated objective and evidence; limits and next evidence are specific. |
For independent study, score your first submission before opening the solution. Then revise it and record the most consequential change. For a class, exchange anonymous memos and require reviewers to reproduce the arithmetic before evaluating the recommendation.
Ian Helfrich tutors econometrics one-to-one, for students taking a course and for researchers designing a study. To arrange sessions on this material, start with a 20-minute intro call.
Read the edition’s scope and limitations →