chc.scm

Stability tier: evolving.

Synthetic control and augmented synthetic control for a single treated unit.

The synthetic control method (SCM, Abadie-Diamond-Hainmueller) builds a counterfactual for one treated unit as a convex combination of donor units matched on the pre-treatment outcome path; the treatment effect is the treated-minus-synthetic gap in the post period. Its weakness is the simplex constraint: when the treated unit lies outside the donors' convex hull no weighting balances the pre-period, so the estimate is biased.

Augmented SCM (ASCM, Ben-Michael-Feller-Rothstein) corrects exactly this: it keeps the SCM donor mix but adds a ridge outcome-model correction for the residual pre-period imbalance, which is allowed to extrapolate. When SCM already balances, the correction is ~0 and ASCM reduces to SCM.

A statistical estimator, so NumPy float64 throughout (like chc.did), independent of the JAX x64 flag -- which must not change an estimate.

synthetic_control_inference adds Abadie, Diamond and Hainmueller's in-space placebo test and an interval from Chernozhukov, Wüthrich and Zhu's conformal test. Both are owned here. diff-diff carries both and is the tests' oracle, but its placebo leaves the treated unit out of every placebo's donor pool, which gives up the test's exact level, and its weights come from a Frank-Wolfe solver that needed 200 000 steps to converge; over this module's exact weights each test is a few lines.

Neither test holds when a design chose the treated unit, as a design that picks its test market does. blank_period_inference gives such a test the reading Abadie and Zhao's design justifies: the weights fit on the earlier periods before treatment, the latest ones left blank, and the gaps after treatment ranked against every arrangement of the blank and post periods. Its serial="ar1" reading keeps that test's level where the errors persist, as a geo panel's do.

SyntheticControlResult dataclass

A fitted synthetic control: post-period effect path, donor mix, and pre-period fit.

SyntheticControlInference dataclass

synthetic_control's estimate, with a placebo test and a conformal interval. Experimental.

placebo_p_value is Abadie, Diamond and Hainmueller's in-space placebo for the null that the treatment moved nothing, built as they built it: each unit's synthetic control from all the others, and the share of the N units whose ratio of post- to pre-period prediction error is at least the treated unit's. A pre-period error below sqrt(eps) of the panel's spread before treatment counts as that much, so units their donors reproduce, as they can when there are more donors than periods before treatment, rank by their error after it. When the treated unit was drawn at random from the units, the test holds its level exactly; it moves in steps of 1 / N.

interval holds a constant effect with probability 1 - alpha: the hull of the effects Chernozhukov, Wüthrich and Zhu's conformal test does not reject, as a search finds them. The test takes the effect out of the treated unit's periods after treatment, refits its synthetic control over every period, and ranks the residuals after treatment against each cyclic shift of the residuals in time. So it needs the residuals exchangeable in time, not the treatment random. Its p-values move in steps of 1 / T: below alpha = 1 / T the interval is unbounded, and it is (nan, nan) when the test rejects every effect the search tries, which can miss a set of accepted effects narrower than its step. The p-value need not fall away from the estimate, which is not refitted after treatment, so the interval can leave the estimate out; and a hull can hold effects the test rejects, the estimate among them.

Measured on factor panels of 20 periods before treatment and 5 after, 1000 a case (scripts/bench_scm_inference.py). With the treated unit drawn at random, the placebo rejected a true null 3.8-4.7% of the time at 5% with 19 donors, and 9.2-9.6% at 10% with 9 or 19. The interval covered the effect 97.2-98.4% of the time at alpha = 0.05 and 93.1-94.0% at 0.10, where the test's own levels, on 25 periods, are 4% and 8%. Neither holds when the treated unit was chosen as the one its donors fit best, as a design that picks its test market chooses: the placebo then rejected 27.8% at 5%, and the interval covered 94.4% and 81.7%.

BlankPeriodInference dataclass

A designed test's synthetic control fitted on the earlier periods before treatment, with a permutation test and split-conformal intervals read from the periods it left blank. Experimental.

estimate is the synthetic control fitted on the fitting periods alone: its pre_rmspe is their error, and its att and overall are the gaps after treatment. blank_gaps are the gaps, under the same weights, in the blank periods, which the weights never saw.

p_value is Abadie and Zhao's (arXiv 2108.02196 v5, eq. 19) for the null that the treatment moved nothing: the share of the arrangements ways of choosing T_post of the blank and post periods whose mean absolute gap is at least the post periods'. It moves in steps of 1 / arrangements, C(T_blank + T_post, T_post), and is never below one step.

Read with serial="ar1", ar_coefficient is an AR(1) coefficient of the gaps, estimated from the donors alone, and p_value ranks the gaps' innovations instead: e_t = u_t - ar_coefficient * u_{t-1} over the blank and post periods, u the gaps and the first lag the last fitting period's. Its statistic is |e_first + (1 - ar_coefficient) * (sum of the other post innovations)|, which a constant effect moves most. The arrangements are each choice of T_post innovations and which of them comes first, T_post C(T_blank + T_post, T_post) of them. ar_coefficient is None in the default reading.

intervals[k] holds the effect in post period k with probability at least 1 - alpha: the gap there, plus or minus the ceil((1 - alpha) (T_blank + 1))-th smallest absolute blank gap, which is infinite when that rank exceeds T_blank, as it does for every alpha < 1 / (T_blank + 1). That is split conformal prediction's finite-sample rank, one above Abadie and Zhao's eq. 20, which reaches 1 - alpha only as T_blank grows.

Measured on factor panels of 19 donors, 15 fitting, 5 blank and 5 post periods, the treated unit the one its donors fit best on the fitting periods, 4000 a case (scripts/bench_scm_inference.py blank). With white-noise factors and errors, the test rejected a true null 4.4% of the time at 5%, where synthetic_control_inference's placebo rejected 23.5% on 400 such panels. With AR(1) factors and errors of coefficient 0.5 it rejected 8.7%, against that placebo's 27.8%: errors that persist break the exchangeability, and the rate grew to 21.5% at 0.8. Factors that persist did not, 4.8% with white-noise errors. The serial="ar1" reading rejected 4.9%, 4.8% and 5.3% at coefficients 0, 0.5 and 0.8 (standard errors 0.35). The intervals, the same in both readings, covered 83.6%, 79.6% and 70.7% at alpha = 0.2; at 0.05 and 0.10 they were unbounded.

synthetic_control(outcomes, treated_unit, n_pre, *, steps=5000)

Classic simplex synthetic control for one treated unit against the remaining donor units.

outcomes is (N, T); treatment starts at period n_pre (so periods 0..n_pre-1 are pre-treatment). Donor weights lie on the probability simplex; att[k] is the treated-minus- synthetic gap in post period k. The weights are the exact optimum, whatever the outcomes' units; steps caps the solver's iterations, and running out raises RuntimeError.

augmented_synthetic_control(outcomes, treated_unit, n_pre, *, ridge=0.1, steps=5000)

Ridge-augmented synthetic control (Ben-Michael-Feller-Rothstein).

Starts from the SCM donor weights, then de-biases each post period by the residual pre-period imbalance passed through a ridge outcome model fit on the donors: Y_1(0)_t = w' Y_post_t + (treated_pre - w' donor_pre)' theta_t, with theta_t the slopes of a ridge regression of the donors' Y_post_t on their pre-period outcomes with a free intercept, as Ben-Michael, Feller and Rothstein's has: theta_t = (Zc'Zc + ridge * v * I)^{-1} Zc' Y_post_t, with Zc each pre-period centred on the donors' mean, or zero where the donors moved apart by rounding alone (chc._units), and v the mean of Zc**2, the donors' variance before treatment, or 1 where Zc is all zero. The intercept cancels from the estimate, since the weights sum to one. The correction vanishes when SCM already balances the pre-period; ridge controls how far the outcome model may extrapolate, and ridge = lam / v is a ridge of lam in the outcomes' squared units. As a share of v it shrinks the same in any units and from any origin: a ridge in the outcomes' squared units outweighed donors logged in small ones, so that at 1e-6 of them the estimate was plain SCM's, and without the intercept the estimate moved with the outcomes' level. weights are the (interpretable) SCM donor weights; the augmentation is an additive outcome correction, not folded into them.

The default share is a tenth, a compromise, since no one share suits every panel. On 200 panels of 30 donors and 25 periods before treatment, the root mean squared error of the effect at a tenth was 0.0526 where the factors are white noise, against 0.0502 at the best share, 1. Where they are random walks it was 0.110, against 0.105 at the best share, 0.03, and 0.359 at 1. With those panels indexed to their first period it was 0.142, against 0.126 at the best, 0.01: a tenth errs by up to 1.133 times the best fixed share. 0.07 would have stayed within 1.1 times it on all three kinds, but the default was set before they were measured, and moving it to suit them would tune it on them. The earlier default, a ridge of 1 in the outcomes' squared units, erred less on these panels, 0.0497, 0.107 and 0.113, by a coincidence of units: their noise is 0.1 in those units, so it was a fixed multiple of the noise variance. At 1e-3 of the units it gave plain SCM's estimate.

synthetic_control_inference(outcomes, treated_unit, n_pre, *, alpha=0.05)

synthetic_control with a placebo test and a conformal interval.

Experimental: it may change or be withdrawn in any release.

Takes synthetic_control's arguments, and computes its estimate. The placebo test fits a synthetic control to every unit. The interval's search tries 401 effects over ten errors either side of the estimate, a twentieth of an error apart, the error being the root mean square of the treated unit's residual, over every period, at the estimate. It pushes the outermost effects not rejected outwards, and bisects each end to a millionth of that error.

Raises:

Type	Description
ValueError	on synthetic_control's refusals, or alpha outside (0, 1).

blank_period_inference(outcomes, treated_unit, n_pre, n_blank, *, alpha=0.05, serial='exchangeable')

A designed geo test read as Abadie and Zhao read it: fit on the earlier periods before treatment, hold out the last n_blank of them, permute over the blank and post periods.

Experimental: it may change or be withdrawn in any release.

outcomes, treated_unit and n_pre are synthetic_control's. The weights fit on periods 0 .. n_pre - n_blank - 1; periods n_pre - n_blank .. n_pre - 1 are blank. The treated unit may be any unit a design chose by its fit on the fitting periods, or on anything else outside the blank and post periods.

Abadie and Zhao's Theorem 2 brackets the test's size, alpha - 1 / arrangements <= P(p <= alpha) <= alpha, when the factors over the blank and post periods are exchangeable and independent of the noise, and each unit's noise is i.i.d. over time; then the gaps over those periods are exchangeable, whatever the fit. The upper end needs only that exchangeability; the lower one needs no ties. So:

• Noise that persists over time breaks it: the mean gap over adjacent post periods then varies more than over scattered ones, and the test rejects too often, 8.7% at 5% with AR(1) errors of coefficient 0.5. Factors that persist do not. serial="ar1" is the remedy, below.
• The design must not look at the blank periods. A unit chosen for its fit there has blank gaps selected small, and the test then rejects too often.
• The blank periods are the latest before treatment, so that a season or a trend moves them as little apart from the post periods as the panel allows. Nothing in the data checks exchangeability; a season that turns at treatment breaks it.
• The blank periods leave the fit: the weights see n_pre - n_blank periods, not n_pre.
• The p-value moves in steps of 1 / C(n_blank + T_post, T_post): with 5 blank and 5 post periods, of 1/252; with 1 blank period, of 1/6, so no test at 5% rejects.

intervals hold under the same exchangeability, the effect being additive and the noise after treatment distributed as before it, one period at a time, not jointly.

serial="ar1" reads gaps that follow an AR(1), u_t = c + rho u_{t-1} + e_t with e i.i.d., as they do when every unit's errors and the factors share one coefficient. Then the innovations over the blank and post periods are i.i.d. given the fitting periods, and with the true rho the test is exact again, whatever the design did with the fitting periods (Chernozhukov, Wüthrich and Zhu, JASA 2021, §2.4.2 and Lemma 7, filter a synthetic control's residuals so before permuting them). rho is estimated:

• Not from the treated unit's gaps over the fitting periods. The fit absorbs their persistence: on the bench's panels their AR(1) estimate averaged -0.13 where the errors' was 0.5.
• From the donors: each donor's synthetic control from the other donors, the treated unit left out, fitted on the first ceil(T_fit / 2) periods; its gaps on every later period; and Han, Phillips and Sul's panel fully aggregated estimator over them (Econometric Theory 30, 2014, §2), which removes each donor's level without the within-estimator's bias, kept in [-1, 1]. An effect on the treated unit does not reach it. It assumes the donors' gaps persist as the treated unit's do.
• An estimate below rho leaves the innovations positively correlated, and the test rejects too often: with rho fixed 0.1 low, 5.8% at 5% at 0.5. On the bench's panels the estimate averaged 0.49 at 0.5 (s.d. 0.08) and 0.77 at 0.8, and the test rejected 4.8% and 5.3%. With 10 blank periods, 5 fitting for the donors, it rejected 5.5% on 12000 panels.
• Its power is for a constant effect: an effect of 1 and 2 on the bench's panels, rejected 14.9% and 45.2% at 0.5, where the exchangeable reading rejected 19.7% and 44.3% at a size of 8.7%, and 27.5% and 76.8% where the errors do not persist, against 13.5% and 48.5%.
• The intervals are the exchangeable reading's, and persistence breaks them: at 0.8 the 80% intervals covered 70.7%.

Raises:

Type	Description
ValueError	on synthetic_control's refusals; when n_blank is not in [1, n_pre - 1]; when alpha is outside (0, 1); when serial is neither reading; when there are more than a million arrangements to enumerate; or, for "ar1", with fewer than two donors or when the donors' gaps cannot give an AR(1) coefficient (fewer than four periods after their fit, or none that move).