chc.switchback

Stability tier: experimental.

Which switchback to run, and how to read it, for a named effect of a lever on a plant with memory.

A switchback randomises a lever over time within a zone, on in some periods and off in others. On a plant with memory the periods are not independent units: what the lever did last period is still in the state. So the design has to say which effect it is for. This module plans for a first-order state per zone,

x_{t+1} = a x_t + b u_t + eps_t,   u_t in {0, 1} with probability 1/2,   y_t = x_t (+ eta_t),

and for the effect of holding the lever on for H periods, tau_H = b (1 - a^H) / (1 - a) (Horizon): the channel b at H = 1, the steady state b / (1 - a) at H = inf, and any horizon between. 0 < a < 1, and nothing here holds outside it.

The results it rests on (the lab's switchback thread, independently verified):

• The design enters only through one number, A1 = sum_{s >= 1} a^(s-1) rho(s), rho the assignment's autocorrelation, when the analysis regresses y_{t+1} on (1, y_t, u_t). That needs a fair, ergodic design and an intercept.
• Orthogonality by design. That regression's plug-in b_hat S_H(a_hat) has variance at least (4 sigma^2 / T) S_H(a)^2, the variance with a known, with equality iff A1 = S_H'(a) / S_H(a): the design makes the persistence estimate orthogonal to the effect. Any other design loses q (A1 - A1*)^2 / D(A1) of it, q = b^2 / (4 sigma^2), and the optimum does not depend on q. So the channel wants an i.i.d. design, a horizon a Markov design that flips with probability a / (1 + 2a) at H = 2, and the steady state never to switch, which a floor on the number of switches stops.
• Several effects at once lose q (A1 - A1*_H)^2 / D(A1) each, with D common to all, so the design that minimises the worst loss sits between the extreme optima.
• Measurement noise makes that regression inconsistent for every H >= 2, and for the channel at any design but the i.i.d. one. IV with (u_t, u_{t-1}) is consistent at every design, and is planned at its own minimax design.
• Without a trusted state model only model-free readings are planned: the channel by the regression at p = 1/2; tau_H by the difference in means over blocks of length H keeping each block's last period, DIM(H, H-1), which is unbiased for tau_H on any linear time-invariant plant, or by the local-projection sum at p = 1/2; the steady state by the block difference in means with the smallest worst mean squared error, whose bias is quoted and not removed, since no model-free design is unbiased for it.

design_switchback returns the arms, and for each effect the analysis, its standard error, its minimum detectable effect and its loss against the effect's own best design, at the least favourable a of the prior. Every variance is asymptotic in T. read_switchback reads an effect off the data a design produced, with a standard error from the data, and restate_mde reads it off the first periods of the run, an internal pilot, and restates the minimum detectable effect the whole run can detect. randomisation_test and randomisation_interval infer from the design's own randomisation instead of from T being large: the test is exact at any number of blocks, and the interval is exact given a range for a.

What is outside the model, and comes back as a warning rather than a refusal: a second state or a longer carryover, drift, spillover between zones, a near 1 on a short run, and fewer than 100 switches. The plug-in's reading tests for a second state, and the block difference's standard error keeps spillover; the plan's variances assume neither. On the zone market of chc.zones, whose incentive also acts through a stock and draws half its recruits from the neighbouring zones, the plug-in read its effects 3% to 14% off, and the block difference spread 1.6-1.7 times as far as planned; the internal pilot reads that spread off the run. The prior's a must lie in (0, 1): for a < 0 the bound is unattainable at even H and the best design depends on q.

SwitchbackAnalysis = Literal['state_aware', 'iv', 'local_projection', 'block_dim'] module-attribute

How an effect is read off the switchback:

• "state_aware": OLS of y_{t+1} on (1, y_t, u_t), and tau_H = b_hat S_H(a_hat);
• "iv": the same regression with instruments (1, u_{t-1}, u_t), and the same plug-in;
• "local_projection": OLS of y_{t+1} + ... + y_{t+H} on (1, u_t, y_t), the coefficient on u_t;
• "block_dim": the difference in means of y_{t+1} over the periods each block keeps.

Alternative = Literal['two-sided', 'greater', 'less'] module-attribute

Which statistics are at least as extreme as the observed one: larger in absolute value, larger, or smaller.

Horizon dataclass

tau_H, the effect of holding the lever on for H periods. H = 1 is the channel b and H = math.inf the steady state b / (1 - a).

Raises:

Type	Description
ValueError	when periods is neither a whole number at least 1 nor math.inf.

PersistencePrior dataclass

What a pilot or the history says before the design: a somewhere in [a_lo, a_hi], the state's shock sd, a guess of the channel b, and the measurement noise's variance as a share of the shock's (0 when the state is observed exactly).

Raises:

Type	Description
ValueError	unless 0 < a_lo <= a_hi < 1, the shock sd is positive, the guess is nonzero, the noise ratio is non-negative, and all are finite.

signal_to_noise property

q = b^2 / (4 sigma^2): what one period of the lever adds, against the shock.

TargetMDE dataclass

Choose the number of zones: the smallest at which every effect's minimum detectable effect is at most fraction of |tau_H|, at the least favourable a.

Raises:

Type	Description
ValueError	unless 0 < fraction and it is finite.

MarkovDesign dataclass

Each period the lever flips with probability flip; 1/2 is i.i.d.

BlockDesign dataclass

Blocks of length periods at one setting, each drawn fair and independently; the analysis drops the first washout periods of each block.

SwitchbackArm dataclass

A design and the share of each zone's periods it runs for.

EstimandReport dataclass

One effect's reading: which arm and analysis, and what it resolves at the least favourable a.

se is the standard error pooled over the zones; mde is (z_{1 - alpha/2} + z_power) se + |bias|; bias is zero for every consistent analysis and the working model's for the steady state read by a block difference in means. loss is the variance over the effect's own best design's in the same mode, minus 1, at its worst a: 0 when the design is aligned to it.

SwitchbackPlan dataclass

The arms to run in every zone, the reading of each effect, and what the plan cannot see. alpha is the two-sided test's level and power the power each effect's minimum detectable effect is for.

SwitchbackReading dataclass

An effect read off a switchback's data.

interval is the 1 - alpha confidence interval: normal about estimate with se, except for the steady state read through the plant, where it is Fieller's for b / (1 - a). Fieller's is (-inf, inf) when the data cannot rule out a unit root, which the delta method's se does not show.

RandomisationTest dataclass

The randomisation test of the sharp null that the lever moves no reading in any zone.

statistic is the reading's estimate / se, as read_switchback reads it, and p_value the chance, over the design's schedules that the reading reads, of a statistic at least as extreme against alternative. When enumerated it is read off all schedules of the design, which are equally likely, less those the reading refuses; otherwise off schedules drawn from it, as (1 + at least as extreme) / (1 + read). Either way it is exact given that the data were read: under the null, P(p_value <= alpha) <= alpha, at any number of blocks and whatever the plant's memory. A reading with neither an effect nor a noise, a constant outcome, has statistic NaN and p_value 1.

design_switchback(estimands, prior, periods, zones=1, *, trust_state_model=True, alpha=0.05, power=0.8, min_switches=40)

The switchback that reads estimands best over periods periods in each zone.

With trust_state_model the plan is a Markov design at the flip probability that minimises the worst loss over the effects and over the prior's persistence interval, read by the state-aware plug-in, or by IV when the state is measured with noise and a horizon is asked for. Without it, only model-free readings are planned (see the module docstring). zones is the number of zones, or a TargetMDE to choose it. min_switches floors how rarely the lever may flip, since never switching is what the steady state asks for.

Raises:

Type	Description
ValueError	on no effects or a repeated one, fewer than 4 * min_switches periods, fewer than one zone, alpha or power outside (0, 1), or a TargetMDE smaller than the working-model bias of a steady state read without a model, which no number of zones can reach.

read_switchback(lever, outcome, estimand, analysis, *, blocks=None, alpha=0.05, se='model')

Read estimand off a switchback's data by analysis.

lever[z, t] is zone z's lever in period t, 0 or 1, and outcome[z, t] its reading at the start of period t: outcome has one more period than lever, and outcome[z, t + 1] is the first reading after lever[z, t]. One zone may be passed as 1-D arrays. The regressions pool the zones as independent replications, each with an intercept.

• "state_aware" and "iv" fit a and b and read b S_H(a), by the delta method on a covariance robust to heteroskedasticity; IV's keeps the lag-one autocovariance of the MA(1) error a state measured with noise has. The plug-in also tests the first-order state it rests on, by whether y_(t-1) and u_(t-1) add to its regression, and warns when the data reject it at 1%; not for the channel at an i.i.d. lever, which it reads right on any linear time-invariant plant. A lever that alternates every period hides a stock it drives from the test, since the stock then alternates with it.
• "local_projection" needs an i.i.d. lever, and its covariance keeps the H - 1 lags of the overlapping sums.
• "block_dim" needs blocks, the design's. Each block's kept periods are centred on the midpoint of the on and off blocks before it, so the difference in means is unbiased at any number of blocks; the blocks before both settings have been seen only centre. It reads tau_H on BlockDesign(H, H - 1), and the steady state on any block, with the working-model bias design_switchback quotes. Each zone's difference is its own and the zones are weighted alike, and the standard error sums the zones' scores block by block, so it keeps what a spillover or a shock common to the zones puts between them.

se is what the standard error is over. "model" reads it off the regression's errors as above. "design" reads it over the lever's own randomisation, with the plant and its noise held fixed (Liang and Recht, Biometrika 2025), for a local projection on a lever drawn by a fair coin each period. The estimate is then the local projection with the design's moments in place of the sample's, no intercept and no y_t: tau_H = sum_(s<H) g_s, with g_s = 2 / (T - s) sum_t (2 u_t - 1) (y_(t+1+s) - y_0), each zone read from its first reading, which precedes the lever, and the zones averaged. Over the coins it is unbiased for tau_H on any linear time-invariant plant, whatever its noise does in time, as long as the noise does not depend on the coins and no zone's lever moves another's outcome. Its variance over the coins holds no product of the noise at two periods for the channel, and none more than H - 1 apart for tau_H: the noise's autocorrelation does not enter it. The standard error is their plug-in of that variance, over 3 log T lags of the impulse response, and is asymptotic in T. It is wider than the model's where the model's holds, since it uses neither the regression's intercept nor y_t.

Raises:

Type

Description

ValueError

when the shapes disagree, the lever is not 0 or 1 or never switches, the outcome is not finite, alpha is outside (0, 1), blocks is given without "block_dim" or missing with it, the blocks do not read tau_H, the lever changes inside a block, a local projection is asked of a steady state or of a lever that is not i.i.d., the steady state is read off a fit with a_hat >= 1, or a variance with lags comes out negative, which the working model's errors cannot produce; and when se is neither "model" nor "design", or "design" is asked of another analysis than the local projection, of a lever that is not a fair coin each period, of a run no longer than the 6 log T lags the plug-in reads, or of an outcome that never moves off its first reading.

restate_mde(plan, estimand, lever, outcome)

estimand's report, with its standard error and minimum detectable effect re-read from the first periods of its arm: an internal pilot.

The plan's variances are the working model's. On a plant with a second state or spillover the model-free readings stay where they were, but those of a horizon or the steady state spread further than planned: 1.04 to 1.73 times on chc.zones' market. lever and outcome are the arm's first periods in every zone, as read_switchback takes them. They are read by the plan's own analysis, and the standard error is carried to the arm's whole length:

• a regression's by its rows, sqrt((n_pilot - L) / (n_run - L)), with L the lags its covariance keeps, which are also the rows it sheds;
• the block difference's by its variance over the run's blocks against its variance over the pilot's. The blocks before a zone has shown both settings are lost, and the next are centred on few blocks before them: rescaled by its periods, a pilot of ten blocks read 1.2 times the run's standard error on the working model.

The restated MDE is c se + |bias|, with bias the plan's and c the power quantile of the noncentral t with nu degrees of freedom and noncentrality z_(1 - alpha/2). The run's power at that MDE, averaged over what the pilot could have read, is then the plan's when the pilot's variance is chi-square on nu. For a regression nu = n / (2 L + 1), with n its rows over every zone; for the block difference, whose standard error squares one sum over the zones a block, it is Satterthwaite's over the blocks. With c = z_(1 - alpha/2) + z_power the power at the restated MDE was 0.71 of 0.80 at a pilot of ten blocks.

On chc.zones' market, over 4000 runs under each matching, the run's power at the restated MDE about the reading's own mean, which leaves the reading's bias out, was 0.81 for the channel from a tenth of its arm, 0.78 to 0.79 for tau_5 and 0.80 to 0.81 for tau_2, and 0.75, 0.79 and 0.80 for the steady state from ten, fifteen and twenty of its forty blocks. The steady state's shortfall at ten is the spillover between zones: it puts a covariance between two zones' scores for the same block that the centring does not inflate, and the block factor treats it as if it did, so a short pilot reads the standard error low.

The test at the end is unchanged, and so is its level: the MDE is restated, the run is not resized. The report's loss is the plan's, since the design is.

Raises:

Type	Description
ValueError	when the plan does not read estimand, the pilot has other zones than the plan or is not shorter than the arm, a block pilot has fewer than ten blocks a zone or a zone that showed one setting in all of them, or read_switchback refuses the pilot.

randomisation_test(lever, outcome, estimand, analysis, design, *, alternative='two-sided', alpha=0.05, draws=9999, seed=0)

Test the sharp null that the lever moves no reading, in any zone and at any lag, by drawing the design's schedule again: Fisher's randomisation test, with the reading's estimate / se as the statistic.

Under the null the outcome does not depend on the schedule, so the observed statistic is one draw from its distribution over the design's schedules, and the p-value is exact at any number of blocks, whatever the plant's memory or the zones' spillover: there is no effect to carry or to spill. The schedules the reading refuses, such as a block schedule with fewer than two blocks after both settings, are left out of that distribution, since the observed one is a schedule the reading read; counting them as never extreme instead would reject up to alpha over the share read, 0.107 at 10% on six blocks. The statistic is studentised, which also keeps the test's level, in large samples, under the weaker null of an effect that averages zero (Wu and Ding 2021).

lever and outcome are as read_switchback takes them, and design is the one that drew the lever: every zone's schedule independently, a Markov design's from a fair first period and a block design's one fair coin a block. A block design with at most 2^14 schedules over the zones is enumerated; otherwise draws schedules are drawn with seed.

On the working model with a = 0.8 and no effect (scripts/bench_switchback_randomisation.py size, 1000 runs each, less those the reading refused), it rejected at 5% 3.6%, 5.2% and 4.6% of the block differences of tau_5 over 6, 8 and 12 blocks, where the Wald interval excluded 0 in 17.2%, 11.9% and 7.2%; 5.8% and 5.4% of the plug-in's tau_3 over 30 and 60 periods of MarkovDesign(0.3), against Wald's 8.7% and 6.1%; and 4.4% of the steady state's over 60 periods at a = 0.9, against Fieller's 5.3%. Against pyfixest 0.60.0's ritest, on 200 logs of an i.i.d. lever over 80 periods, the channel's statistic agreed with its HC1 t to 1e-13. Its p-values, from 999 permutations of the lever where this redraws 4999 schedules, differed by 0.008 on average and 0.054 at most, which the permutations' own noise allows.

Raises:

Type	Description
ValueError	when read_switchback refuses the data, the design cannot have drawn the lever or the block difference is asked of a Markov design, or no outcome could reject at alpha: K fair coins give a two-sided p-value of at least 2^(1 - K), so at 5% a block design needs six coins over the zones two-sided and five one-sided.

randomisation_interval(lever, outcome, estimand, analysis, design, persistence, *, alpha=0.05, draws=999, seed=0)

estimand's 1 - alpha confidence interval, by inverting randomisation_test under the working model's joint null.

A null that names tau_H alone is not sharp on a plant with memory: the schedule moves each reading through the whole path of the lever, b sum_{s >= 1} a^(s - 1) u_(t - s), and two memories with the same tau_H impute different readings. The joint null (a, b) is sharp: it imputes the readings with the lever off, y - b sum a^(s - 1) u_(t - s), which the schedule cannot move, and the test of no effect on them is exact (validation/switchback_randomisation.mac). So the set of every tau_H = b S_H(a) that some a in persistence leaves unrejected at alpha covers tau_H with probability at least 1 - alpha whenever the plant is first-order with its a in that range: at any number of blocks, where the Wald and Fieller intervals are asymptotic. The statistic is read off the imputed readings rather than compared with the effect, so a reading that is biased for its effect, as the block difference is for the steady state, keeps that. The range is the price: it is an assumption, and a long memory read over few blocks is unbounded.

On the working model (scripts/bench_switchback_randomisation.py, 400 runs a case, 1000 for the long memory; a run whose every effect the test rejected counts as a miss):

• the block difference of tau_5 at a = 0.8 over 8, 12 and 20 blocks. Wald covered 0.886, 0.932 and 0.932. Over (0.6, 0.9) this covered 0.995, 0.980 and 0.973, unbounded in 78%, 10% and none of the runs; at the true memory alone 0.956, 0.935 and 0.948, unbounded in 28% of the runs over 8 blocks. At the truth the test rejected 4.0% and 4.9% of 4000 runs over 8 and 12 blocks.
• the plug-in's tau_3 over 40 periods of MarkovDesign(0.3): Wald 0.925, the range 0.990, the true memory 0.953, at 1.35 and 1.14 times Wald's median width.
• the plug-in's steady state at a = 0.9 over 60 and 120 periods: Fieller 0.890 and 0.918, unbounded in 23% and 2% of the runs. Over (0.8, 0.95) this covered 0.993 and 1.000 at 1.37 and 1.57 times Fieller's median width, and at the true memory 0.938 and 0.940 at half of it.
• tau_2 at a = 0.95 over 12 and 24 blocks of 2: Wald 0.919 and 0.934, the true memory 0.949 and 0.947, but unbounded in 94% and 84% of the runs.

A null without the memory, persistence = (0, 0), is not sharp under carryover. For the block difference it still covered 0.950 to 0.966 in every case above, and never unbounded; for the plug-in it covered 0.53 of tau_3 and none of the steady states, where it rejected every effect in 41% and 48% of the runs.

Every null is tested on the same draws schedules, or on all of a block design's when it has at most 2^14, at 21 memories spread evenly across persistence; a memory between two of them is covered by continuity, not exactly. The ends are found from the first effect accepted near the estimate, by steps that double until the test rejects and then bisection to a thousandth of a standard error, each end reported on its rejected side. An accepted set with gaps may come back with a gap filled, or cut at the first rejection a step lands on. An end past 1024 standard errors is infinite.

Raises:

Type	Description
ValueError	when randomisation_test would refuse, persistence is not 0 <= lo <= hi < 1, or the test rejects every effect within eight standard errors of the estimate at every memory in the range: the data are at odds with a first-order plant there, at least near the estimate.