Skip to content

The decision lifecycle

A decision taken from logs goes through six stages:

  1. Identify what the logs can say about the levers.
  2. Plan under the limits the business states.
  3. Evaluate the plan before acting on it.
  4. Experiment where the logs cannot say enough.
  5. Deploy one step at a time.
  6. Adapt when the world moves.

This page files the library by stage. For each stage it lists what is built, where it is documented, and what is not built yet. Every name below is importable from chc. Entries marked experimental may change or be withdrawn in any release; see the stability tiers.

1. Identify: what the logs can say about the levers

  • Panel and prescribe's adjustment set. The causal assumption is a required argument. A graph that does not identify the effect raises NotIdentifiedError instead of returning a schedule. See the quickstart and identification.
  • fit_causal_residual. It fits the lever's channel in a hybrid model by cross-fitted orthogonalisation, or through an instrument when the confounder is latent. When the channel class cannot fit every state, weights (experimental) points it at the states a one-shot decision will be taken at. See chc.dynamics_id.
  • Static effects. estimate_effect_dml, sensitivity_analysis, refute_effect and e_value. See tutorial 3, and tutorial 7 against an experimental benchmark.
  • Designs with a comparison group. Difference-in-differences, in chc.did, and synthetic control, in chc.scm. With the did extra (experimental), a staggered design's estimates come with their uncertainty: standard errors, an interval on the overall effect and a uniform band over the event study, from diff-diff's multiplier bootstrap, with the numbers of units at which they were measured to hold their level. A synthetic control's estimate comes with Abadie, Diamond and Hainmueller's placebo test, exact when the treated unit was drawn at random, and a conformal interval (experimental, no extra).
  • Effects that take time. Impulse responses and delay_estimate, in chc.irf.
  • No adjustment set at all. The sensitivity level Γ bounds the effect instead of pointing at it; see tutorial 5, robust control.

2. Plan: the actions, under what the business states

  • prescribe. From a panel to a schedule with its certificate, in one call. It takes:
  • Lever: a box, a limit on how far it moves in one step, and a quadratic price on its use;
  • Target: a level, or a schedule of levels;
  • Constraint: a bound on a state;
  • Driver: a forecast the plan cannot move;
  • PeriodBudget, as budgets=: at most so much spent per period, with what a unit more would buy read back from Prescription.budget_prices().

See the quickstart and chc.decision. - causal_plan, the solver underneath. It takes the box, linear rows over the whole sequence (LinearConstraint: a budget, a rate limit), a barrier held in the solve (BarrierConstraint) and offline pessimism. See chc.plan. - CausalPlan.shadow_prices(). What each row costs the plan: the budget's price, per unit. - CausalPlan.stationarity and CausalPlan.constraint_violation. How near the plan is to a first-order optimum, and how far it breaks its box and rows. Each is read in the descent's scaled variables, so it is the same in any units, and each has a tolerance and a status. The certificate and the JSON record carry both. - CausalPlan.decision_weight() (experimental). What an error in the lever's one-step channel costs the plan, to second order: W, the Hessian of its regret in the channel's entries. Its entries are the ones the drift monitor watches, so a radius or a detected move can be priced in the plan's own cost. - CausalPlan.planned_cost_bias() (experimental). How far the plan's own cost reads from what it pays the plant, in expectation over the channel's error, with its sign, and the cost corrected for it; chc.misspecification.planned_cost_bias reads the error off a fit's influence. - minimax_action. The robust action over an identified interval, minimising the worst cost or the worst regret. See chc.regret. - chc.regret.minimax_regret_move (experimental). How much of a move whose value is linear to make when confounding up to Γ and the estimate's noise both leave its sign open: the whole move, a share, or none, with the minimax regret. - The case studies, each one decision from its logs to a plan: - pricing under confounding; - an epidemic under hospital capacity; - a pendulum from a confounded log; - media budgets; - marketplace dispatch; - driver supply, end to end; - confounded incentives, closed loop; - a heat pump on a live building emulator.

3. Evaluate: before acting

  • The error tube. CausalPlan.certificate_status and CausalPlan.certified_actions say how far ahead the model's error and the plant's own noise keep the plan inside tolerance, over the plan's RK4 steps. The model's part is a bound where the per-step error supplied is one, and a scale where it is a standard error, as under prescribe; the noise's holds at each step with probability 0.95. See the scopes.
  • certify_safety. Where along a finished plan's predicted path a barrier's guarantee survives unmeasured confounding, checked pointwise, and Γ*: the largest level at which every step along that path still has an admissible action that meets the condition, a ceiling for the problem along the path rather than for the plan.
  • plan_regret_bound. How far the plan's cost can be from the optimum on the planning model, from its own gradient: a certificate on a linear model or a curvature the caller supplies, a diagnostic where the curvature is sampled (status).
  • The reachable tube. chc.reachability computes the backward reachable tube under a partially identified effect.
  • evaluate_plan and certify_evaluation. They estimate what deploying a plan would cost, from logs of another policy, with an interval. Before any cost is read, the certificate says whether the logs can, and refuses by name when a weight the method needs has infinite variance. Four methods: per-decision importance sampling over episodes, marginalised importance sampling, doubly robust, and fitted Q evaluation, which works where every weight is refused. A plan is an AffinePolicy, held for ever, or an AffineSchedule, whose gain and offset change with the step and which only importance sampling over episodes evaluates; AffineSchedule.open_loop builds one from a sequence of actions. Every number is exact for a linear-Gaussian loop and an affine plan, and no more. See chc.evaluation.
  • A prescription's evaluate. It evaluates the prescribed schedule from a later panel, before it is deployed, by importance sampling over episodes: the panel's windows of H + 1 consecutive periods, on the plan's model linearised at their mean. It reads the panel as a target trial emulated from it. The windows start on calendar periods every unit shares, so a unit that leaves the panel does not decide where its windows start. The intervals come from resampling the units, the whole evaluation run again on each draw; on a panel of one unit, with none to resample, the windows' influences are read across their time by cosines, against Student's t (ADR 0083). A draw the certificate refuses is left out; the record counts the draws read, and past 5 % of them refused calls the intervals inconclusive. The estimate reports the first-order bias of its normalised weights and warns where it passes 0.05 of the half-width (ADR 0095). And the plan is set against the policy that logged the panel, the difference's interval read within each draw (ADR 0049). It refuses a plan whose levers were logged on a column outside its state, and one made against driver forecasts. It also asks the panel, as prescribe asks the one it fits, whether the levers read anything besides the state and their recorded parents, or read those through more than a quadratic, and warns when they did (experimental): a premise failing, not a measured cost (ADR 0028). See chc.decision.
  • off_policy_value. It weights one step at a time. So it estimates a candidate's value on the logger's own states, which is the contextual-bandit value. That is not the value of deploying a plan on a plant with memory. See chc.offpolicy.

4. Experiment: where the logs cannot say enough

  • Exploration. optimal_exploration_certificate, adaptive_exploration_certificate and capped_exploration_policy price how much exploration a controller should inject while its effect is not identified, and on what schedule. Each is weighed against the control the exploration costs. See chc.regret.
  • design_experiment (experimental). It says how many units to run in each zone before a one-shot decision, and whether to run any: the regret the experiment removes, against what it spends. The verdict is deploy, experiment or abstain, with the reason. Every number it reports is Monte Carlo's, re-solving the decision on every draw; its local model only chooses the units. A channel the logs do not identify enters as a wide prior. Scope: a static decision over zones, linear in each zone's channel, with a box on the levers. See chc.experiment.
  • design_switchback and read_switchback (experimental). Which switchback to run in a zone whose state carries the lever's past, for a named effect: the channel, the effect of holding the lever on for H periods, or the steady state. The design is aligned to the effect, so the persistence estimate is orthogonal to it, and the plan quotes each effect's standard error and minimum detectable effect at the least favourable persistence. Without a trusted state model only model-free readings are planned. read_switchback reads the effect off the data, with a standard error from the data, and warns when the data reject the first-order state the plug-in rests on. restate_mde re-reads an effect's standard error from the run's first periods, an internal pilot, and restates the minimum detectable effect the whole run can detect, so a plan whose working model missed a second state or spillover is corrected by the run itself. read_switchback(..., se="design") reads a local projection of a lever drawn by a fair coin each period with its standard error over the coins, the noise held fixed, so a noise that persists or trends does not move the interval's level; it is several times wider than the model's. randomisation_test draws the design's schedule again to test the sharp null that the lever moves nothing: exact at any number of blocks and whatever the plant's memory, where read_switchback's interval is asymptotic. randomisation_interval inverts it under the working model's joint null of persistence and channel, exact given a range for the persistence. Scope: a first-order state per zone, and a plan that takes the zones as independent. See chc.switchback.
  • shadow_price_effect (experimental). An experiment on a matching market treats some of the rows, and they compete with the control rows for the same columns, so the naive difference is not what treating every row would do. It reads that global effect off the rows' rents in the experiment's own matching, with a standard error, the second-order bias left at the treated share, which vanishes at one half, and an alarm near the LP limit, where that bias stops being second order. Strata fixed before the assignment post-stratify it. shadow_price_interval is the range the exact LP leaves at eps = 0. See chc.matching.
  • Not built: an experiment design for a dynamic plan's whole decision, beyond one lever's effect.

5. Deploy: one step at a time

  • RecedingHorizon. It runs causal_plan from each measured state, warm-started, and returns the certificate with every step. PeriodBudget caps what each period spends, from a ledger the caller measures. See chc.mpc.
  • robust_safety_filter. It clips one nominal action into the certified interval, at one state. See chc.barrier.
  • DeploymentGate. It runs a candidate policy in shadow of the baseline, per zone, and after each batch says deploy, hold, experiment or roll back. It can be read at any time: at every read, the expected share of wrong deploys across zones is at most alpha. That holds when the propensities were logged at decision time, zones do not spill over, and a decision's reward does not depend on earlier ones. A DecisionLog is the versioned record each decision stores for it: the action, the propensity it was drawn with, a flag for a clipped action and the dither as drawn. With alpha_futility (experimental), an experiment whose candidate is not worth min_effect returns to shadow, each such stop wrong with probability at most alpha_futility. On a plant with memory, block_length and burn_in (experimental) run the experiment in blocks: each block runs one policy, drawn at random, and the gate reads a BlockBatch of them after each block's burn-in, so that a deploy certifies the sustained effect, not the effect of one decision. See chc.gate.

6. Adapt: after deployment

  • SplitConformal. It calibrates an ensemble's next-state intervals on held-out logs. They cover while the logs and the loop are exchangeable, so a run of steps outside them says the model no longer describes the plant. See chc.uncertainty.
  • MissRateAlarm (experimental). It watches for that run. It calibrates a split-conformal radius on held-out one-step innovations, whitened at the noise's AR(1) persistence, and bounds their miss rate by its exact Beta law. It alarms once the deployed innovations miss more often than that bound allows. On a plant the model still describes, it ever alarms with probability at most delta. Unwhitened, AR(1) noise clusters the misses, and the alarm sounded on a correct plant several times as often as delta. See chc.gate.
  • OnlineRadius (experimental). It keeps a radius per horizon and moves it after each scored prediction: up after a miss, down otherwise. On any sequence of scores, also one the plan steers through its own states, each horizon's long-run miss rate comes within a stated bound of alpha. See chc.gate.
  • channel_drift_evalues and DriftAlarm (experimental). They watch a deployed plan's channel through the Gaussian dither its decisions log. Each decision gives e-values that every entry of the channel still lies within its identification radius of the model's. They hold whatever the model gets wrong in the drift, the noise law or the policy. DriftAlarm alarms on them with an average run length of at least its target while the channel holds, and DeploymentGate holds or rolls back a zone on them. Scope: a plant affine in the action over one step, a dither drawn as stated and logged as drawn, applied whole or cut by a box on its own action, and a radius that covers the identification error. See chc.gate.
  • channel_move (experimental). It re-reads the channel off the same dither once it has moved: how far each entry lies from the model's, with a covariance, and with older decisions forgotten at a constant rate. Its price reads the move in the plan's own cost, through CausalPlan.decision_weight(). That gives two expectations: what keeping the plan loses, estimated without bias, and what a plan re-solved on the estimate would lose to its noise. They are not a rule. Compared on one log, they choose on the log's own error. See chc.gate.
  • Not built: a re-solve on the estimate, since causal_plan plans on a continuous-time field and the move is a one-step one; and a rule that chooses between keeping and re-planning.